2024-03-11T18:15:06.942 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-11T18:15:06.942 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591231 --name teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi --verbose --description fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu/{overrides ubuntu_latest}} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/5 tasks/fsstress validater/valgrind} --owner scheduled_teuthology@teuthology -- /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591231/orig.config.yaml 2024-03-11T18:15:06.992 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-11T18:15:07.074 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591231 branch: squid description: fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu/{overrides ubuntu_latest}} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/5 tasks/fsstress validater/valgrind} email: ceph-qa@ceph.com first_in_suite: false job_id: '7591231' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mds beacon grace: 60 mds heartbeat grace: 60 osd heartbeat grace: 60 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds valgrind exit: true mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 mon osd crush smoke test: false osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd fast shutdown: false osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow request - SLOW_OPS - MON_DOWN sha1: 94a539707733325322964c6a187c6a1b41be1daf valgrind: exit_on_first_error: false mds: - --tool=memcheck mon: - --tool=memcheck - --leak-check=full - --show-reachable=yes watchdog: daemon_restart: normal ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: valgrind: - --tool=memcheck - --leak-check=full - --show-reachable=yes check-counter: counters: mds: - mds.exported - mds.imported install: ceph: debuginfo: true flavor: default sha1: 94a539707733325322964c6a187c6a1b41be1daf thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: b6a4997be2b8bb74411268cc42753f29ae9a78ad owner: scheduled_teuthology@teuthology priority: 250 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.y - mds.a - mds.c - mds.e - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - mds.f - osd.4 - osd.5 - osd.6 - osd.7 seed: 8179 sha1: 94a539707733325322964c6a187c6a1b41be1daf sleep_before_teardown: 0 subset: 297/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_b6a4997be2b8bb74411268cc42753f29ae9a78ad/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: b6a4997be2b8bb74411268cc42753f29ae9a78ad syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi090.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIJuV0sxi+snrwXTRt4/zPLb9SRqvHOFAqDX8Znz88gcSqAlDcU59fZUKUi9A3xYhq5fZD0ebLvwMIUVchiTyn8= smithi092.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBElYpEGkVAh1XHw3GH0jCDOU0ZVV0GfXVejBx1pRjf7vK5vqjzj0CWHmynSSiipAFCvhpWZVecthJQz5iDI3x3Y= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - workunit: clients: all: - suites/fsstress.sh timeout: 6h teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-11_14:02:35 tube: smithi user: teuthology verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-11T18:15:07.075 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_b6a4997be2b8bb74411268cc42753f29ae9a78ad/qa; will attempt to use it 2024-03-11T18:15:07.075 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_b6a4997be2b8bb74411268cc42753f29ae9a78ad/qa/tasks 2024-03-11T18:15:07.076 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-11T18:15:07.077 INFO:teuthology.task.internal:Checking packages... 2024-03-11T18:15:07.103 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '94a539707733325322964c6a187c6a1b41be1daf' 2024-03-11T18:15:07.103 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-11T18:15:07.104 INFO:teuthology.packaging:ref: None 2024-03-11T18:15:07.104 INFO:teuthology.packaging:tag: None 2024-03-11T18:15:07.104 INFO:teuthology.packaging:branch: squid 2024-03-11T18:15:07.104 INFO:teuthology.packaging:sha1: 94a539707733325322964c6a187c6a1b41be1daf 2024-03-11T18:15:07.104 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=squid 2024-03-11T18:15:07.332 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1607-g94a53970-1jammy 2024-03-11T18:15:07.333 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-11T18:15:07.340 INFO:teuthology.task.internal:no buildpackages task found 2024-03-11T18:15:07.340 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-11T18:15:07.364 INFO:teuthology.task.internal:Saving configuration 2024-03-11T18:15:07.380 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-11T18:15:07.387 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-11T18:15:07.411 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi090.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591231', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-11 18:08:48.894740', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHwLizRhMz8rGYSOrfPwiLBu5uGMiQJQoOkwX4qdsfV4'} 2024-03-11T18:15:07.434 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi092.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591231', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-11 18:08:48.896341', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAINQXRBfVF4Ayk/iULELD+ELBCPSSXHQjbRlMlMH7meO5'} 2024-03-11T18:15:07.435 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-11T18:15:07.442 INFO:teuthology.task.internal:roles: ubuntu@smithi090.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2024-03-11T18:15:07.442 INFO:teuthology.task.internal:roles: ubuntu@smithi092.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2024-03-11T18:15:07.442 INFO:teuthology.run_tasks:Running task console_log... 2024-03-11T18:15:07.511 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f3d45247ee0>, signals=[15]) 2024-03-11T18:15:07.512 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-11T18:15:07.518 INFO:teuthology.task.internal:Opening connections... 2024-03-11T18:15:07.518 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi090.front.sepia.ceph.com 2024-03-11T18:15:07.520 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-11T18:15:07.599 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi092.front.sepia.ceph.com 2024-03-11T18:15:07.600 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-11T18:15:07.681 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-11T18:15:07.688 DEBUG:teuthology.orchestra.run.smithi090:> uname -m 2024-03-11T18:15:07.693 INFO:teuthology.orchestra.run.smithi090.stdout:x86_64 2024-03-11T18:15:07.693 DEBUG:teuthology.orchestra.run.smithi090:> cat /etc/os-release 2024-03-11T18:15:07.742 INFO:teuthology.orchestra.run.smithi090.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-03-11T18:15:07.742 INFO:teuthology.orchestra.run.smithi090.stdout:NAME="Ubuntu" 2024-03-11T18:15:07.743 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION_ID="22.04" 2024-03-11T18:15:07.743 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-03-11T18:15:07.743 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION_CODENAME=jammy 2024-03-11T18:15:07.743 INFO:teuthology.orchestra.run.smithi090.stdout:ID=ubuntu 2024-03-11T18:15:07.743 INFO:teuthology.orchestra.run.smithi090.stdout:ID_LIKE=debian 2024-03-11T18:15:07.743 INFO:teuthology.orchestra.run.smithi090.stdout:HOME_URL="https://www.ubuntu.com/" 2024-03-11T18:15:07.743 INFO:teuthology.orchestra.run.smithi090.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-03-11T18:15:07.743 INFO:teuthology.orchestra.run.smithi090.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-03-11T18:15:07.743 INFO:teuthology.orchestra.run.smithi090.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-03-11T18:15:07.743 INFO:teuthology.orchestra.run.smithi090.stdout:UBUNTU_CODENAME=jammy 2024-03-11T18:15:07.744 INFO:teuthology.lock.ops:Updating smithi090.front.sepia.ceph.com on lock server 2024-03-11T18:15:07.762 DEBUG:teuthology.orchestra.run.smithi092:> uname -m 2024-03-11T18:15:07.767 INFO:teuthology.orchestra.run.smithi092.stdout:x86_64 2024-03-11T18:15:07.767 DEBUG:teuthology.orchestra.run.smithi092:> cat /etc/os-release 2024-03-11T18:15:07.813 INFO:teuthology.orchestra.run.smithi092.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-03-11T18:15:07.813 INFO:teuthology.orchestra.run.smithi092.stdout:NAME="Ubuntu" 2024-03-11T18:15:07.813 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION_ID="22.04" 2024-03-11T18:15:07.813 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-03-11T18:15:07.813 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION_CODENAME=jammy 2024-03-11T18:15:07.813 INFO:teuthology.orchestra.run.smithi092.stdout:ID=ubuntu 2024-03-11T18:15:07.813 INFO:teuthology.orchestra.run.smithi092.stdout:ID_LIKE=debian 2024-03-11T18:15:07.813 INFO:teuthology.orchestra.run.smithi092.stdout:HOME_URL="https://www.ubuntu.com/" 2024-03-11T18:15:07.813 INFO:teuthology.orchestra.run.smithi092.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-03-11T18:15:07.814 INFO:teuthology.orchestra.run.smithi092.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-03-11T18:15:07.814 INFO:teuthology.orchestra.run.smithi092.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-03-11T18:15:07.814 INFO:teuthology.orchestra.run.smithi092.stdout:UBUNTU_CODENAME=jammy 2024-03-11T18:15:07.814 INFO:teuthology.lock.ops:Updating smithi092.front.sepia.ceph.com on lock server 2024-03-11T18:15:07.837 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-11T18:15:07.847 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-11T18:15:07.853 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-11T18:15:07.854 DEBUG:teuthology.orchestra.run.smithi090:> test '!' -e /home/ubuntu/cephtest 2024-03-11T18:15:07.856 DEBUG:teuthology.orchestra.run.smithi092:> test '!' -e /home/ubuntu/cephtest 2024-03-11T18:15:07.860 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-11T18:15:07.866 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-11T18:15:07.866 DEBUG:teuthology.orchestra.run.smithi090:> test -z $(ls -A /var/lib/ceph) 2024-03-11T18:15:07.904 DEBUG:teuthology.orchestra.run.smithi092:> test -z $(ls -A /var/lib/ceph) 2024-03-11T18:15:07.923 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-11T18:15:08.013 INFO:teuthology.run_tasks:Running task kernel... 2024-03-11T18:15:08.029 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-03-11T18:15:08.030 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-03-11T18:15:08.030 DEBUG:teuthology.orchestra.run.smithi090:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-11T18:15:08.030 DEBUG:teuthology.orchestra.run.smithi092:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-11T18:15:08.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-11T18:15:08.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-11T18:15:08.038 DEBUG:teuthology.orchestra.run.smithi092:> uname -r 2024-03-11T18:15:08.039 DEBUG:teuthology.orchestra.run.smithi090:> uname -r 2024-03-11T18:15:08.085 INFO:teuthology.orchestra.run.smithi092.stdout:5.15.0-56-generic 2024-03-11T18:15:08.086 INFO:teuthology.task.kernel:Running kernel on smithi092: 5.15.0-56-generic 2024-03-11T18:15:08.086 DEBUG:teuthology.orchestra.run.smithi092:> sudo apt-get clean 2024-03-11T18:15:08.087 INFO:teuthology.orchestra.run.smithi090.stdout:5.15.0-56-generic 2024-03-11T18:15:08.087 INFO:teuthology.task.kernel:Running kernel on smithi090: 5.15.0-56-generic 2024-03-11T18:15:08.087 DEBUG:teuthology.orchestra.run.smithi090:> sudo apt-get clean 2024-03-11T18:15:08.211 DEBUG:teuthology.orchestra.run.smithi090:> sudo apt-get update 2024-03-11T18:15:08.222 DEBUG:teuthology.orchestra.run.smithi092:> sudo apt-get update 2024-03-11T18:15:08.382 INFO:teuthology.orchestra.run.smithi090.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-11T18:15:08.382 INFO:teuthology.orchestra.run.smithi090.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-03-11T18:15:08.382 INFO:teuthology.orchestra.run.smithi090.stdout:Get:3 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-03-11T18:15:08.463 INFO:teuthology.orchestra.run.smithi090.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-03-11T18:15:08.525 INFO:teuthology.orchestra.run.smithi092.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-11T18:15:08.525 INFO:teuthology.orchestra.run.smithi092.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-03-11T18:15:08.567 INFO:teuthology.orchestra.run.smithi092.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-03-11T18:15:08.627 INFO:teuthology.orchestra.run.smithi090.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [419 kB] 2024-03-11T18:15:08.700 INFO:teuthology.orchestra.run.smithi090.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [585 kB] 2024-03-11T18:15:08.706 INFO:teuthology.orchestra.run.smithi090.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,241 kB] 2024-03-11T18:15:08.750 INFO:teuthology.orchestra.run.smithi090.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [223 kB] 2024-03-11T18:15:08.752 INFO:teuthology.orchestra.run.smithi090.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-03-11T18:15:08.753 INFO:teuthology.orchestra.run.smithi090.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,526 kB] 2024-03-11T18:15:08.774 INFO:teuthology.orchestra.run.smithi090.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,460 kB] 2024-03-11T18:15:08.776 INFO:teuthology.orchestra.run.smithi090.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [33.7 kB] 2024-03-11T18:15:08.777 INFO:teuthology.orchestra.run.smithi090.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [253 kB] 2024-03-11T18:15:08.781 INFO:teuthology.orchestra.run.smithi090.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-03-11T18:15:08.781 INFO:teuthology.orchestra.run.smithi090.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [597 kB] 2024-03-11T18:15:08.789 INFO:teuthology.orchestra.run.smithi090.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [848 kB] 2024-03-11T18:15:08.801 INFO:teuthology.orchestra.run.smithi090.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [162 kB] 2024-03-11T18:15:08.803 INFO:teuthology.orchestra.run.smithi090.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-03-11T18:15:08.808 INFO:teuthology.orchestra.run.smithi090.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.1 kB] 2024-03-11T18:15:08.845 INFO:teuthology.orchestra.run.smithi090.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-03-11T18:15:08.845 INFO:teuthology.orchestra.run.smithi090.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [283 kB] 2024-03-11T18:15:08.845 INFO:teuthology.orchestra.run.smithi090.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,476 B] 2024-03-11T18:15:08.845 INFO:teuthology.orchestra.run.smithi090.stdout:Get:23 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-03-11T18:15:08.850 INFO:teuthology.orchestra.run.smithi090.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-03-11T18:15:08.851 INFO:teuthology.orchestra.run.smithi090.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,559 kB] 2024-03-11T18:15:08.876 INFO:teuthology.orchestra.run.smithi090.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [35.3 kB] 2024-03-11T18:15:08.877 INFO:teuthology.orchestra.run.smithi090.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [259 kB] 2024-03-11T18:15:08.881 INFO:teuthology.orchestra.run.smithi090.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-03-11T18:15:08.881 INFO:teuthology.orchestra.run.smithi090.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,054 kB] 2024-03-11T18:15:08.894 INFO:teuthology.orchestra.run.smithi090.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [693 kB] 2024-03-11T18:15:08.900 INFO:teuthology.orchestra.run.smithi090.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [238 kB] 2024-03-11T18:15:08.903 INFO:teuthology.orchestra.run.smithi090.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-03-11T18:15:08.903 INFO:teuthology.orchestra.run.smithi090.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.1 kB] 2024-03-11T18:15:08.904 INFO:teuthology.orchestra.run.smithi090.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,160 B] 2024-03-11T18:15:08.918 INFO:teuthology.orchestra.run.smithi090.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.1 kB] 2024-03-11T18:15:08.919 INFO:teuthology.orchestra.run.smithi090.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-03-11T18:15:08.919 INFO:teuthology.orchestra.run.smithi090.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [41.7 kB] 2024-03-11T18:15:08.919 INFO:teuthology.orchestra.run.smithi090.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [33.9 kB] 2024-03-11T18:15:08.920 INFO:teuthology.orchestra.run.smithi090.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [10.5 kB] 2024-03-11T18:15:08.924 INFO:teuthology.orchestra.run.smithi092.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,241 kB] 2024-03-11T18:15:08.928 INFO:teuthology.orchestra.run.smithi092.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-03-11T18:15:08.939 INFO:teuthology.orchestra.run.smithi090.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-03-11T18:15:08.939 INFO:teuthology.orchestra.run.smithi090.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [13.4 kB] 2024-03-11T18:15:08.939 INFO:teuthology.orchestra.run.smithi090.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [24.3 kB] 2024-03-11T18:15:08.940 INFO:teuthology.orchestra.run.smithi090.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-03-11T18:15:08.940 INFO:teuthology.orchestra.run.smithi090.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-03-11T18:15:09.068 INFO:teuthology.orchestra.run.smithi092.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,460 kB] 2024-03-11T18:15:09.339 INFO:teuthology.orchestra.run.smithi092.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [419 kB] 2024-03-11T18:15:09.357 INFO:teuthology.orchestra.run.smithi092.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [223 kB] 2024-03-11T18:15:09.366 INFO:teuthology.orchestra.run.smithi092.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [585 kB] 2024-03-11T18:15:09.367 INFO:teuthology.orchestra.run.smithi092.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-03-11T18:15:09.367 INFO:teuthology.orchestra.run.smithi092.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [33.7 kB] 2024-03-11T18:15:09.369 INFO:teuthology.orchestra.run.smithi092.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,526 kB] 2024-03-11T18:15:09.404 INFO:teuthology.orchestra.run.smithi092.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [283 kB] 2024-03-11T18:15:09.421 INFO:teuthology.orchestra.run.smithi092.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-03-11T18:15:09.421 INFO:teuthology.orchestra.run.smithi092.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,559 kB] 2024-03-11T18:15:09.460 INFO:teuthology.orchestra.run.smithi092.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [253 kB] 2024-03-11T18:15:09.506 INFO:teuthology.orchestra.run.smithi092.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [35.3 kB] 2024-03-11T18:15:09.506 INFO:teuthology.orchestra.run.smithi092.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-03-11T18:15:09.506 INFO:teuthology.orchestra.run.smithi092.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [597 kB] 2024-03-11T18:15:09.507 INFO:teuthology.orchestra.run.smithi092.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [259 kB] 2024-03-11T18:15:09.512 INFO:teuthology.orchestra.run.smithi092.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-03-11T18:15:09.517 INFO:teuthology.orchestra.run.smithi092.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [848 kB] 2024-03-11T18:15:09.678 INFO:teuthology.orchestra.run.smithi092.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [693 kB] 2024-03-11T18:15:09.695 INFO:teuthology.orchestra.run.smithi092.stdout:Get:24 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [162 kB] 2024-03-11T18:15:10.091 INFO:teuthology.orchestra.run.smithi092.stdout:Get:25 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-03-11T18:15:10.097 INFO:teuthology.orchestra.run.smithi092.stdout:Get:26 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.1 kB] 2024-03-11T18:15:10.111 INFO:teuthology.orchestra.run.smithi092.stdout:Get:27 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-03-11T18:15:10.112 INFO:teuthology.orchestra.run.smithi092.stdout:Get:28 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,476 B] 2024-03-11T18:15:10.114 INFO:teuthology.orchestra.run.smithi092.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-03-11T18:15:10.260 INFO:teuthology.orchestra.run.smithi092.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,054 kB] 2024-03-11T18:15:10.370 INFO:teuthology.orchestra.run.smithi092.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [238 kB] 2024-03-11T18:15:10.390 INFO:teuthology.orchestra.run.smithi092.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-03-11T18:15:10.392 INFO:teuthology.orchestra.run.smithi092.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.1 kB] 2024-03-11T18:15:10.395 INFO:teuthology.orchestra.run.smithi092.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,160 B] 2024-03-11T18:15:10.395 INFO:teuthology.orchestra.run.smithi092.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.1 kB] 2024-03-11T18:15:10.396 INFO:teuthology.orchestra.run.smithi092.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-03-11T18:15:10.396 INFO:teuthology.orchestra.run.smithi092.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [33.9 kB] 2024-03-11T18:15:10.398 INFO:teuthology.orchestra.run.smithi092.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [41.7 kB] 2024-03-11T18:15:10.401 INFO:teuthology.orchestra.run.smithi092.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [10.5 kB] 2024-03-11T18:15:10.566 INFO:teuthology.orchestra.run.smithi092.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-03-11T18:15:10.649 INFO:teuthology.orchestra.run.smithi092.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [24.3 kB] 2024-03-11T18:15:10.745 INFO:teuthology.orchestra.run.smithi092.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [13.4 kB] 2024-03-11T18:15:10.773 INFO:teuthology.orchestra.run.smithi092.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-03-11T18:15:10.825 INFO:teuthology.orchestra.run.smithi092.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-03-11T18:15:14.268 INFO:teuthology.orchestra.run.smithi090.stdout:Fetched 12.1 MB in 2s (7,106 kB/s) 2024-03-11T18:15:15.279 INFO:teuthology.orchestra.run.smithi092.stdout:Fetched 12.1 MB in 3s (4,785 kB/s) 2024-03-11T18:15:15.690 INFO:teuthology.orchestra.run.smithi090.stdout:Reading package lists... 2024-03-11T18:15:15.707 DEBUG:teuthology.orchestra.run.smithi090:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-03-11T18:15:15.769 INFO:teuthology.orchestra.run.smithi090.stdout:Reading package lists... 2024-03-11T18:15:15.962 INFO:teuthology.orchestra.run.smithi090.stdout:Building dependency tree... 2024-03-11T18:15:15.963 INFO:teuthology.orchestra.run.smithi090.stdout:Reading state information... 2024-03-11T18:15:16.130 INFO:teuthology.orchestra.run.smithi090.stdout:The following packages were automatically installed and are no longer required: 2024-03-11T18:15:16.130 INFO:teuthology.orchestra.run.smithi090.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-11T18:15:16.130 INFO:teuthology.orchestra.run.smithi090.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-11T18:15:16.131 INFO:teuthology.orchestra.run.smithi090.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-11T18:15:16.131 INFO:teuthology.orchestra.run.smithi090.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-03-11T18:15:16.131 INFO:teuthology.orchestra.run.smithi090.stdout: python2.7-minimal 2024-03-11T18:15:16.131 INFO:teuthology.orchestra.run.smithi090.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-11T18:15:16.132 INFO:teuthology.orchestra.run.smithi090.stdout:The following additional packages will be installed: 2024-03-11T18:15:16.132 INFO:teuthology.orchestra.run.smithi090.stdout: linux-generic linux-headers-5.15.0-100 linux-headers-5.15.0-100-generic 2024-03-11T18:15:16.132 INFO:teuthology.orchestra.run.smithi090.stdout: linux-headers-generic linux-image-5.15.0-100-generic 2024-03-11T18:15:16.133 INFO:teuthology.orchestra.run.smithi090.stdout: linux-modules-5.15.0-100-generic linux-modules-extra-5.15.0-100-generic 2024-03-11T18:15:16.134 INFO:teuthology.orchestra.run.smithi090.stdout:Suggested packages: 2024-03-11T18:15:16.134 INFO:teuthology.orchestra.run.smithi090.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-03-11T18:15:16.134 INFO:teuthology.orchestra.run.smithi090.stdout:Recommended packages: 2024-03-11T18:15:16.134 INFO:teuthology.orchestra.run.smithi090.stdout: thermald 2024-03-11T18:15:16.209 INFO:teuthology.orchestra.run.smithi090.stdout:The following NEW packages will be installed: 2024-03-11T18:15:16.209 INFO:teuthology.orchestra.run.smithi090.stdout: linux-headers-5.15.0-100 linux-headers-5.15.0-100-generic 2024-03-11T18:15:16.209 INFO:teuthology.orchestra.run.smithi090.stdout: linux-image-5.15.0-100-generic linux-modules-5.15.0-100-generic 2024-03-11T18:15:16.209 INFO:teuthology.orchestra.run.smithi090.stdout: linux-modules-extra-5.15.0-100-generic 2024-03-11T18:15:16.211 INFO:teuthology.orchestra.run.smithi090.stdout:The following packages will be upgraded: 2024-03-11T18:15:16.211 INFO:teuthology.orchestra.run.smithi090.stdout: linux-generic linux-headers-generic linux-image-generic 2024-03-11T18:15:16.267 INFO:teuthology.orchestra.run.smithi090.stdout:3 upgraded, 5 newly installed, 0 to remove and 288 not upgraded. 2024-03-11T18:15:16.267 INFO:teuthology.orchestra.run.smithi090.stdout:Need to get 113 MB of archives. 2024-03-11T18:15:16.267 INFO:teuthology.orchestra.run.smithi090.stdout:After this operation, 582 MB of additional disk space will be used. 2024-03-11T18:15:16.267 INFO:teuthology.orchestra.run.smithi090.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-100-generic amd64 5.15.0-100.110 [22.6 MB] 2024-03-11T18:15:16.629 INFO:teuthology.orchestra.run.smithi092.stdout:Reading package lists... 2024-03-11T18:15:16.648 DEBUG:teuthology.orchestra.run.smithi092:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-03-11T18:15:16.713 INFO:teuthology.orchestra.run.smithi092.stdout:Reading package lists... 2024-03-11T18:15:16.859 INFO:teuthology.orchestra.run.smithi090.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-100-generic amd64 5.15.0-100.110 [11.5 MB] 2024-03-11T18:15:16.908 INFO:teuthology.orchestra.run.smithi092.stdout:Building dependency tree... 2024-03-11T18:15:16.909 INFO:teuthology.orchestra.run.smithi092.stdout:Reading state information... 2024-03-11T18:15:17.078 INFO:teuthology.orchestra.run.smithi090.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-100-generic amd64 5.15.0-100.110 [63.8 MB] 2024-03-11T18:15:17.080 INFO:teuthology.orchestra.run.smithi092.stdout:The following packages were automatically installed and are no longer required: 2024-03-11T18:15:17.080 INFO:teuthology.orchestra.run.smithi092.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-11T18:15:17.080 INFO:teuthology.orchestra.run.smithi092.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-11T18:15:17.081 INFO:teuthology.orchestra.run.smithi092.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-11T18:15:17.081 INFO:teuthology.orchestra.run.smithi092.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-03-11T18:15:17.081 INFO:teuthology.orchestra.run.smithi092.stdout: python2.7-minimal 2024-03-11T18:15:17.081 INFO:teuthology.orchestra.run.smithi092.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-11T18:15:17.083 INFO:teuthology.orchestra.run.smithi092.stdout:The following additional packages will be installed: 2024-03-11T18:15:17.083 INFO:teuthology.orchestra.run.smithi092.stdout: linux-generic linux-headers-5.15.0-100 linux-headers-5.15.0-100-generic 2024-03-11T18:15:17.083 INFO:teuthology.orchestra.run.smithi092.stdout: linux-headers-generic linux-image-5.15.0-100-generic 2024-03-11T18:15:17.083 INFO:teuthology.orchestra.run.smithi092.stdout: linux-modules-5.15.0-100-generic linux-modules-extra-5.15.0-100-generic 2024-03-11T18:15:17.084 INFO:teuthology.orchestra.run.smithi092.stdout:Suggested packages: 2024-03-11T18:15:17.084 INFO:teuthology.orchestra.run.smithi092.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-03-11T18:15:17.084 INFO:teuthology.orchestra.run.smithi092.stdout:Recommended packages: 2024-03-11T18:15:17.084 INFO:teuthology.orchestra.run.smithi092.stdout: thermald 2024-03-11T18:15:17.143 INFO:teuthology.orchestra.run.smithi092.stdout:The following NEW packages will be installed: 2024-03-11T18:15:17.143 INFO:teuthology.orchestra.run.smithi092.stdout: linux-headers-5.15.0-100 linux-headers-5.15.0-100-generic 2024-03-11T18:15:17.143 INFO:teuthology.orchestra.run.smithi092.stdout: linux-image-5.15.0-100-generic linux-modules-5.15.0-100-generic 2024-03-11T18:15:17.143 INFO:teuthology.orchestra.run.smithi092.stdout: linux-modules-extra-5.15.0-100-generic 2024-03-11T18:15:17.144 INFO:teuthology.orchestra.run.smithi092.stdout:The following packages will be upgraded: 2024-03-11T18:15:17.145 INFO:teuthology.orchestra.run.smithi092.stdout: linux-generic linux-headers-generic linux-image-generic 2024-03-11T18:15:17.256 INFO:teuthology.orchestra.run.smithi092.stdout:3 upgraded, 5 newly installed, 0 to remove and 288 not upgraded. 2024-03-11T18:15:17.256 INFO:teuthology.orchestra.run.smithi092.stdout:Need to get 113 MB of archives. 2024-03-11T18:15:17.256 INFO:teuthology.orchestra.run.smithi092.stdout:After this operation, 582 MB of additional disk space will be used. 2024-03-11T18:15:17.256 INFO:teuthology.orchestra.run.smithi092.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-100-generic amd64 5.15.0-100.110 [22.6 MB] 2024-03-11T18:15:18.311 INFO:teuthology.orchestra.run.smithi092.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-100-generic amd64 5.15.0-100.110 [11.5 MB] 2024-03-11T18:15:18.538 INFO:teuthology.orchestra.run.smithi092.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-100-generic amd64 5.15.0-100.110 [63.8 MB] 2024-03-11T18:15:18.923 INFO:teuthology.orchestra.run.smithi090.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.100.97 [1,696 B] 2024-03-11T18:15:18.924 INFO:teuthology.orchestra.run.smithi090.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.100.97 [2,454 B] 2024-03-11T18:15:18.924 INFO:teuthology.orchestra.run.smithi090.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-100 all 5.15.0-100.110 [12.3 MB] 2024-03-11T18:15:19.208 INFO:teuthology.orchestra.run.smithi090.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-100-generic amd64 5.15.0-100.110 [2,858 kB] 2024-03-11T18:15:19.286 INFO:teuthology.orchestra.run.smithi090.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.100.97 [2,292 B] 2024-03-11T18:15:19.653 INFO:teuthology.orchestra.run.smithi090.stdout:Fetched 113 MB in 3s (36.9 MB/s) 2024-03-11T18:15:19.813 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package linux-modules-5.15.0-100-generic. 2024-03-11T18:15:19.863 INFO:teuthology.orchestra.run.smithi092.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.100.97 [1,696 B] 2024-03-11T18:15:19.863 INFO:teuthology.orchestra.run.smithi092.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.100.97 [2,454 B] 2024-03-11T18:15:19.863 INFO:teuthology.orchestra.run.smithi092.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-100 all 5.15.0-100.110 [12.3 MB] 2024-03-11T18:15:20.036 INFO:teuthology.orchestra.run.smithi092.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-100-generic amd64 5.15.0-100.110 [2,858 kB] 2024-03-11T18:15:20.074 INFO:teuthology.orchestra.run.smithi092.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.100.97 [2,292 B] 2024-03-11T18:15:20.459 INFO:teuthology.orchestra.run.smithi092.stdout:Fetched 113 MB in 3s (38.7 MB/s) 2024-03-11T18:15:20.651 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package linux-modules-5.15.0-100-generic. 2024-03-11T18:15:22.392 INFO:teuthology.orchestra.run.smithi090.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-03-11T18:15:22.398 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../0-linux-modules-5.15.0-100-generic_5.15.0-100.110_amd64.deb ... 2024-03-11T18:15:22.467 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking linux-modules-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-11T18:15:22.678 INFO:teuthology.orchestra.run.smithi092.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-03-11T18:15:22.682 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../0-linux-modules-5.15.0-100-generic_5.15.0-100.110_amd64.deb ... 2024-03-11T18:15:22.832 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking linux-modules-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-11T18:15:25.349 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package linux-image-5.15.0-100-generic. 2024-03-11T18:15:25.370 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../1-linux-image-5.15.0-100-generic_5.15.0-100.110_amd64.deb ... 2024-03-11T18:15:25.458 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking linux-image-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-11T18:15:25.723 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package linux-image-5.15.0-100-generic. 2024-03-11T18:15:25.744 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../1-linux-image-5.15.0-100-generic_5.15.0-100.110_amd64.deb ... 2024-03-11T18:15:25.830 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking linux-image-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-11T18:15:25.935 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-100-generic. 2024-03-11T18:15:25.956 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-100-generic_5.15.0-100.110_amd64.deb ... 2024-03-11T18:15:26.014 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking linux-modules-extra-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-11T18:15:26.312 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-100-generic. 2024-03-11T18:15:26.332 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-100-generic_5.15.0-100.110_amd64.deb ... 2024-03-11T18:15:26.378 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking linux-modules-extra-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-11T18:15:32.562 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../3-linux-generic_5.15.0.100.97_amd64.deb ... 2024-03-11T18:15:32.680 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking linux-generic (5.15.0.100.97) over (5.15.0.56.54) ... 2024-03-11T18:15:32.988 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.100.97_amd64.deb ... 2024-03-11T18:15:33.084 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking linux-image-generic (5.15.0.100.97) over (5.15.0.56.54) ... 2024-03-11T18:15:33.412 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package linux-headers-5.15.0-100. 2024-03-11T18:15:33.435 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../5-linux-headers-5.15.0-100_5.15.0-100.110_all.deb ... 2024-03-11T18:15:33.479 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking linux-headers-5.15.0-100 (5.15.0-100.110) ... 2024-03-11T18:15:35.368 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../3-linux-generic_5.15.0.100.97_amd64.deb ... 2024-03-11T18:15:35.484 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking linux-generic (5.15.0.100.97) over (5.15.0.56.54) ... 2024-03-11T18:15:35.793 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.100.97_amd64.deb ... 2024-03-11T18:15:35.914 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking linux-image-generic (5.15.0.100.97) over (5.15.0.56.54) ... 2024-03-11T18:15:36.218 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package linux-headers-5.15.0-100. 2024-03-11T18:15:36.240 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../5-linux-headers-5.15.0-100_5.15.0-100.110_all.deb ... 2024-03-11T18:15:36.284 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking linux-headers-5.15.0-100 (5.15.0-100.110) ... 2024-03-11T18:15:39.301 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package linux-headers-5.15.0-100-generic. 2024-03-11T18:15:39.337 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../6-linux-headers-5.15.0-100-generic_5.15.0-100.110_amd64.deb ... 2024-03-11T18:15:39.375 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking linux-headers-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-11T18:15:41.990 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.100.97_amd64.deb ... 2024-03-11T18:15:42.113 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking linux-headers-generic (5.15.0.100.97) over (5.15.0.56.54) ... 2024-03-11T18:15:42.454 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up linux-headers-5.15.0-100 (5.15.0-100.110) ... 2024-03-11T18:15:42.559 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up linux-headers-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-11T18:15:42.660 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up linux-headers-generic (5.15.0.100.97) ... 2024-03-11T18:15:42.772 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up linux-image-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-11T18:15:43.452 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package linux-headers-5.15.0-100-generic. 2024-03-11T18:15:43.489 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../6-linux-headers-5.15.0-100-generic_5.15.0-100.110_amd64.deb ... 2024-03-11T18:15:43.535 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking linux-headers-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-11T18:15:44.204 INFO:teuthology.orchestra.run.smithi092.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-03-11T18:15:44.204 INFO:teuthology.orchestra.run.smithi092.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-03-11T18:15:44.204 INFO:teuthology.orchestra.run.smithi092.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-100-generic 2024-03-11T18:15:44.204 INFO:teuthology.orchestra.run.smithi092.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-100-generic 2024-03-11T18:15:44.593 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up linux-modules-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-11T18:15:46.002 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up linux-modules-extra-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-11T18:15:46.420 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.100.97_amd64.deb ... 2024-03-11T18:15:46.534 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking linux-headers-generic (5.15.0.100.97) over (5.15.0.56.54) ... 2024-03-11T18:15:46.903 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up linux-headers-5.15.0-100 (5.15.0-100.110) ... 2024-03-11T18:15:47.033 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up linux-headers-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-11T18:15:47.168 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up linux-headers-generic (5.15.0.100.97) ... 2024-03-11T18:15:47.305 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up linux-image-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-11T18:15:47.445 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up linux-image-generic (5.15.0.100.97) ... 2024-03-11T18:15:47.571 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up linux-generic (5.15.0.100.97) ... 2024-03-11T18:15:47.697 INFO:teuthology.orchestra.run.smithi092.stdout:Processing triggers for linux-image-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-11T18:15:47.780 INFO:teuthology.orchestra.run.smithi092.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-03-11T18:15:47.780 INFO:teuthology.orchestra.run.smithi092.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-100-generic 2024-03-11T18:15:48.638 INFO:teuthology.orchestra.run.smithi090.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-03-11T18:15:48.638 INFO:teuthology.orchestra.run.smithi090.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-03-11T18:15:48.638 INFO:teuthology.orchestra.run.smithi090.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-100-generic 2024-03-11T18:15:48.638 INFO:teuthology.orchestra.run.smithi090.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-100-generic 2024-03-11T18:15:48.803 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up linux-modules-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-11T18:15:51.038 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up linux-modules-extra-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-11T18:15:52.449 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up linux-image-generic (5.15.0.100.97) ... 2024-03-11T18:15:52.567 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up linux-generic (5.15.0.100.97) ... 2024-03-11T18:15:52.694 INFO:teuthology.orchestra.run.smithi090.stdout:Processing triggers for linux-image-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-11T18:15:52.771 INFO:teuthology.orchestra.run.smithi090.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-03-11T18:15:52.771 INFO:teuthology.orchestra.run.smithi090.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-100-generic 2024-03-11T18:16:01.781 INFO:teuthology.orchestra.run.smithi092.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-03-11T18:16:01.781 INFO:teuthology.orchestra.run.smithi092.stdout:Sourcing file `/etc/default/grub' 2024-03-11T18:16:01.811 INFO:teuthology.orchestra.run.smithi092.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-11T18:16:01.826 INFO:teuthology.orchestra.run.smithi092.stdout:Generating grub configuration file ... 2024-03-11T18:16:02.092 INFO:teuthology.orchestra.run.smithi092.stdout:Found linux image: /boot/vmlinuz-5.15.0-100-generic 2024-03-11T18:16:02.117 INFO:teuthology.orchestra.run.smithi092.stdout:Found initrd image: /boot/initrd.img-5.15.0-100-generic 2024-03-11T18:16:02.453 INFO:teuthology.orchestra.run.smithi092.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-11T18:16:02.459 INFO:teuthology.orchestra.run.smithi092.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-11T18:16:02.527 INFO:teuthology.orchestra.run.smithi092.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-11T18:16:02.532 INFO:teuthology.orchestra.run.smithi092.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-11T18:16:02.692 INFO:teuthology.orchestra.run.smithi092.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-11T18:16:02.692 INFO:teuthology.orchestra.run.smithi092.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-03-11T18:16:02.692 INFO:teuthology.orchestra.run.smithi092.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-11T18:16:02.737 INFO:teuthology.orchestra.run.smithi092.stdout:done 2024-03-11T18:16:03.196 DEBUG:teuthology.orchestra.run.smithi092:> dpkg -s linux-image-generic 2024-03-11T18:16:03.212 INFO:teuthology.orchestra.run.smithi092.stdout:Package: linux-image-generic 2024-03-11T18:16:03.212 INFO:teuthology.orchestra.run.smithi092.stdout:Status: install ok installed 2024-03-11T18:16:03.212 INFO:teuthology.orchestra.run.smithi092.stdout:Priority: optional 2024-03-11T18:16:03.212 INFO:teuthology.orchestra.run.smithi092.stdout:Section: kernel 2024-03-11T18:16:03.212 INFO:teuthology.orchestra.run.smithi092.stdout:Installed-Size: 21 2024-03-11T18:16:03.212 INFO:teuthology.orchestra.run.smithi092.stdout:Maintainer: Ubuntu Kernel Team 2024-03-11T18:16:03.212 INFO:teuthology.orchestra.run.smithi092.stdout:Architecture: amd64 2024-03-11T18:16:03.212 INFO:teuthology.orchestra.run.smithi092.stdout:Source: linux-meta 2024-03-11T18:16:03.212 INFO:teuthology.orchestra.run.smithi092.stdout:Version: 5.15.0.100.97 2024-03-11T18:16:03.212 INFO:teuthology.orchestra.run.smithi092.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.2), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-100), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.2) 2024-03-11T18:16:03.213 INFO:teuthology.orchestra.run.smithi092.stdout:Depends: linux-image-5.15.0-100-generic, linux-modules-extra-5.15.0-100-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-11T18:16:03.213 INFO:teuthology.orchestra.run.smithi092.stdout:Recommends: thermald 2024-03-11T18:16:03.213 INFO:teuthology.orchestra.run.smithi092.stdout:Description: Generic Linux kernel image 2024-03-11T18:16:03.213 INFO:teuthology.orchestra.run.smithi092.stdout: This package will always depend on the latest generic kernel image 2024-03-11T18:16:03.213 INFO:teuthology.orchestra.run.smithi092.stdout: available. 2024-03-11T18:16:03.213 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-100-generic 2024-03-11T18:16:03.213 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-11T18:16:03.213 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-11T18:16:03.213 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-03-11T18:16:03.214 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi092.front.sepia.ceph.com, path=None, version=distro) 2024-03-11T18:16:03.214 DEBUG:teuthology.orchestra.run.smithi092:> sudo apt-get clean 2024-03-11T18:16:03.376 DEBUG:teuthology.orchestra.run.smithi092:> sudo apt-get update 2024-03-11T18:16:03.532 INFO:teuthology.orchestra.run.smithi092.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-11T18:16:03.542 INFO:teuthology.orchestra.run.smithi092.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-11T18:16:03.565 INFO:teuthology.orchestra.run.smithi092.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-11T18:16:03.613 INFO:teuthology.orchestra.run.smithi092.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-11T18:16:04.882 INFO:teuthology.orchestra.run.smithi092.stdout:Reading package lists... 2024-03-11T18:16:04.900 DEBUG:teuthology.orchestra.run.smithi092:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-03-11T18:16:04.959 INFO:teuthology.orchestra.run.smithi092.stdout:Reading package lists... 2024-03-11T18:16:05.152 INFO:teuthology.orchestra.run.smithi092.stdout:Building dependency tree... 2024-03-11T18:16:05.153 INFO:teuthology.orchestra.run.smithi092.stdout:Reading state information... 2024-03-11T18:16:05.324 INFO:teuthology.orchestra.run.smithi092.stdout:linux-image-generic is already the newest version (5.15.0.100.97). 2024-03-11T18:16:05.324 INFO:teuthology.orchestra.run.smithi092.stdout:The following packages were automatically installed and are no longer required: 2024-03-11T18:16:05.324 INFO:teuthology.orchestra.run.smithi092.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-11T18:16:05.324 INFO:teuthology.orchestra.run.smithi092.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-11T18:16:05.324 INFO:teuthology.orchestra.run.smithi092.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-11T18:16:05.324 INFO:teuthology.orchestra.run.smithi092.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-11T18:16:05.325 INFO:teuthology.orchestra.run.smithi092.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-11T18:16:05.325 INFO:teuthology.orchestra.run.smithi092.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-11T18:16:05.364 INFO:teuthology.orchestra.run.smithi092.stdout:0 upgraded, 0 newly installed, 0 to remove and 288 not upgraded. 2024-03-11T18:16:05.365 DEBUG:teuthology.orchestra.run.smithi092:> dpkg -s linux-image-generic 2024-03-11T18:16:05.378 INFO:teuthology.orchestra.run.smithi092.stdout:Package: linux-image-generic 2024-03-11T18:16:05.378 INFO:teuthology.orchestra.run.smithi092.stdout:Status: install ok installed 2024-03-11T18:16:05.378 INFO:teuthology.orchestra.run.smithi092.stdout:Priority: optional 2024-03-11T18:16:05.378 INFO:teuthology.orchestra.run.smithi092.stdout:Section: kernel 2024-03-11T18:16:05.378 INFO:teuthology.orchestra.run.smithi092.stdout:Installed-Size: 21 2024-03-11T18:16:05.378 INFO:teuthology.orchestra.run.smithi092.stdout:Maintainer: Ubuntu Kernel Team 2024-03-11T18:16:05.378 INFO:teuthology.orchestra.run.smithi092.stdout:Architecture: amd64 2024-03-11T18:16:05.378 INFO:teuthology.orchestra.run.smithi092.stdout:Source: linux-meta 2024-03-11T18:16:05.378 INFO:teuthology.orchestra.run.smithi092.stdout:Version: 5.15.0.100.97 2024-03-11T18:16:05.378 INFO:teuthology.orchestra.run.smithi092.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.2), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-100), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.2) 2024-03-11T18:16:05.378 INFO:teuthology.orchestra.run.smithi092.stdout:Depends: linux-image-5.15.0-100-generic, linux-modules-extra-5.15.0-100-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-11T18:16:05.378 INFO:teuthology.orchestra.run.smithi092.stdout:Recommends: thermald 2024-03-11T18:16:05.379 INFO:teuthology.orchestra.run.smithi092.stdout:Description: Generic Linux kernel image 2024-03-11T18:16:05.379 INFO:teuthology.orchestra.run.smithi092.stdout: This package will always depend on the latest generic kernel image 2024-03-11T18:16:05.379 INFO:teuthology.orchestra.run.smithi092.stdout: available. 2024-03-11T18:16:05.379 DEBUG:teuthology.orchestra.run.smithi092:> mktemp 2024-03-11T18:16:05.425 INFO:teuthology.orchestra.run.smithi092.stdout:/tmp/tmp.6UCWrz2m1r 2024-03-11T18:16:05.425 DEBUG:teuthology.orchestra.run.smithi092:> sudo cp /boot/grub/grub.cfg /tmp/tmp.6UCWrz2m1r 2024-03-11T18:16:05.480 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0666 /tmp/tmp.6UCWrz2m1r 2024-03-11T18:16:05.601 DEBUG:teuthology.orchestra.remote:smithi092:/tmp/tmp.6UCWrz2m1r is 10KB 2024-03-11T18:16:05.614 DEBUG:teuthology.orchestra.run.smithi092:> rm -fr /tmp/tmp.6UCWrz2m1r 2024-03-11T18:16:05.619 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-03-11T18:16:05.672 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-11T18:16:05.672 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-03-11T18:16:05.672 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-03-11T18:16:05.740 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-100-generic 2024-03-11T18:16:05.740 DEBUG:teuthology.orchestra.run.smithi092:> sudo update-grub 2024-03-11T18:16:06.876 INFO:teuthology.orchestra.run.smithi092.stderr:Sourcing file `/etc/default/grub' 2024-03-11T18:16:06.877 INFO:teuthology.orchestra.run.smithi092.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-11T18:16:06.879 INFO:teuthology.orchestra.run.smithi092.stderr:Generating grub configuration file ... 2024-03-11T18:16:07.056 INFO:teuthology.orchestra.run.smithi092.stderr:Found linux image: /boot/vmlinuz-5.15.0-100-generic 2024-03-11T18:16:07.062 INFO:teuthology.orchestra.run.smithi092.stderr:Found initrd image: /boot/initrd.img-5.15.0-100-generic 2024-03-11T18:16:07.346 INFO:teuthology.orchestra.run.smithi092.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-11T18:16:07.351 INFO:teuthology.orchestra.run.smithi092.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-11T18:16:07.396 INFO:teuthology.orchestra.run.smithi092.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-11T18:16:07.402 INFO:teuthology.orchestra.run.smithi092.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-11T18:16:07.556 INFO:teuthology.orchestra.run.smithi092.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-11T18:16:07.556 INFO:teuthology.orchestra.run.smithi092.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-03-11T18:16:07.556 INFO:teuthology.orchestra.run.smithi092.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-11T18:16:07.578 INFO:teuthology.orchestra.run.smithi092.stderr:done 2024-03-11T18:16:07.581 DEBUG:teuthology.orchestra.run.smithi092:> sudo shutdown -r now 2024-03-11T18:16:07.674 INFO:teuthology.orchestra.run.smithi090.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-03-11T18:16:07.674 INFO:teuthology.orchestra.run.smithi090.stdout:Sourcing file `/etc/default/grub' 2024-03-11T18:16:07.705 INFO:teuthology.orchestra.run.smithi090.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-11T18:16:07.729 INFO:teuthology.orchestra.run.smithi090.stdout:Generating grub configuration file ... 2024-03-11T18:16:08.077 INFO:teuthology.orchestra.run.smithi090.stdout:Found linux image: /boot/vmlinuz-5.15.0-100-generic 2024-03-11T18:16:08.113 INFO:teuthology.orchestra.run.smithi090.stdout:Found initrd image: /boot/initrd.img-5.15.0-100-generic 2024-03-11T18:16:08.466 INFO:teuthology.orchestra.run.smithi090.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-11T18:16:08.472 INFO:teuthology.orchestra.run.smithi090.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-11T18:16:08.548 INFO:teuthology.orchestra.run.smithi090.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-11T18:16:08.554 INFO:teuthology.orchestra.run.smithi090.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-11T18:16:08.801 INFO:teuthology.orchestra.run.smithi090.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-11T18:16:08.801 INFO:teuthology.orchestra.run.smithi090.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-03-11T18:16:08.802 INFO:teuthology.orchestra.run.smithi090.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-11T18:16:08.911 INFO:teuthology.orchestra.run.smithi090.stdout:done 2024-03-11T18:16:09.433 DEBUG:teuthology.orchestra.run.smithi090:> dpkg -s linux-image-generic 2024-03-11T18:16:09.452 INFO:teuthology.orchestra.run.smithi090.stdout:Package: linux-image-generic 2024-03-11T18:16:09.452 INFO:teuthology.orchestra.run.smithi090.stdout:Status: install ok installed 2024-03-11T18:16:09.452 INFO:teuthology.orchestra.run.smithi090.stdout:Priority: optional 2024-03-11T18:16:09.452 INFO:teuthology.orchestra.run.smithi090.stdout:Section: kernel 2024-03-11T18:16:09.452 INFO:teuthology.orchestra.run.smithi090.stdout:Installed-Size: 21 2024-03-11T18:16:09.452 INFO:teuthology.orchestra.run.smithi090.stdout:Maintainer: Ubuntu Kernel Team 2024-03-11T18:16:09.452 INFO:teuthology.orchestra.run.smithi090.stdout:Architecture: amd64 2024-03-11T18:16:09.452 INFO:teuthology.orchestra.run.smithi090.stdout:Source: linux-meta 2024-03-11T18:16:09.452 INFO:teuthology.orchestra.run.smithi090.stdout:Version: 5.15.0.100.97 2024-03-11T18:16:09.452 INFO:teuthology.orchestra.run.smithi090.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.2), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-100), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.2) 2024-03-11T18:16:09.453 INFO:teuthology.orchestra.run.smithi090.stdout:Depends: linux-image-5.15.0-100-generic, linux-modules-extra-5.15.0-100-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-11T18:16:09.453 INFO:teuthology.orchestra.run.smithi090.stdout:Recommends: thermald 2024-03-11T18:16:09.453 INFO:teuthology.orchestra.run.smithi090.stdout:Description: Generic Linux kernel image 2024-03-11T18:16:09.453 INFO:teuthology.orchestra.run.smithi090.stdout: This package will always depend on the latest generic kernel image 2024-03-11T18:16:09.453 INFO:teuthology.orchestra.run.smithi090.stdout: available. 2024-03-11T18:16:09.453 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-100-generic 2024-03-11T18:16:09.453 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-11T18:16:09.453 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-11T18:16:09.453 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-03-11T18:16:09.453 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi090.front.sepia.ceph.com, path=None, version=distro) 2024-03-11T18:16:09.454 DEBUG:teuthology.orchestra.run.smithi090:> sudo apt-get clean 2024-03-11T18:16:09.630 DEBUG:teuthology.orchestra.run.smithi090:> sudo apt-get update 2024-03-11T18:16:09.759 INFO:teuthology.orchestra.run.smithi090.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-03-11T18:16:09.881 INFO:teuthology.orchestra.run.smithi090.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-11T18:16:09.965 INFO:teuthology.orchestra.run.smithi090.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-03-11T18:16:10.328 INFO:teuthology.orchestra.run.smithi090.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-11T18:16:10.515 INFO:teuthology.orchestra.run.smithi090.stdout:Fetched 229 kB in 1s (314 kB/s) 2024-03-11T18:16:11.478 INFO:teuthology.orchestra.run.smithi090.stdout:Reading package lists... 2024-03-11T18:16:11.496 DEBUG:teuthology.orchestra.run.smithi090:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-03-11T18:16:11.547 INFO:teuthology.orchestra.run.smithi090.stdout:Reading package lists... 2024-03-11T18:16:11.767 INFO:teuthology.orchestra.run.smithi090.stdout:Building dependency tree... 2024-03-11T18:16:11.768 INFO:teuthology.orchestra.run.smithi090.stdout:Reading state information... 2024-03-11T18:16:11.917 INFO:teuthology.orchestra.run.smithi090.stdout:linux-image-generic is already the newest version (5.15.0.100.97). 2024-03-11T18:16:11.918 INFO:teuthology.orchestra.run.smithi090.stdout:The following packages were automatically installed and are no longer required: 2024-03-11T18:16:11.918 INFO:teuthology.orchestra.run.smithi090.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-11T18:16:11.918 INFO:teuthology.orchestra.run.smithi090.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-11T18:16:11.918 INFO:teuthology.orchestra.run.smithi090.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-11T18:16:11.918 INFO:teuthology.orchestra.run.smithi090.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-11T18:16:11.918 INFO:teuthology.orchestra.run.smithi090.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-11T18:16:11.918 INFO:teuthology.orchestra.run.smithi090.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-11T18:16:11.956 INFO:teuthology.orchestra.run.smithi090.stdout:0 upgraded, 0 newly installed, 0 to remove and 288 not upgraded. 2024-03-11T18:16:11.958 DEBUG:teuthology.orchestra.run.smithi090:> dpkg -s linux-image-generic 2024-03-11T18:16:12.017 INFO:teuthology.orchestra.run.smithi090.stdout:Package: linux-image-generic 2024-03-11T18:16:12.017 INFO:teuthology.orchestra.run.smithi090.stdout:Status: install ok installed 2024-03-11T18:16:12.017 INFO:teuthology.orchestra.run.smithi090.stdout:Priority: optional 2024-03-11T18:16:12.017 INFO:teuthology.orchestra.run.smithi090.stdout:Section: kernel 2024-03-11T18:16:12.017 INFO:teuthology.orchestra.run.smithi090.stdout:Installed-Size: 21 2024-03-11T18:16:12.017 INFO:teuthology.orchestra.run.smithi090.stdout:Maintainer: Ubuntu Kernel Team 2024-03-11T18:16:12.017 INFO:teuthology.orchestra.run.smithi090.stdout:Architecture: amd64 2024-03-11T18:16:12.018 INFO:teuthology.orchestra.run.smithi090.stdout:Source: linux-meta 2024-03-11T18:16:12.018 INFO:teuthology.orchestra.run.smithi090.stdout:Version: 5.15.0.100.97 2024-03-11T18:16:12.018 INFO:teuthology.orchestra.run.smithi090.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.2), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-100), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.2) 2024-03-11T18:16:12.018 INFO:teuthology.orchestra.run.smithi090.stdout:Depends: linux-image-5.15.0-100-generic, linux-modules-extra-5.15.0-100-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-11T18:16:12.018 INFO:teuthology.orchestra.run.smithi090.stdout:Recommends: thermald 2024-03-11T18:16:12.018 INFO:teuthology.orchestra.run.smithi090.stdout:Description: Generic Linux kernel image 2024-03-11T18:16:12.018 INFO:teuthology.orchestra.run.smithi090.stdout: This package will always depend on the latest generic kernel image 2024-03-11T18:16:12.018 INFO:teuthology.orchestra.run.smithi090.stdout: available. 2024-03-11T18:16:12.018 DEBUG:teuthology.orchestra.run.smithi090:> mktemp 2024-03-11T18:16:12.062 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/tmp.hOVVCtiuwo 2024-03-11T18:16:12.062 DEBUG:teuthology.orchestra.run.smithi090:> sudo cp /boot/grub/grub.cfg /tmp/tmp.hOVVCtiuwo 2024-03-11T18:16:12.133 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 0666 /tmp/tmp.hOVVCtiuwo 2024-03-11T18:16:12.262 DEBUG:teuthology.orchestra.remote:smithi090:/tmp/tmp.hOVVCtiuwo is 10KB 2024-03-11T18:16:12.274 DEBUG:teuthology.orchestra.run.smithi090:> rm -fr /tmp/tmp.hOVVCtiuwo 2024-03-11T18:16:12.280 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-03-11T18:16:12.334 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-11T18:16:12.334 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-03-11T18:16:12.334 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-03-11T18:16:12.431 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-100-generic 2024-03-11T18:16:12.432 DEBUG:teuthology.orchestra.run.smithi090:> sudo update-grub 2024-03-11T18:16:14.100 INFO:teuthology.orchestra.run.smithi090.stderr:Sourcing file `/etc/default/grub' 2024-03-11T18:16:14.120 INFO:teuthology.orchestra.run.smithi090.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-11T18:16:14.128 INFO:teuthology.orchestra.run.smithi090.stderr:Generating grub configuration file ... 2024-03-11T18:16:14.377 INFO:teuthology.orchestra.run.smithi090.stderr:Found linux image: /boot/vmlinuz-5.15.0-100-generic 2024-03-11T18:16:14.383 INFO:teuthology.orchestra.run.smithi090.stderr:Found initrd image: /boot/initrd.img-5.15.0-100-generic 2024-03-11T18:16:14.746 INFO:teuthology.orchestra.run.smithi090.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-11T18:16:14.752 INFO:teuthology.orchestra.run.smithi090.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-11T18:16:14.794 INFO:teuthology.orchestra.run.smithi090.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-11T18:16:14.800 INFO:teuthology.orchestra.run.smithi090.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-11T18:16:14.928 INFO:teuthology.orchestra.run.smithi090.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-11T18:16:14.928 INFO:teuthology.orchestra.run.smithi090.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-03-11T18:16:14.928 INFO:teuthology.orchestra.run.smithi090.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-11T18:16:14.955 INFO:teuthology.orchestra.run.smithi090.stderr:done 2024-03-11T18:16:14.957 DEBUG:teuthology.orchestra.run.smithi090:> sudo shutdown -r now 2024-03-11T18:16:37.609 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-03-11T18:16:37.610 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-03-11T18:16:37.611 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-11T18:16:44.966 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-03-11T18:16:44.966 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-03-11T18:16:44.967 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-11T18:16:56.094 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.92 2024-03-11T18:17:03.518 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.90 2024-03-11T18:17:05.096 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-03-11T18:17:05.097 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-11T18:17:08.158 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.92 2024-03-11T18:17:12.524 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-03-11T18:17:12.525 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-11T18:17:15.586 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.90 2024-03-11T18:17:20.162 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-03-11T18:17:20.163 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-11T18:17:27.594 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-03-11T18:17:27.595 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-11T18:17:30.654 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.90 2024-03-11T18:17:38.590 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.92 2024-03-11T18:17:45.660 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-03-11T18:17:45.661 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-11T18:17:53.598 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-03-11T18:17:53.599 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-11T18:17:54.122 DEBUG:teuthology.orchestra.run.smithi092:> true 2024-03-11T18:17:54.895 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-03-11T18:17:54.895 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-100-generic"... 2024-03-11T18:17:54.896 DEBUG:teuthology.orchestra.run.smithi092:> uname -r 2024-03-11T18:17:54.942 INFO:teuthology.orchestra.run.smithi092.stdout:5.15.0-100-generic 2024-03-11T18:17:54.942 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-100-generic vs 5.15.0-100-generic 2024-03-11T18:17:54.942 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-11T18:17:54.943 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-03-11T18:17:55.943 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-03-11T18:17:55.944 DEBUG:teuthology.orchestra.run.smithi092:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-11T18:17:56.102 INFO:teuthology.orchestra.run.smithi092.stdout:ttyS1 2024-03-11T18:17:56.136 DEBUG:teuthology.parallel:result is None 2024-03-11T18:18:04.194 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.90 2024-03-11T18:18:22.195 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-03-11T18:18:22.197 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-11T18:18:22.536 DEBUG:teuthology.orchestra.run.smithi090:> true 2024-03-11T18:18:23.083 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-03-11T18:18:23.083 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-100-generic"... 2024-03-11T18:18:23.084 DEBUG:teuthology.orchestra.run.smithi090:> uname -r 2024-03-11T18:18:23.129 INFO:teuthology.orchestra.run.smithi090.stdout:5.15.0-100-generic 2024-03-11T18:18:23.130 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-100-generic vs 5.15.0-100-generic 2024-03-11T18:18:23.130 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-11T18:18:23.130 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-03-11T18:18:24.131 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-03-11T18:18:24.131 DEBUG:teuthology.orchestra.run.smithi090:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-11T18:18:24.255 INFO:teuthology.orchestra.run.smithi090.stdout:ttyS1 2024-03-11T18:18:24.287 DEBUG:teuthology.parallel:result is None 2024-03-11T18:18:24.288 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-11T18:18:24.295 INFO:teuthology.task.internal:Creating test directory... 2024-03-11T18:18:24.295 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-11T18:18:24.298 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-11T18:18:24.304 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-11T18:18:24.313 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-11T18:18:24.319 INFO:teuthology.task.internal:Creating archive directory... 2024-03-11T18:18:24.319 DEBUG:teuthology.orchestra.run.smithi090:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-11T18:18:24.346 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-11T18:18:24.373 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-11T18:18:24.380 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-11T18:18:24.381 DEBUG:teuthology.orchestra.run.smithi090:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-11T18:18:24.399 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-11T18:18:24.416 INFO:teuthology.orchestra.run.smithi090.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-11T18:18:24.424 INFO:teuthology.orchestra.run.smithi090.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-11T18:18:24.436 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-11T18:18:24.444 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-11T18:18:24.446 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-11T18:18:24.453 INFO:teuthology.task.internal:Configuring sudo... 2024-03-11T18:18:24.453 DEBUG:teuthology.orchestra.run.smithi090:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-11T18:18:24.470 DEBUG:teuthology.orchestra.run.smithi092:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-11T18:18:24.508 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-11T18:18:24.518 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-11T18:18:24.518 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-11T18:18:24.526 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-11T18:18:24.554 DEBUG:teuthology.orchestra.run.smithi090:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-11T18:18:24.577 DEBUG:teuthology.orchestra.run.smithi090:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-11T18:18:24.626 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-11T18:18:24.626 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-11T18:18:24.718 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-11T18:18:24.724 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-11T18:18:24.770 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-11T18:18:24.770 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-11T18:18:24.844 DEBUG:teuthology.orchestra.run.smithi090:> sudo service rsyslog restart 2024-03-11T18:18:24.848 DEBUG:teuthology.orchestra.run.smithi092:> sudo service rsyslog restart 2024-03-11T18:18:24.915 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-11T18:18:24.922 INFO:teuthology.task.internal:Starting timer... 2024-03-11T18:18:24.922 INFO:teuthology.run_tasks:Running task pcp... 2024-03-11T18:18:24.934 INFO:teuthology.run_tasks:Running task selinux... 2024-03-11T18:18:24.943 DEBUG:teuthology.task.selinux:Excluding smithi090: OS 'ubuntu' does not support SELinux 2024-03-11T18:18:24.943 DEBUG:teuthology.task.selinux:Excluding smithi092: OS 'ubuntu' does not support SELinux 2024-03-11T18:18:24.944 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-11T18:18:24.944 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-03-11T18:18:24.944 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-11T18:18:24.944 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-11T18:18:24.954 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-11T18:18:24.960 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-11T18:18:25.071 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-11T18:18:25.090 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-11T18:18:25.092 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi090.front.sepia.ceph.com,smithi092.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-11T18:25:29.304 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi090.front.sepia.ceph.com'), Remote(name='ubuntu@smithi092.front.sepia.ceph.com')] 2024-03-11T18:25:29.306 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-03-11T18:25:29.307 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-11T18:25:29.387 DEBUG:teuthology.orchestra.run.smithi090:> true 2024-03-11T18:25:29.466 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-03-11T18:25:29.466 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-03-11T18:25:29.467 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-11T18:25:29.545 DEBUG:teuthology.orchestra.run.smithi092:> true 2024-03-11T18:25:29.618 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi092.front.sepia.ceph.com' 2024-03-11T18:25:29.618 INFO:teuthology.run_tasks:Running task clock... 2024-03-11T18:25:29.629 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-11T18:25:29.629 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-11T18:25:29.630 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-11T18:25:29.632 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-11T18:25:29.632 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-11T18:25:29.659 INFO:teuthology.orchestra.run.smithi090.stdout:11 Mar 18:25:29 ntpd[17168]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-03-11T18:25:29.659 INFO:teuthology.orchestra.run.smithi090.stdout:11 Mar 18:25:29 ntpd[17168]: Command line: ntpd -gq 2024-03-11T18:25:29.659 INFO:teuthology.orchestra.run.smithi090.stdout:11 Mar 18:25:29 ntpd[17168]: ---------------------------------------------------- 2024-03-11T18:25:29.659 INFO:teuthology.orchestra.run.smithi090.stdout:11 Mar 18:25:29 ntpd[17168]: ntp-4 is maintained by Network Time Foundation, 2024-03-11T18:25:29.659 INFO:teuthology.orchestra.run.smithi090.stdout:11 Mar 18:25:29 ntpd[17168]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-03-11T18:25:29.659 INFO:teuthology.orchestra.run.smithi090.stdout:11 Mar 18:25:29 ntpd[17168]: corporation. Support and training for ntp-4 are 2024-03-11T18:25:29.659 INFO:teuthology.orchestra.run.smithi090.stdout:11 Mar 18:25:29 ntpd[17168]: available at https://www.nwtime.org/support 2024-03-11T18:25:29.659 INFO:teuthology.orchestra.run.smithi090.stdout:11 Mar 18:25:29 ntpd[17168]: ---------------------------------------------------- 2024-03-11T18:25:29.661 INFO:teuthology.orchestra.run.smithi090.stdout:11 Mar 18:25:29 ntpd[17168]: proto: precision = 0.044 usec (-24) 2024-03-11T18:25:29.661 INFO:teuthology.orchestra.run.smithi090.stdout:11 Mar 18:25:29 ntpd[17168]: basedate set to 2022-02-04 2024-03-11T18:25:29.661 INFO:teuthology.orchestra.run.smithi090.stdout:11 Mar 18:25:29 ntpd[17168]: gps base set to 2022-02-06 (week 2196) 2024-03-11T18:25:29.661 INFO:teuthology.orchestra.run.smithi090.stdout:11 Mar 18:25:29 ntpd[17168]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-11T18:25:29.661 INFO:teuthology.orchestra.run.smithi090.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-11T18:25:29.662 INFO:teuthology.orchestra.run.smithi090.stderr:restrict ::: KOD does nothing without LIMITED. 2024-03-11T18:25:29.663 INFO:teuthology.orchestra.run.smithi090.stdout:11 Mar 18:25:29 ntpd[17168]: restrict ::: KOD does nothing without LIMITED. 2024-03-11T18:25:29.663 INFO:teuthology.orchestra.run.smithi090.stdout:11 Mar 18:25:29 ntpd[17168]: Listen and drop on 0 v6wildcard [::]:123 2024-03-11T18:25:29.663 INFO:teuthology.orchestra.run.smithi090.stdout:11 Mar 18:25:29 ntpd[17168]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-03-11T18:25:29.663 INFO:teuthology.orchestra.run.smithi090.stdout:11 Mar 18:25:29 ntpd[17168]: Listen normally on 2 lo 127.0.0.1:123 2024-03-11T18:25:29.663 INFO:teuthology.orchestra.run.smithi090.stdout:11 Mar 18:25:29 ntpd[17168]: Listen normally on 3 enp3s0f1 172.21.15.90:123 2024-03-11T18:25:29.663 INFO:teuthology.orchestra.run.smithi090.stdout:11 Mar 18:25:29 ntpd[17168]: Listen normally on 4 lo [::1]:123 2024-03-11T18:25:29.663 INFO:teuthology.orchestra.run.smithi090.stdout:11 Mar 18:25:29 ntpd[17168]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d767%5]:123 2024-03-11T18:25:29.663 INFO:teuthology.orchestra.run.smithi090.stdout:11 Mar 18:25:29 ntpd[17168]: Listening on routing socket on fd #22 for interface updates 2024-03-11T18:25:29.683 INFO:teuthology.orchestra.run.smithi092.stdout:11 Mar 18:25:29 ntpd[17182]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-03-11T18:25:29.683 INFO:teuthology.orchestra.run.smithi092.stdout:11 Mar 18:25:29 ntpd[17182]: Command line: ntpd -gq 2024-03-11T18:25:29.683 INFO:teuthology.orchestra.run.smithi092.stdout:11 Mar 18:25:29 ntpd[17182]: ---------------------------------------------------- 2024-03-11T18:25:29.683 INFO:teuthology.orchestra.run.smithi092.stdout:11 Mar 18:25:29 ntpd[17182]: ntp-4 is maintained by Network Time Foundation, 2024-03-11T18:25:29.683 INFO:teuthology.orchestra.run.smithi092.stdout:11 Mar 18:25:29 ntpd[17182]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-03-11T18:25:29.683 INFO:teuthology.orchestra.run.smithi092.stdout:11 Mar 18:25:29 ntpd[17182]: corporation. Support and training for ntp-4 are 2024-03-11T18:25:29.683 INFO:teuthology.orchestra.run.smithi092.stdout:11 Mar 18:25:29 ntpd[17182]: available at https://www.nwtime.org/support 2024-03-11T18:25:29.684 INFO:teuthology.orchestra.run.smithi092.stdout:11 Mar 18:25:29 ntpd[17182]: ---------------------------------------------------- 2024-03-11T18:25:29.684 INFO:teuthology.orchestra.run.smithi092.stdout:11 Mar 18:25:29 ntpd[17182]: proto: precision = 0.045 usec (-24) 2024-03-11T18:25:29.684 INFO:teuthology.orchestra.run.smithi092.stdout:11 Mar 18:25:29 ntpd[17182]: basedate set to 2022-02-04 2024-03-11T18:25:29.684 INFO:teuthology.orchestra.run.smithi092.stdout:11 Mar 18:25:29 ntpd[17182]: gps base set to 2022-02-06 (week 2196) 2024-03-11T18:25:29.684 INFO:teuthology.orchestra.run.smithi092.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-11T18:25:29.685 INFO:teuthology.orchestra.run.smithi092.stdout:11 Mar 18:25:29 ntpd[17182]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-11T18:25:29.685 INFO:teuthology.orchestra.run.smithi092.stdout:11 Mar 18:25:29 ntpd[17182]: restrict ::: KOD does nothing without LIMITED. 2024-03-11T18:25:29.685 INFO:teuthology.orchestra.run.smithi092.stderr:restrict ::: KOD does nothing without LIMITED. 2024-03-11T18:25:29.686 INFO:teuthology.orchestra.run.smithi092.stdout:11 Mar 18:25:29 ntpd[17182]: Listen and drop on 0 v6wildcard [::]:123 2024-03-11T18:25:29.686 INFO:teuthology.orchestra.run.smithi092.stdout:11 Mar 18:25:29 ntpd[17182]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-03-11T18:25:29.686 INFO:teuthology.orchestra.run.smithi092.stdout:11 Mar 18:25:29 ntpd[17182]: Listen normally on 2 lo 127.0.0.1:123 2024-03-11T18:25:29.687 INFO:teuthology.orchestra.run.smithi092.stdout:11 Mar 18:25:29 ntpd[17182]: Listen normally on 3 enp3s0f1 172.21.15.92:123 2024-03-11T18:25:29.687 INFO:teuthology.orchestra.run.smithi092.stdout:11 Mar 18:25:29 ntpd[17182]: Listen normally on 4 lo [::1]:123 2024-03-11T18:25:29.687 INFO:teuthology.orchestra.run.smithi092.stdout:11 Mar 18:25:29 ntpd[17182]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:cd45%5]:123 2024-03-11T18:25:29.687 INFO:teuthology.orchestra.run.smithi092.stdout:11 Mar 18:25:29 ntpd[17182]: Listening on routing socket on fd #22 for interface updates 2024-03-11T18:25:30.660 INFO:teuthology.orchestra.run.smithi090.stderr:11 Mar 18:25:30 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-11T18:25:30.660 INFO:teuthology.orchestra.run.smithi090.stderr:11 Mar 18:25:30 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240311: Permission denied 2024-03-11T18:25:30.660 INFO:teuthology.orchestra.run.smithi090.stderr:11 Mar 18:25:30 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-11T18:25:30.660 INFO:teuthology.orchestra.run.smithi090.stderr:11 Mar 18:25:30 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240311: Permission denied 2024-03-11T18:25:30.660 INFO:teuthology.orchestra.run.smithi090.stderr:11 Mar 18:25:30 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-11T18:25:30.660 INFO:teuthology.orchestra.run.smithi090.stderr:11 Mar 18:25:30 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240311: Permission denied 2024-03-11T18:25:30.660 INFO:teuthology.orchestra.run.smithi090.stderr:11 Mar 18:25:30 ntpd[17168]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-11T18:25:30.661 INFO:teuthology.orchestra.run.smithi090.stderr:11 Mar 18:25:30 ntpd[17168]: can't open /var/log/ntpstats/peerstats.20240311: Permission denied 2024-03-11T18:25:30.683 INFO:teuthology.orchestra.run.smithi092.stderr:11 Mar 18:25:30 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-11T18:25:30.683 INFO:teuthology.orchestra.run.smithi092.stderr:11 Mar 18:25:30 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240311: Permission denied 2024-03-11T18:25:30.683 INFO:teuthology.orchestra.run.smithi092.stderr:11 Mar 18:25:30 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-11T18:25:30.683 INFO:teuthology.orchestra.run.smithi092.stderr:11 Mar 18:25:30 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240311: Permission denied 2024-03-11T18:25:30.683 INFO:teuthology.orchestra.run.smithi092.stderr:11 Mar 18:25:30 ntpd[17182]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-11T18:25:30.683 INFO:teuthology.orchestra.run.smithi092.stderr:11 Mar 18:25:30 ntpd[17182]: can't open /var/log/ntpstats/peerstats.20240311: Permission denied 2024-03-11T18:25:32.660 INFO:teuthology.orchestra.run.smithi090.stderr:11 Mar 18:25:32 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-11T18:25:32.660 INFO:teuthology.orchestra.run.smithi090.stderr:11 Mar 18:25:32 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240311: Permission denied 2024-03-11T18:25:32.660 INFO:teuthology.orchestra.run.smithi090.stderr:11 Mar 18:25:32 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-11T18:25:32.660 INFO:teuthology.orchestra.run.smithi090.stderr:11 Mar 18:25:32 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240311: Permission denied 2024-03-11T18:25:32.660 INFO:teuthology.orchestra.run.smithi090.stderr:11 Mar 18:25:32 ntpd[17168]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-11T18:25:32.660 INFO:teuthology.orchestra.run.smithi090.stderr:11 Mar 18:25:32 ntpd[17168]: can't open /var/log/ntpstats/peerstats.20240311: Permission denied 2024-03-11T18:25:32.683 INFO:teuthology.orchestra.run.smithi092.stderr:11 Mar 18:25:32 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-11T18:25:32.683 INFO:teuthology.orchestra.run.smithi092.stderr:11 Mar 18:25:32 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240311: Permission denied 2024-03-11T18:25:32.683 INFO:teuthology.orchestra.run.smithi092.stderr:11 Mar 18:25:32 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-11T18:25:32.683 INFO:teuthology.orchestra.run.smithi092.stderr:11 Mar 18:25:32 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240311: Permission denied 2024-03-11T18:25:32.683 INFO:teuthology.orchestra.run.smithi092.stderr:11 Mar 18:25:32 ntpd[17182]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-11T18:25:32.683 INFO:teuthology.orchestra.run.smithi092.stderr:11 Mar 18:25:32 ntpd[17182]: can't open /var/log/ntpstats/peerstats.20240311: Permission denied 2024-03-11T18:25:33.683 INFO:teuthology.orchestra.run.smithi092.stderr:11 Mar 18:25:33 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-11T18:25:33.683 INFO:teuthology.orchestra.run.smithi092.stderr:11 Mar 18:25:33 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240311: Permission denied 2024-03-11T18:25:33.683 INFO:teuthology.orchestra.run.smithi092.stderr:11 Mar 18:25:33 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-11T18:25:33.684 INFO:teuthology.orchestra.run.smithi092.stderr:11 Mar 18:25:33 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240311: Permission denied 2024-03-11T18:25:33.684 INFO:teuthology.orchestra.run.smithi092.stderr:11 Mar 18:25:33 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-11T18:25:33.684 INFO:teuthology.orchestra.run.smithi092.stderr:11 Mar 18:25:33 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240311: Permission denied 2024-03-11T18:25:33.684 INFO:teuthology.orchestra.run.smithi092.stderr:11 Mar 18:25:33 ntpd[17182]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-11T18:25:33.684 INFO:teuthology.orchestra.run.smithi092.stderr:11 Mar 18:25:33 ntpd[17182]: can't open /var/log/ntpstats/peerstats.20240311: Permission denied 2024-03-11T18:25:34.661 INFO:teuthology.orchestra.run.smithi090.stderr:11 Mar 18:25:34 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-11T18:25:34.661 INFO:teuthology.orchestra.run.smithi090.stderr:11 Mar 18:25:34 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240311: Permission denied 2024-03-11T18:25:34.661 INFO:teuthology.orchestra.run.smithi090.stderr:11 Mar 18:25:34 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-11T18:25:34.661 INFO:teuthology.orchestra.run.smithi090.stderr:11 Mar 18:25:34 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240311: Permission denied 2024-03-11T18:25:34.661 INFO:teuthology.orchestra.run.smithi090.stderr:11 Mar 18:25:34 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-11T18:25:34.662 INFO:teuthology.orchestra.run.smithi090.stderr:11 Mar 18:25:34 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240311: Permission denied 2024-03-11T18:25:34.662 INFO:teuthology.orchestra.run.smithi090.stderr:11 Mar 18:25:34 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-11T18:25:34.662 INFO:teuthology.orchestra.run.smithi090.stderr:11 Mar 18:25:34 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240311: Permission denied 2024-03-11T18:25:34.662 INFO:teuthology.orchestra.run.smithi090.stderr:11 Mar 18:25:34 ntpd[17168]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-11T18:25:34.662 INFO:teuthology.orchestra.run.smithi090.stderr:11 Mar 18:25:34 ntpd[17168]: can't open /var/log/ntpstats/peerstats.20240311: Permission denied 2024-03-11T18:25:34.662 INFO:teuthology.orchestra.run.smithi090.stderr:11 Mar 18:25:34 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-11T18:25:34.662 INFO:teuthology.orchestra.run.smithi090.stderr:11 Mar 18:25:34 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240311: Permission denied 2024-03-11T18:25:34.662 INFO:teuthology.orchestra.run.smithi090.stderr:11 Mar 18:25:34 ntpd[17168]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-11T18:25:34.662 INFO:teuthology.orchestra.run.smithi090.stderr:11 Mar 18:25:34 ntpd[17168]: can't open /var/log/ntpstats/peerstats.20240311: Permission denied 2024-03-11T18:25:34.662 INFO:teuthology.orchestra.run.smithi090.stderr:11 Mar 18:25:34 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-11T18:25:34.662 INFO:teuthology.orchestra.run.smithi090.stderr:11 Mar 18:25:34 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240311: Permission denied 2024-03-11T18:25:34.662 INFO:teuthology.orchestra.run.smithi090.stderr:11 Mar 18:25:34 ntpd[17168]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-11T18:25:34.662 INFO:teuthology.orchestra.run.smithi090.stderr:11 Mar 18:25:34 ntpd[17168]: can't open /var/log/ntpstats/peerstats.20240311: Permission denied 2024-03-11T18:25:34.683 INFO:teuthology.orchestra.run.smithi092.stderr:11 Mar 18:25:34 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-11T18:25:34.683 INFO:teuthology.orchestra.run.smithi092.stderr:11 Mar 18:25:34 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240311: Permission denied 2024-03-11T18:25:34.684 INFO:teuthology.orchestra.run.smithi092.stderr:11 Mar 18:25:34 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-11T18:25:34.684 INFO:teuthology.orchestra.run.smithi092.stderr:11 Mar 18:25:34 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240311: Permission denied 2024-03-11T18:25:34.684 INFO:teuthology.orchestra.run.smithi092.stderr:11 Mar 18:25:34 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-11T18:25:34.684 INFO:teuthology.orchestra.run.smithi092.stderr:11 Mar 18:25:34 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240311: Permission denied 2024-03-11T18:25:34.684 INFO:teuthology.orchestra.run.smithi092.stderr:11 Mar 18:25:34 ntpd[17182]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-11T18:25:34.684 INFO:teuthology.orchestra.run.smithi092.stderr:11 Mar 18:25:34 ntpd[17182]: can't open /var/log/ntpstats/peerstats.20240311: Permission denied 2024-03-11T18:25:34.684 INFO:teuthology.orchestra.run.smithi092.stderr:11 Mar 18:25:34 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-11T18:25:34.684 INFO:teuthology.orchestra.run.smithi092.stderr:11 Mar 18:25:34 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240311: Permission denied 2024-03-11T18:25:34.684 INFO:teuthology.orchestra.run.smithi092.stderr:11 Mar 18:25:34 ntpd[17182]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-11T18:25:34.684 INFO:teuthology.orchestra.run.smithi092.stderr:11 Mar 18:25:34 ntpd[17182]: can't open /var/log/ntpstats/peerstats.20240311: Permission denied 2024-03-11T18:25:35.683 INFO:teuthology.orchestra.run.smithi092.stderr:11 Mar 18:25:35 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-11T18:25:35.683 INFO:teuthology.orchestra.run.smithi092.stderr:11 Mar 18:25:35 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240311: Permission denied 2024-03-11T18:25:35.683 INFO:teuthology.orchestra.run.smithi092.stderr:11 Mar 18:25:35 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-11T18:25:35.683 INFO:teuthology.orchestra.run.smithi092.stderr:11 Mar 18:25:35 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240311: Permission denied 2024-03-11T18:25:35.683 INFO:teuthology.orchestra.run.smithi092.stderr:11 Mar 18:25:35 ntpd[17182]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-11T18:25:35.683 INFO:teuthology.orchestra.run.smithi092.stderr:11 Mar 18:25:35 ntpd[17182]: can't open /var/log/ntpstats/peerstats.20240311: Permission denied 2024-03-11T18:25:36.661 INFO:teuthology.orchestra.run.smithi090.stderr:11 Mar 18:25:36 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-11T18:25:36.661 INFO:teuthology.orchestra.run.smithi090.stderr:11 Mar 18:25:36 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240311: Permission denied 2024-03-11T18:25:36.661 INFO:teuthology.orchestra.run.smithi090.stderr:11 Mar 18:25:36 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-11T18:25:36.661 INFO:teuthology.orchestra.run.smithi090.stderr:11 Mar 18:25:36 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240311: Permission denied 2024-03-11T18:25:36.661 INFO:teuthology.orchestra.run.smithi090.stderr:11 Mar 18:25:36 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-11T18:25:36.661 INFO:teuthology.orchestra.run.smithi090.stderr:11 Mar 18:25:36 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240311: Permission denied 2024-03-11T18:25:36.661 INFO:teuthology.orchestra.run.smithi090.stderr:11 Mar 18:25:36 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-11T18:25:36.661 INFO:teuthology.orchestra.run.smithi090.stderr:11 Mar 18:25:36 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240311: Permission denied 2024-03-11T18:25:36.661 INFO:teuthology.orchestra.run.smithi090.stderr:11 Mar 18:25:36 ntpd[17168]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-11T18:25:36.661 INFO:teuthology.orchestra.run.smithi090.stderr:11 Mar 18:25:36 ntpd[17168]: can't open /var/log/ntpstats/peerstats.20240311: Permission denied 2024-03-11T18:25:36.662 INFO:teuthology.orchestra.run.smithi090.stderr:11 Mar 18:25:36 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-11T18:25:36.662 INFO:teuthology.orchestra.run.smithi090.stderr:11 Mar 18:25:36 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240311: Permission denied 2024-03-11T18:25:36.662 INFO:teuthology.orchestra.run.smithi090.stderr:11 Mar 18:25:36 ntpd[17168]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-11T18:25:36.662 INFO:teuthology.orchestra.run.smithi090.stderr:11 Mar 18:25:36 ntpd[17168]: can't open /var/log/ntpstats/peerstats.20240311: Permission denied 2024-03-11T18:25:36.662 INFO:teuthology.orchestra.run.smithi090.stdout:11 Mar 18:25:36 ntpd[17168]: ntpd: time slew +0.002502 s 2024-03-11T18:25:36.662 INFO:teuthology.orchestra.run.smithi090.stdout:ntpd: time slew +0.002502s 2024-03-11T18:25:36.663 INFO:teuthology.orchestra.run.smithi090.stderr:11 Mar 18:25:36 ntpd[17168]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-03-11T18:25:36.663 INFO:teuthology.orchestra.run.smithi090.stderr:11 Mar 18:25:36 ntpd[17168]: can't open /var/log/ntpstats/loopstats.20240311: Permission denied 2024-03-11T18:25:36.683 INFO:teuthology.orchestra.run.smithi092.stderr:11 Mar 18:25:36 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-11T18:25:36.683 INFO:teuthology.orchestra.run.smithi092.stderr:11 Mar 18:25:36 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240311: Permission denied 2024-03-11T18:25:36.683 INFO:teuthology.orchestra.run.smithi092.stderr:11 Mar 18:25:36 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-11T18:25:36.683 INFO:teuthology.orchestra.run.smithi092.stderr:11 Mar 18:25:36 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240311: Permission denied 2024-03-11T18:25:36.683 INFO:teuthology.orchestra.run.smithi092.stderr:11 Mar 18:25:36 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-11T18:25:36.683 INFO:teuthology.orchestra.run.smithi092.stderr:11 Mar 18:25:36 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240311: Permission denied 2024-03-11T18:25:36.683 INFO:teuthology.orchestra.run.smithi092.stderr:11 Mar 18:25:36 ntpd[17182]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-11T18:25:36.683 INFO:teuthology.orchestra.run.smithi092.stderr:11 Mar 18:25:36 ntpd[17182]: can't open /var/log/ntpstats/peerstats.20240311: Permission denied 2024-03-11T18:25:36.684 INFO:teuthology.orchestra.run.smithi092.stdout:11 Mar 18:25:36 ntpd[17182]: ntpd: time slew -0.001142 s 2024-03-11T18:25:36.684 INFO:teuthology.orchestra.run.smithi092.stdout:ntpd: time slew -0.001142s 2024-03-11T18:25:36.684 INFO:teuthology.orchestra.run.smithi092.stderr:11 Mar 18:25:36 ntpd[17182]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-03-11T18:25:36.684 INFO:teuthology.orchestra.run.smithi092.stderr:11 Mar 18:25:36 ntpd[17182]: can't open /var/log/ntpstats/loopstats.20240311: Permission denied 2024-03-11T18:25:36.708 INFO:teuthology.orchestra.run.smithi090.stdout: remote refid st t when poll reach delay offset jitter 2024-03-11T18:25:36.708 INFO:teuthology.orchestra.run.smithi090.stdout:============================================================================== 2024-03-11T18:25:36.708 INFO:teuthology.orchestra.run.smithi090.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-11T18:25:36.708 INFO:teuthology.orchestra.run.smithi090.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-11T18:25:36.708 INFO:teuthology.orchestra.run.smithi090.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-11T18:25:36.709 INFO:teuthology.orchestra.run.smithi090.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-11T18:25:36.737 INFO:teuthology.orchestra.run.smithi092.stdout: remote refid st t when poll reach delay offset jitter 2024-03-11T18:25:36.737 INFO:teuthology.orchestra.run.smithi092.stdout:============================================================================== 2024-03-11T18:25:36.737 INFO:teuthology.orchestra.run.smithi092.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-11T18:25:36.737 INFO:teuthology.orchestra.run.smithi092.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-11T18:25:36.737 INFO:teuthology.orchestra.run.smithi092.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-11T18:25:36.737 INFO:teuthology.orchestra.run.smithi092.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-11T18:25:36.738 INFO:teuthology.run_tasks:Running task install... 2024-03-11T18:25:36.749 DEBUG:teuthology.task.install:project ceph 2024-03-11T18:25:36.749 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'debuginfo': True, 'flavor': 'default', 'sha1': '94a539707733325322964c6a187c6a1b41be1daf'}} 2024-03-11T18:25:36.749 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'debuginfo': True, 'flavor': 'default', 'sha1': '94a539707733325322964c6a187c6a1b41be1daf'} 2024-03-11T18:25:36.749 INFO:teuthology.task.install:Using flavor: default 2024-03-11T18:25:36.755 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse', 'ceph-common-dbg', 'ceph-fuse-dbg', 'ceph-mds-dbg', 'ceph-mgr-dbg', 'ceph-mon-dbg', 'ceph-osd-dbg', 'ceph-test-dbg', 'libcephfs2-dbg', 'librados2-dbg', 'libradosstriper1-dbg', 'librbd1-dbg', 'librgw2-dbg', 'radosgw-dbg', 'rbd-fuse-dbg', 'rbd-mirror-dbg', 'rbd-nbd-dbg'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd', 'ceph-base-debuginfo', 'ceph-common-debuginfo', 'ceph-immutable-object-cache-debuginfo', 'ceph-radosgw-debuginfo', 'ceph-test-debuginfo', 'ceph-base-debuginfo', 'ceph-mgr-debuginfo', 'ceph-mds-debuginfo', 'ceph-mon-debuginfo', 'ceph-osd-debuginfo', 'ceph-fuse-debuginfo', 'librados-devel-debuginfo', 'libcephfs2-debuginfo', 'librados2-debuginfo', 'librbd1-debuginfo', 'python3-cephfs-debuginfo', 'python3-rados-debuginfo', 'python3-rbd-debuginfo', 'python3-rgw-debuginfo', 'rbd-fuse-debuginfo', 'rbd-mirror-debuginfo', 'rbd-nbd-debuginfo']} 2024-03-11T18:25:36.755 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-11T18:25:36.756 DEBUG:teuthology.orchestra.run.smithi090:> sudo apt-key list | grep Ceph 2024-03-11T18:25:36.756 DEBUG:teuthology.orchestra.run.smithi092:> sudo apt-key list | grep Ceph 2024-03-11T18:25:36.808 INFO:teuthology.orchestra.run.smithi090.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-03-11T18:25:36.835 INFO:teuthology.orchestra.run.smithi090.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-03-11T18:25:36.835 INFO:teuthology.orchestra.run.smithi090.stdout:uid [ unknown] Ceph.com (release key) 2024-03-11T18:25:36.836 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, ceph-common-dbg, ceph-fuse-dbg, ceph-mds-dbg, ceph-mgr-dbg, ceph-mon-dbg, ceph-osd-dbg, ceph-test-dbg, libcephfs2-dbg, librados2-dbg, libradosstriper1-dbg, librbd1-dbg, librgw2-dbg, radosgw-dbg, rbd-fuse-dbg, rbd-mirror-dbg, rbd-nbd-dbg, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-03-11T18:25:36.836 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-03-11T18:25:36.836 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=94a539707733325322964c6a187c6a1b41be1daf 2024-03-11T18:25:36.840 INFO:teuthology.orchestra.run.smithi092.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-03-11T18:25:36.870 INFO:teuthology.orchestra.run.smithi092.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-03-11T18:25:36.870 INFO:teuthology.orchestra.run.smithi092.stdout:uid [ unknown] Ceph.com (release key) 2024-03-11T18:25:36.871 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, ceph-common-dbg, ceph-fuse-dbg, ceph-mds-dbg, ceph-mgr-dbg, ceph-mon-dbg, ceph-osd-dbg, ceph-test-dbg, libcephfs2-dbg, librados2-dbg, libradosstriper1-dbg, librbd1-dbg, librgw2-dbg, radosgw-dbg, rbd-fuse-dbg, rbd-mirror-dbg, rbd-nbd-dbg, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-03-11T18:25:36.871 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-03-11T18:25:36.871 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=94a539707733325322964c6a187c6a1b41be1daf 2024-03-11T18:25:37.026 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default/ 2024-03-11T18:25:37.027 INFO:teuthology.task.install.deb:Package version is 19.0.0-1607-g94a53970-1jammy 2024-03-11T18:25:37.060 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default/ 2024-03-11T18:25:37.060 INFO:teuthology.task.install.deb:Package version is 19.0.0-1607-g94a53970-1jammy 2024-03-11T18:25:37.200 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-11T18:25:37.200 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-03-11T18:25:37.214 DEBUG:teuthology.orchestra.run.smithi092:> sudo apt-get update 2024-03-11T18:25:37.230 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-11T18:25:37.230 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-03-11T18:25:37.244 DEBUG:teuthology.orchestra.run.smithi090:> sudo apt-get update 2024-03-11T18:25:37.387 INFO:teuthology.orchestra.run.smithi092.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-11T18:25:37.387 INFO:teuthology.orchestra.run.smithi092.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-11T18:25:37.398 INFO:teuthology.orchestra.run.smithi092.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-11T18:25:37.414 INFO:teuthology.orchestra.run.smithi090.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-11T18:25:37.418 INFO:teuthology.orchestra.run.smithi092.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-11T18:25:37.427 INFO:teuthology.orchestra.run.smithi090.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-11T18:25:37.434 INFO:teuthology.orchestra.run.smithi092.stdout:Ign:5 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy InRelease 2024-03-11T18:25:37.448 INFO:teuthology.orchestra.run.smithi090.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-11T18:25:37.456 INFO:teuthology.orchestra.run.smithi092.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-03-11T18:25:37.477 INFO:teuthology.orchestra.run.smithi092.stdout:Ign:7 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy Release.gpg 2024-03-11T18:25:37.499 INFO:teuthology.orchestra.run.smithi092.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,408 B] 2024-03-11T18:25:37.524 INFO:teuthology.orchestra.run.smithi090.stdout:Ign:4 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy InRelease 2024-03-11T18:25:37.525 INFO:teuthology.orchestra.run.smithi092.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-03-11T18:25:37.530 INFO:teuthology.orchestra.run.smithi090.stdout:Hit:5 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-11T18:25:37.561 INFO:teuthology.orchestra.run.smithi090.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-03-11T18:25:37.597 INFO:teuthology.orchestra.run.smithi090.stdout:Ign:7 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy Release.gpg 2024-03-11T18:25:37.634 INFO:teuthology.orchestra.run.smithi090.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-03-11T18:25:37.675 INFO:teuthology.orchestra.run.smithi090.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,408 B] 2024-03-11T18:25:37.736 INFO:teuthology.orchestra.run.smithi092.stdout:Fetched 28.8 kB in 0s (84.5 kB/s) 2024-03-11T18:25:37.753 INFO:teuthology.orchestra.run.smithi090.stdout:Fetched 28.8 kB in 0s (85.8 kB/s) 2024-03-11T18:25:38.687 INFO:teuthology.orchestra.run.smithi092.stdout:Reading package lists... 2024-03-11T18:25:38.703 DEBUG:teuthology.orchestra.run.smithi092:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-1607-g94a53970-1jammy cephadm=19.0.0-1607-g94a53970-1jammy ceph-mds=19.0.0-1607-g94a53970-1jammy ceph-mgr=19.0.0-1607-g94a53970-1jammy ceph-common=19.0.0-1607-g94a53970-1jammy ceph-fuse=19.0.0-1607-g94a53970-1jammy ceph-test=19.0.0-1607-g94a53970-1jammy ceph-volume=19.0.0-1607-g94a53970-1jammy radosgw=19.0.0-1607-g94a53970-1jammy python3-rados=19.0.0-1607-g94a53970-1jammy python3-rgw=19.0.0-1607-g94a53970-1jammy python3-cephfs=19.0.0-1607-g94a53970-1jammy python3-rbd=19.0.0-1607-g94a53970-1jammy libcephfs2=19.0.0-1607-g94a53970-1jammy libcephfs-dev=19.0.0-1607-g94a53970-1jammy librados2=19.0.0-1607-g94a53970-1jammy librbd1=19.0.0-1607-g94a53970-1jammy rbd-fuse=19.0.0-1607-g94a53970-1jammy ceph-common-dbg=19.0.0-1607-g94a53970-1jammy ceph-fuse-dbg=19.0.0-1607-g94a53970-1jammy ceph-mds-dbg=19.0.0-1607-g94a53970-1jammy ceph-mgr-dbg=19.0.0-1607-g94a53970-1jammy ceph-mon-dbg=19.0.0-1607-g94a53970-1jammy ceph-osd-dbg=19.0.0-1607-g94a53970-1jammy ceph-test-dbg=19.0.0-1607-g94a53970-1jammy libcephfs2-dbg=19.0.0-1607-g94a53970-1jammy librados2-dbg=19.0.0-1607-g94a53970-1jammy libradosstriper1-dbg=19.0.0-1607-g94a53970-1jammy librbd1-dbg=19.0.0-1607-g94a53970-1jammy librgw2-dbg=19.0.0-1607-g94a53970-1jammy radosgw-dbg=19.0.0-1607-g94a53970-1jammy rbd-fuse-dbg=19.0.0-1607-g94a53970-1jammy rbd-mirror-dbg=19.0.0-1607-g94a53970-1jammy rbd-nbd-dbg=19.0.0-1607-g94a53970-1jammy python3-cephfs=19.0.0-1607-g94a53970-1jammy cephfs-shell=19.0.0-1607-g94a53970-1jammy cephfs-top=19.0.0-1607-g94a53970-1jammy cephfs-mirror=19.0.0-1607-g94a53970-1jammy 2024-03-11T18:25:38.722 INFO:teuthology.orchestra.run.smithi090.stdout:Reading package lists... 2024-03-11T18:25:38.738 DEBUG:teuthology.orchestra.run.smithi090:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-1607-g94a53970-1jammy cephadm=19.0.0-1607-g94a53970-1jammy ceph-mds=19.0.0-1607-g94a53970-1jammy ceph-mgr=19.0.0-1607-g94a53970-1jammy ceph-common=19.0.0-1607-g94a53970-1jammy ceph-fuse=19.0.0-1607-g94a53970-1jammy ceph-test=19.0.0-1607-g94a53970-1jammy ceph-volume=19.0.0-1607-g94a53970-1jammy radosgw=19.0.0-1607-g94a53970-1jammy python3-rados=19.0.0-1607-g94a53970-1jammy python3-rgw=19.0.0-1607-g94a53970-1jammy python3-cephfs=19.0.0-1607-g94a53970-1jammy python3-rbd=19.0.0-1607-g94a53970-1jammy libcephfs2=19.0.0-1607-g94a53970-1jammy libcephfs-dev=19.0.0-1607-g94a53970-1jammy librados2=19.0.0-1607-g94a53970-1jammy librbd1=19.0.0-1607-g94a53970-1jammy rbd-fuse=19.0.0-1607-g94a53970-1jammy ceph-common-dbg=19.0.0-1607-g94a53970-1jammy ceph-fuse-dbg=19.0.0-1607-g94a53970-1jammy ceph-mds-dbg=19.0.0-1607-g94a53970-1jammy ceph-mgr-dbg=19.0.0-1607-g94a53970-1jammy ceph-mon-dbg=19.0.0-1607-g94a53970-1jammy ceph-osd-dbg=19.0.0-1607-g94a53970-1jammy ceph-test-dbg=19.0.0-1607-g94a53970-1jammy libcephfs2-dbg=19.0.0-1607-g94a53970-1jammy librados2-dbg=19.0.0-1607-g94a53970-1jammy libradosstriper1-dbg=19.0.0-1607-g94a53970-1jammy librbd1-dbg=19.0.0-1607-g94a53970-1jammy librgw2-dbg=19.0.0-1607-g94a53970-1jammy radosgw-dbg=19.0.0-1607-g94a53970-1jammy rbd-fuse-dbg=19.0.0-1607-g94a53970-1jammy rbd-mirror-dbg=19.0.0-1607-g94a53970-1jammy rbd-nbd-dbg=19.0.0-1607-g94a53970-1jammy python3-cephfs=19.0.0-1607-g94a53970-1jammy cephfs-shell=19.0.0-1607-g94a53970-1jammy cephfs-top=19.0.0-1607-g94a53970-1jammy cephfs-mirror=19.0.0-1607-g94a53970-1jammy 2024-03-11T18:25:38.754 INFO:teuthology.orchestra.run.smithi092.stdout:Reading package lists... 2024-03-11T18:25:38.788 INFO:teuthology.orchestra.run.smithi090.stdout:Reading package lists... 2024-03-11T18:25:38.924 INFO:teuthology.orchestra.run.smithi092.stdout:Building dependency tree... 2024-03-11T18:25:38.925 INFO:teuthology.orchestra.run.smithi092.stdout:Reading state information... 2024-03-11T18:25:38.960 INFO:teuthology.orchestra.run.smithi090.stdout:Building dependency tree... 2024-03-11T18:25:38.961 INFO:teuthology.orchestra.run.smithi090.stdout:Reading state information... 2024-03-11T18:25:39.082 INFO:teuthology.orchestra.run.smithi092.stdout:The following packages were automatically installed and are no longer required: 2024-03-11T18:25:39.082 INFO:teuthology.orchestra.run.smithi092.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-11T18:25:39.082 INFO:teuthology.orchestra.run.smithi092.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-11T18:25:39.082 INFO:teuthology.orchestra.run.smithi092.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-11T18:25:39.083 INFO:teuthology.orchestra.run.smithi092.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-11T18:25:39.083 INFO:teuthology.orchestra.run.smithi092.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-11T18:25:39.083 INFO:teuthology.orchestra.run.smithi092.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-11T18:25:39.084 INFO:teuthology.orchestra.run.smithi092.stdout:The following additional packages will be installed: 2024-03-11T18:25:39.084 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-03-11T18:25:39.084 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-03-11T18:25:39.084 INFO:teuthology.orchestra.run.smithi092.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-03-11T18:25:39.084 INFO:teuthology.orchestra.run.smithi092.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-03-11T18:25:39.085 INFO:teuthology.orchestra.run.smithi092.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-03-11T18:25:39.085 INFO:teuthology.orchestra.run.smithi092.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-03-11T18:25:39.085 INFO:teuthology.orchestra.run.smithi092.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-03-11T18:25:39.085 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-03-11T18:25:39.085 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-03-11T18:25:39.085 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-03-11T18:25:39.086 INFO:teuthology.orchestra.run.smithi092.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-03-11T18:25:39.086 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-03-11T18:25:39.086 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-03-11T18:25:39.086 INFO:teuthology.orchestra.run.smithi092.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-03-11T18:25:39.086 INFO:teuthology.orchestra.run.smithi092.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-03-11T18:25:39.086 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-03-11T18:25:39.086 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-03-11T18:25:39.086 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-03-11T18:25:39.086 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-03-11T18:25:39.086 INFO:teuthology.orchestra.run.smithi092.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-03-11T18:25:39.086 INFO:teuthology.orchestra.run.smithi092.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-03-11T18:25:39.086 INFO:teuthology.orchestra.run.smithi092.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-03-11T18:25:39.086 INFO:teuthology.orchestra.run.smithi092.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-03-11T18:25:39.086 INFO:teuthology.orchestra.run.smithi092.stdout: qttranslations5-l10n rbd-mirror rbd-nbd socat unzip xmlstarlet zip 2024-03-11T18:25:39.087 INFO:teuthology.orchestra.run.smithi092.stdout:Suggested packages: 2024-03-11T18:25:39.087 INFO:teuthology.orchestra.run.smithi092.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-03-11T18:25:39.087 INFO:teuthology.orchestra.run.smithi092.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-03-11T18:25:39.087 INFO:teuthology.orchestra.run.smithi092.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-03-11T18:25:39.087 INFO:teuthology.orchestra.run.smithi092.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-03-11T18:25:39.087 INFO:teuthology.orchestra.run.smithi092.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-03-11T18:25:39.087 INFO:teuthology.orchestra.run.smithi092.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-03-11T18:25:39.087 INFO:teuthology.orchestra.run.smithi092.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-03-11T18:25:39.087 INFO:teuthology.orchestra.run.smithi092.stdout:Recommended packages: 2024-03-11T18:25:39.087 INFO:teuthology.orchestra.run.smithi092.stdout: btrfs-tools 2024-03-11T18:25:39.118 INFO:teuthology.orchestra.run.smithi090.stdout:The following packages were automatically installed and are no longer required: 2024-03-11T18:25:39.118 INFO:teuthology.orchestra.run.smithi090.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-11T18:25:39.118 INFO:teuthology.orchestra.run.smithi090.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-11T18:25:39.118 INFO:teuthology.orchestra.run.smithi090.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-11T18:25:39.118 INFO:teuthology.orchestra.run.smithi090.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-11T18:25:39.119 INFO:teuthology.orchestra.run.smithi090.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-11T18:25:39.119 INFO:teuthology.orchestra.run.smithi090.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-11T18:25:39.119 INFO:teuthology.orchestra.run.smithi090.stdout:The following additional packages will be installed: 2024-03-11T18:25:39.119 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-03-11T18:25:39.119 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-03-11T18:25:39.120 INFO:teuthology.orchestra.run.smithi090.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-03-11T18:25:39.120 INFO:teuthology.orchestra.run.smithi090.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-03-11T18:25:39.120 INFO:teuthology.orchestra.run.smithi090.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-03-11T18:25:39.121 INFO:teuthology.orchestra.run.smithi090.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-03-11T18:25:39.121 INFO:teuthology.orchestra.run.smithi090.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-03-11T18:25:39.121 INFO:teuthology.orchestra.run.smithi090.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-03-11T18:25:39.121 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-03-11T18:25:39.121 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-03-11T18:25:39.121 INFO:teuthology.orchestra.run.smithi090.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-03-11T18:25:39.121 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-03-11T18:25:39.121 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-03-11T18:25:39.121 INFO:teuthology.orchestra.run.smithi090.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-03-11T18:25:39.121 INFO:teuthology.orchestra.run.smithi090.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-03-11T18:25:39.121 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-03-11T18:25:39.121 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-03-11T18:25:39.121 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-03-11T18:25:39.122 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-03-11T18:25:39.122 INFO:teuthology.orchestra.run.smithi090.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-03-11T18:25:39.122 INFO:teuthology.orchestra.run.smithi090.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-03-11T18:25:39.122 INFO:teuthology.orchestra.run.smithi090.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-03-11T18:25:39.122 INFO:teuthology.orchestra.run.smithi090.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-03-11T18:25:39.122 INFO:teuthology.orchestra.run.smithi090.stdout: qttranslations5-l10n rbd-mirror rbd-nbd socat unzip xmlstarlet zip 2024-03-11T18:25:39.122 INFO:teuthology.orchestra.run.smithi090.stdout:Suggested packages: 2024-03-11T18:25:39.122 INFO:teuthology.orchestra.run.smithi090.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-03-11T18:25:39.122 INFO:teuthology.orchestra.run.smithi090.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-03-11T18:25:39.122 INFO:teuthology.orchestra.run.smithi090.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-03-11T18:25:39.122 INFO:teuthology.orchestra.run.smithi090.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-03-11T18:25:39.123 INFO:teuthology.orchestra.run.smithi090.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-03-11T18:25:39.123 INFO:teuthology.orchestra.run.smithi090.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-03-11T18:25:39.123 INFO:teuthology.orchestra.run.smithi090.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-03-11T18:25:39.123 INFO:teuthology.orchestra.run.smithi090.stdout:Recommended packages: 2024-03-11T18:25:39.123 INFO:teuthology.orchestra.run.smithi090.stdout: btrfs-tools 2024-03-11T18:25:39.213 INFO:teuthology.orchestra.run.smithi092.stdout:The following NEW packages will be installed: 2024-03-11T18:25:39.214 INFO:teuthology.orchestra.run.smithi092.stdout: ceph ceph-base ceph-common ceph-common-dbg ceph-fuse ceph-fuse-dbg ceph-mds 2024-03-11T18:25:39.214 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mds-dbg ceph-mgr ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-dbg 2024-03-11T18:25:39.214 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2024-03-11T18:25:39.214 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mon ceph-mon-dbg ceph-osd ceph-osd-dbg ceph-test ceph-test-dbg 2024-03-11T18:25:39.214 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-volume cephadm cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev 2024-03-11T18:25:39.214 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs2 libcephfs2-dbg libdouble-conversion3 libjq1 liblttng-ust1 2024-03-11T18:25:39.214 INFO:teuthology.orchestra.run.smithi092.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-03-11T18:25:39.214 INFO:teuthology.orchestra.run.smithi092.stdout: libqt5network5 librados2 librados2-dbg libradosstriper1 libradosstriper1-dbg 2024-03-11T18:25:39.215 INFO:teuthology.orchestra.run.smithi092.stdout: librbd1 librbd1-dbg librdkafka1 libreadline-dev librgw2 librgw2-dbg 2024-03-11T18:25:39.215 INFO:teuthology.orchestra.run.smithi092.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-03-11T18:25:39.215 INFO:teuthology.orchestra.run.smithi092.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-03-11T18:25:39.216 INFO:teuthology.orchestra.run.smithi092.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-03-11T18:25:39.216 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-03-11T18:25:39.216 INFO:teuthology.orchestra.run.smithi092.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-03-11T18:25:39.216 INFO:teuthology.orchestra.run.smithi092.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-03-11T18:25:39.216 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-03-11T18:25:39.216 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-03-11T18:25:39.216 INFO:teuthology.orchestra.run.smithi092.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-03-11T18:25:39.216 INFO:teuthology.orchestra.run.smithi092.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-03-11T18:25:39.216 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-03-11T18:25:39.216 INFO:teuthology.orchestra.run.smithi092.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-03-11T18:25:39.216 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-03-11T18:25:39.216 INFO:teuthology.orchestra.run.smithi092.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-03-11T18:25:39.216 INFO:teuthology.orchestra.run.smithi092.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-03-11T18:25:39.216 INFO:teuthology.orchestra.run.smithi092.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-03-11T18:25:39.216 INFO:teuthology.orchestra.run.smithi092.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-03-11T18:25:39.216 INFO:teuthology.orchestra.run.smithi092.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-03-11T18:25:39.217 INFO:teuthology.orchestra.run.smithi092.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-03-11T18:25:39.217 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw 2024-03-11T18:25:39.217 INFO:teuthology.orchestra.run.smithi092.stdout: radosgw-dbg rbd-fuse rbd-fuse-dbg rbd-mirror rbd-mirror-dbg rbd-nbd 2024-03-11T18:25:39.217 INFO:teuthology.orchestra.run.smithi092.stdout: rbd-nbd-dbg socat unzip xmlstarlet zip 2024-03-11T18:25:39.252 INFO:teuthology.orchestra.run.smithi090.stdout:The following NEW packages will be installed: 2024-03-11T18:25:39.252 INFO:teuthology.orchestra.run.smithi090.stdout: ceph ceph-base ceph-common ceph-common-dbg ceph-fuse ceph-fuse-dbg ceph-mds 2024-03-11T18:25:39.252 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mds-dbg ceph-mgr ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-dbg 2024-03-11T18:25:39.253 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2024-03-11T18:25:39.253 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mon ceph-mon-dbg ceph-osd ceph-osd-dbg ceph-test ceph-test-dbg 2024-03-11T18:25:39.253 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-volume cephadm cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev 2024-03-11T18:25:39.254 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs2 libcephfs2-dbg libdouble-conversion3 libjq1 liblttng-ust1 2024-03-11T18:25:39.254 INFO:teuthology.orchestra.run.smithi090.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-03-11T18:25:39.254 INFO:teuthology.orchestra.run.smithi090.stdout: libqt5network5 librados2 librados2-dbg libradosstriper1 libradosstriper1-dbg 2024-03-11T18:25:39.254 INFO:teuthology.orchestra.run.smithi090.stdout: librbd1 librbd1-dbg librdkafka1 libreadline-dev librgw2 librgw2-dbg 2024-03-11T18:25:39.254 INFO:teuthology.orchestra.run.smithi090.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-03-11T18:25:39.254 INFO:teuthology.orchestra.run.smithi090.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-03-11T18:25:39.255 INFO:teuthology.orchestra.run.smithi090.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-03-11T18:25:39.255 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-03-11T18:25:39.255 INFO:teuthology.orchestra.run.smithi090.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-03-11T18:25:39.255 INFO:teuthology.orchestra.run.smithi090.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-03-11T18:25:39.255 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-03-11T18:25:39.255 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-03-11T18:25:39.255 INFO:teuthology.orchestra.run.smithi090.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-03-11T18:25:39.255 INFO:teuthology.orchestra.run.smithi090.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-03-11T18:25:39.255 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-03-11T18:25:39.255 INFO:teuthology.orchestra.run.smithi090.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-03-11T18:25:39.255 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-03-11T18:25:39.255 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-03-11T18:25:39.255 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-03-11T18:25:39.255 INFO:teuthology.orchestra.run.smithi090.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-03-11T18:25:39.255 INFO:teuthology.orchestra.run.smithi090.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-03-11T18:25:39.256 INFO:teuthology.orchestra.run.smithi090.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-03-11T18:25:39.256 INFO:teuthology.orchestra.run.smithi090.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-03-11T18:25:39.256 INFO:teuthology.orchestra.run.smithi090.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw 2024-03-11T18:25:39.256 INFO:teuthology.orchestra.run.smithi090.stdout: radosgw-dbg rbd-fuse rbd-fuse-dbg rbd-mirror rbd-mirror-dbg rbd-nbd 2024-03-11T18:25:39.256 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-nbd-dbg socat unzip xmlstarlet zip 2024-03-11T18:25:39.282 INFO:teuthology.orchestra.run.smithi092.stdout:0 upgraded, 142 newly installed, 0 to remove and 275 not upgraded. 2024-03-11T18:25:39.282 INFO:teuthology.orchestra.run.smithi092.stdout:Need to get 3,736 MB of archives. 2024-03-11T18:25:39.282 INFO:teuthology.orchestra.run.smithi092.stdout:After this operation, 4,547 MB of additional disk space will be used. 2024-03-11T18:25:39.282 INFO:teuthology.orchestra.run.smithi092.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-03-11T18:25:39.303 INFO:teuthology.orchestra.run.smithi092.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-03-11T18:25:39.315 INFO:teuthology.orchestra.run.smithi092.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-03-11T18:25:39.318 INFO:teuthology.orchestra.run.smithi090.stdout:0 upgraded, 142 newly installed, 0 to remove and 275 not upgraded. 2024-03-11T18:25:39.319 INFO:teuthology.orchestra.run.smithi090.stdout:Need to get 3,736 MB of archives. 2024-03-11T18:25:39.319 INFO:teuthology.orchestra.run.smithi090.stdout:After this operation, 4,547 MB of additional disk space will be used. 2024-03-11T18:25:39.319 INFO:teuthology.orchestra.run.smithi090.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-03-11T18:25:39.339 INFO:teuthology.orchestra.run.smithi090.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-03-11T18:25:39.350 INFO:teuthology.orchestra.run.smithi090.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-03-11T18:25:39.358 INFO:teuthology.orchestra.run.smithi092.stdout:Get:4 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-1607-g94a53970-1jammy [3,532 kB] 2024-03-11T18:25:39.366 INFO:teuthology.orchestra.run.smithi092.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-03-11T18:25:39.391 INFO:teuthology.orchestra.run.smithi092.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-03-11T18:25:39.394 INFO:teuthology.orchestra.run.smithi092.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-03-11T18:25:39.398 INFO:teuthology.orchestra.run.smithi090.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-03-11T18:25:39.406 INFO:teuthology.orchestra.run.smithi092.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-03-11T18:25:39.422 INFO:teuthology.orchestra.run.smithi090.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-03-11T18:25:39.426 INFO:teuthology.orchestra.run.smithi090.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-03-11T18:25:39.439 INFO:teuthology.orchestra.run.smithi090.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-03-11T18:25:39.459 INFO:teuthology.orchestra.run.smithi092.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-03-11T18:25:39.464 INFO:teuthology.orchestra.run.smithi092.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-03-11T18:25:39.476 INFO:teuthology.orchestra.run.smithi092.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-03-11T18:25:39.478 INFO:teuthology.orchestra.run.smithi092.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-03-11T18:25:39.479 INFO:teuthology.orchestra.run.smithi092.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-03-11T18:25:39.481 INFO:teuthology.orchestra.run.smithi090.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-1607-g94a53970-1jammy [3,532 kB] 2024-03-11T18:25:39.487 INFO:teuthology.orchestra.run.smithi092.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-03-11T18:25:39.489 INFO:teuthology.orchestra.run.smithi092.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-03-11T18:25:39.491 INFO:teuthology.orchestra.run.smithi092.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-03-11T18:25:39.492 INFO:teuthology.orchestra.run.smithi092.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-03-11T18:25:39.493 INFO:teuthology.orchestra.run.smithi092.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-03-11T18:25:39.498 INFO:teuthology.orchestra.run.smithi090.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-03-11T18:25:39.502 INFO:teuthology.orchestra.run.smithi090.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-03-11T18:25:39.515 INFO:teuthology.orchestra.run.smithi090.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-03-11T18:25:39.519 INFO:teuthology.orchestra.run.smithi090.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-03-11T18:25:39.520 INFO:teuthology.orchestra.run.smithi090.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-03-11T18:25:39.525 INFO:teuthology.orchestra.run.smithi092.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-03-11T18:25:39.527 INFO:teuthology.orchestra.run.smithi092.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-03-11T18:25:39.528 INFO:teuthology.orchestra.run.smithi090.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-03-11T18:25:39.529 INFO:teuthology.orchestra.run.smithi092.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-03-11T18:25:39.529 INFO:teuthology.orchestra.run.smithi092.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-03-11T18:25:39.529 INFO:teuthology.orchestra.run.smithi092.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-03-11T18:25:39.531 INFO:teuthology.orchestra.run.smithi090.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-03-11T18:25:39.531 INFO:teuthology.orchestra.run.smithi092.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-03-11T18:25:39.533 INFO:teuthology.orchestra.run.smithi090.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-03-11T18:25:39.534 INFO:teuthology.orchestra.run.smithi090.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-03-11T18:25:39.534 INFO:teuthology.orchestra.run.smithi090.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-03-11T18:25:39.546 INFO:teuthology.orchestra.run.smithi092.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-03-11T18:25:39.547 INFO:teuthology.orchestra.run.smithi092.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-03-11T18:25:39.547 INFO:teuthology.orchestra.run.smithi092.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-03-11T18:25:39.548 INFO:teuthology.orchestra.run.smithi092.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-03-11T18:25:39.553 INFO:teuthology.orchestra.run.smithi090.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-03-11T18:25:39.556 INFO:teuthology.orchestra.run.smithi090.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-03-11T18:25:39.558 INFO:teuthology.orchestra.run.smithi090.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-03-11T18:25:39.559 INFO:teuthology.orchestra.run.smithi090.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-03-11T18:25:39.566 INFO:teuthology.orchestra.run.smithi092.stdout:Get:29 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-1607-g94a53970-1jammy [3,212 kB] 2024-03-11T18:25:39.584 INFO:teuthology.orchestra.run.smithi092.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-03-11T18:25:39.585 INFO:teuthology.orchestra.run.smithi092.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-03-11T18:25:39.588 INFO:teuthology.orchestra.run.smithi092.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-03-11T18:25:39.588 INFO:teuthology.orchestra.run.smithi092.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-03-11T18:25:39.589 INFO:teuthology.orchestra.run.smithi092.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-03-11T18:25:39.589 INFO:teuthology.orchestra.run.smithi092.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-03-11T18:25:39.592 INFO:teuthology.orchestra.run.smithi090.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-03-11T18:25:39.594 INFO:teuthology.orchestra.run.smithi090.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-03-11T18:25:39.594 INFO:teuthology.orchestra.run.smithi090.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-03-11T18:25:39.594 INFO:teuthology.orchestra.run.smithi090.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-03-11T18:25:39.595 INFO:teuthology.orchestra.run.smithi090.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-03-11T18:25:39.595 INFO:teuthology.orchestra.run.smithi090.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-03-11T18:25:39.630 INFO:teuthology.orchestra.run.smithi092.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-03-11T18:25:39.631 INFO:teuthology.orchestra.run.smithi092.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-03-11T18:25:39.631 INFO:teuthology.orchestra.run.smithi092.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-03-11T18:25:39.634 INFO:teuthology.orchestra.run.smithi092.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-03-11T18:25:39.638 INFO:teuthology.orchestra.run.smithi090.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-03-11T18:25:39.639 INFO:teuthology.orchestra.run.smithi090.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-03-11T18:25:39.644 INFO:teuthology.orchestra.run.smithi090.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-03-11T18:25:39.645 INFO:teuthology.orchestra.run.smithi090.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-03-11T18:25:39.667 INFO:teuthology.orchestra.run.smithi092.stdout:Get:40 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-1607-g94a53970-1jammy [710 kB] 2024-03-11T18:25:39.674 INFO:teuthology.orchestra.run.smithi092.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-03-11T18:25:39.675 INFO:teuthology.orchestra.run.smithi092.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-03-11T18:25:39.686 INFO:teuthology.orchestra.run.smithi092.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-03-11T18:25:39.686 INFO:teuthology.orchestra.run.smithi092.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-03-11T18:25:39.687 INFO:teuthology.orchestra.run.smithi092.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-03-11T18:25:39.688 INFO:teuthology.orchestra.run.smithi092.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-03-11T18:25:39.689 INFO:teuthology.orchestra.run.smithi092.stdout:Get:47 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-1607-g94a53970-1jammy [356 kB] 2024-03-11T18:25:39.693 INFO:teuthology.orchestra.run.smithi090.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-03-11T18:25:39.694 INFO:teuthology.orchestra.run.smithi090.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-03-11T18:25:39.695 INFO:teuthology.orchestra.run.smithi090.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-03-11T18:25:39.696 INFO:teuthology.orchestra.run.smithi090.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-03-11T18:25:39.697 INFO:teuthology.orchestra.run.smithi090.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-03-11T18:25:39.699 INFO:teuthology.orchestra.run.smithi090.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-03-11T18:25:39.703 INFO:teuthology.orchestra.run.smithi092.stdout:Get:48 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-1607-g94a53970-1jammy [32.8 kB] 2024-03-11T18:25:39.704 INFO:teuthology.orchestra.run.smithi092.stdout:Get:49 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-1607-g94a53970-1jammy [180 kB] 2024-03-11T18:25:39.708 INFO:teuthology.orchestra.run.smithi092.stdout:Get:50 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-1607-g94a53970-1jammy [64.8 kB] 2024-03-11T18:25:39.710 INFO:teuthology.orchestra.run.smithi092.stdout:Get:51 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-1607-g94a53970-1jammy [330 kB] 2024-03-11T18:25:39.718 INFO:teuthology.orchestra.run.smithi092.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-03-11T18:25:39.721 INFO:teuthology.orchestra.run.smithi092.stdout:Get:53 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-1607-g94a53970-1jammy [6,568 kB] 2024-03-11T18:25:39.722 INFO:teuthology.orchestra.run.smithi090.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-03-11T18:25:39.722 INFO:teuthology.orchestra.run.smithi092.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-03-11T18:25:39.722 INFO:teuthology.orchestra.run.smithi090.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-03-11T18:25:39.723 INFO:teuthology.orchestra.run.smithi092.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-03-11T18:25:39.726 INFO:teuthology.orchestra.run.smithi092.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-03-11T18:25:39.733 INFO:teuthology.orchestra.run.smithi090.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-03-11T18:25:39.733 INFO:teuthology.orchestra.run.smithi090.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-03-11T18:25:39.738 INFO:teuthology.orchestra.run.smithi092.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-03-11T18:25:39.743 INFO:teuthology.orchestra.run.smithi092.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-03-11T18:25:39.744 INFO:teuthology.orchestra.run.smithi092.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-03-11T18:25:39.746 INFO:teuthology.orchestra.run.smithi092.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-03-11T18:25:39.746 INFO:teuthology.orchestra.run.smithi090.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-03-11T18:25:39.746 INFO:teuthology.orchestra.run.smithi090.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-03-11T18:25:39.749 INFO:teuthology.orchestra.run.smithi090.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-03-11T18:25:39.750 INFO:teuthology.orchestra.run.smithi092.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-03-11T18:25:39.753 INFO:teuthology.orchestra.run.smithi090.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-03-11T18:25:39.754 INFO:teuthology.orchestra.run.smithi090.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-03-11T18:25:39.757 INFO:teuthology.orchestra.run.smithi090.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-03-11T18:25:39.775 INFO:teuthology.orchestra.run.smithi090.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-03-11T18:25:39.779 INFO:teuthology.orchestra.run.smithi090.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-03-11T18:25:39.781 INFO:teuthology.orchestra.run.smithi090.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-03-11T18:25:39.783 INFO:teuthology.orchestra.run.smithi090.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-03-11T18:25:39.796 INFO:teuthology.orchestra.run.smithi090.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-03-11T18:25:39.824 INFO:teuthology.orchestra.run.smithi092.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-03-11T18:25:39.825 INFO:teuthology.orchestra.run.smithi092.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-03-11T18:25:39.826 INFO:teuthology.orchestra.run.smithi092.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-03-11T18:25:39.827 INFO:teuthology.orchestra.run.smithi092.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-03-11T18:25:39.828 INFO:teuthology.orchestra.run.smithi092.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-03-11T18:25:39.866 INFO:teuthology.orchestra.run.smithi092.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-03-11T18:25:39.869 INFO:teuthology.orchestra.run.smithi092.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-03-11T18:25:39.869 INFO:teuthology.orchestra.run.smithi092.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-03-11T18:25:39.884 INFO:teuthology.orchestra.run.smithi090.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-03-11T18:25:39.885 INFO:teuthology.orchestra.run.smithi090.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-03-11T18:25:39.888 INFO:teuthology.orchestra.run.smithi090.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-03-11T18:25:39.889 INFO:teuthology.orchestra.run.smithi090.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-03-11T18:25:39.892 INFO:teuthology.orchestra.run.smithi090.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-03-11T18:25:39.903 INFO:teuthology.orchestra.run.smithi092.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-03-11T18:25:39.903 INFO:teuthology.orchestra.run.smithi092.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-03-11T18:25:39.919 INFO:teuthology.orchestra.run.smithi092.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-03-11T18:25:39.920 INFO:teuthology.orchestra.run.smithi092.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-03-11T18:25:39.921 INFO:teuthology.orchestra.run.smithi092.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-03-11T18:25:39.921 INFO:teuthology.orchestra.run.smithi090.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-03-11T18:25:39.921 INFO:teuthology.orchestra.run.smithi092.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-03-11T18:25:39.922 INFO:teuthology.orchestra.run.smithi092.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-03-11T18:25:39.922 INFO:teuthology.orchestra.run.smithi092.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-03-11T18:25:39.923 INFO:teuthology.orchestra.run.smithi090.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-03-11T18:25:39.924 INFO:teuthology.orchestra.run.smithi090.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-03-11T18:25:39.927 INFO:teuthology.orchestra.run.smithi092.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-03-11T18:25:39.940 INFO:teuthology.orchestra.run.smithi092.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-03-11T18:25:39.941 INFO:teuthology.orchestra.run.smithi092.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-03-11T18:25:39.942 INFO:teuthology.orchestra.run.smithi090.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-03-11T18:25:39.942 INFO:teuthology.orchestra.run.smithi090.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-03-11T18:25:39.943 INFO:teuthology.orchestra.run.smithi092.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-03-11T18:25:39.945 INFO:teuthology.orchestra.run.smithi092.stdout:Get:82 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-1607-g94a53970-1jammy [112 kB] 2024-03-11T18:25:39.947 INFO:teuthology.orchestra.run.smithi090.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-03-11T18:25:39.947 INFO:teuthology.orchestra.run.smithi092.stdout:Get:83 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-1607-g94a53970-1jammy [465 kB] 2024-03-11T18:25:39.948 INFO:teuthology.orchestra.run.smithi090.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-03-11T18:25:39.948 INFO:teuthology.orchestra.run.smithi090.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-03-11T18:25:39.948 INFO:teuthology.orchestra.run.smithi090.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-03-11T18:25:39.949 INFO:teuthology.orchestra.run.smithi090.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-03-11T18:25:39.949 INFO:teuthology.orchestra.run.smithi090.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-03-11T18:25:39.953 INFO:teuthology.orchestra.run.smithi090.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-03-11T18:25:39.960 INFO:teuthology.orchestra.run.smithi092.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-03-11T18:25:39.963 INFO:teuthology.orchestra.run.smithi092.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-03-11T18:25:39.964 INFO:teuthology.orchestra.run.smithi092.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-03-11T18:25:39.965 INFO:teuthology.orchestra.run.smithi092.stdout:Get:87 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-1607-g94a53970-1jammy [24.1 MB] 2024-03-11T18:25:39.965 INFO:teuthology.orchestra.run.smithi092.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-03-11T18:25:39.966 INFO:teuthology.orchestra.run.smithi092.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-03-11T18:25:39.967 INFO:teuthology.orchestra.run.smithi090.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-03-11T18:25:39.969 INFO:teuthology.orchestra.run.smithi090.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-03-11T18:25:39.970 INFO:teuthology.orchestra.run.smithi090.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-03-11T18:25:39.971 INFO:teuthology.orchestra.run.smithi092.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-03-11T18:25:39.972 INFO:teuthology.orchestra.run.smithi092.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-03-11T18:25:39.982 INFO:teuthology.orchestra.run.smithi092.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-03-11T18:25:39.982 INFO:teuthology.orchestra.run.smithi092.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-03-11T18:25:39.983 INFO:teuthology.orchestra.run.smithi092.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-03-11T18:25:39.988 INFO:teuthology.orchestra.run.smithi090.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-03-11T18:25:39.990 INFO:teuthology.orchestra.run.smithi090.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-03-11T18:25:39.991 INFO:teuthology.orchestra.run.smithi090.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-03-11T18:25:39.992 INFO:teuthology.orchestra.run.smithi090.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-03-11T18:25:39.992 INFO:teuthology.orchestra.run.smithi090.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-03-11T18:25:39.997 INFO:teuthology.orchestra.run.smithi090.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-03-11T18:25:39.998 INFO:teuthology.orchestra.run.smithi090.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-03-11T18:25:40.002 INFO:teuthology.orchestra.run.smithi092.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-03-11T18:25:40.004 INFO:teuthology.orchestra.run.smithi092.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-03-11T18:25:40.005 INFO:teuthology.orchestra.run.smithi092.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-03-11T18:25:40.008 INFO:teuthology.orchestra.run.smithi090.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-03-11T18:25:40.008 INFO:teuthology.orchestra.run.smithi090.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-03-11T18:25:40.009 INFO:teuthology.orchestra.run.smithi090.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-03-11T18:25:40.014 INFO:teuthology.orchestra.run.smithi092.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-03-11T18:25:40.015 INFO:teuthology.orchestra.run.smithi092.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-03-11T18:25:40.015 INFO:teuthology.orchestra.run.smithi092.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-03-11T18:25:40.018 INFO:teuthology.orchestra.run.smithi092.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-03-11T18:25:40.023 INFO:teuthology.orchestra.run.smithi092.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-03-11T18:25:40.028 INFO:teuthology.orchestra.run.smithi090.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-03-11T18:25:40.030 INFO:teuthology.orchestra.run.smithi090.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-03-11T18:25:40.031 INFO:teuthology.orchestra.run.smithi090.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-03-11T18:25:40.041 INFO:teuthology.orchestra.run.smithi090.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-03-11T18:25:40.042 INFO:teuthology.orchestra.run.smithi090.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-03-11T18:25:40.042 INFO:teuthology.orchestra.run.smithi090.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-03-11T18:25:40.045 INFO:teuthology.orchestra.run.smithi090.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-03-11T18:25:40.052 INFO:teuthology.orchestra.run.smithi090.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-03-11T18:25:40.894 INFO:teuthology.orchestra.run.smithi092.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-1607-g94a53970-1jammy [5,073 kB] 2024-03-11T18:25:40.971 INFO:teuthology.orchestra.run.smithi092.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-1607-g94a53970-1jammy [243 kB] 2024-03-11T18:25:40.978 INFO:teuthology.orchestra.run.smithi092.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-1607-g94a53970-1jammy [124 kB] 2024-03-11T18:25:40.983 INFO:teuthology.orchestra.run.smithi092.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-1607-g94a53970-1jammy [1,519 kB] 2024-03-11T18:25:41.037 INFO:teuthology.orchestra.run.smithi092.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-1607-g94a53970-1jammy [6,240 kB] 2024-03-11T18:25:41.253 INFO:teuthology.orchestra.run.smithi092.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-1607-g94a53970-1jammy [22.7 MB] 2024-03-11T18:25:42.100 INFO:teuthology.orchestra.run.smithi090.stdout:Get:92 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-1607-g94a53970-1jammy [3,212 kB] 2024-03-11T18:25:42.127 INFO:teuthology.orchestra.run.smithi092.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-1607-g94a53970-1jammy [14.0 kB] 2024-03-11T18:25:42.128 INFO:teuthology.orchestra.run.smithi092.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-1607-g94a53970-1jammy [865 kB] 2024-03-11T18:25:42.145 INFO:teuthology.orchestra.run.smithi092.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-1607-g94a53970-1jammy [2,185 kB] 2024-03-11T18:25:42.182 INFO:teuthology.orchestra.run.smithi092.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-1607-g94a53970-1jammy [776 kB] 2024-03-11T18:25:42.192 INFO:teuthology.orchestra.run.smithi092.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-1607-g94a53970-1jammy [144 kB] 2024-03-11T18:25:42.194 INFO:teuthology.orchestra.run.smithi092.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-1607-g94a53970-1jammy [1,921 kB] 2024-03-11T18:25:42.249 INFO:teuthology.orchestra.run.smithi092.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-1607-g94a53970-1jammy [8,625 kB] 2024-03-11T18:25:42.551 INFO:teuthology.orchestra.run.smithi092.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-1607-g94a53970-1jammy [14.2 kB] 2024-03-11T18:25:42.552 INFO:teuthology.orchestra.run.smithi092.stdout:Get:117 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-1607-g94a53970-1jammy [49.9 MB] 2024-03-11T18:25:43.778 INFO:teuthology.orchestra.run.smithi090.stdout:Get:93 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-1607-g94a53970-1jammy [710 kB] 2024-03-11T18:25:44.167 INFO:teuthology.orchestra.run.smithi090.stdout:Get:94 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-1607-g94a53970-1jammy [356 kB] 2024-03-11T18:25:44.348 INFO:teuthology.orchestra.run.smithi090.stdout:Get:95 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-1607-g94a53970-1jammy [32.8 kB] 2024-03-11T18:25:44.360 INFO:teuthology.orchestra.run.smithi090.stdout:Get:96 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-1607-g94a53970-1jammy [180 kB] 2024-03-11T18:25:44.421 INFO:teuthology.orchestra.run.smithi090.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-1607-g94a53970-1jammy [64.8 kB] 2024-03-11T18:25:44.455 INFO:teuthology.orchestra.run.smithi090.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-1607-g94a53970-1jammy [330 kB] 2024-03-11T18:25:44.530 INFO:teuthology.orchestra.run.smithi090.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-1607-g94a53970-1jammy [6,568 kB] 2024-03-11T18:25:44.550 INFO:teuthology.orchestra.run.smithi092.stdout:Get:118 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-1607-g94a53970-1jammy [121 kB] 2024-03-11T18:25:44.553 INFO:teuthology.orchestra.run.smithi092.stdout:Get:119 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-1607-g94a53970-1jammy [226 kB] 2024-03-11T18:25:44.558 INFO:teuthology.orchestra.run.smithi092.stdout:Get:120 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-1607-g94a53970-1jammy [34.0 kB] 2024-03-11T18:25:44.559 INFO:teuthology.orchestra.run.smithi092.stdout:Get:121 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-1607-g94a53970-1jammy [23.4 kB] 2024-03-11T18:25:44.560 INFO:teuthology.orchestra.run.smithi092.stdout:Get:122 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-1607-g94a53970-1jammy [38.4 kB] 2024-03-11T18:25:44.561 INFO:teuthology.orchestra.run.smithi092.stdout:Get:123 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-1607-g94a53970-1jammy [13.0 MB] 2024-03-11T18:25:44.939 INFO:teuthology.orchestra.run.smithi092.stdout:Get:124 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-1607-g94a53970-1jammy [91.5 kB] 2024-03-11T18:25:44.941 INFO:teuthology.orchestra.run.smithi092.stdout:Get:125 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 rbd-mirror amd64 19.0.0-1607-g94a53970-1jammy [3,121 kB] 2024-03-11T18:25:45.007 INFO:teuthology.orchestra.run.smithi092.stdout:Get:126 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 rbd-nbd amd64 19.0.0-1607-g94a53970-1jammy [179 kB] 2024-03-11T18:25:45.013 INFO:teuthology.orchestra.run.smithi092.stdout:Get:127 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common-dbg amd64 19.0.0-1607-g94a53970-1jammy [722 MB] 2024-03-11T18:25:48.066 INFO:teuthology.orchestra.run.smithi090.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-1607-g94a53970-1jammy [112 kB] 2024-03-11T18:25:48.118 INFO:teuthology.orchestra.run.smithi090.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-1607-g94a53970-1jammy [465 kB] 2024-03-11T18:25:48.326 INFO:teuthology.orchestra.run.smithi090.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-1607-g94a53970-1jammy [24.1 MB] 2024-03-11T18:25:55.323 INFO:teuthology.orchestra.run.smithi090.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-1607-g94a53970-1jammy [5,073 kB] 2024-03-11T18:25:56.167 INFO:teuthology.orchestra.run.smithi090.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-1607-g94a53970-1jammy [243 kB] 2024-03-11T18:25:56.219 INFO:teuthology.orchestra.run.smithi090.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-1607-g94a53970-1jammy [124 kB] 2024-03-11T18:25:56.228 INFO:teuthology.orchestra.run.smithi090.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-1607-g94a53970-1jammy [1,519 kB] 2024-03-11T18:25:56.414 INFO:teuthology.orchestra.run.smithi090.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-1607-g94a53970-1jammy [6,240 kB] 2024-03-11T18:25:57.132 INFO:teuthology.orchestra.run.smithi090.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-1607-g94a53970-1jammy [22.7 MB] 2024-03-11T18:25:59.092 INFO:teuthology.orchestra.run.smithi090.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-1607-g94a53970-1jammy [14.0 kB] 2024-03-11T18:25:59.093 INFO:teuthology.orchestra.run.smithi090.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-1607-g94a53970-1jammy [865 kB] 2024-03-11T18:25:59.110 INFO:teuthology.orchestra.run.smithi090.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-1607-g94a53970-1jammy [2,185 kB] 2024-03-11T18:25:59.195 INFO:teuthology.orchestra.run.smithi090.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-1607-g94a53970-1jammy [776 kB] 2024-03-11T18:25:59.246 INFO:teuthology.orchestra.run.smithi090.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-1607-g94a53970-1jammy [144 kB] 2024-03-11T18:25:59.254 INFO:teuthology.orchestra.run.smithi090.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-1607-g94a53970-1jammy [1,921 kB] 2024-03-11T18:25:59.372 INFO:teuthology.orchestra.run.smithi090.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-1607-g94a53970-1jammy [8,625 kB] 2024-03-11T18:25:59.906 INFO:teuthology.orchestra.run.smithi090.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-1607-g94a53970-1jammy [14.2 kB] 2024-03-11T18:25:59.907 INFO:teuthology.orchestra.run.smithi090.stdout:Get:117 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-1607-g94a53970-1jammy [49.9 MB] 2024-03-11T18:26:03.881 INFO:teuthology.orchestra.run.smithi090.stdout:Get:118 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-1607-g94a53970-1jammy [121 kB] 2024-03-11T18:26:03.884 INFO:teuthology.orchestra.run.smithi090.stdout:Get:119 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-1607-g94a53970-1jammy [226 kB] 2024-03-11T18:26:03.889 INFO:teuthology.orchestra.run.smithi090.stdout:Get:120 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-1607-g94a53970-1jammy [34.0 kB] 2024-03-11T18:26:03.890 INFO:teuthology.orchestra.run.smithi090.stdout:Get:121 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-1607-g94a53970-1jammy [23.4 kB] 2024-03-11T18:26:03.891 INFO:teuthology.orchestra.run.smithi090.stdout:Get:122 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-1607-g94a53970-1jammy [38.4 kB] 2024-03-11T18:26:03.892 INFO:teuthology.orchestra.run.smithi090.stdout:Get:123 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-1607-g94a53970-1jammy [13.0 MB] 2024-03-11T18:26:04.725 INFO:teuthology.orchestra.run.smithi090.stdout:Get:124 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-1607-g94a53970-1jammy [91.5 kB] 2024-03-11T18:26:04.727 INFO:teuthology.orchestra.run.smithi090.stdout:Get:125 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 rbd-mirror amd64 19.0.0-1607-g94a53970-1jammy [3,121 kB] 2024-03-11T18:26:05.405 INFO:teuthology.orchestra.run.smithi090.stdout:Get:126 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 rbd-nbd amd64 19.0.0-1607-g94a53970-1jammy [179 kB] 2024-03-11T18:26:05.411 INFO:teuthology.orchestra.run.smithi090.stdout:Get:127 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common-dbg amd64 19.0.0-1607-g94a53970-1jammy [722 MB] 2024-03-11T18:26:23.341 INFO:teuthology.orchestra.run.smithi092.stdout:Get:128 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse-dbg amd64 19.0.0-1607-g94a53970-1jammy [18.1 MB] 2024-03-11T18:26:24.518 INFO:teuthology.orchestra.run.smithi092.stdout:Get:129 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds-dbg amd64 19.0.0-1607-g94a53970-1jammy [77.7 MB] 2024-03-11T18:26:30.441 INFO:teuthology.orchestra.run.smithi092.stdout:Get:130 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dbg amd64 19.0.0-1607-g94a53970-1jammy [41.4 MB] 2024-03-11T18:26:32.827 INFO:teuthology.orchestra.run.smithi092.stdout:Get:131 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon-dbg amd64 19.0.0-1607-g94a53970-1jammy [158 MB] 2024-03-11T18:26:43.113 INFO:teuthology.orchestra.run.smithi092.stdout:Get:132 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd-dbg amd64 19.0.0-1607-g94a53970-1jammy [537 MB] 2024-03-11T18:27:05.592 INFO:teuthology.orchestra.run.smithi090.stdout:Get:128 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse-dbg amd64 19.0.0-1607-g94a53970-1jammy [18.1 MB] 2024-03-11T18:27:07.048 INFO:teuthology.orchestra.run.smithi090.stdout:Get:129 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds-dbg amd64 19.0.0-1607-g94a53970-1jammy [77.7 MB] 2024-03-11T18:27:15.286 INFO:teuthology.orchestra.run.smithi090.stdout:Get:130 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dbg amd64 19.0.0-1607-g94a53970-1jammy [41.4 MB] 2024-03-11T18:27:19.254 INFO:teuthology.orchestra.run.smithi090.stdout:Get:131 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon-dbg amd64 19.0.0-1607-g94a53970-1jammy [158 MB] 2024-03-11T18:27:26.351 INFO:teuthology.orchestra.run.smithi092.stdout:Get:133 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test-dbg amd64 19.0.0-1607-g94a53970-1jammy [1,178 MB] 2024-03-11T18:27:34.447 INFO:teuthology.orchestra.run.smithi090.stdout:Get:132 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd-dbg amd64 19.0.0-1607-g94a53970-1jammy [537 MB] 2024-03-11T18:28:10.262 INFO:teuthology.orchestra.run.smithi090.stdout:Get:133 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test-dbg amd64 19.0.0-1607-g94a53970-1jammy [1,178 MB] 2024-03-11T18:28:50.266 INFO:teuthology.orchestra.run.smithi092.stdout:Get:134 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2-dbg amd64 19.0.0-1607-g94a53970-1jammy [16.3 MB] 2024-03-11T18:28:51.056 INFO:teuthology.orchestra.run.smithi092.stdout:Get:135 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 librados2-dbg amd64 19.0.0-1607-g94a53970-1jammy [81.4 MB] 2024-03-11T18:28:56.341 INFO:teuthology.orchestra.run.smithi092.stdout:Get:136 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1-dbg amd64 19.0.0-1607-g94a53970-1jammy [11.2 MB] 2024-03-11T18:28:56.774 INFO:teuthology.orchestra.run.smithi092.stdout:Get:137 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 librbd1-dbg amd64 19.0.0-1607-g94a53970-1jammy [103 MB] 2024-03-11T18:29:02.905 INFO:teuthology.orchestra.run.smithi092.stdout:Get:138 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 librgw2-dbg amd64 19.0.0-1607-g94a53970-1jammy [173 MB] 2024-03-11T18:29:12.740 INFO:teuthology.orchestra.run.smithi092.stdout:Get:139 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 radosgw-dbg amd64 19.0.0-1607-g94a53970-1jammy [338 MB] 2024-03-11T18:29:32.160 INFO:teuthology.orchestra.run.smithi092.stdout:Get:140 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse-dbg amd64 19.0.0-1607-g94a53970-1jammy [1,696 kB] 2024-03-11T18:29:32.191 INFO:teuthology.orchestra.run.smithi092.stdout:Get:141 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 rbd-mirror-dbg amd64 19.0.0-1607-g94a53970-1jammy [97.1 MB] 2024-03-11T18:29:37.486 INFO:teuthology.orchestra.run.smithi092.stdout:Get:142 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 rbd-nbd-dbg amd64 19.0.0-1607-g94a53970-1jammy [3,003 kB] 2024-03-11T18:29:38.157 INFO:teuthology.orchestra.run.smithi092.stdout:Fetched 3,736 MB in 3min 58s (15.7 MB/s) 2024-03-11T18:29:38.298 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-pyperclip. 2024-03-11T18:29:38.394 INFO:teuthology.orchestra.run.smithi092.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165997 files and directories currently installed.) 2024-03-11T18:29:38.400 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-03-11T18:29:38.438 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-03-11T18:29:38.531 INFO:teuthology.orchestra.run.smithi090.stdout:Get:134 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2-dbg amd64 19.0.0-1607-g94a53970-1jammy [16.3 MB] 2024-03-11T18:29:38.749 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-wcwidth. 2024-03-11T18:29:38.775 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-03-11T18:29:38.823 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-11T18:29:39.101 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-cmd2. 2024-03-11T18:29:39.128 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-03-11T18:29:39.159 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-03-11T18:29:39.340 INFO:teuthology.orchestra.run.smithi090.stdout:Get:135 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 librados2-dbg amd64 19.0.0-1607-g94a53970-1jammy [81.4 MB] 2024-03-11T18:29:39.437 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-03-11T18:29:39.452 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-03-11T18:29:39.521 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-11T18:29:39.849 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-03-11T18:29:39.875 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-03-11T18:29:39.915 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-11T18:29:40.227 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-03-11T18:29:40.253 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-03-11T18:29:40.301 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-11T18:29:40.721 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-03-11T18:29:40.748 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-11T18:29:40.796 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-11T18:29:41.200 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-03-11T18:29:41.226 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-11T18:29:41.266 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-11T18:29:41.586 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-03-11T18:29:41.612 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-11T18:29:41.644 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-11T18:29:41.972 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-03-11T18:29:41.998 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-03-11T18:29:42.038 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-11T18:29:42.358 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package librados2. 2024-03-11T18:29:42.384 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../010-librados2_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:29:42.424 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking librados2 (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:29:42.894 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package librbd1. 2024-03-11T18:29:42.920 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../011-librbd1_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:29:42.960 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking librbd1 (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:29:43.422 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libcephfs2. 2024-03-11T18:29:43.448 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:29:43.488 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libcephfs2 (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:29:43.765 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-rados. 2024-03-11T18:29:43.792 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../013-python3-rados_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:29:43.840 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-rados (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:29:44.176 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-03-11T18:29:44.203 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-1607-g94a53970-1jammy_all.deb ... 2024-03-11T18:29:44.243 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-ceph-argparse (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:29:44.244 INFO:teuthology.orchestra.run.smithi090.stdout:Get:136 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1-dbg amd64 19.0.0-1607-g94a53970-1jammy [11.2 MB] 2024-03-11T18:29:44.495 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-cephfs. 2024-03-11T18:29:44.521 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:29:44.562 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-cephfs (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:29:44.773 INFO:teuthology.orchestra.run.smithi090.stdout:Get:137 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 librbd1-dbg amd64 19.0.0-1607-g94a53970-1jammy [103 MB] 2024-03-11T18:29:44.873 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-ceph-common. 2024-03-11T18:29:44.899 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-1607-g94a53970-1jammy_all.deb ... 2024-03-11T18:29:44.939 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-ceph-common (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:29:45.259 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-prettytable. 2024-03-11T18:29:45.285 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-03-11T18:29:45.325 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-03-11T18:29:45.595 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-rbd. 2024-03-11T18:29:45.621 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:29:45.661 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-rbd (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:29:45.964 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-03-11T18:29:45.990 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-03-11T18:29:46.022 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-11T18:29:46.300 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-03-11T18:29:46.314 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-03-11T18:29:46.358 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-03-11T18:29:46.644 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-03-11T18:29:46.659 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-03-11T18:29:46.702 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-11T18:29:47.039 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package lua5.1. 2024-03-11T18:29:47.065 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-03-11T18:29:47.105 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-03-11T18:29:47.391 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package lua-any. 2024-03-11T18:29:47.418 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-03-11T18:29:47.483 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking lua-any (27ubuntu1) ... 2024-03-11T18:29:47.727 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package zip. 2024-03-11T18:29:47.743 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-03-11T18:29:47.785 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking zip (3.0-12build2) ... 2024-03-11T18:29:48.105 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package unzip. 2024-03-11T18:29:48.131 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-03-11T18:29:48.171 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-03-11T18:29:48.567 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package luarocks. 2024-03-11T18:29:48.593 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-03-11T18:29:48.633 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-03-11T18:29:49.007 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package librgw2. 2024-03-11T18:29:49.034 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../027-librgw2_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:29:49.074 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking librgw2 (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:29:49.584 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-rgw. 2024-03-11T18:29:49.600 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:29:49.634 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-rgw (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:29:49.948 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package liboath0:amd64. 2024-03-11T18:29:49.974 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-03-11T18:29:50.014 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-03-11T18:29:50.310 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libradosstriper1. 2024-03-11T18:29:50.335 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:29:50.375 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libradosstriper1 (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:29:50.804 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-common. 2024-03-11T18:29:50.830 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../031-ceph-common_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:29:50.870 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-common (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:29:52.134 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-base. 2024-03-11T18:29:52.160 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../032-ceph-base_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:29:52.202 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-base (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:29:52.730 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-03-11T18:29:52.756 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-03-11T18:29:52.796 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-03-11T18:29:53.099 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-cheroot. 2024-03-11T18:29:53.126 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-03-11T18:29:53.166 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-11T18:29:53.362 INFO:teuthology.orchestra.run.smithi090.stdout:Get:138 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 librgw2-dbg amd64 19.0.0-1607-g94a53970-1jammy [173 MB] 2024-03-11T18:29:53.477 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-03-11T18:29:53.504 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-03-11T18:29:53.535 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-03-11T18:29:53.830 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-jaraco.text. 2024-03-11T18:29:53.857 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-03-11T18:29:53.897 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-03-11T18:29:54.200 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-03-11T18:29:54.226 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-03-11T18:29:54.266 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-03-11T18:29:54.536 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-tempora. 2024-03-11T18:29:54.562 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-03-11T18:29:54.594 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-03-11T18:29:54.906 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-portend. 2024-03-11T18:29:54.932 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-03-11T18:29:54.972 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-03-11T18:29:55.275 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-03-11T18:29:55.301 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-03-11T18:29:55.341 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-03-11T18:29:55.619 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-cherrypy3. 2024-03-11T18:29:55.646 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-03-11T18:29:55.686 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-03-11T18:29:56.032 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-natsort. 2024-03-11T18:29:56.058 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-03-11T18:29:56.098 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-03-11T18:29:56.434 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-logutils. 2024-03-11T18:29:56.461 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-03-11T18:29:56.501 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-03-11T18:29:56.745 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-markupsafe. 2024-03-11T18:29:56.772 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-03-11T18:29:56.820 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-03-11T18:29:57.107 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-mako. 2024-03-11T18:29:57.133 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-03-11T18:29:57.173 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-11T18:29:57.468 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-simplegeneric. 2024-03-11T18:29:57.483 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-03-11T18:29:57.518 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-03-11T18:29:57.796 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-singledispatch. 2024-03-11T18:29:57.822 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-03-11T18:29:57.862 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-03-11T18:29:58.157 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-webob. 2024-03-11T18:29:58.183 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-03-11T18:29:58.223 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-03-11T18:29:58.585 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-waitress. 2024-03-11T18:29:58.613 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-03-11T18:29:58.655 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-11T18:29:58.963 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-tempita. 2024-03-11T18:29:58.989 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-03-11T18:29:59.029 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-03-11T18:29:59.291 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-paste. 2024-03-11T18:29:59.317 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-03-11T18:29:59.357 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-03-11T18:29:59.652 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-03-11T18:29:59.667 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-03-11T18:29:59.711 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-03-11T18:30:00.013 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-pastedeploy. 2024-03-11T18:30:00.041 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-03-11T18:30:00.080 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-03-11T18:30:00.350 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-webtest. 2024-03-11T18:30:00.377 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-03-11T18:30:00.408 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-03-11T18:30:00.719 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-pecan. 2024-03-11T18:30:00.745 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-03-11T18:30:00.777 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-03-11T18:30:01.116 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-certifi. 2024-03-11T18:30:01.142 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-03-11T18:30:01.182 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-03-11T18:30:01.493 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-idna. 2024-03-11T18:30:01.520 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-03-11T18:30:01.560 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-idna (3.3-1) ... 2024-03-11T18:30:01.855 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-urllib3. 2024-03-11T18:30:01.881 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-03-11T18:30:01.921 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-11T18:30:02.200 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-requests. 2024-03-11T18:30:02.226 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-03-11T18:30:02.266 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-11T18:30:02.527 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-werkzeug. 2024-03-11T18:30:02.543 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-03-11T18:30:02.577 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-11T18:30:03.130 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-03-11T18:30:03.157 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-1607-g94a53970-1jammy_all.deb ... 2024-03-11T18:30:03.188 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-mgr-modules-core (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:30:03.569 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-03-11T18:30:03.596 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:30:03.635 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:30:03.963 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-bcrypt. 2024-03-11T18:30:03.990 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-03-11T18:30:04.029 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-03-11T18:30:04.299 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-openssl. 2024-03-11T18:30:04.326 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-03-11T18:30:04.357 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-03-11T18:30:04.694 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-mgr. 2024-03-11T18:30:04.721 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:30:04.760 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-mgr (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:30:05.189 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-mon. 2024-03-11T18:30:05.216 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:30:05.255 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-mon (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:30:05.842 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-osd. 2024-03-11T18:30:05.868 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:30:05.908 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-osd (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:30:07.036 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph. 2024-03-11T18:30:07.063 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../068-ceph_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:30:07.103 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:30:07.381 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-fuse. 2024-03-11T18:30:07.407 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:30:07.447 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-fuse (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:30:07.775 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-mds. 2024-03-11T18:30:07.802 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:30:07.841 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-mds (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:30:08.211 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package cephadm. 2024-03-11T18:30:08.238 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../071-cephadm_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:30:08.277 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking cephadm (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:30:08.606 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-asyncssh. 2024-03-11T18:30:08.632 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-03-11T18:30:08.672 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-03-11T18:30:08.958 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python-babel-localedata. 2024-03-11T18:30:08.974 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-03-11T18:30:09.016 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-11T18:30:10.080 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-babel. 2024-03-11T18:30:10.107 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-03-11T18:30:10.146 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-03-11T18:30:10.466 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-jinja2. 2024-03-11T18:30:10.493 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-03-11T18:30:10.532 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-11T18:30:10.885 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-03-11T18:30:10.913 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-1607-g94a53970-1jammy_all.deb ... 2024-03-11T18:30:10.943 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-mgr-cephadm (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:30:11.289 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-repoze.lru. 2024-03-11T18:30:11.316 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-03-11T18:30:11.347 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-03-11T18:30:11.625 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-routes. 2024-03-11T18:30:11.652 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-03-11T18:30:11.691 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-03-11T18:30:12.003 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-03-11T18:30:12.029 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-1607-g94a53970-1jammy_all.deb ... 2024-03-11T18:30:12.069 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-mgr-dashboard (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:30:12.506 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-03-11T18:30:12.533 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-03-11T18:30:12.572 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-11T18:30:13.076 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-joblib. 2024-03-11T18:30:13.103 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-03-11T18:30:13.142 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-03-11T18:30:13.504 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-03-11T18:30:13.531 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-03-11T18:30:13.690 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-03-11T18:30:13.993 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-sklearn. 2024-03-11T18:30:14.020 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-03-11T18:30:14.059 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-11T18:30:14.727 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-03-11T18:30:14.754 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-1607-g94a53970-1jammy_all.deb ... 2024-03-11T18:30:14.793 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:30:15.755 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-cachetools. 2024-03-11T18:30:15.782 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-03-11T18:30:15.821 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-03-11T18:30:16.125 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-pyasn1. 2024-03-11T18:30:16.152 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-03-11T18:30:16.191 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-03-11T18:30:16.528 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-03-11T18:30:16.555 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-03-11T18:30:16.594 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-03-11T18:30:16.906 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-rsa. 2024-03-11T18:30:16.933 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-03-11T18:30:16.972 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-rsa (4.8-1) ... 2024-03-11T18:30:17.259 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-google-auth. 2024-03-11T18:30:17.275 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-03-11T18:30:17.309 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-03-11T18:30:17.417 INFO:teuthology.orchestra.run.smithi090.stdout:Get:139 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 radosgw-dbg amd64 19.0.0-1607-g94a53970-1jammy [338 MB] 2024-03-11T18:30:17.604 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-03-11T18:30:17.631 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-03-11T18:30:17.671 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-11T18:30:18.024 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-websocket. 2024-03-11T18:30:18.051 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-03-11T18:30:18.090 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-03-11T18:30:18.402 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-kubernetes. 2024-03-11T18:30:18.429 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-03-11T18:30:18.468 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-11T18:30:19.166 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-03-11T18:30:19.182 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-1607-g94a53970-1jammy_all.deb ... 2024-03-11T18:30:19.216 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:30:19.535 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libonig5:amd64. 2024-03-11T18:30:19.562 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-03-11T18:30:19.601 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-11T18:30:19.947 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libjq1:amd64. 2024-03-11T18:30:19.974 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-03-11T18:30:20.013 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-11T18:30:20.266 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package jq. 2024-03-11T18:30:20.281 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-03-11T18:30:20.324 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-03-11T18:30:20.552 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package socat. 2024-03-11T18:30:20.567 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-03-11T18:30:20.610 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-03-11T18:30:20.947 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package xmlstarlet. 2024-03-11T18:30:20.962 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-03-11T18:30:21.005 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-03-11T18:30:21.343 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-test. 2024-03-11T18:30:21.370 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../099-ceph-test_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:30:21.409 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-test (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:30:23.563 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-volume. 2024-03-11T18:30:23.590 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-1607-g94a53970-1jammy_all.deb ... 2024-03-11T18:30:23.630 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-volume (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:30:24.000 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package cephfs-mirror. 2024-03-11T18:30:24.028 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:30:24.067 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking cephfs-mirror (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:30:24.378 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-colorama. 2024-03-11T18:30:24.405 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-03-11T18:30:24.444 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-03-11T18:30:24.689 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package cephfs-shell. 2024-03-11T18:30:24.716 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-1607-g94a53970-1jammy_all.deb ... 2024-03-11T18:30:24.747 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking cephfs-shell (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:30:24.991 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package cephfs-top. 2024-03-11T18:30:25.007 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-1607-g94a53970-1jammy_all.deb ... 2024-03-11T18:30:25.049 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking cephfs-top (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:30:25.302 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libcephfs-dev. 2024-03-11T18:30:25.318 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:30:25.360 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libcephfs-dev (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:30:25.688 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package lua-socket:amd64. 2024-03-11T18:30:25.716 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-03-11T18:30:25.755 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-11T18:30:26.118 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package lua-sec:amd64. 2024-03-11T18:30:26.145 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-03-11T18:30:26.184 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-03-11T18:30:26.513 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package nvme-cli. 2024-03-11T18:30:26.541 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-03-11T18:30:26.571 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-03-11T18:30:26.977 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package pkg-config. 2024-03-11T18:30:27.004 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-03-11T18:30:27.043 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-03-11T18:30:27.313 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-03-11T18:30:27.340 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-03-11T18:30:27.379 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-03-11T18:30:27.824 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-iniconfig. 2024-03-11T18:30:27.852 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-03-11T18:30:27.890 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-03-11T18:30:28.202 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-pastescript. 2024-03-11T18:30:28.229 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-03-11T18:30:28.268 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-03-11T18:30:28.605 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-pluggy. 2024-03-11T18:30:28.633 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-03-11T18:30:28.672 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-03-11T18:30:28.933 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-psutil. 2024-03-11T18:30:28.961 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-03-11T18:30:28.999 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-03-11T18:30:29.302 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-py. 2024-03-11T18:30:29.318 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-03-11T18:30:29.360 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-py (1.10.0-1) ... 2024-03-11T18:30:29.641 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-pygments. 2024-03-11T18:30:29.657 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-03-11T18:30:29.690 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-03-11T18:30:30.145 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-pyinotify. 2024-03-11T18:30:30.172 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-03-11T18:30:30.211 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-03-11T18:30:30.522 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-toml. 2024-03-11T18:30:30.549 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-03-11T18:30:30.590 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-03-11T18:30:30.891 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-pytest. 2024-03-11T18:30:30.919 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-03-11T18:30:30.986 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-03-11T18:30:31.278 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-simplejson. 2024-03-11T18:30:31.294 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-03-11T18:30:31.336 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-03-11T18:30:31.580 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-03-11T18:30:31.610 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-03-11T18:30:31.647 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-03-11T18:30:32.293 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package radosgw. 2024-03-11T18:30:32.320 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../122-radosgw_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:30:32.358 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking radosgw (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:30:33.145 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package rbd-fuse. 2024-03-11T18:30:33.161 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:30:33.203 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking rbd-fuse (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:30:33.548 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package rbd-mirror. 2024-03-11T18:30:33.576 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../124-rbd-mirror_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:30:33.614 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking rbd-mirror (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:30:34.350 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package rbd-nbd. 2024-03-11T18:30:34.374 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../125-rbd-nbd_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:30:34.416 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking rbd-nbd (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:30:34.653 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-common-dbg. 2024-03-11T18:30:34.680 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../126-ceph-common-dbg_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:30:34.719 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-common-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:30:41.806 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-fuse-dbg. 2024-03-11T18:30:41.821 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../127-ceph-fuse-dbg_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:30:41.864 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-fuse-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:30:42.283 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-mds-dbg. 2024-03-11T18:30:42.299 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../128-ceph-mds-dbg_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:30:42.341 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-mds-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:30:43.343 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-mgr-dbg. 2024-03-11T18:30:43.359 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../129-ceph-mgr-dbg_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:30:43.401 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-mgr-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:30:44.096 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-mon-dbg. 2024-03-11T18:30:44.123 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../130-ceph-mon-dbg_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:30:44.154 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-mon-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:30:44.686 INFO:teuthology.orchestra.run.smithi090.stdout:Get:140 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse-dbg amd64 19.0.0-1607-g94a53970-1jammy [1,696 kB] 2024-03-11T18:30:44.718 INFO:teuthology.orchestra.run.smithi090.stdout:Get:141 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 rbd-mirror-dbg amd64 19.0.0-1607-g94a53970-1jammy [97.1 MB] 2024-03-11T18:30:45.898 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-osd-dbg. 2024-03-11T18:30:45.914 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../131-ceph-osd-dbg_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:30:45.956 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-osd-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:30:48.971 INFO:teuthology.orchestra.run.smithi090.stdout:Get:142 https://1.chacra.ceph.com/r/ceph/squid/94a539707733325322964c6a187c6a1b41be1daf/ubuntu/jammy/flavors/default jammy/main amd64 rbd-nbd-dbg amd64 19.0.0-1607-g94a53970-1jammy [3,003 kB] 2024-03-11T18:30:49.636 INFO:teuthology.orchestra.run.smithi090.stdout:Fetched 3,736 MB in 5min 10s (12.1 MB/s) 2024-03-11T18:30:49.832 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-pyperclip. 2024-03-11T18:30:49.932 INFO:teuthology.orchestra.run.smithi090.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165997 files and directories currently installed.) 2024-03-11T18:30:49.937 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-03-11T18:30:49.980 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-03-11T18:30:50.258 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-wcwidth. 2024-03-11T18:30:50.284 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-03-11T18:30:50.324 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-11T18:30:50.646 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-cmd2. 2024-03-11T18:30:50.672 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-03-11T18:30:50.712 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-03-11T18:30:51.041 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-03-11T18:30:51.067 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-03-11T18:30:51.149 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-11T18:30:51.174 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-test-dbg. 2024-03-11T18:30:51.190 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../132-ceph-test-dbg_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:30:51.232 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-test-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:30:51.471 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-03-11T18:30:51.498 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-03-11T18:30:51.538 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-11T18:30:51.875 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-03-11T18:30:51.901 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-03-11T18:30:51.942 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-11T18:30:52.380 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-03-11T18:30:52.406 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-11T18:30:52.455 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-11T18:30:52.867 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-03-11T18:30:52.893 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-11T18:30:52.934 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-11T18:30:53.289 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-03-11T18:30:53.315 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-11T18:30:53.347 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-11T18:30:53.667 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-03-11T18:30:53.691 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-03-11T18:30:53.725 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-11T18:30:54.096 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package librados2. 2024-03-11T18:30:54.122 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../010-librados2_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:30:54.162 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking librados2 (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:30:54.617 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package librbd1. 2024-03-11T18:30:54.643 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../011-librbd1_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:30:54.683 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking librbd1 (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:30:55.120 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package libcephfs2. 2024-03-11T18:30:55.146 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:30:55.186 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking libcephfs2 (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:30:55.473 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-rados. 2024-03-11T18:30:55.499 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../013-python3-rados_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:30:55.540 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-rados (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:30:55.819 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-03-11T18:30:55.845 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-1607-g94a53970-1jammy_all.deb ... 2024-03-11T18:30:55.885 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-ceph-argparse (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:30:56.113 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-cephfs. 2024-03-11T18:30:56.140 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:30:56.180 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-cephfs (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:30:56.501 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-ceph-common. 2024-03-11T18:30:56.528 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-1607-g94a53970-1jammy_all.deb ... 2024-03-11T18:30:56.567 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-ceph-common (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:30:56.896 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-prettytable. 2024-03-11T18:30:56.922 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-03-11T18:30:56.954 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-03-11T18:30:57.191 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-rbd. 2024-03-11T18:30:57.217 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:30:57.257 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-rbd (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:30:57.621 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-03-11T18:30:57.647 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-03-11T18:30:57.687 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-11T18:30:57.982 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-03-11T18:30:57.998 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-03-11T18:30:58.032 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-03-11T18:30:58.345 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-03-11T18:30:58.360 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-03-11T18:30:58.403 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-11T18:30:58.740 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package lua5.1. 2024-03-11T18:30:58.766 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-03-11T18:30:58.807 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-03-11T18:30:59.162 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package lua-any. 2024-03-11T18:30:59.188 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-03-11T18:30:59.227 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking lua-any (27ubuntu1) ... 2024-03-11T18:30:59.473 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package zip. 2024-03-11T18:30:59.499 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-03-11T18:30:59.540 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking zip (3.0-12build2) ... 2024-03-11T18:30:59.826 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package unzip. 2024-03-11T18:30:59.842 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-03-11T18:30:59.876 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-03-11T18:31:00.274 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package luarocks. 2024-03-11T18:31:00.300 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-03-11T18:31:00.341 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-03-11T18:31:00.692 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package librgw2. 2024-03-11T18:31:00.719 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../027-librgw2_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:31:00.759 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking librgw2 (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:01.345 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-rgw. 2024-03-11T18:31:01.360 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:31:01.403 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-rgw (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:01.750 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package liboath0:amd64. 2024-03-11T18:31:01.776 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-03-11T18:31:01.816 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-03-11T18:31:02.153 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package libradosstriper1. 2024-03-11T18:31:02.179 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:31:02.228 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking libradosstriper1 (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:02.456 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libcephfs2-dbg. 2024-03-11T18:31:02.473 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../133-libcephfs2-dbg_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:31:02.514 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libcephfs2-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:02.690 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package ceph-common. 2024-03-11T18:31:02.717 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../031-ceph-common_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:31:02.765 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking ceph-common (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:02.950 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package librados2-dbg. 2024-03-11T18:31:02.976 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../134-librados2-dbg_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:31:03.017 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking librados2-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:04.036 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libradosstriper1-dbg. 2024-03-11T18:31:04.052 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../135-libradosstriper1-dbg_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:31:04.086 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libradosstriper1-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:04.223 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package ceph-base. 2024-03-11T18:31:04.249 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../032-ceph-base_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:31:04.332 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking ceph-base (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:04.463 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package librbd1-dbg. 2024-03-11T18:31:04.491 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../136-librbd1-dbg_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:31:04.530 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking librbd1-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:04.962 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-03-11T18:31:04.989 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-03-11T18:31:05.029 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-03-11T18:31:05.316 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-cheroot. 2024-03-11T18:31:05.342 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-03-11T18:31:05.382 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-11T18:31:05.712 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-03-11T18:31:05.739 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-03-11T18:31:05.787 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-03-11T18:31:05.966 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package librgw2-dbg. 2024-03-11T18:31:05.994 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../137-librgw2-dbg_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:31:06.032 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking librgw2-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:06.074 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-jaraco.text. 2024-03-11T18:31:06.100 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-03-11T18:31:06.140 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-03-11T18:31:06.435 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-03-11T18:31:06.462 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-03-11T18:31:06.502 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-03-11T18:31:06.781 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-tempora. 2024-03-11T18:31:06.808 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-03-11T18:31:06.848 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-03-11T18:31:07.143 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-portend. 2024-03-11T18:31:07.170 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-03-11T18:31:07.209 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-03-11T18:31:07.506 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-03-11T18:31:07.532 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-03-11T18:31:07.581 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-03-11T18:31:07.860 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package radosgw-dbg. 2024-03-11T18:31:07.867 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-cherrypy3. 2024-03-11T18:31:07.888 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../138-radosgw-dbg_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:31:07.894 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-03-11T18:31:07.926 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking radosgw-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:07.934 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-03-11T18:31:08.263 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-natsort. 2024-03-11T18:31:08.290 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-03-11T18:31:08.329 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-03-11T18:31:08.601 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-logutils. 2024-03-11T18:31:08.617 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-03-11T18:31:08.651 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-03-11T18:31:08.971 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-markupsafe. 2024-03-11T18:31:08.997 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-03-11T18:31:09.038 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-03-11T18:31:09.334 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-mako. 2024-03-11T18:31:09.360 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-03-11T18:31:09.426 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-11T18:31:09.738 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-simplegeneric. 2024-03-11T18:31:09.764 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-03-11T18:31:09.813 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-03-11T18:31:10.158 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-singledispatch. 2024-03-11T18:31:10.184 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-03-11T18:31:10.234 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-03-11T18:31:10.595 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-webob. 2024-03-11T18:31:10.622 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-03-11T18:31:10.670 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-03-11T18:31:11.057 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-waitress. 2024-03-11T18:31:11.084 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-03-11T18:31:11.143 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-11T18:31:11.237 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package rbd-fuse-dbg. 2024-03-11T18:31:11.264 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../139-rbd-fuse-dbg_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:31:11.303 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking rbd-fuse-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:11.504 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-tempita. 2024-03-11T18:31:11.530 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-03-11T18:31:11.578 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-03-11T18:31:11.580 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package rbd-mirror-dbg. 2024-03-11T18:31:11.596 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../140-rbd-mirror-dbg_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:31:11.639 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking rbd-mirror-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:11.940 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-paste. 2024-03-11T18:31:11.967 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-03-11T18:31:12.015 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-03-11T18:31:12.345 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-03-11T18:31:12.372 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-03-11T18:31:12.411 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-03-11T18:31:12.673 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-pastedeploy. 2024-03-11T18:31:12.700 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-03-11T18:31:12.740 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-03-11T18:31:12.858 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package rbd-nbd-dbg. 2024-03-11T18:31:12.886 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../141-rbd-nbd-dbg_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:31:12.924 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking rbd-nbd-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:13.052 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-webtest. 2024-03-11T18:31:13.078 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-03-11T18:31:13.118 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-03-11T18:31:13.238 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-03-11T18:31:13.423 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-pecan. 2024-03-11T18:31:13.441 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-03-11T18:31:13.481 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-03-11T18:31:13.496 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-11T18:31:13.614 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-03-11T18:31:13.854 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-certifi. 2024-03-11T18:31:13.878 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-03-11T18:31:13.880 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-03-11T18:31:13.920 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-03-11T18:31:14.185 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-03-11T18:31:14.233 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-idna. 2024-03-11T18:31:14.259 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-03-11T18:31:14.299 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-idna (3.3-1) ... 2024-03-11T18:31:14.594 INFO:teuthology.orchestra.run.smithi092.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-03-11T18:31:14.611 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-urllib3. 2024-03-11T18:31:14.638 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-03-11T18:31:14.666 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up cephadm (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:14.677 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-11T18:31:14.991 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-requests. 2024-03-11T18:31:15.017 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-03-11T18:31:15.057 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-11T18:31:15.209 INFO:teuthology.orchestra.run.smithi092.stdout:Adding system user cephadm....done 2024-03-11T18:31:15.344 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-werkzeug. 2024-03-11T18:31:15.355 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-11T18:31:15.371 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-03-11T18:31:15.411 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-11T18:31:15.624 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-03-11T18:31:15.723 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-03-11T18:31:15.747 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-1607-g94a53970-1jammy_all.deb ... 2024-03-11T18:31:15.789 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking ceph-mgr-modules-core (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:15.875 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-03-11T18:31:16.001 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-03-11T18:31:16.238 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-03-11T18:31:16.260 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-03-11T18:31:16.264 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:31:16.313 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:16.545 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-03-11T18:31:16.634 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-bcrypt. 2024-03-11T18:31:16.660 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-03-11T18:31:16.662 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-py (1.10.0-1) ... 2024-03-11T18:31:16.700 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-03-11T18:31:16.989 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-03-11T18:31:16.997 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-openssl. 2024-03-11T18:31:17.022 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-03-11T18:31:17.062 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-03-11T18:31:17.357 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-03-11T18:31:17.407 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package ceph-mgr. 2024-03-11T18:31:17.434 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:31:17.474 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking ceph-mgr (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:17.608 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-03-11T18:31:17.859 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-03-11T18:31:17.862 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package ceph-mon. 2024-03-11T18:31:17.889 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:31:17.929 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking ceph-mon (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:18.127 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-03-11T18:31:18.454 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-03-11T18:31:18.583 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package ceph-osd. 2024-03-11T18:31:18.609 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:31:18.649 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking ceph-osd (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:18.705 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-03-11T18:31:18.973 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-03-11T18:31:19.224 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-ceph-argparse (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:19.483 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-11T18:31:19.610 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-11T18:31:19.736 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-03-11T18:31:19.771 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package ceph. 2024-03-11T18:31:19.797 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../068-ceph_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:31:19.845 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking ceph (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:19.995 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-03-11T18:31:20.121 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-03-11T18:31:20.190 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package ceph-fuse. 2024-03-11T18:31:20.217 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:31:20.257 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking ceph-fuse (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:20.381 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-03-11T18:31:20.901 INFO:teuthology.orchestra.run.smithi092.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-03-11T18:31:20.937 INFO:teuthology.orchestra.run.smithi092.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-03-11T18:31:20.992 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-11T18:31:21.002 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package ceph-mds. 2024-03-11T18:31:21.029 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:31:21.069 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking ceph-mds (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:21.118 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-03-11T18:31:21.474 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package cephadm. 2024-03-11T18:31:21.486 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-11T18:31:21.500 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../071-cephadm_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:31:21.540 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking cephadm (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:21.612 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-03-11T18:31:21.860 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-asyncssh. 2024-03-11T18:31:21.879 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-03-11T18:31:21.887 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-03-11T18:31:21.926 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-03-11T18:31:22.139 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-03-11T18:31:22.264 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python-babel-localedata. 2024-03-11T18:31:22.279 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-03-11T18:31:22.322 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-11T18:31:22.449 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up zip (3.0-12build2) ... 2024-03-11T18:31:22.575 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-03-11T18:31:23.268 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-03-11T18:31:23.520 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-03-11T18:31:23.621 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-03-11T18:31:23.654 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-babel. 2024-03-11T18:31:23.680 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-03-11T18:31:23.720 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-03-11T18:31:23.747 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-11T18:31:24.025 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-jinja2. 2024-03-11T18:31:24.048 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-03-11T18:31:24.051 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-03-11T18:31:24.091 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-11T18:31:24.441 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-03-11T18:31:24.444 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-03-11T18:31:24.471 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-1607-g94a53970-1jammy_all.deb ... 2024-03-11T18:31:24.511 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking ceph-mgr-cephadm (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:24.692 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-03-11T18:31:24.891 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-repoze.lru. 2024-03-11T18:31:24.918 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-03-11T18:31:24.958 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-03-11T18:31:25.068 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-11T18:31:25.261 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-routes. 2024-03-11T18:31:25.288 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-03-11T18:31:25.327 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-03-11T18:31:25.361 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-03-11T18:31:25.662 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-11T18:31:25.673 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-03-11T18:31:25.700 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-1607-g94a53970-1jammy_all.deb ... 2024-03-11T18:31:25.748 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking ceph-mgr-dashboard (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:26.005 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-03-11T18:31:26.211 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-03-11T18:31:26.238 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-03-11T18:31:26.256 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-03-11T18:31:26.278 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-11T18:31:26.383 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-ceph-common (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:26.684 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-idna (3.3-1) ... 2024-03-11T18:31:26.799 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-joblib. 2024-03-11T18:31:26.825 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-03-11T18:31:26.865 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-03-11T18:31:27.001 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-11T18:31:27.220 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-03-11T18:31:27.247 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-03-11T18:31:27.381 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-03-11T18:31:27.676 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-sklearn. 2024-03-11T18:31:27.703 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-03-11T18:31:27.742 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-11T18:31:28.183 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-03-11T18:31:28.451 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-11T18:31:28.585 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-toml (0.10.2-1) ... 2024-03-11T18:31:28.614 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-03-11T18:31:28.642 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-1607-g94a53970-1jammy_all.deb ... 2024-03-11T18:31:28.689 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:28.845 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-11T18:31:28.954 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-03-11T18:31:29.063 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-11T18:31:29.365 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-03-11T18:31:29.633 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-03-11T18:31:29.701 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-cachetools. 2024-03-11T18:31:29.728 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-03-11T18:31:29.767 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-03-11T18:31:29.892 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-11T18:31:30.018 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-03-11T18:31:30.130 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-pyasn1. 2024-03-11T18:31:30.144 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-03-11T18:31:30.197 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-03-11T18:31:30.328 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-03-11T18:31:30.568 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-03-11T18:31:30.587 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-03-11T18:31:30.595 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-03-11T18:31:30.643 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-03-11T18:31:30.855 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-03-11T18:31:30.979 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-rsa. 2024-03-11T18:31:31.007 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-03-11T18:31:31.054 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-rsa (4.8-1) ... 2024-03-11T18:31:31.115 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-03-11T18:31:31.374 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-03-11T18:31:31.418 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-google-auth. 2024-03-11T18:31:31.445 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-03-11T18:31:31.493 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-03-11T18:31:31.642 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-11T18:31:31.768 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-03-11T18:31:31.864 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-03-11T18:31:31.891 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-03-11T18:31:31.938 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-11T18:31:32.052 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-11T18:31:32.178 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-11T18:31:32.268 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-websocket. 2024-03-11T18:31:32.295 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-03-11T18:31:32.343 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-03-11T18:31:32.454 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-03-11T18:31:32.706 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-kubernetes. 2024-03-11T18:31:32.732 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-03-11T18:31:32.780 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-11T18:31:32.781 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-03-11T18:31:33.032 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-11T18:31:33.158 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-03-11T18:31:33.284 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-03-11T18:31:33.374 INFO:teuthology.orchestra.run.smithi092.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-03-11T18:31:33.530 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-03-11T18:31:33.558 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-1607-g94a53970-1jammy_all.deb ... 2024-03-11T18:31:33.596 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:33.610 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-11T18:31:33.720 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-03-11T18:31:33.925 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package libonig5:amd64. 2024-03-11T18:31:33.952 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-03-11T18:31:33.999 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-11T18:31:34.113 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-03-11T18:31:34.329 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package libjq1:amd64. 2024-03-11T18:31:34.357 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-03-11T18:31:34.364 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up lua-any (27ubuntu1) ... 2024-03-11T18:31:34.396 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-11T18:31:34.499 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-portend (3.0.0-1) ... 2024-03-11T18:31:34.674 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package jq. 2024-03-11T18:31:34.689 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-03-11T18:31:34.732 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-03-11T18:31:34.758 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-11T18:31:34.884 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-11T18:31:35.021 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package socat. 2024-03-11T18:31:35.047 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-03-11T18:31:35.086 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-03-11T18:31:35.202 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-03-11T18:31:35.415 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package xmlstarlet. 2024-03-11T18:31:35.439 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-03-11T18:31:35.482 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-03-11T18:31:35.495 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-11T18:31:35.746 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-03-11T18:31:35.846 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package ceph-test. 2024-03-11T18:31:35.862 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../099-ceph-test_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:31:35.904 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking ceph-test (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:35.966 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-rsa (4.8-1) ... 2024-03-11T18:31:36.267 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-03-11T18:31:36.761 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-11T18:31:37.161 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-03-11T18:31:37.554 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-03-11T18:31:37.839 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-03-11T18:31:38.167 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-11T18:31:38.293 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up librados2 (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:38.401 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package ceph-volume. 2024-03-11T18:31:38.418 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libsqlite3-mod-ceph (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:38.428 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-1607-g94a53970-1jammy_all.deb ... 2024-03-11T18:31:38.468 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking ceph-volume (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:38.536 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-03-11T18:31:38.849 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package cephfs-mirror. 2024-03-11T18:31:38.856 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libcephfs2 (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:38.876 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:31:38.915 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking cephfs-mirror (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:38.982 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libradosstriper1 (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:39.108 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-03-11T18:31:39.244 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-colorama. 2024-03-11T18:31:39.271 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-03-11T18:31:39.318 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-03-11T18:31:39.384 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up librados2-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:39.485 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up librbd1 (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:39.571 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package cephfs-shell. 2024-03-11T18:31:39.587 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-1607-g94a53970-1jammy_all.deb ... 2024-03-11T18:31:39.603 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libcephfs2-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:39.621 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking cephfs-shell (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:39.729 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-mgr-modules-core (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:39.864 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-fuse (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:39.884 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package cephfs-top. 2024-03-11T18:31:39.900 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-1607-g94a53970-1jammy_all.deb ... 2024-03-11T18:31:39.942 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking cephfs-top (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:40.094 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-11T18:31:40.094 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-11T18:31:40.220 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package libcephfs-dev. 2024-03-11T18:31:40.236 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:31:40.278 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking libcephfs-dev (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:40.548 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libradosstriper1-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:40.615 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package lua-socket:amd64. 2024-03-11T18:31:40.643 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-03-11T18:31:40.682 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-11T18:31:40.691 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libcephfs-dev (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:40.817 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-fuse-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:40.943 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-rados (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:41.038 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package lua-sec:amd64. 2024-03-11T18:31:41.044 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up librgw2 (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:41.066 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-03-11T18:31:41.104 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-03-11T18:31:41.161 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up cephfs-top (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:41.288 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-11T18:31:41.444 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package nvme-cli. 2024-03-11T18:31:41.472 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-03-11T18:31:41.510 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-03-11T18:31:41.891 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package pkg-config. 2024-03-11T18:31:41.919 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-03-11T18:31:41.957 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-03-11T18:31:42.220 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-03-11T18:31:42.238 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-03-11T18:31:42.278 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-03-11T18:31:42.416 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up librbd1-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:42.542 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-rbd (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:42.660 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up librgw2-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:42.761 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up rbd-fuse (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:42.773 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-iniconfig. 2024-03-11T18:31:42.801 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-03-11T18:31:42.840 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-03-11T18:31:42.887 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up rbd-nbd (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:43.022 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-rgw (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:43.135 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-pastescript. 2024-03-11T18:31:43.147 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-cephfs (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:43.162 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-03-11T18:31:43.201 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-03-11T18:31:43.273 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up rbd-nbd-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:43.399 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up cephfs-shell (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:43.506 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-pluggy. 2024-03-11T18:31:43.525 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up rbd-fuse-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:43.533 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-03-11T18:31:43.581 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-03-11T18:31:43.651 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-common (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:43.851 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-psutil. 2024-03-11T18:31:43.878 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-03-11T18:31:43.917 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-03-11T18:31:44.161 INFO:teuthology.orchestra.run.smithi092.stdout:Adding group ceph....done 2024-03-11T18:31:44.255 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-py. 2024-03-11T18:31:44.283 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-03-11T18:31:44.321 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-py (1.10.0-1) ... 2024-03-11T18:31:44.513 INFO:teuthology.orchestra.run.smithi092.stdout:Adding system user ceph....done 2024-03-11T18:31:44.650 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-pygments. 2024-03-11T18:31:44.678 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-03-11T18:31:44.681 INFO:teuthology.orchestra.run.smithi092.stdout:Setting system user ceph properties....done 2024-03-11T18:31:44.704 INFO:teuthology.orchestra.run.smithi092.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-03-11T18:31:44.725 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-03-11T18:31:44.842 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-03-11T18:31:45.169 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-03-11T18:31:45.251 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-pyinotify. 2024-03-11T18:31:45.279 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-03-11T18:31:45.322 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-03-11T18:31:45.603 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up cephfs-mirror (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:45.617 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-toml. 2024-03-11T18:31:45.633 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-03-11T18:31:45.675 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-03-11T18:31:45.808 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-03-11T18:31:45.808 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-03-11T18:31:45.970 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-pytest. 2024-03-11T18:31:45.998 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-03-11T18:31:46.036 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-03-11T18:31:46.290 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-test (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:46.350 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package python3-simplejson. 2024-03-11T18:31:46.378 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-03-11T18:31:46.415 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up radosgw (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:46.416 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-03-11T18:31:46.677 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-03-11T18:31:46.694 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-03-11T18:31:46.727 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-03-11T18:31:46.866 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-11T18:31:46.867 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-11T18:31:47.343 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-base (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:47.567 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package radosgw. 2024-03-11T18:31:47.594 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../122-radosgw_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:31:47.633 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking radosgw (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:47.668 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-03-11T18:31:48.111 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-mds (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:48.344 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-11T18:31:48.344 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-11T18:31:48.436 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package rbd-fuse. 2024-03-11T18:31:48.452 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:31:48.494 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking rbd-fuse (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:48.781 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package rbd-mirror. 2024-03-11T18:31:48.797 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../124-rbd-mirror_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:31:48.813 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-common-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:48.831 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking rbd-mirror (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:48.939 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up rbd-mirror (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:49.150 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target -> /lib/systemd/system/ceph-rbd-mirror.target. 2024-03-11T18:31:49.150 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target -> /lib/systemd/system/ceph-rbd-mirror.target. 2024-03-11T18:31:49.277 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package rbd-nbd. 2024-03-11T18:31:49.293 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../125-rbd-nbd_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:31:49.335 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking rbd-nbd (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:49.604 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package ceph-common-dbg. 2024-03-11T18:31:49.621 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../126-ceph-common-dbg_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:31:49.650 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-mgr (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:49.662 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking ceph-common-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:49.868 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-11T18:31:49.868 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-11T18:31:50.317 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-mds-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:50.443 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up radosgw-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:50.543 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-test-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:50.769 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-osd (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:51.066 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-11T18:31:51.066 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-11T18:31:51.538 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-mgr-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:51.672 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-osd-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:51.798 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up rbd-mirror-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:51.925 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-mgr-k8sevents (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:52.050 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:52.211 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-mon (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:52.421 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-11T18:31:52.421 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-11T18:31:52.870 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-mgr-cephadm (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:53.021 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:53.156 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-mgr-dashboard (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:53.307 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-mon-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:53.433 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-volume (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:53.777 INFO:teuthology.orchestra.run.smithi092.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-03-11T18:31:53.886 INFO:teuthology.orchestra.run.smithi092.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-11T18:31:54.022 INFO:teuthology.orchestra.run.smithi092.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-11T18:31:55.131 INFO:teuthology.orchestra.run.smithi092.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-03-11T18:31:55.790 INFO:teuthology.orchestra.run.smithi092.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-11T18:31:55.793 DEBUG:teuthology.orchestra.run.smithi092:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-03-11T18:31:55.898 INFO:teuthology.orchestra.run.smithi092.stdout:Reading package lists... 2024-03-11T18:31:56.101 INFO:teuthology.orchestra.run.smithi092.stdout:Building dependency tree... 2024-03-11T18:31:56.102 INFO:teuthology.orchestra.run.smithi092.stdout:Reading state information... 2024-03-11T18:31:56.280 INFO:teuthology.orchestra.run.smithi092.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-03-11T18:31:56.280 INFO:teuthology.orchestra.run.smithi092.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-03-11T18:31:56.280 INFO:teuthology.orchestra.run.smithi092.stdout:util-linux is already the newest version (2.37.2-4ubuntu3). 2024-03-11T18:31:56.280 INFO:teuthology.orchestra.run.smithi092.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3). 2024-03-11T18:31:56.280 INFO:teuthology.orchestra.run.smithi092.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-03-11T18:31:56.280 INFO:teuthology.orchestra.run.smithi092.stdout:The following packages were automatically installed and are no longer required: 2024-03-11T18:31:56.280 INFO:teuthology.orchestra.run.smithi092.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-11T18:31:56.280 INFO:teuthology.orchestra.run.smithi092.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-11T18:31:56.280 INFO:teuthology.orchestra.run.smithi092.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-11T18:31:56.281 INFO:teuthology.orchestra.run.smithi092.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-11T18:31:56.281 INFO:teuthology.orchestra.run.smithi092.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-11T18:31:56.281 INFO:teuthology.orchestra.run.smithi092.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-11T18:31:56.282 INFO:teuthology.orchestra.run.smithi092.stdout:The following additional packages will be installed: 2024-03-11T18:31:56.282 INFO:teuthology.orchestra.run.smithi092.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-03-11T18:31:56.282 INFO:teuthology.orchestra.run.smithi092.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-03-11T18:31:56.282 INFO:teuthology.orchestra.run.smithi092.stdout: libqmi-proxy libssl3 libteamdctl0 modemmanager network-manager-pptp 2024-03-11T18:31:56.282 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-03-11T18:31:56.282 INFO:teuthology.orchestra.run.smithi092.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-03-11T18:31:56.283 INFO:teuthology.orchestra.run.smithi092.stdout:Suggested packages: 2024-03-11T18:31:56.283 INFO:teuthology.orchestra.run.smithi092.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-03-11T18:31:56.283 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial wpagui 2024-03-11T18:31:56.283 INFO:teuthology.orchestra.run.smithi092.stdout: libengine-pkcs11-openssl 2024-03-11T18:31:56.321 INFO:teuthology.orchestra.run.smithi092.stdout:The following NEW packages will be installed: 2024-03-11T18:31:56.322 INFO:teuthology.orchestra.run.smithi092.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-03-11T18:31:56.322 INFO:teuthology.orchestra.run.smithi092.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-03-11T18:31:56.322 INFO:teuthology.orchestra.run.smithi092.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-03-11T18:31:56.322 INFO:teuthology.orchestra.run.smithi092.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-03-11T18:31:56.322 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-03-11T18:31:56.323 INFO:teuthology.orchestra.run.smithi092.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-03-11T18:31:56.323 INFO:teuthology.orchestra.run.smithi092.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-03-11T18:31:56.323 INFO:teuthology.orchestra.run.smithi092.stdout:The following packages will be upgraded: 2024-03-11T18:31:56.323 INFO:teuthology.orchestra.run.smithi092.stdout: libssl3 2024-03-11T18:31:56.503 INFO:teuthology.orchestra.run.smithi092.stdout:1 upgraded, 39 newly installed, 0 to remove and 274 not upgraded. 2024-03-11T18:31:56.504 INFO:teuthology.orchestra.run.smithi092.stdout:Need to get 54.6 MB of archives. 2024-03-11T18:31:56.504 INFO:teuthology.orchestra.run.smithi092.stdout:After this operation, 202 MB of additional disk space will be used. 2024-03-11T18:31:56.504 INFO:teuthology.orchestra.run.smithi092.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-03-11T18:31:56.981 INFO:teuthology.orchestra.run.smithi092.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-03-11T18:31:57.238 INFO:teuthology.orchestra.run.smithi092.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-03-11T18:31:57.273 INFO:teuthology.orchestra.run.smithi092.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-03-11T18:31:57.274 INFO:teuthology.orchestra.run.smithi092.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-03-11T18:31:57.279 INFO:teuthology.orchestra.run.smithi092.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-03-11T18:31:57.283 INFO:teuthology.orchestra.run.smithi092.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-03-11T18:31:57.284 INFO:teuthology.orchestra.run.smithi092.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-03-11T18:31:57.286 INFO:teuthology.orchestra.run.smithi092.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-03-11T18:31:57.289 INFO:teuthology.orchestra.run.smithi092.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-03-11T18:31:57.289 INFO:teuthology.orchestra.run.smithi092.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-03-11T18:31:57.454 INFO:teuthology.orchestra.run.smithi092.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-03-11T18:31:57.774 INFO:teuthology.orchestra.run.smithi092.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-03-11T18:31:58.510 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package ceph-fuse-dbg. 2024-03-11T18:31:58.526 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../127-ceph-fuse-dbg_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:31:58.568 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking ceph-fuse-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:58.951 INFO:teuthology.orchestra.run.smithi092.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-03-11T18:31:58.956 INFO:teuthology.orchestra.run.smithi092.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-03-11T18:31:58.957 INFO:teuthology.orchestra.run.smithi092.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-03-11T18:31:58.963 INFO:teuthology.orchestra.run.smithi092.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-03-11T18:31:58.964 INFO:teuthology.orchestra.run.smithi092.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-03-11T18:31:58.975 INFO:teuthology.orchestra.run.smithi092.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-03-11T18:31:58.979 INFO:teuthology.orchestra.run.smithi092.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-03-11T18:31:58.996 INFO:teuthology.orchestra.run.smithi092.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-03-11T18:31:58.996 INFO:teuthology.orchestra.run.smithi092.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-03-11T18:31:59.080 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package ceph-mds-dbg. 2024-03-11T18:31:59.107 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../128-ceph-mds-dbg_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:31:59.154 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking ceph-mds-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:31:59.230 INFO:teuthology.orchestra.run.smithi092.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-03-11T18:31:59.313 INFO:teuthology.orchestra.run.smithi092.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-03-11T18:31:59.794 INFO:teuthology.orchestra.run.smithi092.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-03-11T18:31:59.924 INFO:teuthology.orchestra.run.smithi092.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-03-11T18:31:59.935 INFO:teuthology.orchestra.run.smithi092.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-03-11T18:31:59.937 INFO:teuthology.orchestra.run.smithi092.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-03-11T18:31:59.938 INFO:teuthology.orchestra.run.smithi092.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-03-11T18:31:59.939 INFO:teuthology.orchestra.run.smithi092.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-03-11T18:31:59.972 INFO:teuthology.orchestra.run.smithi092.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-03-11T18:31:59.976 INFO:teuthology.orchestra.run.smithi092.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-03-11T18:32:00.332 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package ceph-mgr-dbg. 2024-03-11T18:32:00.360 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../129-ceph-mgr-dbg_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:32:00.399 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking ceph-mgr-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:32:00.528 INFO:teuthology.orchestra.run.smithi092.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-03-11T18:32:00.691 INFO:teuthology.orchestra.run.smithi092.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-03-11T18:32:00.774 INFO:teuthology.orchestra.run.smithi092.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-03-11T18:32:00.774 INFO:teuthology.orchestra.run.smithi092.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-03-11T18:32:00.859 INFO:teuthology.orchestra.run.smithi092.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-03-11T18:32:00.868 INFO:teuthology.orchestra.run.smithi092.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-03-11T18:32:01.176 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package ceph-mon-dbg. 2024-03-11T18:32:01.204 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../130-ceph-mon-dbg_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:32:01.244 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking ceph-mon-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:32:01.335 INFO:teuthology.orchestra.run.smithi092.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-03-11T18:32:01.337 INFO:teuthology.orchestra.run.smithi092.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-03-11T18:32:01.745 INFO:teuthology.orchestra.run.smithi092.stdout:Preconfiguring packages ... 2024-03-11T18:32:01.900 INFO:teuthology.orchestra.run.smithi092.stdout:Fetched 54.6 MB in 5s (10.9 MB/s) 2024-03-11T18:32:02.039 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package flex. 2024-03-11T18:32:02.141 INFO:teuthology.orchestra.run.smithi092.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173939 files and directories currently installed.) 2024-03-11T18:32:02.146 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-03-11T18:32:02.195 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking flex (2.6.4-8build2) ... 2024-03-11T18:32:02.896 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-03-11T18:32:03.068 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-03-11T18:32:03.239 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package ceph-osd-dbg. 2024-03-11T18:32:03.266 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../131-ceph-osd-dbg_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:32:03.313 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking ceph-osd-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:32:03.479 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-11T18:32:04.003 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package bison. 2024-03-11T18:32:04.107 INFO:teuthology.orchestra.run.smithi092.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174012 files and directories currently installed.) 2024-03-11T18:32:04.113 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-03-11T18:32:04.155 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-03-11T18:32:04.564 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-03-11T18:32:04.593 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-03-11T18:32:04.631 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-03-11T18:32:04.959 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package dump. 2024-03-11T18:32:04.986 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-03-11T18:32:05.025 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking dump (0.4b47-3) ... 2024-03-11T18:32:05.312 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package indent. 2024-03-11T18:32:05.328 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-03-11T18:32:05.370 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-03-11T18:32:05.631 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-03-11T18:32:05.659 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-03-11T18:32:05.698 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-03-11T18:32:06.018 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-03-11T18:32:06.046 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-03-11T18:32:06.084 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-03-11T18:32:06.345 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-03-11T18:32:06.373 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-03-11T18:32:06.412 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-03-11T18:32:06.765 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libfl2:amd64. 2024-03-11T18:32:06.793 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-03-11T18:32:06.832 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-03-11T18:32:07.101 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-03-11T18:32:07.117 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-03-11T18:32:07.159 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-03-11T18:32:07.397 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libjson-perl. 2024-03-11T18:32:07.422 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-03-11T18:32:07.461 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-03-11T18:32:08.158 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libllvm14:amd64. 2024-03-11T18:32:08.186 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-03-11T18:32:08.224 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-03-11T18:32:09.800 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package ceph-test-dbg. 2024-03-11T18:32:09.816 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../132-ceph-test-dbg_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:32:09.858 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking ceph-test-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:32:09.930 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-03-11T18:32:09.955 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-03-11T18:32:09.993 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-03-11T18:32:10.262 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libmbim-proxy. 2024-03-11T18:32:10.278 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-03-11T18:32:10.321 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-03-11T18:32:10.657 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-03-11T18:32:10.684 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-03-11T18:32:10.723 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-03-11T18:32:11.093 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libndp0:amd64. 2024-03-11T18:32:11.120 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-03-11T18:32:11.153 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-03-11T18:32:11.487 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libnm0:amd64. 2024-03-11T18:32:11.515 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-03-11T18:32:11.545 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-03-11T18:32:11.898 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libpq5:amd64. 2024-03-11T18:32:11.926 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-11T18:32:11.965 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-03-11T18:32:12.318 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-03-11T18:32:12.345 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-03-11T18:32:12.384 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-03-11T18:32:12.695 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libqmi-proxy. 2024-03-11T18:32:12.722 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-03-11T18:32:12.761 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-03-11T18:32:13.166 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-03-11T18:32:13.194 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-03-11T18:32:13.232 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-11T18:32:13.769 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-03-11T18:32:13.797 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-03-11T18:32:13.835 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-03-11T18:32:14.180 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package modemmanager. 2024-03-11T18:32:14.208 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-03-11T18:32:14.238 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-03-11T18:32:14.861 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package network-manager. 2024-03-11T18:32:14.889 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-03-11T18:32:14.935 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-03-11T18:32:15.560 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ppp. 2024-03-11T18:32:15.588 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-03-11T18:32:15.845 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-03-11T18:32:16.147 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package pptp-linux. 2024-03-11T18:32:16.175 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-03-11T18:32:16.214 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-03-11T18:32:16.467 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package network-manager-pptp. 2024-03-11T18:32:16.494 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-03-11T18:32:16.533 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-03-11T18:32:16.836 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package postgresql-client-common. 2024-03-11T18:32:16.864 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-03-11T18:32:16.902 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking postgresql-client-common (238) ... 2024-03-11T18:32:17.190 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package postgresql-client-14. 2024-03-11T18:32:17.218 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-11T18:32:17.248 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-11T18:32:17.828 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package postgresql-common. 2024-03-11T18:32:17.855 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-03-11T18:32:17.914 INFO:teuthology.orchestra.run.smithi092.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-03-11T18:32:17.942 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking postgresql-common (238) ... 2024-03-11T18:32:18.468 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package postgresql-14. 2024-03-11T18:32:18.497 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-11T18:32:18.555 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-11T18:32:19.938 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package postgresql. 2024-03-11T18:32:19.967 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-03-11T18:32:20.005 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking postgresql (14+238) ... 2024-03-11T18:32:20.231 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package postgresql-client. 2024-03-11T18:32:20.248 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-03-11T18:32:20.290 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking postgresql-client (14+238) ... 2024-03-11T18:32:20.542 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package postgresql-contrib. 2024-03-11T18:32:20.570 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-03-11T18:32:20.617 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking postgresql-contrib (14+238) ... 2024-03-11T18:32:20.944 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package tcl8.6. 2024-03-11T18:32:20.972 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-03-11T18:32:21.011 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-03-11T18:32:21.289 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package tcl. 2024-03-11T18:32:21.317 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-03-11T18:32:21.361 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-03-11T18:32:21.708 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package wpasupplicant. 2024-03-11T18:32:21.736 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-03-11T18:32:21.774 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-03-11T18:32:22.154 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-03-11T18:32:22.170 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-03-11T18:32:22.214 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-03-11T18:32:22.574 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package usb-modeswitch. 2024-03-11T18:32:22.602 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-03-11T18:32:22.649 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-03-11T18:32:22.945 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up postgresql-client-common (238) ... 2024-03-11T18:32:23.163 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up flex (2.6.4-8build2) ... 2024-03-11T18:32:23.245 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package libcephfs2-dbg. 2024-03-11T18:32:23.272 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../133-libcephfs2-dbg_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:32:23.289 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-03-11T18:32:23.311 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking libcephfs2-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:32:23.398 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-03-11T18:32:23.524 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-03-11T18:32:23.650 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-03-11T18:32:23.776 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-11T18:32:23.790 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package librados2-dbg. 2024-03-11T18:32:23.806 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../134-librados2-dbg_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:32:23.848 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking librados2-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:32:24.324 INFO:teuthology.orchestra.run.smithi092.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-03-11T18:32:24.415 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-03-11T18:32:24.787 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-03-11T18:32:24.787 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-03-11T18:32:25.084 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package libradosstriper1-dbg. 2024-03-11T18:32:25.100 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../135-libradosstriper1-dbg_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:32:25.134 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking libradosstriper1-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:32:25.268 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-03-11T18:32:25.394 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-11T18:32:25.520 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-03-11T18:32:25.654 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-03-11T18:32:25.780 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-03-11T18:32:25.803 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package librbd1-dbg. 2024-03-11T18:32:25.831 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../136-librbd1-dbg_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:32:25.870 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking librbd1-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:32:25.889 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-03-11T18:32:26.015 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-03-11T18:32:26.662 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-03-11T18:32:26.788 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up dump (0.4b47-3) ... 2024-03-11T18:32:26.881 INFO:teuthology.orchestra.run.smithi092.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-03-11T18:32:27.030 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-03-11T18:32:27.174 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-03-11T18:32:27.300 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up tcl (8.6.11+1build2) ... 2024-03-11T18:32:27.324 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package librgw2-dbg. 2024-03-11T18:32:27.340 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../137-librgw2-dbg_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:32:27.382 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking librgw2-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:32:27.426 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libjson-perl (4.04000-1) ... 2024-03-11T18:32:27.553 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-03-11T18:32:27.642 INFO:teuthology.orchestra.run.smithi092.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-03-11T18:32:27.712 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-03-11T18:32:27.863 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-03-11T18:32:28.039 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-03-11T18:32:28.157 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-03-11T18:32:28.733 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-03-11T18:32:28.901 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up postgresql-client (14+238) ... 2024-03-11T18:32:29.027 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-03-11T18:32:29.477 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-11T18:32:29.477 INFO:teuthology.orchestra.run.smithi092.stdout:The following network interfaces were found in /etc/network/interfaces 2024-03-11T18:32:29.477 INFO:teuthology.orchestra.run.smithi092.stdout:which means they are currently configured by ifupdown: 2024-03-11T18:32:29.477 INFO:teuthology.orchestra.run.smithi092.stdout:- enp3s0f1 2024-03-11T18:32:29.477 INFO:teuthology.orchestra.run.smithi092.stdout:If you want to manage those interfaces with NetworkManager instead 2024-03-11T18:32:29.477 INFO:teuthology.orchestra.run.smithi092.stdout:remove their configuration from /etc/network/interfaces. 2024-03-11T18:32:29.477 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-11T18:32:29.543 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package radosgw-dbg. 2024-03-11T18:32:29.559 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../138-radosgw-dbg_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:32:29.587 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-03-11T18:32:29.593 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking radosgw-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:32:29.909 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-03-11T18:32:30.205 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-03-11T18:32:30.680 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-03-11T18:32:30.814 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-03-11T18:32:30.940 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-03-11T18:32:31.066 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up postgresql-common (238) ... 2024-03-11T18:32:31.936 INFO:teuthology.orchestra.run.smithi092.stdout:Adding user postgres to group ssl-cert 2024-03-11T18:32:32.173 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-11T18:32:32.173 INFO:teuthology.orchestra.run.smithi092.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-03-11T18:32:32.365 INFO:teuthology.orchestra.run.smithi092.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-03-11T18:32:32.365 INFO:teuthology.orchestra.run.smithi092.stdout:Removing obsolete dictionary files: 2024-03-11T18:32:32.951 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-03-11T18:32:33.326 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-11T18:32:33.657 INFO:teuthology.orchestra.run.smithi092.stdout:Creating new PostgreSQL cluster 14/main ... 2024-03-11T18:32:33.668 INFO:teuthology.orchestra.run.smithi092.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-03-11T18:32:33.732 INFO:teuthology.orchestra.run.smithi092.stdout:The files belonging to this database system will be owned by user "postgres". 2024-03-11T18:32:33.732 INFO:teuthology.orchestra.run.smithi092.stdout:This user must also own the server process. 2024-03-11T18:32:33.732 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-11T18:32:33.732 INFO:teuthology.orchestra.run.smithi092.stdout:The database cluster will be initialized with locale "en_US". 2024-03-11T18:32:33.732 INFO:teuthology.orchestra.run.smithi092.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-03-11T18:32:33.733 INFO:teuthology.orchestra.run.smithi092.stdout:The default text search configuration will be set to "english". 2024-03-11T18:32:33.733 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-11T18:32:33.733 INFO:teuthology.orchestra.run.smithi092.stdout:Data page checksums are disabled. 2024-03-11T18:32:33.733 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-11T18:32:33.733 INFO:teuthology.orchestra.run.smithi092.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-03-11T18:32:33.733 INFO:teuthology.orchestra.run.smithi092.stdout:creating subdirectories ... ok 2024-03-11T18:32:33.733 INFO:teuthology.orchestra.run.smithi092.stdout:selecting dynamic shared memory implementation ... posix 2024-03-11T18:32:33.748 INFO:teuthology.orchestra.run.smithi092.stdout:selecting default max_connections ... 100 2024-03-11T18:32:33.769 INFO:teuthology.orchestra.run.smithi092.stdout:selecting default shared_buffers ... 128MB 2024-03-11T18:32:33.779 INFO:teuthology.orchestra.run.smithi092.stdout:selecting default time zone ... Etc/UTC 2024-03-11T18:32:33.779 INFO:teuthology.orchestra.run.smithi092.stdout:creating configuration files ... ok 2024-03-11T18:32:33.821 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package rbd-fuse-dbg. 2024-03-11T18:32:33.837 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../139-rbd-fuse-dbg_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:32:33.880 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking rbd-fuse-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:32:33.894 INFO:teuthology.orchestra.run.smithi092.stdout:running bootstrap script ... ok 2024-03-11T18:32:34.175 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package rbd-mirror-dbg. 2024-03-11T18:32:34.191 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../140-rbd-mirror-dbg_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:32:34.233 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking rbd-mirror-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:32:34.561 INFO:teuthology.orchestra.run.smithi092.stdout:performing post-bootstrap initialization ... ok 2024-03-11T18:32:35.215 INFO:teuthology.orchestra.run.smithi092.stdout:syncing data to disk ... ok 2024-03-11T18:32:35.601 INFO:teuthology.orchestra.run.smithi092.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-03-11T18:32:35.653 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package rbd-nbd-dbg. 2024-03-11T18:32:35.669 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../141-rbd-nbd-dbg_19.0.0-1607-g94a53970-1jammy_amd64.deb ... 2024-03-11T18:32:35.703 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking rbd-nbd-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:32:36.060 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-03-11T18:32:36.310 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-11T18:32:36.453 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-03-11T18:32:36.718 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-03-11T18:32:37.081 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-03-11T18:32:37.479 INFO:teuthology.orchestra.run.smithi090.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-03-11T18:32:37.580 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up cephadm (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:32:38.167 INFO:teuthology.orchestra.run.smithi090.stdout:Adding system user cephadm....done 2024-03-11T18:32:38.313 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-11T18:32:38.352 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-03-11T18:32:38.477 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up postgresql-contrib (14+238) ... 2024-03-11T18:32:38.589 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-03-11T18:32:38.603 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up postgresql (14+238) ... 2024-03-11T18:32:38.721 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-03-11T18:32:38.849 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-03-11T18:32:38.855 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-03-11T18:32:38.977 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-03-11T18:32:39.107 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-03-11T18:32:39.107 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-03-11T18:32:39.230 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-03-11T18:32:39.471 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-03-11T18:32:39.597 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-py (1.10.0-1) ... 2024-03-11T18:32:39.650 INFO:teuthology.orchestra.run.smithi092.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-03-11T18:32:39.899 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-03-11T18:32:39.919 INFO:teuthology.orchestra.run.smithi092.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-11T18:32:40.071 INFO:teuthology.orchestra.run.smithi092.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-11T18:32:40.277 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-03-11T18:32:40.512 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-03-11T18:32:40.747 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-03-11T18:32:41.023 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-03-11T18:32:41.291 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-03-11T18:32:41.441 INFO:teuthology.orchestra.run.smithi092.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-03-11T18:32:41.566 INFO:teuthology.orchestra.run.smithi092.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-03-11T18:32:41.576 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-03-11T18:32:41.970 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-03-11T18:32:42.221 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-ceph-argparse (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:32:42.271 INFO:teuthology.orchestra.run.smithi092.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-11T18:32:42.277 DEBUG:teuthology.parallel:result is None 2024-03-11T18:32:42.481 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-11T18:32:42.607 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-11T18:32:42.733 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-03-11T18:32:42.993 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-03-11T18:32:43.119 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-03-11T18:32:43.371 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-03-11T18:32:43.829 INFO:teuthology.orchestra.run.smithi090.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-03-11T18:32:43.861 INFO:teuthology.orchestra.run.smithi090.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-03-11T18:32:43.925 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-11T18:32:44.051 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-03-11T18:32:44.411 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-11T18:32:44.520 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-03-11T18:32:44.773 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-03-11T18:32:45.032 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-03-11T18:32:45.342 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up zip (3.0-12build2) ... 2024-03-11T18:32:45.468 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-03-11T18:32:46.195 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-03-11T18:32:46.430 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-03-11T18:32:46.557 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-03-11T18:32:46.708 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-11T18:32:47.010 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-03-11T18:32:47.394 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-03-11T18:32:47.637 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-03-11T18:32:47.997 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-11T18:32:48.283 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-03-11T18:32:48.584 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-11T18:32:48.936 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-03-11T18:32:49.179 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-03-11T18:32:49.288 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-ceph-common (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:32:49.581 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-idna (3.3-1) ... 2024-03-11T18:32:49.916 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-11T18:32:51.082 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-03-11T18:32:51.334 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-11T18:32:51.470 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-toml (0.10.2-1) ... 2024-03-11T18:32:51.729 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-11T18:32:51.864 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-03-11T18:32:51.990 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-11T18:32:52.274 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-03-11T18:32:52.534 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-03-11T18:32:52.786 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-11T18:32:52.913 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-03-11T18:32:53.214 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-03-11T18:32:53.465 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-03-11T18:32:53.725 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-03-11T18:32:53.984 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-03-11T18:32:54.236 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-03-11T18:32:54.505 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-11T18:32:54.614 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-03-11T18:32:54.874 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-11T18:32:55.000 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-11T18:32:55.293 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-03-11T18:32:55.602 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-03-11T18:32:55.845 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-11T18:32:55.956 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-03-11T18:32:56.082 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-03-11T18:32:56.172 INFO:teuthology.orchestra.run.smithi090.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-03-11T18:32:56.562 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-11T18:32:57.052 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-03-11T18:32:57.445 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-03-11T18:32:57.688 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up lua-any (27ubuntu1) ... 2024-03-11T18:32:57.807 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-portend (3.0.0-1) ... 2024-03-11T18:32:58.058 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-11T18:32:58.184 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-11T18:32:58.502 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-03-11T18:32:58.795 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-11T18:32:59.055 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-03-11T18:32:59.276 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-rsa (4.8-1) ... 2024-03-11T18:32:59.545 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-03-11T18:32:59.813 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-11T18:33:00.064 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-03-11T18:33:00.434 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-03-11T18:33:00.721 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-03-11T18:33:01.055 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-11T18:33:01.181 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up librados2 (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:33:01.282 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up libsqlite3-mod-ceph (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:33:01.400 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-03-11T18:33:01.738 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up libcephfs2 (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:33:01.864 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up libradosstriper1 (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:33:01.982 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-03-11T18:33:02.267 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up librados2-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:33:02.393 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up librbd1 (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:33:02.521 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up libcephfs2-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:33:02.647 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up ceph-mgr-modules-core (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:33:02.756 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up ceph-fuse (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:33:02.955 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-11T18:33:02.955 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-11T18:33:03.391 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up libradosstriper1-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:33:03.509 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up libcephfs-dev (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:33:03.635 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up ceph-fuse-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:33:03.763 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-rados (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:33:03.889 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up librgw2 (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:33:04.015 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up cephfs-top (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:33:04.125 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-11T18:33:05.354 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up librbd1-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:33:05.481 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-rbd (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:33:05.599 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up librgw2-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:33:05.700 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up rbd-fuse (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:33:05.830 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up rbd-nbd (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:33:05.952 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-rgw (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:33:06.079 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up python3-cephfs (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:33:06.205 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up rbd-nbd-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:33:06.332 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up cephfs-shell (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:33:06.450 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up rbd-fuse-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:33:06.551 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up ceph-common (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:33:07.086 INFO:teuthology.orchestra.run.smithi090.stdout:Adding group ceph....done 2024-03-11T18:33:07.522 INFO:teuthology.orchestra.run.smithi090.stdout:Adding system user ceph....done 2024-03-11T18:33:07.700 INFO:teuthology.orchestra.run.smithi090.stdout:Setting system user ceph properties....done 2024-03-11T18:33:07.715 INFO:teuthology.orchestra.run.smithi090.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-03-11T18:33:07.850 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-03-11T18:33:08.142 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-03-11T18:33:08.581 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up cephfs-mirror (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:33:09.286 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-03-11T18:33:09.286 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-03-11T18:33:09.707 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up ceph-test (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:33:09.818 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up radosgw (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:33:10.275 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-11T18:33:10.275 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-11T18:33:10.720 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up ceph-base (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:33:11.062 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-03-11T18:33:11.505 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up ceph-mds (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:33:11.736 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-11T18:33:11.737 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-11T18:33:12.207 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up ceph-common-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:33:12.487 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up rbd-mirror (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:33:12.714 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target -> /lib/systemd/system/ceph-rbd-mirror.target. 2024-03-11T18:33:12.714 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target -> /lib/systemd/system/ceph-rbd-mirror.target. 2024-03-11T18:33:13.171 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up ceph-mgr (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:33:13.380 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-11T18:33:13.381 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-11T18:33:13.831 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up ceph-mds-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:33:13.932 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up radosgw-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:33:14.058 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up ceph-test-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:33:14.184 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up ceph-osd (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:33:14.455 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-11T18:33:14.455 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-11T18:33:14.929 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up ceph-mgr-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:33:15.055 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up ceph-osd-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:33:15.181 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up rbd-mirror-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:33:15.307 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up ceph-mgr-k8sevents (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:33:15.434 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:33:15.560 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up ceph-mon (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:33:15.779 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-11T18:33:15.779 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-11T18:33:16.212 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up ceph-mgr-cephadm (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:33:16.364 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up ceph (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:33:16.490 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up ceph-mgr-dashboard (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:33:16.641 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up ceph-mon-dbg (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:33:16.767 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up ceph-volume (19.0.0-1607-g94a53970-1jammy) ... 2024-03-11T18:33:17.103 INFO:teuthology.orchestra.run.smithi090.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-03-11T18:33:17.204 INFO:teuthology.orchestra.run.smithi090.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-11T18:33:17.350 INFO:teuthology.orchestra.run.smithi090.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-11T18:33:19.149 INFO:teuthology.orchestra.run.smithi090.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-03-11T18:33:19.922 INFO:teuthology.orchestra.run.smithi090.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-11T18:33:19.925 DEBUG:teuthology.orchestra.run.smithi090:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-03-11T18:33:20.029 INFO:teuthology.orchestra.run.smithi090.stdout:Reading package lists... 2024-03-11T18:33:20.225 INFO:teuthology.orchestra.run.smithi090.stdout:Building dependency tree... 2024-03-11T18:33:20.226 INFO:teuthology.orchestra.run.smithi090.stdout:Reading state information... 2024-03-11T18:33:20.399 INFO:teuthology.orchestra.run.smithi090.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-03-11T18:33:20.400 INFO:teuthology.orchestra.run.smithi090.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-03-11T18:33:20.400 INFO:teuthology.orchestra.run.smithi090.stdout:util-linux is already the newest version (2.37.2-4ubuntu3). 2024-03-11T18:33:20.400 INFO:teuthology.orchestra.run.smithi090.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3). 2024-03-11T18:33:20.400 INFO:teuthology.orchestra.run.smithi090.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-03-11T18:33:20.400 INFO:teuthology.orchestra.run.smithi090.stdout:The following packages were automatically installed and are no longer required: 2024-03-11T18:33:20.400 INFO:teuthology.orchestra.run.smithi090.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-11T18:33:20.400 INFO:teuthology.orchestra.run.smithi090.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-11T18:33:20.401 INFO:teuthology.orchestra.run.smithi090.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-11T18:33:20.401 INFO:teuthology.orchestra.run.smithi090.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-11T18:33:20.401 INFO:teuthology.orchestra.run.smithi090.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-11T18:33:20.401 INFO:teuthology.orchestra.run.smithi090.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-11T18:33:20.403 INFO:teuthology.orchestra.run.smithi090.stdout:The following additional packages will be installed: 2024-03-11T18:33:20.403 INFO:teuthology.orchestra.run.smithi090.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-03-11T18:33:20.403 INFO:teuthology.orchestra.run.smithi090.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-03-11T18:33:20.403 INFO:teuthology.orchestra.run.smithi090.stdout: libqmi-proxy libssl3 libteamdctl0 modemmanager network-manager-pptp 2024-03-11T18:33:20.403 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-03-11T18:33:20.403 INFO:teuthology.orchestra.run.smithi090.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-03-11T18:33:20.403 INFO:teuthology.orchestra.run.smithi090.stdout:Suggested packages: 2024-03-11T18:33:20.403 INFO:teuthology.orchestra.run.smithi090.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-03-11T18:33:20.403 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial wpagui 2024-03-11T18:33:20.403 INFO:teuthology.orchestra.run.smithi090.stdout: libengine-pkcs11-openssl 2024-03-11T18:33:20.440 INFO:teuthology.orchestra.run.smithi090.stdout:The following NEW packages will be installed: 2024-03-11T18:33:20.441 INFO:teuthology.orchestra.run.smithi090.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-03-11T18:33:20.441 INFO:teuthology.orchestra.run.smithi090.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-03-11T18:33:20.441 INFO:teuthology.orchestra.run.smithi090.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-03-11T18:33:20.442 INFO:teuthology.orchestra.run.smithi090.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-03-11T18:33:20.442 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-03-11T18:33:20.442 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-03-11T18:33:20.442 INFO:teuthology.orchestra.run.smithi090.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-03-11T18:33:20.442 INFO:teuthology.orchestra.run.smithi090.stdout:The following packages will be upgraded: 2024-03-11T18:33:20.443 INFO:teuthology.orchestra.run.smithi090.stdout: libssl3 2024-03-11T18:33:20.522 INFO:teuthology.orchestra.run.smithi090.stdout:1 upgraded, 39 newly installed, 0 to remove and 274 not upgraded. 2024-03-11T18:33:20.522 INFO:teuthology.orchestra.run.smithi090.stdout:Need to get 54.6 MB of archives. 2024-03-11T18:33:20.522 INFO:teuthology.orchestra.run.smithi090.stdout:After this operation, 202 MB of additional disk space will be used. 2024-03-11T18:33:20.522 INFO:teuthology.orchestra.run.smithi090.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-03-11T18:33:20.638 INFO:teuthology.orchestra.run.smithi090.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-03-11T18:33:20.706 INFO:teuthology.orchestra.run.smithi090.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-03-11T18:33:20.721 INFO:teuthology.orchestra.run.smithi090.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-03-11T18:33:20.722 INFO:teuthology.orchestra.run.smithi090.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-03-11T18:33:20.724 INFO:teuthology.orchestra.run.smithi090.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-03-11T18:33:20.726 INFO:teuthology.orchestra.run.smithi090.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-03-11T18:33:20.726 INFO:teuthology.orchestra.run.smithi090.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-03-11T18:33:20.728 INFO:teuthology.orchestra.run.smithi090.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-03-11T18:33:20.729 INFO:teuthology.orchestra.run.smithi090.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-03-11T18:33:20.730 INFO:teuthology.orchestra.run.smithi090.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-03-11T18:33:20.731 INFO:teuthology.orchestra.run.smithi090.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-03-11T18:33:20.750 INFO:teuthology.orchestra.run.smithi090.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-03-11T18:33:21.103 INFO:teuthology.orchestra.run.smithi090.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-03-11T18:33:21.105 INFO:teuthology.orchestra.run.smithi090.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-03-11T18:33:21.105 INFO:teuthology.orchestra.run.smithi090.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-03-11T18:33:21.108 INFO:teuthology.orchestra.run.smithi090.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-03-11T18:33:21.108 INFO:teuthology.orchestra.run.smithi090.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-03-11T18:33:21.113 INFO:teuthology.orchestra.run.smithi090.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-03-11T18:33:21.164 INFO:teuthology.orchestra.run.smithi090.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-03-11T18:33:21.172 INFO:teuthology.orchestra.run.smithi090.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-03-11T18:33:21.172 INFO:teuthology.orchestra.run.smithi090.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-03-11T18:33:21.198 INFO:teuthology.orchestra.run.smithi090.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-03-11T18:33:21.198 INFO:teuthology.orchestra.run.smithi090.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-03-11T18:33:21.209 INFO:teuthology.orchestra.run.smithi090.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-03-11T18:33:21.229 INFO:teuthology.orchestra.run.smithi090.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-03-11T18:33:21.232 INFO:teuthology.orchestra.run.smithi090.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-03-11T18:33:21.233 INFO:teuthology.orchestra.run.smithi090.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-03-11T18:33:21.233 INFO:teuthology.orchestra.run.smithi090.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-03-11T18:33:21.234 INFO:teuthology.orchestra.run.smithi090.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-03-11T18:33:21.257 INFO:teuthology.orchestra.run.smithi090.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-03-11T18:33:21.260 INFO:teuthology.orchestra.run.smithi090.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-03-11T18:33:21.596 INFO:teuthology.orchestra.run.smithi090.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-03-11T18:33:21.597 INFO:teuthology.orchestra.run.smithi090.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-03-11T18:33:21.597 INFO:teuthology.orchestra.run.smithi090.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-03-11T18:33:21.597 INFO:teuthology.orchestra.run.smithi090.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-03-11T18:33:21.597 INFO:teuthology.orchestra.run.smithi090.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-03-11T18:33:21.597 INFO:teuthology.orchestra.run.smithi090.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-03-11T18:33:21.616 INFO:teuthology.orchestra.run.smithi090.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-03-11T18:33:21.617 INFO:teuthology.orchestra.run.smithi090.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-03-11T18:33:22.016 INFO:teuthology.orchestra.run.smithi090.stdout:Preconfiguring packages ... 2024-03-11T18:33:22.195 INFO:teuthology.orchestra.run.smithi090.stdout:Fetched 54.6 MB in 1s (46.8 MB/s) 2024-03-11T18:33:22.316 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package flex. 2024-03-11T18:33:22.406 INFO:teuthology.orchestra.run.smithi090.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173939 files and directories currently installed.) 2024-03-11T18:33:22.412 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-03-11T18:33:22.456 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking flex (2.6.4-8build2) ... 2024-03-11T18:33:23.050 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-03-11T18:33:23.214 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-03-11T18:33:23.659 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-11T18:33:24.201 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package bison. 2024-03-11T18:33:24.305 INFO:teuthology.orchestra.run.smithi090.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174012 files and directories currently installed.) 2024-03-11T18:33:24.311 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-03-11T18:33:24.361 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-03-11T18:33:24.822 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-03-11T18:33:24.850 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-03-11T18:33:24.897 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-03-11T18:33:25.309 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package dump. 2024-03-11T18:33:25.337 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-03-11T18:33:25.376 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking dump (0.4b47-3) ... 2024-03-11T18:33:25.647 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package indent. 2024-03-11T18:33:25.664 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-03-11T18:33:25.706 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-03-11T18:33:25.958 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-03-11T18:33:25.975 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-03-11T18:33:26.008 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-03-11T18:33:26.329 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-03-11T18:33:26.357 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-03-11T18:33:26.396 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-03-11T18:33:26.675 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-03-11T18:33:26.703 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-03-11T18:33:26.750 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-03-11T18:33:27.087 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package libfl2:amd64. 2024-03-11T18:33:27.114 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-03-11T18:33:27.153 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-03-11T18:33:27.399 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-03-11T18:33:27.415 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-03-11T18:33:27.457 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-03-11T18:33:27.719 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package libjson-perl. 2024-03-11T18:33:27.734 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-03-11T18:33:27.776 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-03-11T18:33:28.096 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package libllvm14:amd64. 2024-03-11T18:33:28.124 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-03-11T18:33:28.163 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-03-11T18:33:30.109 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-03-11T18:33:30.137 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-03-11T18:33:30.184 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-03-11T18:33:30.445 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package libmbim-proxy. 2024-03-11T18:33:30.473 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-03-11T18:33:30.512 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-03-11T18:33:30.831 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-03-11T18:33:30.859 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-03-11T18:33:30.898 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-03-11T18:33:31.252 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package libndp0:amd64. 2024-03-11T18:33:31.280 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-03-11T18:33:31.319 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-03-11T18:33:31.639 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package libnm0:amd64. 2024-03-11T18:33:31.666 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-03-11T18:33:31.705 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-03-11T18:33:32.067 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package libpq5:amd64. 2024-03-11T18:33:32.094 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-11T18:33:32.134 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-03-11T18:33:32.462 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-03-11T18:33:32.490 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-03-11T18:33:32.529 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-03-11T18:33:32.865 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package libqmi-proxy. 2024-03-11T18:33:32.881 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-03-11T18:33:32.923 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-03-11T18:33:33.168 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-03-11T18:33:33.195 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-03-11T18:33:33.234 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-11T18:33:33.781 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-03-11T18:33:33.809 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-03-11T18:33:33.848 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-03-11T18:33:34.176 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package modemmanager. 2024-03-11T18:33:34.203 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-03-11T18:33:34.242 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-03-11T18:33:34.875 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package network-manager. 2024-03-11T18:33:34.903 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-03-11T18:33:34.950 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-03-11T18:33:35.584 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package ppp. 2024-03-11T18:33:35.612 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-03-11T18:33:35.876 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-03-11T18:33:36.174 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package pptp-linux. 2024-03-11T18:33:36.202 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-03-11T18:33:36.240 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-03-11T18:33:36.485 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package network-manager-pptp. 2024-03-11T18:33:36.501 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-03-11T18:33:36.543 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-03-11T18:33:36.846 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package postgresql-client-common. 2024-03-11T18:33:36.874 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-03-11T18:33:36.913 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking postgresql-client-common (238) ... 2024-03-11T18:33:37.210 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package postgresql-client-14. 2024-03-11T18:33:37.237 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-11T18:33:37.276 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-11T18:33:37.965 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package postgresql-common. 2024-03-11T18:33:37.993 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-03-11T18:33:38.049 INFO:teuthology.orchestra.run.smithi090.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-03-11T18:33:38.081 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking postgresql-common (238) ... 2024-03-11T18:33:38.549 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package postgresql-14. 2024-03-11T18:33:38.577 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-11T18:33:38.628 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-11T18:33:40.356 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package postgresql. 2024-03-11T18:33:40.373 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-03-11T18:33:40.415 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking postgresql (14+238) ... 2024-03-11T18:33:40.667 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package postgresql-client. 2024-03-11T18:33:40.683 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-03-11T18:33:40.725 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking postgresql-client (14+238) ... 2024-03-11T18:33:40.944 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package postgresql-contrib. 2024-03-11T18:33:40.961 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-03-11T18:33:41.004 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking postgresql-contrib (14+238) ... 2024-03-11T18:33:41.307 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package tcl8.6. 2024-03-11T18:33:41.335 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-03-11T18:33:41.373 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-03-11T18:33:41.668 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package tcl. 2024-03-11T18:33:41.696 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-03-11T18:33:41.748 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-03-11T18:33:42.046 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package wpasupplicant. 2024-03-11T18:33:42.062 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-03-11T18:33:42.097 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-03-11T18:33:42.469 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-03-11T18:33:42.494 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-03-11T18:33:42.535 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-03-11T18:33:42.864 INFO:teuthology.orchestra.run.smithi090.stdout:Selecting previously unselected package usb-modeswitch. 2024-03-11T18:33:42.880 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-03-11T18:33:42.923 INFO:teuthology.orchestra.run.smithi090.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-03-11T18:33:43.245 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up postgresql-client-common (238) ... 2024-03-11T18:33:43.455 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up flex (2.6.4-8build2) ... 2024-03-11T18:33:43.564 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-03-11T18:33:43.675 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-03-11T18:33:43.784 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-03-11T18:33:43.885 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-03-11T18:33:44.012 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-11T18:33:44.473 INFO:teuthology.orchestra.run.smithi090.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-03-11T18:33:44.552 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-03-11T18:33:44.934 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-03-11T18:33:44.935 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-03-11T18:33:45.407 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-03-11T18:33:45.532 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-11T18:33:45.658 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-03-11T18:33:45.777 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-03-11T18:33:45.897 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-03-11T18:33:46.021 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-03-11T18:33:46.147 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-03-11T18:33:46.787 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-03-11T18:33:46.913 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up dump (0.4b47-3) ... 2024-03-11T18:33:47.007 INFO:teuthology.orchestra.run.smithi090.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-03-11T18:33:47.074 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-03-11T18:33:47.200 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-03-11T18:33:47.326 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up tcl (8.6.11+1build2) ... 2024-03-11T18:33:47.602 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up libjson-perl (4.04000-1) ... 2024-03-11T18:33:47.704 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-03-11T18:33:47.794 INFO:teuthology.orchestra.run.smithi090.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-03-11T18:33:47.864 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-03-11T18:33:47.991 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-03-11T18:33:48.193 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-03-11T18:33:48.318 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-03-11T18:33:48.436 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-03-11T18:33:48.605 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up postgresql-client (14+238) ... 2024-03-11T18:33:48.706 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-03-11T18:33:49.313 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:33:49.313 INFO:teuthology.orchestra.run.smithi090.stdout:The following network interfaces were found in /etc/network/interfaces 2024-03-11T18:33:49.313 INFO:teuthology.orchestra.run.smithi090.stdout:which means they are currently configured by ifupdown: 2024-03-11T18:33:49.313 INFO:teuthology.orchestra.run.smithi090.stdout:- enp3s0f1 2024-03-11T18:33:49.313 INFO:teuthology.orchestra.run.smithi090.stdout:If you want to manage those interfaces with NetworkManager instead 2024-03-11T18:33:49.313 INFO:teuthology.orchestra.run.smithi090.stdout:remove their configuration from /etc/network/interfaces. 2024-03-11T18:33:49.314 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:33:49.442 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-03-11T18:33:49.773 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-03-11T18:33:50.087 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-03-11T18:33:50.576 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-03-11T18:33:50.728 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-03-11T18:33:50.854 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-03-11T18:33:50.980 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up postgresql-common (238) ... 2024-03-11T18:33:51.844 INFO:teuthology.orchestra.run.smithi090.stdout:Adding user postgres to group ssl-cert 2024-03-11T18:33:52.096 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:33:52.096 INFO:teuthology.orchestra.run.smithi090.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-03-11T18:33:52.363 INFO:teuthology.orchestra.run.smithi090.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-03-11T18:33:52.364 INFO:teuthology.orchestra.run.smithi090.stdout:Removing obsolete dictionary files: 2024-03-11T18:33:52.904 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-03-11T18:33:53.327 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-11T18:33:53.652 INFO:teuthology.orchestra.run.smithi090.stdout:Creating new PostgreSQL cluster 14/main ... 2024-03-11T18:33:53.667 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-03-11T18:33:53.719 INFO:teuthology.orchestra.run.smithi090.stdout:The files belonging to this database system will be owned by user "postgres". 2024-03-11T18:33:53.719 INFO:teuthology.orchestra.run.smithi090.stdout:This user must also own the server process. 2024-03-11T18:33:53.719 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:33:53.719 INFO:teuthology.orchestra.run.smithi090.stdout:The database cluster will be initialized with locale "en_US". 2024-03-11T18:33:53.719 INFO:teuthology.orchestra.run.smithi090.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-03-11T18:33:53.719 INFO:teuthology.orchestra.run.smithi090.stdout:The default text search configuration will be set to "english". 2024-03-11T18:33:53.719 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:33:53.719 INFO:teuthology.orchestra.run.smithi090.stdout:Data page checksums are disabled. 2024-03-11T18:33:53.719 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:33:53.719 INFO:teuthology.orchestra.run.smithi090.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-03-11T18:33:53.720 INFO:teuthology.orchestra.run.smithi090.stdout:creating subdirectories ... ok 2024-03-11T18:33:53.720 INFO:teuthology.orchestra.run.smithi090.stdout:selecting dynamic shared memory implementation ... posix 2024-03-11T18:33:53.731 INFO:teuthology.orchestra.run.smithi090.stdout:selecting default max_connections ... 100 2024-03-11T18:33:53.749 INFO:teuthology.orchestra.run.smithi090.stdout:selecting default shared_buffers ... 128MB 2024-03-11T18:33:53.768 INFO:teuthology.orchestra.run.smithi090.stdout:selecting default time zone ... Etc/UTC 2024-03-11T18:33:53.770 INFO:teuthology.orchestra.run.smithi090.stdout:creating configuration files ... ok 2024-03-11T18:33:53.890 INFO:teuthology.orchestra.run.smithi090.stdout:running bootstrap script ... ok 2024-03-11T18:33:54.564 INFO:teuthology.orchestra.run.smithi090.stdout:performing post-bootstrap initialization ... ok 2024-03-11T18:33:55.424 INFO:teuthology.orchestra.run.smithi090.stdout:syncing data to disk ... ok 2024-03-11T18:33:55.814 INFO:teuthology.orchestra.run.smithi090.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-03-11T18:33:58.569 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-03-11T18:33:58.687 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up postgresql-contrib (14+238) ... 2024-03-11T18:33:58.813 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up postgresql (14+238) ... 2024-03-11T18:33:58.939 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-03-11T18:33:59.065 INFO:teuthology.orchestra.run.smithi090.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-03-11T18:33:59.319 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-03-11T18:33:59.319 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-03-11T18:33:59.861 INFO:teuthology.orchestra.run.smithi090.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-03-11T18:34:00.205 INFO:teuthology.orchestra.run.smithi090.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-11T18:34:00.357 INFO:teuthology.orchestra.run.smithi090.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-11T18:34:01.671 INFO:teuthology.orchestra.run.smithi090.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-03-11T18:34:01.763 INFO:teuthology.orchestra.run.smithi090.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-03-11T18:34:02.258 INFO:teuthology.orchestra.run.smithi090.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-11T18:34:02.264 DEBUG:teuthology.parallel:result is None 2024-03-11T18:34:02.265 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=94a539707733325322964c6a187c6a1b41be1daf 2024-03-11T18:34:02.495 DEBUG:teuthology.orchestra.run.smithi090:> dpkg-query -W -f '${Version}' ceph 2024-03-11T18:34:02.515 INFO:teuthology.orchestra.run.smithi090.stdout:19.0.0-1607-g94a53970-1jammy 2024-03-11T18:34:02.515 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1607-g94a53970-1jammy 2024-03-11T18:34:02.515 INFO:teuthology.task.install:The correct ceph version 19.0.0-1607-g94a53970-1jammy is installed. 2024-03-11T18:34:02.517 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=94a539707733325322964c6a187c6a1b41be1daf 2024-03-11T18:34:02.747 DEBUG:teuthology.orchestra.run.smithi092:> dpkg-query -W -f '${Version}' ceph 2024-03-11T18:34:02.767 INFO:teuthology.orchestra.run.smithi092.stdout:19.0.0-1607-g94a53970-1jammy 2024-03-11T18:34:02.767 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1607-g94a53970-1jammy 2024-03-11T18:34:02.767 INFO:teuthology.task.install:The correct ceph version 19.0.0-1607-g94a53970-1jammy is installed. 2024-03-11T18:34:02.769 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-11T18:34:02.769 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-11T18:34:02.769 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-11T18:34:02.783 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-11T18:34:02.784 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-11T18:34:02.822 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-11T18:34:02.822 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-11T18:34:02.822 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/usr/bin/daemon-helper 2024-03-11T18:34:02.840 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-11T18:34:02.898 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-11T18:34:02.898 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/daemon-helper 2024-03-11T18:34:02.911 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-11T18:34:02.965 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-11T18:34:02.966 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-11T18:34:02.966 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-11T18:34:02.980 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-11T18:34:03.036 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-11T18:34:03.036 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-11T18:34:03.048 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-11T18:34:03.104 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-11T18:34:03.104 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-11T18:34:03.104 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/usr/bin/stdin-killer 2024-03-11T18:34:03.117 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-11T18:34:03.172 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-11T18:34:03.172 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/stdin-killer 2024-03-11T18:34:03.185 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-11T18:34:03.239 INFO:teuthology.run_tasks:Running task ceph... 2024-03-11T18:34:03.335 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-03-11T18:34:03.336 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 777 /var/log/ceph 2024-03-11T18:34:03.338 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 777 /var/log/ceph 2024-03-11T18:34:03.351 INFO:tasks.ceph:Disabling ceph logrotate... 2024-03-11T18:34:03.351 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-11T18:34:03.393 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-11T18:34:03.406 INFO:tasks.ceph:Creating extra log directories... 2024-03-11T18:34:03.407 DEBUG:teuthology.orchestra.run.smithi090:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-11T18:34:03.449 DEBUG:teuthology.orchestra.run.smithi092:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-11T18:34:03.468 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-03-11T18:34:03.468 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-03-11T18:34:03.468 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-03-11T18:34:03.468 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-11T18:34:03.468 DEBUG:teuthology.orchestra.run.smithi090:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-11T18:34:03.509 DEBUG:teuthology.orchestra.run.smithi090:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-11T18:34:03.583 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-11T18:34:03.583 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-11T18:34:03.590 DEBUG:teuthology.orchestra.run.smithi092:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-11T18:34:03.670 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-03-11T18:34:03.670 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mds beacon grace': 60, 'mds heartbeat grace': 60, 'osd heartbeat grace': 60}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds valgrind exit': True, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon osd crush smoke test': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow request', 'SLOW_OPS', 'MON_DOWN'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-03-11T18:34:03.670 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591231', 'branch': 'squid', 'description': 'fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu/{overrides ubuntu_latest}} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/5 tasks/fsstress validater/valgrind}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7591231', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mds beacon grace': 60, 'mds heartbeat grace': 60, 'osd heartbeat grace': 60}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds valgrind exit': True, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon osd crush smoke test': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow request', 'SLOW_OPS', 'MON_DOWN'], 'sha1': '94a539707733325322964c6a187c6a1b41be1daf', 'valgrind': {'exit_on_first_error': False, 'mds': ['--tool=memcheck'], 'mon': ['--tool=memcheck', '--leak-check=full', '--show-reachable=yes']}, 'watchdog': {'daemon_restart': 'normal'}}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'ceph-fuse': {'client.0': {'valgrind': ['--tool=memcheck', '--leak-check=full', '--show-reachable=yes']}}, 'check-counter': {'counters': {'mds': ['mds.exported', 'mds.imported']}}, 'install': {'ceph': {'debuginfo': True, 'flavor': 'default', 'sha1': '94a539707733325322964c6a187c6a1b41be1daf'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'b6a4997be2b8bb74411268cc42753f29ae9a78ad'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 250, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7']], 'seed': 8179, 'sha1': '94a539707733325322964c6a187c6a1b41be1daf', 'sleep_before_teardown': 0, 'subset': '297/512', 'suite': 'fs', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_b6a4997be2b8bb74411268cc42753f29ae9a78ad/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'b6a4997be2b8bb74411268cc42753f29ae9a78ad', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi090.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIJuV0sxi+snrwXTRt4/zPLb9SRqvHOFAqDX8Znz88gcSqAlDcU59fZUKUi9A3xYhq5fZD0ebLvwMIUVchiTyn8=', 'smithi092.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBElYpEGkVAh1XHw3GH0jCDOU0ZVV0GfXVejBx1pRjf7vK5vqjzj0CWHmynSSiipAFCvhpWZVecthJQz5iDI3x3Y='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'debuginfo': True, 'flavor': 'default', 'sha1': '94a539707733325322964c6a187c6a1b41be1daf'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mds beacon grace': 60, 'mds heartbeat grace': 60, 'osd heartbeat grace': 60}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds valgrind exit': True, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon osd crush smoke test': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow request', 'SLOW_OPS', 'MON_DOWN'], 'sha1': '94a539707733325322964c6a187c6a1b41be1daf', 'valgrind': {'exit_on_first_error': False, 'mds': ['--tool=memcheck'], 'mon': ['--tool=memcheck', '--leak-check=full', '--show-reachable=yes']}, 'watchdog': {'daemon_restart': 'normal'}, 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'workunit': {'clients': {'all': ['suites/fsstress.sh']}, 'timeout': '6h'}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e691533f9cbb33d85b2187bba20d7102f098636d', 'timestamp': '2024-03-11_14:02:35', 'tube': 'smithi', 'user': 'teuthology', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-03-11T18:34:03.671 DEBUG:teuthology.orchestra.run.smithi090:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-11T18:34:03.673 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-11T18:34:03.714 DEBUG:teuthology.orchestra.run.smithi090:> sudo install -d -m0777 -- /var/run/ceph 2024-03-11T18:34:03.721 DEBUG:teuthology.orchestra.run.smithi092:> sudo install -d -m0777 -- /var/run/ceph 2024-03-11T18:34:03.769 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-11T18:34:03.769 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/scratch_devs of=/dev/stdout 2024-03-11T18:34:03.780 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-11T18:34:03.780 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_1 2024-03-11T18:34:03.829 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-11T18:34:03.829 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-11T18:34:03.829 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 5h/5d Inode: 876 Links: 1 2024-03-11T18:34:03.829 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-11T18:34:03.829 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-03-11 18:22:11.975632708 +0000 2024-03-11T18:34:03.829 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-03-11 18:22:11.711635058 +0000 2024-03-11T18:34:03.829 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-03-11 18:22:11.711635058 +0000 2024-03-11T18:34:03.829 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: - 2024-03-11T18:34:03.829 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-11T18:34:03.887 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-03-11T18:34:03.887 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-03-11T18:34:03.887 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000389841 s, 1.3 MB/s 2024-03-11T18:34:03.888 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-11T18:34:03.942 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_2 2024-03-11T18:34:03.989 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-11T18:34:03.989 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-11T18:34:03.989 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 5h/5d Inode: 892 Links: 1 2024-03-11T18:34:03.989 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-11T18:34:03.989 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-03-11 18:22:12.483628186 +0000 2024-03-11T18:34:03.990 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-03-11 18:22:12.227630464 +0000 2024-03-11T18:34:03.990 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-03-11 18:22:12.227630464 +0000 2024-03-11T18:34:03.990 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: - 2024-03-11T18:34:03.990 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-11T18:34:04.042 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-03-11T18:34:04.042 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-03-11T18:34:04.043 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000316291 s, 1.6 MB/s 2024-03-11T18:34:04.044 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-11T18:34:04.096 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_3 2024-03-11T18:34:04.144 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-11T18:34:04.144 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-11T18:34:04.144 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 5h/5d Inode: 905 Links: 1 2024-03-11T18:34:04.144 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-11T18:34:04.145 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-03-11 18:22:12.983623735 +0000 2024-03-11T18:34:04.145 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-03-11 18:22:12.711626157 +0000 2024-03-11T18:34:04.145 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-03-11 18:22:12.711626157 +0000 2024-03-11T18:34:04.145 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: - 2024-03-11T18:34:04.145 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-11T18:34:04.199 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-03-11T18:34:04.200 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-03-11T18:34:04.200 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000431323 s, 1.2 MB/s 2024-03-11T18:34:04.201 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-11T18:34:04.252 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_4 2024-03-11T18:34:04.301 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-11T18:34:04.301 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-11T18:34:04.301 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 5h/5d Inode: 926 Links: 1 2024-03-11T18:34:04.301 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-11T18:34:04.301 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-03-11 18:22:13.487619249 +0000 2024-03-11T18:34:04.301 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-03-11 18:22:13.239621456 +0000 2024-03-11T18:34:04.301 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-03-11 18:22:13.239621456 +0000 2024-03-11T18:34:04.301 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: - 2024-03-11T18:34:04.301 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-11T18:34:04.355 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-03-11T18:34:04.355 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-03-11T18:34:04.355 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000388847 s, 1.3 MB/s 2024-03-11T18:34:04.356 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-11T18:34:04.406 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-11T18:34:04.406 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-11T18:34:04.406 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/scratch_devs of=/dev/stdout 2024-03-11T18:34:04.412 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-11T18:34:04.412 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_1 2024-03-11T18:34:04.461 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-11T18:34:04.461 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-11T18:34:04.462 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 867 Links: 1 2024-03-11T18:34:04.462 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-11T18:34:04.462 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-03-11 18:22:08.901596528 +0000 2024-03-11T18:34:04.462 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-03-11 18:22:08.641598850 +0000 2024-03-11T18:34:04.462 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-03-11 18:22:08.641598850 +0000 2024-03-11T18:34:04.462 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: - 2024-03-11T18:34:04.462 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-11T18:34:04.515 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-03-11T18:34:04.516 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-03-11T18:34:04.516 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000432811 s, 1.2 MB/s 2024-03-11T18:34:04.517 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-11T18:34:04.571 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_2 2024-03-11T18:34:04.617 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-11T18:34:04.617 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-11T18:34:04.617 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 878 Links: 1 2024-03-11T18:34:04.617 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-11T18:34:04.617 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-03-11 18:22:09.429591811 +0000 2024-03-11T18:34:04.617 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-03-11 18:22:09.133594454 +0000 2024-03-11T18:34:04.617 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-03-11 18:22:09.133594454 +0000 2024-03-11T18:34:04.617 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: - 2024-03-11T18:34:04.618 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-11T18:34:04.671 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-03-11T18:34:04.672 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-03-11T18:34:04.672 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000372173 s, 1.4 MB/s 2024-03-11T18:34:04.673 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-11T18:34:04.722 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_3 2024-03-11T18:34:04.769 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-11T18:34:04.769 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-11T18:34:04.769 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 903 Links: 1 2024-03-11T18:34:04.769 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-11T18:34:04.769 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-03-11 18:22:09.941587238 +0000 2024-03-11T18:34:04.769 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-03-11 18:22:09.665589703 +0000 2024-03-11T18:34:04.769 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-03-11 18:22:09.665589703 +0000 2024-03-11T18:34:04.769 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: - 2024-03-11T18:34:04.769 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-11T18:34:04.823 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-03-11T18:34:04.823 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-03-11T18:34:04.823 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000414726 s, 1.2 MB/s 2024-03-11T18:34:04.824 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-11T18:34:04.874 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_4 2024-03-11T18:34:04.920 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-11T18:34:04.921 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-11T18:34:04.921 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 5h/5d Inode: 915 Links: 1 2024-03-11T18:34:04.921 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-11T18:34:04.921 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2024-03-11 18:22:10.449582701 +0000 2024-03-11T18:34:04.921 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2024-03-11 18:22:10.201584916 +0000 2024-03-11T18:34:04.921 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2024-03-11 18:22:10.201584916 +0000 2024-03-11T18:34:04.921 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: - 2024-03-11T18:34:04.921 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-11T18:34:04.974 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2024-03-11T18:34:04.974 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2024-03-11T18:34:04.974 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000316158 s, 1.6 MB/s 2024-03-11T18:34:04.976 DEBUG:teuthology.orchestra.run.smithi092:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-11T18:34:05.026 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-11T18:34:05.026 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi090.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi092.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-11T18:34:05.026 INFO:tasks.ceph:Generating config... 2024-03-11T18:34:05.027 INFO:tasks.ceph:[client] client mount timeout = 600 2024-03-11T18:34:05.027 INFO:tasks.ceph:[client] debug client = 20 2024-03-11T18:34:05.027 INFO:tasks.ceph:[client] debug ms = 1 2024-03-11T18:34:05.027 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-03-11T18:34:05.027 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-03-11T18:34:05.027 INFO:tasks.ceph:[global] mds beacon grace = 60 2024-03-11T18:34:05.028 INFO:tasks.ceph:[global] mds heartbeat grace = 60 2024-03-11T18:34:05.028 INFO:tasks.ceph:[global] osd heartbeat grace = 60 2024-03-11T18:34:05.028 INFO:tasks.ceph:[mds] debug mds = 20 2024-03-11T18:34:05.028 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-03-11T18:34:05.028 INFO:tasks.ceph:[mds] debug ms = 1 2024-03-11T18:34:05.028 INFO:tasks.ceph:[mds] mds debug frag = True 2024-03-11T18:34:05.028 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-03-11T18:34:05.028 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-03-11T18:34:05.028 INFO:tasks.ceph:[mds] mds valgrind exit = True 2024-03-11T18:34:05.028 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-03-11T18:34:05.028 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-03-11T18:34:05.028 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-03-11T18:34:05.028 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-03-11T18:34:05.028 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-03-11T18:34:05.028 INFO:tasks.ceph:[mgr] debug ms = 1 2024-03-11T18:34:05.029 INFO:tasks.ceph:[mon] debug mon = 20 2024-03-11T18:34:05.029 INFO:tasks.ceph:[mon] debug ms = 1 2024-03-11T18:34:05.029 INFO:tasks.ceph:[mon] debug paxos = 20 2024-03-11T18:34:05.029 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-03-11T18:34:05.029 INFO:tasks.ceph:[mon] mon osd crush smoke test = False 2024-03-11T18:34:05.029 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-03-11T18:34:05.029 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-03-11T18:34:05.029 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-03-11T18:34:05.029 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-03-11T18:34:05.029 INFO:tasks.ceph:[osd] debug ms = 1 2024-03-11T18:34:05.029 INFO:tasks.ceph:[osd] debug osd = 20 2024-03-11T18:34:05.029 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-03-11T18:34:05.029 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-03-11T18:34:05.029 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-03-11T18:34:05.030 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-03-11T18:34:05.030 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-03-11T18:34:05.030 INFO:tasks.ceph:[osd] osd fast shutdown = False 2024-03-11T18:34:05.030 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-03-11T18:34:05.030 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-03-11T18:34:05.030 INFO:tasks.ceph:Setting up mon.a... 2024-03-11T18:34:05.030 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-03-11T18:34:05.064 INFO:teuthology.orchestra.run.smithi090.stdout:creating /etc/ceph/ceph.keyring 2024-03-11T18:34:05.096 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-03-11T18:34:05.174 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-11T18:34:05.231 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.90'), ('mon.b', '172.21.15.92'), ('mon.c', '[v2:172.21.15.92:3301,v1:172.21.15.92:6790]')] 2024-03-11T18:34:05.232 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.90,172.21.15.92,[v2:172.21.15.92:3301,v1:172.21.15.92:6790]', 'mds beacon grace': 60, 'mds heartbeat grace': 60, 'osd heartbeat grace': 60}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120, 'mon osd crush smoke test': False}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.e': {}, 'mds.b': {}, 'mds.d': {}, 'mds.f': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds valgrind exit': True, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-03-11T18:34:05.233 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-11T18:34:05.233 DEBUG:teuthology.orchestra.run.smithi090:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-03-11T18:34:05.280 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.90 --add b 172.21.15.92 --addv c '[v2:172.21.15.92:3301,v1:172.21.15.92:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-03-11T18:34:05.364 INFO:teuthology.orchestra.run.smithi090.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-03-11T18:34:05.364 INFO:teuthology.orchestra.run.smithi090.stdout:monmaptool: generated fsid 76d56b10-6c29-476a-804b-8482ea90f9e2 2024-03-11T18:34:05.364 INFO:teuthology.orchestra.run.smithi090.stdout:setting min_mon_release = quincy 2024-03-11T18:34:05.364 INFO:teuthology.orchestra.run.smithi090.stdout:epoch 0 2024-03-11T18:34:05.364 INFO:teuthology.orchestra.run.smithi090.stdout:fsid 76d56b10-6c29-476a-804b-8482ea90f9e2 2024-03-11T18:34:05.364 INFO:teuthology.orchestra.run.smithi090.stdout:last_changed 2024-03-11T18:34:05.361089+0000 2024-03-11T18:34:05.364 INFO:teuthology.orchestra.run.smithi090.stdout:created 2024-03-11T18:34:05.361089+0000 2024-03-11T18:34:05.364 INFO:teuthology.orchestra.run.smithi090.stdout:min_mon_release 17 (quincy) 2024-03-11T18:34:05.365 INFO:teuthology.orchestra.run.smithi090.stdout:election_strategy: 1 2024-03-11T18:34:05.365 INFO:teuthology.orchestra.run.smithi090.stdout:0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-03-11T18:34:05.365 INFO:teuthology.orchestra.run.smithi090.stdout:1: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.b 2024-03-11T18:34:05.365 INFO:teuthology.orchestra.run.smithi090.stdout:2: [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] mon.c 2024-03-11T18:34:05.365 INFO:teuthology.orchestra.run.smithi090.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-03-11T18:34:05.366 DEBUG:teuthology.orchestra.run.smithi090:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-03-11T18:34:05.412 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 76d56b10-6c29-476a-804b-8482ea90f9e2... 2024-03-11T18:34:05.413 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-11T18:34:05.457 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-11T18:34:05.485 INFO:teuthology.orchestra.run.smithi090.stdout:[global] 2024-03-11T18:34:05.486 INFO:teuthology.orchestra.run.smithi090.stdout: chdir = "" 2024-03-11T18:34:05.486 INFO:teuthology.orchestra.run.smithi090.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-11T18:34:05.486 INFO:teuthology.orchestra.run.smithi090.stdout: auth supported = cephx 2024-03-11T18:34:05.486 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:34:05.486 INFO:teuthology.orchestra.run.smithi090.stdout: filestore xattr use omap = true 2024-03-11T18:34:05.486 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:34:05.486 INFO:teuthology.orchestra.run.smithi090.stdout: mon clock drift allowed = 1.000 2024-03-11T18:34:05.486 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:34:05.486 INFO:teuthology.orchestra.run.smithi090.stdout: osd crush chooseleaf type = 0 2024-03-11T18:34:05.486 INFO:teuthology.orchestra.run.smithi090.stdout: auth debug = true 2024-03-11T18:34:05.486 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:34:05.486 INFO:teuthology.orchestra.run.smithi090.stdout: ms die on old message = true 2024-03-11T18:34:05.487 INFO:teuthology.orchestra.run.smithi090.stdout: ms die on bug = true 2024-03-11T18:34:05.487 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:34:05.487 INFO:teuthology.orchestra.run.smithi090.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-11T18:34:05.487 INFO:teuthology.orchestra.run.smithi090.stdout: mon pg warn max object skew = 0 2024-03-11T18:34:05.487 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:34:05.487 INFO:teuthology.orchestra.run.smithi090.stdout: # disable pg_autoscaler by default for new pools 2024-03-11T18:34:05.487 INFO:teuthology.orchestra.run.smithi090.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-11T18:34:05.487 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:34:05.487 INFO:teuthology.orchestra.run.smithi090.stdout: osd pool default size = 2 2024-03-11T18:34:05.487 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:34:05.487 INFO:teuthology.orchestra.run.smithi090.stdout: mon osd allow primary affinity = true 2024-03-11T18:34:05.487 INFO:teuthology.orchestra.run.smithi090.stdout: mon osd allow pg remap = true 2024-03-11T18:34:05.487 INFO:teuthology.orchestra.run.smithi090.stdout: mon warn on legacy crush tunables = false 2024-03-11T18:34:05.487 INFO:teuthology.orchestra.run.smithi090.stdout: mon warn on crush straw calc version zero = false 2024-03-11T18:34:05.488 INFO:teuthology.orchestra.run.smithi090.stdout: mon warn on no sortbitwise = false 2024-03-11T18:34:05.488 INFO:teuthology.orchestra.run.smithi090.stdout: mon warn on osd down out interval zero = false 2024-03-11T18:34:05.488 INFO:teuthology.orchestra.run.smithi090.stdout: mon warn on too few osds = false 2024-03-11T18:34:05.488 INFO:teuthology.orchestra.run.smithi090.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-11T18:34:05.488 INFO:teuthology.orchestra.run.smithi090.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-11T18:34:05.488 INFO:teuthology.orchestra.run.smithi090.stdout: mon_allow_pool_size_one = true 2024-03-11T18:34:05.488 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:34:05.488 INFO:teuthology.orchestra.run.smithi090.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-11T18:34:05.488 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:34:05.488 INFO:teuthology.orchestra.run.smithi090.stdout: osd default data pool replay window = 5 2024-03-11T18:34:05.488 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:34:05.488 INFO:teuthology.orchestra.run.smithi090.stdout: mon allow pool delete = true 2024-03-11T18:34:05.488 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:34:05.488 INFO:teuthology.orchestra.run.smithi090.stdout: mon cluster log file level = debug 2024-03-11T18:34:05.489 INFO:teuthology.orchestra.run.smithi090.stdout: debug asserts on shutdown = true 2024-03-11T18:34:05.489 INFO:teuthology.orchestra.run.smithi090.stdout: mon health detail to clog = false 2024-03-11T18:34:05.489 INFO:teuthology.orchestra.run.smithi090.stdout: mon host = "172.21.15.90,172.21.15.92,[v2:172.21.15.92:3301,v1:172.21.15.92:6790]" 2024-03-11T18:34:05.489 INFO:teuthology.orchestra.run.smithi090.stdout: mds beacon grace = 60 2024-03-11T18:34:05.489 INFO:teuthology.orchestra.run.smithi090.stdout: mds heartbeat grace = 60 2024-03-11T18:34:05.489 INFO:teuthology.orchestra.run.smithi090.stdout: osd heartbeat grace = 60 2024-03-11T18:34:05.489 INFO:teuthology.orchestra.run.smithi090.stdout: fsid = 76d56b10-6c29-476a-804b-8482ea90f9e2 2024-03-11T18:34:05.489 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:34:05.489 INFO:teuthology.orchestra.run.smithi090.stdout:[osd] 2024-03-11T18:34:05.489 INFO:teuthology.orchestra.run.smithi090.stdout: osd journal size = 100 2024-03-11T18:34:05.489 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:34:05.489 INFO:teuthology.orchestra.run.smithi090.stdout: osd scrub load threshold = 5.0 2024-03-11T18:34:05.489 INFO:teuthology.orchestra.run.smithi090.stdout: osd scrub max interval = 600 2024-03-11T18:34:05.489 INFO:teuthology.orchestra.run.smithi090.stdout: osd mclock profile = high_recovery_ops 2024-03-11T18:34:05.489 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:34:05.490 INFO:teuthology.orchestra.run.smithi090.stdout: osd recover clone overlap = true 2024-03-11T18:34:05.490 INFO:teuthology.orchestra.run.smithi090.stdout: osd recovery max chunk = 1048576 2024-03-11T18:34:05.490 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:34:05.490 INFO:teuthology.orchestra.run.smithi090.stdout: osd debug shutdown = true 2024-03-11T18:34:05.490 INFO:teuthology.orchestra.run.smithi090.stdout: osd debug op order = true 2024-03-11T18:34:05.490 INFO:teuthology.orchestra.run.smithi090.stdout: osd debug verify stray on activate = true 2024-03-11T18:34:05.490 INFO:teuthology.orchestra.run.smithi090.stdout: osd debug trim objects = true 2024-03-11T18:34:05.490 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:34:05.490 INFO:teuthology.orchestra.run.smithi090.stdout: osd open classes on start = true 2024-03-11T18:34:05.490 INFO:teuthology.orchestra.run.smithi090.stdout: osd debug pg log writeout = true 2024-03-11T18:34:05.490 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:34:05.490 INFO:teuthology.orchestra.run.smithi090.stdout: osd deep scrub update digest min age = 30 2024-03-11T18:34:05.491 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:34:05.491 INFO:teuthology.orchestra.run.smithi090.stdout: osd map max advance = 10 2024-03-11T18:34:05.491 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:34:05.491 INFO:teuthology.orchestra.run.smithi090.stdout: journal zero on create = true 2024-03-11T18:34:05.491 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:34:05.491 INFO:teuthology.orchestra.run.smithi090.stdout: filestore ondisk finisher threads = 3 2024-03-11T18:34:05.491 INFO:teuthology.orchestra.run.smithi090.stdout: filestore apply finisher threads = 3 2024-03-11T18:34:05.491 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:34:05.491 INFO:teuthology.orchestra.run.smithi090.stdout: bdev debug aio = true 2024-03-11T18:34:05.491 INFO:teuthology.orchestra.run.smithi090.stdout: osd debug misdirected ops = true 2024-03-11T18:34:05.491 INFO:teuthology.orchestra.run.smithi090.stdout: bluestore block size = 96636764160 2024-03-11T18:34:05.491 INFO:teuthology.orchestra.run.smithi090.stdout: bluestore fsck on mount = True 2024-03-11T18:34:05.491 INFO:teuthology.orchestra.run.smithi090.stdout: debug bluefs = 20 2024-03-11T18:34:05.491 INFO:teuthology.orchestra.run.smithi090.stdout: debug bluestore = 20 2024-03-11T18:34:05.491 INFO:teuthology.orchestra.run.smithi090.stdout: debug ms = 1 2024-03-11T18:34:05.492 INFO:teuthology.orchestra.run.smithi090.stdout: debug osd = 20 2024-03-11T18:34:05.492 INFO:teuthology.orchestra.run.smithi090.stdout: debug rocksdb = 10 2024-03-11T18:34:05.492 INFO:teuthology.orchestra.run.smithi090.stdout: mon osd backfillfull_ratio = 0.85 2024-03-11T18:34:05.492 INFO:teuthology.orchestra.run.smithi090.stdout: mon osd full ratio = 0.9 2024-03-11T18:34:05.492 INFO:teuthology.orchestra.run.smithi090.stdout: mon osd nearfull ratio = 0.8 2024-03-11T18:34:05.492 INFO:teuthology.orchestra.run.smithi090.stdout: osd failsafe full ratio = 0.95 2024-03-11T18:34:05.492 INFO:teuthology.orchestra.run.smithi090.stdout: osd fast shutdown = False 2024-03-11T18:34:05.492 INFO:teuthology.orchestra.run.smithi090.stdout: osd objectstore = bluestore 2024-03-11T18:34:05.492 INFO:teuthology.orchestra.run.smithi090.stdout: osd op complaint time = 180 2024-03-11T18:34:05.492 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:34:05.492 INFO:teuthology.orchestra.run.smithi090.stdout:[mgr] 2024-03-11T18:34:05.492 INFO:teuthology.orchestra.run.smithi090.stdout: debug ms = 1 2024-03-11T18:34:05.492 INFO:teuthology.orchestra.run.smithi090.stdout: debug mgr = 20 2024-03-11T18:34:05.492 INFO:teuthology.orchestra.run.smithi090.stdout: debug mon = 20 2024-03-11T18:34:05.493 INFO:teuthology.orchestra.run.smithi090.stdout: debug auth = 20 2024-03-11T18:34:05.493 INFO:teuthology.orchestra.run.smithi090.stdout: mon reweight min pgs per osd = 4 2024-03-11T18:34:05.493 INFO:teuthology.orchestra.run.smithi090.stdout: mon reweight min bytes per osd = 10 2024-03-11T18:34:05.493 INFO:teuthology.orchestra.run.smithi090.stdout: mgr/telemetry/nag = false 2024-03-11T18:34:05.493 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:34:05.493 INFO:teuthology.orchestra.run.smithi090.stdout:[mon] 2024-03-11T18:34:05.493 INFO:teuthology.orchestra.run.smithi090.stdout: debug ms = 1 2024-03-11T18:34:05.493 INFO:teuthology.orchestra.run.smithi090.stdout: debug mon = 20 2024-03-11T18:34:05.493 INFO:teuthology.orchestra.run.smithi090.stdout: debug paxos = 20 2024-03-11T18:34:05.493 INFO:teuthology.orchestra.run.smithi090.stdout: debug auth = 20 2024-03-11T18:34:05.493 INFO:teuthology.orchestra.run.smithi090.stdout: mon data avail warn = 5 2024-03-11T18:34:05.493 INFO:teuthology.orchestra.run.smithi090.stdout: mon mgr mkfs grace = 240 2024-03-11T18:34:05.493 INFO:teuthology.orchestra.run.smithi090.stdout: mon reweight min pgs per osd = 4 2024-03-11T18:34:05.493 INFO:teuthology.orchestra.run.smithi090.stdout: mon osd reporter subtree level = osd 2024-03-11T18:34:05.493 INFO:teuthology.orchestra.run.smithi090.stdout: mon osd prime pg temp = true 2024-03-11T18:34:05.493 INFO:teuthology.orchestra.run.smithi090.stdout: mon reweight min bytes per osd = 10 2024-03-11T18:34:05.494 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:34:05.494 INFO:teuthology.orchestra.run.smithi090.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-11T18:34:05.494 INFO:teuthology.orchestra.run.smithi090.stdout: auth mon ticket ttl = 660 # 11m 2024-03-11T18:34:05.494 INFO:teuthology.orchestra.run.smithi090.stdout: auth service ticket ttl = 240 # 4m 2024-03-11T18:34:05.494 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:34:05.494 INFO:teuthology.orchestra.run.smithi090.stdout: # don't complain about insecure global_id in the test suite 2024-03-11T18:34:05.494 INFO:teuthology.orchestra.run.smithi090.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-11T18:34:05.494 INFO:teuthology.orchestra.run.smithi090.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-11T18:34:05.494 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:34:05.494 INFO:teuthology.orchestra.run.smithi090.stdout: # 1m isn't quite enough 2024-03-11T18:34:05.494 INFO:teuthology.orchestra.run.smithi090.stdout: mon_down_mkfs_grace = 2m 2024-03-11T18:34:05.494 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:34:05.494 INFO:teuthology.orchestra.run.smithi090.stdout: mon_warn_on_filestore_osds = false 2024-03-11T18:34:05.494 INFO:teuthology.orchestra.run.smithi090.stdout: mon op complaint time = 120 2024-03-11T18:34:05.495 INFO:teuthology.orchestra.run.smithi090.stdout: mon osd crush smoke test = False 2024-03-11T18:34:05.495 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:34:05.495 INFO:teuthology.orchestra.run.smithi090.stdout:[client] 2024-03-11T18:34:05.495 INFO:teuthology.orchestra.run.smithi090.stdout: rgw cache enabled = true 2024-03-11T18:34:05.495 INFO:teuthology.orchestra.run.smithi090.stdout: rgw enable ops log = true 2024-03-11T18:34:05.495 INFO:teuthology.orchestra.run.smithi090.stdout: rgw enable usage log = true 2024-03-11T18:34:05.495 INFO:teuthology.orchestra.run.smithi090.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-11T18:34:05.495 INFO:teuthology.orchestra.run.smithi090.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-11T18:34:05.495 INFO:teuthology.orchestra.run.smithi090.stdout: client mount timeout = 600 2024-03-11T18:34:05.495 INFO:teuthology.orchestra.run.smithi090.stdout: debug client = 20 2024-03-11T18:34:05.495 INFO:teuthology.orchestra.run.smithi090.stdout: debug ms = 1 2024-03-11T18:34:05.495 INFO:teuthology.orchestra.run.smithi090.stdout: rados mon op timeout = 900 2024-03-11T18:34:05.495 INFO:teuthology.orchestra.run.smithi090.stdout: rados osd op timeout = 900 2024-03-11T18:34:05.495 INFO:teuthology.orchestra.run.smithi090.stdout:[mon.a] 2024-03-11T18:34:05.495 INFO:teuthology.orchestra.run.smithi090.stdout:[mon.b] 2024-03-11T18:34:05.496 INFO:teuthology.orchestra.run.smithi090.stdout:[mon.c] 2024-03-11T18:34:05.496 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.a] 2024-03-11T18:34:05.496 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.c] 2024-03-11T18:34:05.496 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.e] 2024-03-11T18:34:05.496 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.b] 2024-03-11T18:34:05.496 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.d] 2024-03-11T18:34:05.496 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.f] 2024-03-11T18:34:05.496 INFO:teuthology.orchestra.run.smithi090.stdout:[mds] 2024-03-11T18:34:05.496 INFO:teuthology.orchestra.run.smithi090.stdout: debug mds = 20 2024-03-11T18:34:05.496 INFO:teuthology.orchestra.run.smithi090.stdout: debug mds balancer = 20 2024-03-11T18:34:05.496 INFO:teuthology.orchestra.run.smithi090.stdout: debug ms = 1 2024-03-11T18:34:05.496 INFO:teuthology.orchestra.run.smithi090.stdout: mds debug frag = True 2024-03-11T18:34:05.496 INFO:teuthology.orchestra.run.smithi090.stdout: mds debug scatterstat = True 2024-03-11T18:34:05.497 INFO:teuthology.orchestra.run.smithi090.stdout: mds op complaint time = 180 2024-03-11T18:34:05.497 INFO:teuthology.orchestra.run.smithi090.stdout: mds valgrind exit = True 2024-03-11T18:34:05.497 INFO:teuthology.orchestra.run.smithi090.stdout: mds verify scatter = True 2024-03-11T18:34:05.497 INFO:teuthology.orchestra.run.smithi090.stdout: osd op complaint time = 180 2024-03-11T18:34:05.497 INFO:teuthology.orchestra.run.smithi090.stdout: rados mon op timeout = 900 2024-03-11T18:34:05.497 INFO:teuthology.orchestra.run.smithi090.stdout: rados osd op timeout = 900 2024-03-11T18:34:05.498 INFO:teuthology.orchestra.run.smithi092.stdout:[global] 2024-03-11T18:34:05.498 INFO:teuthology.orchestra.run.smithi092.stdout: chdir = "" 2024-03-11T18:34:05.498 INFO:teuthology.orchestra.run.smithi092.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-11T18:34:05.498 INFO:teuthology.orchestra.run.smithi092.stdout: auth supported = cephx 2024-03-11T18:34:05.498 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-11T18:34:05.498 INFO:teuthology.orchestra.run.smithi092.stdout: filestore xattr use omap = true 2024-03-11T18:34:05.498 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-11T18:34:05.498 INFO:teuthology.orchestra.run.smithi092.stdout: mon clock drift allowed = 1.000 2024-03-11T18:34:05.498 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-11T18:34:05.498 INFO:teuthology.orchestra.run.smithi092.stdout: osd crush chooseleaf type = 0 2024-03-11T18:34:05.498 INFO:teuthology.orchestra.run.smithi092.stdout: auth debug = true 2024-03-11T18:34:05.498 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-11T18:34:05.498 INFO:teuthology.orchestra.run.smithi092.stdout: ms die on old message = true 2024-03-11T18:34:05.499 INFO:teuthology.orchestra.run.smithi092.stdout: ms die on bug = true 2024-03-11T18:34:05.499 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-11T18:34:05.499 INFO:teuthology.orchestra.run.smithi092.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-11T18:34:05.499 INFO:teuthology.orchestra.run.smithi092.stdout: mon pg warn max object skew = 0 2024-03-11T18:34:05.499 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-11T18:34:05.499 INFO:teuthology.orchestra.run.smithi092.stdout: # disable pg_autoscaler by default for new pools 2024-03-11T18:34:05.499 INFO:teuthology.orchestra.run.smithi092.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-11T18:34:05.499 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-11T18:34:05.499 INFO:teuthology.orchestra.run.smithi092.stdout: osd pool default size = 2 2024-03-11T18:34:05.499 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-11T18:34:05.499 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd allow primary affinity = true 2024-03-11T18:34:05.499 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd allow pg remap = true 2024-03-11T18:34:05.499 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on legacy crush tunables = false 2024-03-11T18:34:05.499 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on crush straw calc version zero = false 2024-03-11T18:34:05.500 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on no sortbitwise = false 2024-03-11T18:34:05.500 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on osd down out interval zero = false 2024-03-11T18:34:05.500 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on too few osds = false 2024-03-11T18:34:05.500 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-11T18:34:05.500 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-11T18:34:05.500 INFO:teuthology.orchestra.run.smithi092.stdout: mon_allow_pool_size_one = true 2024-03-11T18:34:05.500 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-11T18:34:05.500 INFO:teuthology.orchestra.run.smithi092.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-11T18:34:05.500 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-11T18:34:05.500 INFO:teuthology.orchestra.run.smithi092.stdout: osd default data pool replay window = 5 2024-03-11T18:34:05.500 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-11T18:34:05.500 INFO:teuthology.orchestra.run.smithi092.stdout: mon allow pool delete = true 2024-03-11T18:34:05.500 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-11T18:34:05.500 INFO:teuthology.orchestra.run.smithi092.stdout: mon cluster log file level = debug 2024-03-11T18:34:05.501 INFO:teuthology.orchestra.run.smithi092.stdout: debug asserts on shutdown = true 2024-03-11T18:34:05.501 INFO:teuthology.orchestra.run.smithi092.stdout: mon health detail to clog = false 2024-03-11T18:34:05.501 INFO:teuthology.orchestra.run.smithi092.stdout: mon host = "172.21.15.90,172.21.15.92,[v2:172.21.15.92:3301,v1:172.21.15.92:6790]" 2024-03-11T18:34:05.501 INFO:teuthology.orchestra.run.smithi092.stdout: mds beacon grace = 60 2024-03-11T18:34:05.501 INFO:teuthology.orchestra.run.smithi092.stdout: mds heartbeat grace = 60 2024-03-11T18:34:05.501 INFO:teuthology.orchestra.run.smithi092.stdout: osd heartbeat grace = 60 2024-03-11T18:34:05.501 INFO:teuthology.orchestra.run.smithi092.stdout: fsid = 76d56b10-6c29-476a-804b-8482ea90f9e2 2024-03-11T18:34:05.501 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-11T18:34:05.501 INFO:teuthology.orchestra.run.smithi092.stdout:[osd] 2024-03-11T18:34:05.501 INFO:teuthology.orchestra.run.smithi092.stdout: osd journal size = 100 2024-03-11T18:34:05.501 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-11T18:34:05.501 INFO:teuthology.orchestra.run.smithi092.stdout: osd scrub load threshold = 5.0 2024-03-11T18:34:05.501 INFO:teuthology.orchestra.run.smithi092.stdout: osd scrub max interval = 600 2024-03-11T18:34:05.501 INFO:teuthology.orchestra.run.smithi092.stdout: osd mclock profile = high_recovery_ops 2024-03-11T18:34:05.502 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-11T18:34:05.502 INFO:teuthology.orchestra.run.smithi092.stdout: osd recover clone overlap = true 2024-03-11T18:34:05.502 INFO:teuthology.orchestra.run.smithi092.stdout: osd recovery max chunk = 1048576 2024-03-11T18:34:05.502 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-11T18:34:05.502 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug shutdown = true 2024-03-11T18:34:05.502 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug op order = true 2024-03-11T18:34:05.502 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug verify stray on activate = true 2024-03-11T18:34:05.502 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug trim objects = true 2024-03-11T18:34:05.502 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-11T18:34:05.502 INFO:teuthology.orchestra.run.smithi092.stdout: osd open classes on start = true 2024-03-11T18:34:05.502 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug pg log writeout = true 2024-03-11T18:34:05.502 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-11T18:34:05.502 INFO:teuthology.orchestra.run.smithi092.stdout: osd deep scrub update digest min age = 30 2024-03-11T18:34:05.503 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-11T18:34:05.503 INFO:teuthology.orchestra.run.smithi092.stdout: osd map max advance = 10 2024-03-11T18:34:05.503 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-11T18:34:05.503 INFO:teuthology.orchestra.run.smithi092.stdout: journal zero on create = true 2024-03-11T18:34:05.503 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-11T18:34:05.503 INFO:teuthology.orchestra.run.smithi092.stdout: filestore ondisk finisher threads = 3 2024-03-11T18:34:05.503 INFO:teuthology.orchestra.run.smithi092.stdout: filestore apply finisher threads = 3 2024-03-11T18:34:05.503 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-11T18:34:05.503 INFO:teuthology.orchestra.run.smithi092.stdout: bdev debug aio = true 2024-03-11T18:34:05.503 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug misdirected ops = true 2024-03-11T18:34:05.503 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore block size = 96636764160 2024-03-11T18:34:05.503 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore fsck on mount = True 2024-03-11T18:34:05.503 INFO:teuthology.orchestra.run.smithi092.stdout: debug bluefs = 20 2024-03-11T18:34:05.503 INFO:teuthology.orchestra.run.smithi092.stdout: debug bluestore = 20 2024-03-11T18:34:05.504 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2024-03-11T18:34:05.504 INFO:teuthology.orchestra.run.smithi092.stdout: debug osd = 20 2024-03-11T18:34:05.504 INFO:teuthology.orchestra.run.smithi092.stdout: debug rocksdb = 10 2024-03-11T18:34:05.504 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd backfillfull_ratio = 0.85 2024-03-11T18:34:05.504 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd full ratio = 0.9 2024-03-11T18:34:05.504 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd nearfull ratio = 0.8 2024-03-11T18:34:05.504 INFO:teuthology.orchestra.run.smithi092.stdout: osd failsafe full ratio = 0.95 2024-03-11T18:34:05.504 INFO:teuthology.orchestra.run.smithi092.stdout: osd fast shutdown = False 2024-03-11T18:34:05.504 INFO:teuthology.orchestra.run.smithi092.stdout: osd objectstore = bluestore 2024-03-11T18:34:05.504 INFO:teuthology.orchestra.run.smithi092.stdout: osd op complaint time = 180 2024-03-11T18:34:05.504 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-11T18:34:05.504 INFO:teuthology.orchestra.run.smithi092.stdout:[mgr] 2024-03-11T18:34:05.504 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2024-03-11T18:34:05.504 INFO:teuthology.orchestra.run.smithi092.stdout: debug mgr = 20 2024-03-11T18:34:05.505 INFO:teuthology.orchestra.run.smithi092.stdout: debug mon = 20 2024-03-11T18:34:05.505 INFO:teuthology.orchestra.run.smithi092.stdout: debug auth = 20 2024-03-11T18:34:05.505 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min pgs per osd = 4 2024-03-11T18:34:05.505 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min bytes per osd = 10 2024-03-11T18:34:05.505 INFO:teuthology.orchestra.run.smithi092.stdout: mgr/telemetry/nag = false 2024-03-11T18:34:05.505 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-11T18:34:05.505 INFO:teuthology.orchestra.run.smithi092.stdout:[mon] 2024-03-11T18:34:05.505 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2024-03-11T18:34:05.505 INFO:teuthology.orchestra.run.smithi092.stdout: debug mon = 20 2024-03-11T18:34:05.505 INFO:teuthology.orchestra.run.smithi092.stdout: debug paxos = 20 2024-03-11T18:34:05.505 INFO:teuthology.orchestra.run.smithi092.stdout: debug auth = 20 2024-03-11T18:34:05.505 INFO:teuthology.orchestra.run.smithi092.stdout: mon data avail warn = 5 2024-03-11T18:34:05.505 INFO:teuthology.orchestra.run.smithi092.stdout: mon mgr mkfs grace = 240 2024-03-11T18:34:05.505 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min pgs per osd = 4 2024-03-11T18:34:05.505 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd reporter subtree level = osd 2024-03-11T18:34:05.506 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd prime pg temp = true 2024-03-11T18:34:05.506 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min bytes per osd = 10 2024-03-11T18:34:05.506 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-11T18:34:05.506 INFO:teuthology.orchestra.run.smithi092.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-11T18:34:05.506 INFO:teuthology.orchestra.run.smithi092.stdout: auth mon ticket ttl = 660 # 11m 2024-03-11T18:34:05.506 INFO:teuthology.orchestra.run.smithi092.stdout: auth service ticket ttl = 240 # 4m 2024-03-11T18:34:05.506 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-11T18:34:05.506 INFO:teuthology.orchestra.run.smithi092.stdout: # don't complain about insecure global_id in the test suite 2024-03-11T18:34:05.506 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-11T18:34:05.506 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-11T18:34:05.506 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-11T18:34:05.506 INFO:teuthology.orchestra.run.smithi092.stdout: # 1m isn't quite enough 2024-03-11T18:34:05.506 INFO:teuthology.orchestra.run.smithi092.stdout: mon_down_mkfs_grace = 2m 2024-03-11T18:34:05.507 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-11T18:34:05.507 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_filestore_osds = false 2024-03-11T18:34:05.507 INFO:teuthology.orchestra.run.smithi092.stdout: mon op complaint time = 120 2024-03-11T18:34:05.507 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd crush smoke test = False 2024-03-11T18:34:05.507 INFO:teuthology.orchestra.run.smithi092.stdout: 2024-03-11T18:34:05.507 INFO:teuthology.orchestra.run.smithi092.stdout:[client] 2024-03-11T18:34:05.507 INFO:teuthology.orchestra.run.smithi092.stdout: rgw cache enabled = true 2024-03-11T18:34:05.507 INFO:teuthology.orchestra.run.smithi092.stdout: rgw enable ops log = true 2024-03-11T18:34:05.507 INFO:teuthology.orchestra.run.smithi092.stdout: rgw enable usage log = true 2024-03-11T18:34:05.507 INFO:teuthology.orchestra.run.smithi092.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-11T18:34:05.507 INFO:teuthology.orchestra.run.smithi092.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-11T18:34:05.507 INFO:teuthology.orchestra.run.smithi092.stdout: client mount timeout = 600 2024-03-11T18:34:05.507 INFO:teuthology.orchestra.run.smithi092.stdout: debug client = 20 2024-03-11T18:34:05.507 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2024-03-11T18:34:05.507 INFO:teuthology.orchestra.run.smithi092.stdout: rados mon op timeout = 900 2024-03-11T18:34:05.508 INFO:teuthology.orchestra.run.smithi092.stdout: rados osd op timeout = 900 2024-03-11T18:34:05.508 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.a] 2024-03-11T18:34:05.508 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.b] 2024-03-11T18:34:05.508 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.c] 2024-03-11T18:34:05.508 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.a] 2024-03-11T18:34:05.508 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.c] 2024-03-11T18:34:05.508 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.e] 2024-03-11T18:34:05.508 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.b] 2024-03-11T18:34:05.508 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.d] 2024-03-11T18:34:05.508 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.f] 2024-03-11T18:34:05.508 INFO:teuthology.orchestra.run.smithi092.stdout:[mds] 2024-03-11T18:34:05.508 INFO:teuthology.orchestra.run.smithi092.stdout: debug mds = 20 2024-03-11T18:34:05.508 INFO:teuthology.orchestra.run.smithi092.stdout: debug mds balancer = 20 2024-03-11T18:34:05.508 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2024-03-11T18:34:05.508 INFO:teuthology.orchestra.run.smithi092.stdout: mds debug frag = True 2024-03-11T18:34:05.509 INFO:teuthology.orchestra.run.smithi092.stdout: mds debug scatterstat = True 2024-03-11T18:34:05.509 INFO:teuthology.orchestra.run.smithi092.stdout: mds op complaint time = 180 2024-03-11T18:34:05.509 INFO:teuthology.orchestra.run.smithi092.stdout: mds valgrind exit = True 2024-03-11T18:34:05.509 INFO:teuthology.orchestra.run.smithi092.stdout: mds verify scatter = True 2024-03-11T18:34:05.509 INFO:teuthology.orchestra.run.smithi092.stdout: osd op complaint time = 180 2024-03-11T18:34:05.509 INFO:teuthology.orchestra.run.smithi092.stdout: rados mon op timeout = 900 2024-03-11T18:34:05.509 INFO:teuthology.orchestra.run.smithi092.stdout: rados osd op timeout = 900 2024-03-11T18:34:05.511 INFO:tasks.ceph:Creating admin key on mon.a... 2024-03-11T18:34:05.511 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-03-11T18:34:05.602 INFO:tasks.ceph:Copying monmap to all nodes... 2024-03-11T18:34:05.602 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-11T18:34:05.602 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-03-11T18:34:05.608 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-11T18:34:05.608 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-03-11T18:34:05.656 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi090.front.sepia.ceph.com 2024-03-11T18:34:05.656 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-11T18:34:05.656 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-11T18:34:05.656 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-11T18:34:05.719 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-11T18:34:05.720 DEBUG:teuthology.orchestra.run.smithi090:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-11T18:34:05.770 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi092.front.sepia.ceph.com 2024-03-11T18:34:05.771 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-11T18:34:05.771 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-11T18:34:05.771 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-11T18:34:05.794 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-11T18:34:05.794 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-11T18:34:05.841 INFO:tasks.ceph:Setting up mon nodes... 2024-03-11T18:34:05.841 INFO:tasks.ceph:Setting up mgr nodes... 2024-03-11T18:34:05.841 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-03-11T18:34:05.884 INFO:teuthology.orchestra.run.smithi090.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-03-11T18:34:05.887 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-03-11T18:34:05.930 INFO:teuthology.orchestra.run.smithi092.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-03-11T18:34:06.031 INFO:tasks.ceph:Setting up mds nodes... 2024-03-11T18:34:06.031 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-03-11T18:34:06.076 INFO:teuthology.orchestra.run.smithi090.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-03-11T18:34:06.079 DEBUG:teuthology.orchestra.run.smithi090:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-03-11T18:34:06.091 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-03-11T18:34:06.168 INFO:teuthology.orchestra.run.smithi090.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-03-11T18:34:06.171 DEBUG:teuthology.orchestra.run.smithi090:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-03-11T18:34:06.224 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /var/lib/ceph/mds/ceph-e && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.e /var/lib/ceph/mds/ceph-e/keyring 2024-03-11T18:34:06.306 INFO:teuthology.orchestra.run.smithi090.stdout:creating /var/lib/ceph/mds/ceph-e/keyring 2024-03-11T18:34:06.308 DEBUG:teuthology.orchestra.run.smithi090:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-e 2024-03-11T18:34:06.364 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-03-11T18:34:06.409 INFO:teuthology.orchestra.run.smithi092.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-03-11T18:34:06.412 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-03-11T18:34:06.425 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-03-11T18:34:06.509 INFO:teuthology.orchestra.run.smithi092.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-03-11T18:34:06.512 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-03-11T18:34:06.568 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mds/ceph-f && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.f /var/lib/ceph/mds/ceph-f/keyring 2024-03-11T18:34:06.653 INFO:teuthology.orchestra.run.smithi092.stdout:creating /var/lib/ceph/mds/ceph-f/keyring 2024-03-11T18:34:06.656 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-f 2024-03-11T18:34:06.712 INFO:tasks.ceph_client:Setting up client nodes... 2024-03-11T18:34:06.712 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-11T18:34:06.748 INFO:teuthology.orchestra.run.smithi090.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-03-11T18:34:06.759 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-03-11T18:34:06.759 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi090.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi092.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-11T18:34:06.759 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-03-11T18:34:06.812 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-11T18:34:06.812 INFO:tasks.ceph:role: osd.0 2024-03-11T18:34:06.812 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi090.front.sepia.ceph.com 2024-03-11T18:34:06.812 DEBUG:teuthology.orchestra.run.smithi090:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-11T18:34:06.876 INFO:teuthology.orchestra.run.smithi090.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-11T18:34:06.876 INFO:teuthology.orchestra.run.smithi090.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-11T18:34:06.876 INFO:teuthology.orchestra.run.smithi090.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-11T18:34:06.876 INFO:teuthology.orchestra.run.smithi090.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-11T18:34:06.876 INFO:teuthology.orchestra.run.smithi090.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-11T18:34:06.876 INFO:teuthology.orchestra.run.smithi090.stdout: = sunit=0 swidth=0 blks 2024-03-11T18:34:06.877 INFO:teuthology.orchestra.run.smithi090.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-11T18:34:06.877 INFO:teuthology.orchestra.run.smithi090.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-11T18:34:06.877 INFO:teuthology.orchestra.run.smithi090.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-11T18:34:06.877 INFO:teuthology.orchestra.run.smithi090.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-11T18:34:07.323 INFO:teuthology.orchestra.run.smithi090.stdout:Discarding blocks...Done. 2024-03-11T18:34:07.325 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi090.front.sepia.ceph.com -o noatime 2024-03-11T18:34:07.325 DEBUG:teuthology.orchestra.run.smithi090:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-03-11T18:34:07.422 DEBUG:teuthology.orchestra.run.smithi090:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-03-11T18:34:07.431 INFO:teuthology.orchestra.run.smithi090.stderr:sudo: /sbin/restorecon: command not found 2024-03-11T18:34:07.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-11T18:34:07.432 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-03-11T18:34:07.485 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-11T18:34:07.485 INFO:tasks.ceph:role: osd.1 2024-03-11T18:34:07.485 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi090.front.sepia.ceph.com 2024-03-11T18:34:07.486 DEBUG:teuthology.orchestra.run.smithi090:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-11T18:34:07.547 INFO:teuthology.orchestra.run.smithi090.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-11T18:34:07.547 INFO:teuthology.orchestra.run.smithi090.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-11T18:34:07.547 INFO:teuthology.orchestra.run.smithi090.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-11T18:34:07.547 INFO:teuthology.orchestra.run.smithi090.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-11T18:34:07.547 INFO:teuthology.orchestra.run.smithi090.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-11T18:34:07.547 INFO:teuthology.orchestra.run.smithi090.stdout: = sunit=0 swidth=0 blks 2024-03-11T18:34:07.547 INFO:teuthology.orchestra.run.smithi090.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-11T18:34:07.548 INFO:teuthology.orchestra.run.smithi090.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-11T18:34:07.548 INFO:teuthology.orchestra.run.smithi090.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-11T18:34:07.548 INFO:teuthology.orchestra.run.smithi090.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-11T18:34:08.018 INFO:teuthology.orchestra.run.smithi090.stdout:Discarding blocks...Done. 2024-03-11T18:34:08.020 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi090.front.sepia.ceph.com -o noatime 2024-03-11T18:34:08.020 DEBUG:teuthology.orchestra.run.smithi090:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-03-11T18:34:08.179 DEBUG:teuthology.orchestra.run.smithi090:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-03-11T18:34:08.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-11T18:34:08.191 INFO:teuthology.orchestra.run.smithi090.stderr:sudo: /sbin/restorecon: command not found 2024-03-11T18:34:08.191 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-03-11T18:34:08.243 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-11T18:34:08.244 INFO:tasks.ceph:role: osd.2 2024-03-11T18:34:08.244 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi090.front.sepia.ceph.com 2024-03-11T18:34:08.244 DEBUG:teuthology.orchestra.run.smithi090:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-11T18:34:08.311 INFO:teuthology.orchestra.run.smithi090.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-11T18:34:08.311 INFO:teuthology.orchestra.run.smithi090.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-11T18:34:08.311 INFO:teuthology.orchestra.run.smithi090.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-11T18:34:08.311 INFO:teuthology.orchestra.run.smithi090.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-11T18:34:08.311 INFO:teuthology.orchestra.run.smithi090.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-11T18:34:08.311 INFO:teuthology.orchestra.run.smithi090.stdout: = sunit=0 swidth=0 blks 2024-03-11T18:34:08.311 INFO:teuthology.orchestra.run.smithi090.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-11T18:34:08.311 INFO:teuthology.orchestra.run.smithi090.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-11T18:34:08.312 INFO:teuthology.orchestra.run.smithi090.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-11T18:34:08.312 INFO:teuthology.orchestra.run.smithi090.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-11T18:34:08.775 INFO:teuthology.orchestra.run.smithi090.stdout:Discarding blocks...Done. 2024-03-11T18:34:08.777 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi090.front.sepia.ceph.com -o noatime 2024-03-11T18:34:08.778 DEBUG:teuthology.orchestra.run.smithi090:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-03-11T18:34:08.921 DEBUG:teuthology.orchestra.run.smithi090:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-03-11T18:34:08.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-11T18:34:08.933 INFO:teuthology.orchestra.run.smithi090.stderr:sudo: /sbin/restorecon: command not found 2024-03-11T18:34:08.933 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-03-11T18:34:08.988 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-11T18:34:08.988 INFO:tasks.ceph:role: osd.3 2024-03-11T18:34:08.988 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi090.front.sepia.ceph.com 2024-03-11T18:34:08.988 DEBUG:teuthology.orchestra.run.smithi090:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-11T18:34:09.055 INFO:teuthology.orchestra.run.smithi090.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-11T18:34:09.056 INFO:teuthology.orchestra.run.smithi090.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-11T18:34:09.056 INFO:teuthology.orchestra.run.smithi090.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-11T18:34:09.056 INFO:teuthology.orchestra.run.smithi090.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-11T18:34:09.056 INFO:teuthology.orchestra.run.smithi090.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-11T18:34:09.056 INFO:teuthology.orchestra.run.smithi090.stdout: = sunit=0 swidth=0 blks 2024-03-11T18:34:09.056 INFO:teuthology.orchestra.run.smithi090.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-11T18:34:09.056 INFO:teuthology.orchestra.run.smithi090.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-11T18:34:09.056 INFO:teuthology.orchestra.run.smithi090.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-11T18:34:09.056 INFO:teuthology.orchestra.run.smithi090.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-11T18:34:09.480 INFO:teuthology.orchestra.run.smithi090.stdout:Discarding blocks...Done. 2024-03-11T18:34:09.483 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi090.front.sepia.ceph.com -o noatime 2024-03-11T18:34:09.483 DEBUG:teuthology.orchestra.run.smithi090:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-03-11T18:34:09.572 DEBUG:teuthology.orchestra.run.smithi090:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-03-11T18:34:09.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-11T18:34:09.584 INFO:teuthology.orchestra.run.smithi090.stderr:sudo: /sbin/restorecon: command not found 2024-03-11T18:34:09.584 DEBUG:teuthology.orchestra.run.smithi090:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-11T18:34:09.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:09.720+0000 7f9e2a6487c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-03-11T18:34:09.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:09.720+0000 7f9e2a6487c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-03-11T18:34:09.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:09.720+0000 7f9e2a6487c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-11T18:34:09.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:09.720+0000 7f9e2a6487c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-11T18:34:09.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:09.720+0000 7f9e2a6487c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-11T18:34:09.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:09.720+0000 7f9e2a6487c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-03-11T18:34:12.701 DEBUG:teuthology.orchestra.run.smithi090:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-11T18:34:12.717 DEBUG:teuthology.orchestra.run.smithi090:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-11T18:34:12.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:12.788+0000 7f02e3e887c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-03-11T18:34:12.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:12.788+0000 7f02e3e887c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-03-11T18:34:12.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:12.788+0000 7f02e3e887c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-11T18:34:12.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:12.788+0000 7f02e3e887c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-11T18:34:12.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:12.788+0000 7f02e3e887c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-11T18:34:12.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:12.788+0000 7f02e3e887c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-03-11T18:34:15.760 DEBUG:teuthology.orchestra.run.smithi090:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-11T18:34:15.816 DEBUG:teuthology.orchestra.run.smithi090:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-11T18:34:15.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:15.896+0000 7f531158f7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-03-11T18:34:15.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:15.896+0000 7f531158f7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-03-11T18:34:15.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:15.896+0000 7f531158f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-11T18:34:15.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:15.896+0000 7f531158f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-11T18:34:15.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:15.896+0000 7f531158f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-11T18:34:15.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:15.900+0000 7f531158f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-03-11T18:34:18.883 DEBUG:teuthology.orchestra.run.smithi090:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-11T18:34:18.941 DEBUG:teuthology.orchestra.run.smithi090:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-11T18:34:19.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:19.016+0000 7f925feb97c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-03-11T18:34:19.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:19.016+0000 7f925feb97c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-03-11T18:34:19.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:19.016+0000 7f925feb97c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-11T18:34:19.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:19.016+0000 7f925feb97c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-11T18:34:19.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:19.016+0000 7f925feb97c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-11T18:34:19.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:19.020+0000 7f925feb97c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-03-11T18:34:22.007 DEBUG:teuthology.orchestra.run.smithi090:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-11T18:34:22.064 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-03-11T18:34:22.078 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-11T18:34:22.078 INFO:tasks.ceph:role: osd.4 2024-03-11T18:34:22.078 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi092.front.sepia.ceph.com 2024-03-11T18:34:22.078 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-11T18:34:22.140 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-11T18:34:22.140 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-11T18:34:22.140 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-11T18:34:22.140 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-11T18:34:22.141 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-11T18:34:22.141 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2024-03-11T18:34:22.141 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-11T18:34:22.141 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-11T18:34:22.141 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-11T18:34:22.141 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-11T18:34:22.596 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2024-03-11T18:34:22.598 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2024-03-11T18:34:22.598 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-03-11T18:34:22.700 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-03-11T18:34:22.709 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: /sbin/restorecon: command not found 2024-03-11T18:34:22.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-11T18:34:22.710 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-03-11T18:34:22.763 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-11T18:34:22.763 INFO:tasks.ceph:role: osd.5 2024-03-11T18:34:22.763 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi092.front.sepia.ceph.com 2024-03-11T18:34:22.763 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-11T18:34:22.824 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-11T18:34:22.824 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-11T18:34:22.824 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-11T18:34:22.824 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-11T18:34:22.824 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-11T18:34:22.825 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2024-03-11T18:34:22.825 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-11T18:34:22.825 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-11T18:34:22.825 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-11T18:34:22.825 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-11T18:34:23.295 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2024-03-11T18:34:23.296 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2024-03-11T18:34:23.297 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-03-11T18:34:23.446 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-03-11T18:34:23.457 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: /sbin/restorecon: command not found 2024-03-11T18:34:23.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-11T18:34:23.458 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-03-11T18:34:23.510 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-11T18:34:23.510 INFO:tasks.ceph:role: osd.6 2024-03-11T18:34:23.510 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi092.front.sepia.ceph.com 2024-03-11T18:34:23.510 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-11T18:34:23.575 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-11T18:34:23.576 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-11T18:34:23.576 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-11T18:34:23.576 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-11T18:34:23.576 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-11T18:34:23.576 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2024-03-11T18:34:23.576 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-11T18:34:23.577 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-11T18:34:23.577 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-11T18:34:23.577 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-11T18:34:24.068 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2024-03-11T18:34:24.071 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2024-03-11T18:34:24.071 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-03-11T18:34:24.235 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-03-11T18:34:24.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-11T18:34:24.247 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: /sbin/restorecon: command not found 2024-03-11T18:34:24.247 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-03-11T18:34:24.299 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-11T18:34:24.300 INFO:tasks.ceph:role: osd.7 2024-03-11T18:34:24.300 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi092.front.sepia.ceph.com 2024-03-11T18:34:24.300 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-11T18:34:24.364 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-11T18:34:24.364 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-11T18:34:24.364 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-11T18:34:24.364 INFO:teuthology.orchestra.run.smithi092.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-11T18:34:24.365 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-11T18:34:24.365 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2024-03-11T18:34:24.365 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-11T18:34:24.365 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-11T18:34:24.365 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-11T18:34:24.365 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-11T18:34:24.803 INFO:teuthology.orchestra.run.smithi092.stdout:Discarding blocks...Done. 2024-03-11T18:34:24.805 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2024-03-11T18:34:24.805 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-03-11T18:34:24.892 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-03-11T18:34:24.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-11T18:34:24.949 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: /sbin/restorecon: command not found 2024-03-11T18:34:24.950 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-11T18:34:25.147 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:25.142+0000 7ff4b82827c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-03-11T18:34:25.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:25.142+0000 7ff4b82827c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-03-11T18:34:25.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:25.142+0000 7ff4b82827c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-11T18:34:25.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:25.142+0000 7ff4b82827c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-11T18:34:25.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:25.142+0000 7ff4b82827c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-11T18:34:25.148 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:25.146+0000 7ff4b82827c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-03-11T18:34:28.118 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-11T18:34:28.132 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-11T18:34:28.213 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:28.210+0000 7fd90827f7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-03-11T18:34:28.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:28.210+0000 7fd90827f7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-03-11T18:34:28.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:28.210+0000 7fd90827f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-11T18:34:28.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:28.210+0000 7fd90827f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-11T18:34:28.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:28.210+0000 7fd90827f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-11T18:34:28.214 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:28.210+0000 7fd90827f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-03-11T18:34:31.179 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-11T18:34:31.232 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-11T18:34:31.313 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:31.310+0000 7f242ee4f7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-03-11T18:34:31.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:31.310+0000 7f242ee4f7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-03-11T18:34:31.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:31.310+0000 7f242ee4f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-11T18:34:31.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:31.310+0000 7f242ee4f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-11T18:34:31.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:31.310+0000 7f242ee4f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-11T18:34:31.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:31.310+0000 7f242ee4f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-03-11T18:34:33.672 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:34:33.675 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:34:34.302 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-03-11T18:34:34.316 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-11T18:34:34.397 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:34.394+0000 7f2c44c7d7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-03-11T18:34:34.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:34.394+0000 7f2c44c7d7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-03-11T18:34:34.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:34.394+0000 7f2c44c7d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-11T18:34:34.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:34.394+0000 7f2c44c7d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-11T18:34:34.398 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:34.394+0000 7f2c44c7d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-11T18:34:34.402 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:34.394+0000 7f2c44c7d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-03-11T18:34:37.366 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-03-11T18:34:37.420 INFO:tasks.ceph:Reading keys from all nodes... 2024-03-11T18:34:37.421 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-11T18:34:37.421 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-03-11T18:34:37.434 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-11T18:34:37.434 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-03-11T18:34:37.489 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-11T18:34:37.489 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-03-11T18:34:37.543 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-11T18:34:37.543 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/mds/ceph-e/keyring of=/dev/stdout 2024-03-11T18:34:37.595 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-11T18:34:37.595 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-03-11T18:34:37.647 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-11T18:34:37.648 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-03-11T18:34:37.703 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-11T18:34:37.703 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-03-11T18:34:37.755 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-11T18:34:37.755 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-03-11T18:34:37.807 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-11T18:34:37.807 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-03-11T18:34:37.821 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-11T18:34:37.821 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-03-11T18:34:37.875 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-11T18:34:37.875 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-03-11T18:34:37.931 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-11T18:34:37.931 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/mds/ceph-f/keyring of=/dev/stdout 2024-03-11T18:34:37.982 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-11T18:34:37.983 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-03-11T18:34:38.034 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-11T18:34:38.035 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-03-11T18:34:38.088 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-11T18:34:38.088 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-03-11T18:34:38.143 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-11T18:34:38.143 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-03-11T18:34:38.195 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-11T18:34:38.195 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-03-11T18:34:38.201 INFO:tasks.ceph:Adding keys to all mons... 2024-03-11T18:34:38.201 DEBUG:teuthology.orchestra.run.smithi090:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-11T18:34:38.246 DEBUG:teuthology.orchestra.run.smithi092:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-11T18:34:38.255 INFO:teuthology.orchestra.run.smithi090.stdout:[mgr.y] 2024-03-11T18:34:38.255 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQCdTu9ltQ+MNBAAifnWqtfQO5K0Zg74YujHow== 2024-03-11T18:34:38.255 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.a] 2024-03-11T18:34:38.255 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQCeTu9lj/peBBAAnG4Vx/Gxky1JTaGUFmssMw== 2024-03-11T18:34:38.255 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.c] 2024-03-11T18:34:38.256 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQCeTu9lcWfaCRAAdvopZ9esksXGQRzR1mwG0A== 2024-03-11T18:34:38.256 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.e] 2024-03-11T18:34:38.256 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQCeTu9lk1MSEhAATfhowl8tbaTKtfsGzScqyg== 2024-03-11T18:34:38.256 INFO:teuthology.orchestra.run.smithi090.stdout:[osd.0] 2024-03-11T18:34:38.256 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQChTu9lOwH7KhAAWZLByeGnuHKPUxJCgVC4yw== 2024-03-11T18:34:38.256 INFO:teuthology.orchestra.run.smithi090.stdout:[osd.1] 2024-03-11T18:34:38.256 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQCkTu9lytEXLxAAsO+TlE8fQcSHrsHm82bUow== 2024-03-11T18:34:38.256 INFO:teuthology.orchestra.run.smithi090.stdout:[osd.2] 2024-03-11T18:34:38.256 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQCnTu9l+rSjNRAAnw0bq7YEZn8l5+Ay2zXk2w== 2024-03-11T18:34:38.256 INFO:teuthology.orchestra.run.smithi090.stdout:[osd.3] 2024-03-11T18:34:38.256 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQCrTu9l94wzARAAwoNrS3h/kAK0x2OvORGG7g== 2024-03-11T18:34:38.256 INFO:teuthology.orchestra.run.smithi090.stdout:[mgr.x] 2024-03-11T18:34:38.257 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQCdTu9lY9d+NxAAsy6nimlfZRSke/sLyBSRJA== 2024-03-11T18:34:38.257 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.b] 2024-03-11T18:34:38.257 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQCeTu9lpBt0GBAAURs9KxqWjUugSTd3DM/gEA== 2024-03-11T18:34:38.257 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.d] 2024-03-11T18:34:38.257 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQCeTu9lqWlmHhAAMG5TsklRx+OkiJMAqkPMzg== 2024-03-11T18:34:38.257 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.f] 2024-03-11T18:34:38.257 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQCeTu9lX34DJxAAjWBEsSNB97Yoq9kzMdenKQ== 2024-03-11T18:34:38.257 INFO:teuthology.orchestra.run.smithi090.stdout:[osd.4] 2024-03-11T18:34:38.257 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQCxTu9l2FrUCBAAKUbxJFOjAoDm3KLavtS4rQ== 2024-03-11T18:34:38.257 INFO:teuthology.orchestra.run.smithi090.stdout:[osd.5] 2024-03-11T18:34:38.257 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQC0Tu9lKprHDBAAYDRw7fqLmb8rgKiWFqnjMg== 2024-03-11T18:34:38.257 INFO:teuthology.orchestra.run.smithi090.stdout:[osd.6] 2024-03-11T18:34:38.257 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQC3Tu9lVda+EhAApsYyTGY+r96k1Kz+J2iawQ== 2024-03-11T18:34:38.257 INFO:teuthology.orchestra.run.smithi090.stdout:[osd.7] 2024-03-11T18:34:38.258 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQC6Tu9lsLTAFxAAZLTcA0IBHafqEmNqnAK5ng== 2024-03-11T18:34:38.258 INFO:teuthology.orchestra.run.smithi090.stdout:[client.0] 2024-03-11T18:34:38.258 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQCeTu9lrc1wLBAAIaPhXaMFdzoEp6Kh5ptjxQ== 2024-03-11T18:34:38.259 INFO:teuthology.orchestra.run.smithi092.stdout:[mgr.y] 2024-03-11T18:34:38.259 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCdTu9ltQ+MNBAAifnWqtfQO5K0Zg74YujHow== 2024-03-11T18:34:38.259 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.a] 2024-03-11T18:34:38.259 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCeTu9lj/peBBAAnG4Vx/Gxky1JTaGUFmssMw== 2024-03-11T18:34:38.259 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.c] 2024-03-11T18:34:38.260 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCeTu9lcWfaCRAAdvopZ9esksXGQRzR1mwG0A== 2024-03-11T18:34:38.260 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.e] 2024-03-11T18:34:38.260 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCeTu9lk1MSEhAATfhowl8tbaTKtfsGzScqyg== 2024-03-11T18:34:38.260 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.0] 2024-03-11T18:34:38.260 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQChTu9lOwH7KhAAWZLByeGnuHKPUxJCgVC4yw== 2024-03-11T18:34:38.260 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.1] 2024-03-11T18:34:38.260 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCkTu9lytEXLxAAsO+TlE8fQcSHrsHm82bUow== 2024-03-11T18:34:38.260 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.2] 2024-03-11T18:34:38.260 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCnTu9l+rSjNRAAnw0bq7YEZn8l5+Ay2zXk2w== 2024-03-11T18:34:38.260 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.3] 2024-03-11T18:34:38.260 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCrTu9l94wzARAAwoNrS3h/kAK0x2OvORGG7g== 2024-03-11T18:34:38.260 INFO:teuthology.orchestra.run.smithi092.stdout:[mgr.x] 2024-03-11T18:34:38.260 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCdTu9lY9d+NxAAsy6nimlfZRSke/sLyBSRJA== 2024-03-11T18:34:38.260 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.b] 2024-03-11T18:34:38.260 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCeTu9lpBt0GBAAURs9KxqWjUugSTd3DM/gEA== 2024-03-11T18:34:38.261 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.d] 2024-03-11T18:34:38.261 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCeTu9lqWlmHhAAMG5TsklRx+OkiJMAqkPMzg== 2024-03-11T18:34:38.261 INFO:teuthology.orchestra.run.smithi092.stdout:[mds.f] 2024-03-11T18:34:38.261 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCeTu9lX34DJxAAjWBEsSNB97Yoq9kzMdenKQ== 2024-03-11T18:34:38.261 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.4] 2024-03-11T18:34:38.261 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCxTu9l2FrUCBAAKUbxJFOjAoDm3KLavtS4rQ== 2024-03-11T18:34:38.261 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.5] 2024-03-11T18:34:38.261 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQC0Tu9lKprHDBAAYDRw7fqLmb8rgKiWFqnjMg== 2024-03-11T18:34:38.261 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.6] 2024-03-11T18:34:38.261 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQC3Tu9lVda+EhAApsYyTGY+r96k1Kz+J2iawQ== 2024-03-11T18:34:38.261 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.7] 2024-03-11T18:34:38.261 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQC6Tu9lsLTAFxAAZLTcA0IBHafqEmNqnAK5ng== 2024-03-11T18:34:38.261 INFO:teuthology.orchestra.run.smithi092.stdout:[client.0] 2024-03-11T18:34:38.262 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQCeTu9lrc1wLBAAIaPhXaMFdzoEp6Kh5ptjxQ== 2024-03-11T18:34:38.263 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-11T18:34:38.304 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-11T18:34:38.350 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-11T18:34:38.390 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-11T18:34:38.436 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-11T18:34:38.439 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-11T18:34:38.523 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-11T18:34:38.525 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-11T18:34:38.608 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-11T18:34:38.610 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-11T18:34:38.691 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-11T18:34:38.694 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-11T18:34:38.775 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-11T18:34:38.778 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-11T18:34:38.859 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-11T18:34:38.861 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-11T18:34:38.947 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-11T18:34:38.949 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-11T18:34:39.028 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-11T18:34:39.033 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-11T18:34:39.116 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-11T18:34:39.118 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-11T18:34:39.201 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-11T18:34:39.203 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-11T18:34:39.291 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-11T18:34:39.294 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-11T18:34:39.336 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-11T18:34:39.338 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-11T18:34:39.416 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-11T18:34:39.422 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-11T18:34:39.501 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-11T18:34:39.505 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-11T18:34:39.583 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-11T18:34:39.590 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-11T18:34:39.670 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-03-11T18:34:39.670 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-03-11T18:34:39.683 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-11T18:34:39.769 DEBUG:teuthology.orchestra.run.smithi090:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-03-11T18:34:39.778 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-03-11T18:34:39.792 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-11T18:34:39.889 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-03-11T18:34:39.899 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-03-11T18:34:39.953 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-11T18:34:40.042 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-03-11T18:34:40.053 DEBUG:teuthology.orchestra.run.smithi090:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-11T18:34:40.055 DEBUG:teuthology.orchestra.run.smithi092:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-11T18:34:40.100 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-03-11T18:34:40.101 DEBUG:tasks.ceph_manager:running mon.a under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.a.log', '--time-stamp=yes', '--vgdb=yes', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2024-03-11T18:34:40.101 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-11T18:34:40.101 DEBUG:teuthology.orchestra.run.smithi090:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a 2024-03-11T18:34:40.103 INFO:tasks.ceph.mon.a:Started 2024-03-11T18:34:40.103 DEBUG:tasks.ceph_manager:running mon.b under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.b.log', '--time-stamp=yes', '--vgdb=yes', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2024-03-11T18:34:40.104 INFO:tasks.ceph.mon.b:Restarting daemon 2024-03-11T18:34:40.104 DEBUG:teuthology.orchestra.run.smithi092:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b 2024-03-11T18:34:40.145 INFO:tasks.ceph.mon.b:Started 2024-03-11T18:34:40.145 DEBUG:tasks.ceph_manager:running mon.c under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.c.log', '--time-stamp=yes', '--vgdb=yes', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2024-03-11T18:34:40.146 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-11T18:34:40.146 DEBUG:teuthology.orchestra.run.smithi092:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c 2024-03-11T18:34:40.149 INFO:tasks.ceph.mon.c:Started 2024-03-11T18:34:40.149 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-03-11T18:34:40.149 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-03-11T18:34:40.150 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster ceph -i y 2024-03-11T18:34:40.152 INFO:tasks.ceph.mgr.y:Started 2024-03-11T18:34:40.152 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-03-11T18:34:40.152 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster ceph -i x 2024-03-11T18:34:40.154 INFO:tasks.ceph.mgr.x:Started 2024-03-11T18:34:40.155 INFO:tasks.ceph:Setting crush tunables to default 2024-03-11T18:34:40.155 DEBUG:teuthology.orchestra.run.smithi090:> sudo ceph --cluster ceph osd crush tunables default 2024-03-11T18:34:40.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:40.260+0000 7f21e7afc640 1 Processor -- start 2024-03-11T18:34:40.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:40.260+0000 7f21e7afc640 1 -- start start 2024-03-11T18:34:40.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:40.260+0000 7f21e7afc640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21e01550c0 0x7f21e00c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:34:40.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:40.260+0000 7f21e7afc640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f21e01554a0 0x7f21e015ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:34:40.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:40.260+0000 7f21e7afc640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21e015bfb0 con 0x7f21e006fe60 2024-03-11T18:34:40.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:40.260+0000 7f21e7afc640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21e00c2ae0 con 0x7f21e01554a0 2024-03-11T18:34:40.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:40.260+0000 7f21e7afc640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21e00c2560 con 0x7f21e01550c0 2024-03-11T18:34:40.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:40.260+0000 7f21e5097640 1 -- >> v2:172.21.15.90:3300/0 conn(0x7f21e01554a0 msgr2=0x7f21e015ba70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.90:3300/0 2024-03-11T18:34:40.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:40.260+0000 7f21e6099640 1 -- >> v1:172.21.15.90:6789/0 conn(0x7f21e006fe60 legacy=0x7f21e00c2dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.90:6789/0 2024-03-11T18:34:40.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:40.260+0000 7f21e5097640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f21e01554a0 0x7f21e015ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-11T18:34:40.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:40.260+0000 7f21e5898640 1 -- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21e01550c0 msgr2=0x7f21e00c8fb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3301/0 2024-03-11T18:34:40.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:40.260+0000 7f21e5898640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21e01550c0 0x7f21e00c8fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-11T18:34:40.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:40.464+0000 7f21e6099640 1 -- >> v1:172.21.15.90:6789/0 conn(0x7f21e006fe60 legacy=0x7f21e00c2dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.90:6789/0 2024-03-11T18:34:40.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:40.464+0000 7f21e5097640 1 -- >> v2:172.21.15.90:3300/0 conn(0x7f21e01554a0 msgr2=0x7f21e015ba70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.90:3300/0 2024-03-11T18:34:40.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:40.464+0000 7f21e5097640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f21e01554a0 0x7f21e015ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-11T18:34:40.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:40.464+0000 7f21e5898640 1 -- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21e01550c0 msgr2=0x7f21e00c8fb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3301/0 2024-03-11T18:34:40.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:40.464+0000 7f21e5898640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21e01550c0 0x7f21e00c8fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-11T18:34:40.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:40.868+0000 7f21e6099640 1 -- >> v1:172.21.15.90:6789/0 conn(0x7f21e006fe60 legacy=0x7f21e00c2dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.90:6789/0 2024-03-11T18:34:40.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:40.868+0000 7f21e5097640 1 -- >> v2:172.21.15.90:3300/0 conn(0x7f21e01554a0 msgr2=0x7f21e015ba70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.90:3300/0 2024-03-11T18:34:40.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:40.868+0000 7f21e5097640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f21e01554a0 0x7f21e015ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-11T18:34:40.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:40.868+0000 7f21e5898640 1 -- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21e01550c0 msgr2=0x7f21e00c8fb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3301/0 2024-03-11T18:34:40.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:40.868+0000 7f21e5898640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21e01550c0 0x7f21e00c8fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-11T18:34:41.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:41.672+0000 7f21e6099640 1 -- >> v1:172.21.15.90:6789/0 conn(0x7f21e006fe60 legacy=0x7f21e00c2dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.90:6789/0 2024-03-11T18:34:41.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:41.672+0000 7f21e5097640 1 -- >> v2:172.21.15.90:3300/0 conn(0x7f21e01554a0 msgr2=0x7f21e015ba70 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.90:3300/0 2024-03-11T18:34:41.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:41.672+0000 7f21e5898640 1 -- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21e01550c0 msgr2=0x7f21e00c8fb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3301/0 2024-03-11T18:34:41.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:41.672+0000 7f21e5097640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f21e01554a0 0x7f21e015ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-11T18:34:41.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:41.672+0000 7f21e5898640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21e01550c0 0x7f21e00c8fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-11T18:34:43.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:43.260+0000 7f21df7fe640 1 -- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21e01550c0 msgr2=0x7f21e00c8fb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T18:34:43.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:43.260+0000 7f21df7fe640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21e01550c0 0x7f21e00c8fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:34:43.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:43.260+0000 7f21df7fe640 1 -- >> v2:172.21.15.90:3300/0 conn(0x7f21e01554a0 msgr2=0x7f21e015ba70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T18:34:43.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:43.260+0000 7f21df7fe640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f21e01554a0 0x7f21e015ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:34:43.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:43.260+0000 7f21df7fe640 1 -- >> v1:172.21.15.90:6789/0 conn(0x7f21e006fe60 legacy=0x7f21e00c2dd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T18:34:43.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:43.260+0000 7f21df7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f21cc0036d0 0x7f21cc005b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:34:43.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:43.260+0000 7f21df7fe640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21e0157cc0 con 0x7f21cc0061a0 2024-03-11T18:34:43.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:43.260+0000 7f21df7fe640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21e00c2ae0 con 0x7f21cc0036d0 2024-03-11T18:34:43.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:43.260+0000 7f21e5898640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f21cc0036d0 msgr2=0x7f21cc005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-03-11T18:34:43.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:43.260+0000 7f21e5097640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f21cc0061a0 legacy=0x7f21cc008660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-03-11T18:34:43.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:43.260+0000 7f21e5898640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f21cc0036d0 0x7f21cc005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-11T18:34:43.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:43.464+0000 7f21e5898640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f21cc0036d0 msgr2=0x7f21cc005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-03-11T18:34:43.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:43.464+0000 7f21e5097640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f21cc0061a0 legacy=0x7f21cc008660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-03-11T18:34:43.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:43.464+0000 7f21e5898640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f21cc0036d0 0x7f21cc005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-11T18:34:43.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:43.868+0000 7f21e5898640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f21cc0036d0 msgr2=0x7f21cc005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-03-11T18:34:43.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:43.868+0000 7f21e5097640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f21cc0061a0 legacy=0x7f21cc008660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-03-11T18:34:43.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:43.868+0000 7f21e5898640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f21cc0036d0 0x7f21cc005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-11T18:34:44.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:44.672+0000 7f21e5097640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f21cc0061a0 legacy=0x7f21cc008660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-03-11T18:34:44.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:44.672+0000 7f21e5898640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f21cc0036d0 msgr2=0x7f21cc005b50 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-03-11T18:34:44.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:44.672+0000 7f21e5898640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f21cc0036d0 0x7f21cc005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-11T18:34:46.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:46.260+0000 7f21df7fe640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f21cc0036d0 msgr2=0x7f21cc005b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T18:34:46.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:46.260+0000 7f21df7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f21cc0036d0 0x7f21cc005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:34:46.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:46.260+0000 7f21df7fe640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f21cc0061a0 legacy=0x7f21cc008660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T18:34:46.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:46.260+0000 7f21e6099640 1 -- reap_dead start 2024-03-11T18:34:46.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:46.264+0000 7f21df7fe640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f21cc0099e0 0x7f21cc00e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:34:46.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:46.264+0000 7f21df7fe640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21e015bfb0 con 0x7f21cc005bf0 2024-03-11T18:34:46.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:46.264+0000 7f21df7fe640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21e0157cc0 con 0x7f21cc008e30 2024-03-11T18:34:46.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:46.264+0000 7f21df7fe640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21e00c2ae0 con 0x7f21cc0099e0 2024-03-11T18:34:46.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:46.264+0000 7f21e5097640 1 -- >> v1:172.21.15.90:6789/0 conn(0x7f21cc005bf0 legacy=0x7f21cc008660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.90:6789/0 2024-03-11T18:34:46.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:46.264+0000 7f21e6099640 1 -- >> v2:172.21.15.90:3300/0 conn(0x7f21cc0099e0 msgr2=0x7f21cc00e0f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.90:3300/0 2024-03-11T18:34:46.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:46.264+0000 7f21e5898640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f21cc008e30 legacy=0x7f21cc009210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-03-11T18:34:46.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:46.264+0000 7f21e6099640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f21cc0099e0 0x7f21cc00e0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-11T18:34:46.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:46.468+0000 7f21e6099640 1 -- >> v2:172.21.15.90:3300/0 conn(0x7f21cc0099e0 msgr2=0x7f21cc00e0f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.90:3300/0 2024-03-11T18:34:46.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:46.468+0000 7f21e5097640 1 -- >> v1:172.21.15.90:6789/0 conn(0x7f21cc005bf0 legacy=0x7f21cc008660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.90:6789/0 2024-03-11T18:34:46.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:46.468+0000 7f21e6099640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f21cc0099e0 0x7f21cc00e0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-11T18:34:46.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:46.468+0000 7f21e5898640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f21cc008e30 legacy=0x7f21cc009210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-03-11T18:34:46.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:46.872+0000 7f21e6099640 1 -- >> v2:172.21.15.90:3300/0 conn(0x7f21cc0099e0 msgr2=0x7f21cc00e0f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.90:3300/0 2024-03-11T18:34:46.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:46.872+0000 7f21e5097640 1 -- >> v1:172.21.15.90:6789/0 conn(0x7f21cc005bf0 legacy=0x7f21cc008660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.90:6789/0 2024-03-11T18:34:46.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:46.872+0000 7f21e6099640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f21cc0099e0 0x7f21cc00e0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-11T18:34:46.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:46.872+0000 7f21e5898640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f21cc008e30 legacy=0x7f21cc009210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-03-11T18:34:47.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:47.676+0000 7f21e6099640 1 -- >> v2:172.21.15.90:3300/0 conn(0x7f21cc0099e0 msgr2=0x7f21cc00e0f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.90:3300/0 2024-03-11T18:34:47.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:47.676+0000 7f21e5097640 1 -- >> v1:172.21.15.90:6789/0 conn(0x7f21cc005bf0 legacy=0x7f21cc008660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.90:6789/0 2024-03-11T18:34:47.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:47.676+0000 7f21e6099640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f21cc0099e0 0x7f21cc00e0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-11T18:34:47.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:47.676+0000 7f21e5898640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f21cc008e30 legacy=0x7f21cc009210 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.92:6789/0 2024-03-11T18:34:48.426 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:00:08.185 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T18:34:48.572 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:00:08.258 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T18:34:48.575 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:00:08.270 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T18:34:48.749 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:00:08.508 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T18:34:48.899 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:00:08.585 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T18:34:48.902 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:00:08.596 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T18:34:49.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:49.264+0000 7f21df7fe640 1 -- >> v2:172.21.15.90:3300/0 conn(0x7f21cc0099e0 msgr2=0x7f21cc00e0f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T18:34:49.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:49.264+0000 7f21df7fe640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f21cc0099e0 0x7f21cc00e0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:34:49.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:49.264+0000 7f21df7fe640 1 -- >> v1:172.21.15.92:6789/0 conn(0x7f21cc008e30 legacy=0x7f21cc009210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T18:34:49.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:49.264+0000 7f21df7fe640 1 -- >> v1:172.21.15.90:6789/0 conn(0x7f21cc005bf0 legacy=0x7f21cc008660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T18:34:49.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:49.264+0000 7f21df7fe640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21cc00e8d0 0x7f21cc00ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:34:49.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:49.264+0000 7f21df7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f21cc00f2f0 0x7f21cc00f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:34:49.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:49.264+0000 7f21df7fe640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21e00c2ae0 con 0x7f21cc00f2f0 2024-03-11T18:34:49.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:49.264+0000 7f21df7fe640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21e00c2560 con 0x7f21cc00e8d0 2024-03-11T18:34:49.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:49.264+0000 7f21e5898640 1 -- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21cc00e8d0 msgr2=0x7f21cc00ecb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3301/0 2024-03-11T18:34:49.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:49.264+0000 7f21e5898640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21cc00e8d0 0x7f21cc00ecb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-11T18:34:49.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:49.264+0000 7f21e5097640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f21cc00f2f0 msgr2=0x7f21cc00f870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-03-11T18:34:49.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:49.264+0000 7f21e5097640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f21cc00f2f0 0x7f21cc00f870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-11T18:34:49.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:49.468+0000 7f21e5898640 1 -- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21cc00e8d0 msgr2=0x7f21cc00ecb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3301/0 2024-03-11T18:34:49.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:49.468+0000 7f21e5097640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f21cc00f2f0 msgr2=0x7f21cc00f870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-03-11T18:34:49.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:49.468+0000 7f21e5898640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21cc00e8d0 0x7f21cc00ecb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-11T18:34:49.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:49.468+0000 7f21e5097640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f21cc00f2f0 0x7f21cc00f870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-11T18:34:49.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:49.872+0000 7f21e5097640 1 -- >> v2:172.21.15.92:3300/0 conn(0x7f21cc00f2f0 msgr2=0x7f21cc00f870 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3300/0 2024-03-11T18:34:49.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:49.872+0000 7f21e5898640 1 -- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21cc00e8d0 msgr2=0x7f21cc00ecb0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.92:3301/0 2024-03-11T18:34:49.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:49.872+0000 7f21e5097640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f21cc00f2f0 0x7f21cc00f870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-11T18:34:49.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:49.872+0000 7f21e5898640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21cc00e8d0 0x7f21cc00ecb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-11T18:34:50.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:50.704+0000 7f21e5898640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21cc00e8d0 0x7f21cc00ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:34:50.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:50.720+0000 7f21e5898640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21cc00e8d0 0x7f21cc00ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.90:60836/0 (socket says 172.21.15.90:60836) 2024-03-11T18:34:50.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:50.720+0000 7f21e5898640 1 -- 172.21.15.90:0/2332021069 learned_addr learned my addr 172.21.15.90:0/2332021069 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:34:50.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:50.720+0000 7f21e5898640 1 -- 172.21.15.90:0/2332021069 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21cc00e8d0 msgr2=0x7f21cc00ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-11T18:34:50.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:50.720+0000 7f21e5898640 1 -- 172.21.15.90:0/2332021069 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21cc00e8d0 msgr2=0x7f21cc00ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-11T18:34:50.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:50.720+0000 7f21e5898640 1 --2- 172.21.15.90:0/2332021069 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21cc00e8d0 0x7f21cc00ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-11T18:34:50.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:50.720+0000 7f21e5898640 1 --2- 172.21.15.90:0/2332021069 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21cc00e8d0 0x7f21cc00ecb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-11T18:34:51.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:51.216+0000 7f21e5097640 1 --2- 172.21.15.90:0/2332021069 >> v2:172.21.15.92:3300/0 conn(0x7f21cc00f2f0 0x7f21cc00f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:34:51.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:51.260+0000 7f21e5097640 1 -- 172.21.15.90:0/2332021069 >> v2:172.21.15.92:3300/0 conn(0x7f21cc00f2f0 msgr2=0x7f21cc00f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-11T18:34:51.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:51.260+0000 7f21e5097640 1 -- 172.21.15.90:0/2332021069 >> v2:172.21.15.92:3300/0 conn(0x7f21cc00f2f0 msgr2=0x7f21cc00f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-11T18:34:51.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:51.260+0000 7f21e5097640 1 --2- 172.21.15.90:0/2332021069 >> v2:172.21.15.92:3300/0 conn(0x7f21cc00f2f0 0x7f21cc00f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-11T18:34:51.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:51.260+0000 7f21e5097640 1 --2- 172.21.15.90:0/2332021069 >> v2:172.21.15.92:3300/0 conn(0x7f21cc00f2f0 0x7f21cc00f870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-11T18:34:52.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:52.264+0000 7f21df7fe640 1 -- 172.21.15.90:0/2332021069 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21cc00e8d0 msgr2=0x7f21cc00ecb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T18:34:52.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:52.264+0000 7f21df7fe640 1 --2- 172.21.15.90:0/2332021069 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21cc00e8d0 0x7f21cc00ecb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:34:52.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:52.264+0000 7f21df7fe640 1 -- 172.21.15.90:0/2332021069 >> v2:172.21.15.92:3300/0 conn(0x7f21cc00f2f0 msgr2=0x7f21cc00f870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T18:34:52.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:52.264+0000 7f21df7fe640 1 --2- 172.21.15.90:0/2332021069 >> v2:172.21.15.92:3300/0 conn(0x7f21cc00f2f0 0x7f21cc00f870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:34:52.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:52.264+0000 7f21df7fe640 1 --2- 172.21.15.90:0/2332021069 >> v2:172.21.15.92:3300/0 conn(0x7f21cc00f2f0 0x7f21cc00f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:34:52.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:52.264+0000 7f21df7fe640 1 --2- 172.21.15.90:0/2332021069 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21cc00e8d0 0x7f21cc00ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:34:52.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:52.264+0000 7f21e6099640 1 -- 172.21.15.90:0/2332021069 reap_dead start 2024-03-11T18:34:52.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:52.264+0000 7f21df7fe640 1 -- 172.21.15.90:0/2332021069 --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21cc0108e0 con 0x7f21cc005bf0 2024-03-11T18:34:52.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:52.264+0000 7f21df7fe640 1 -- 172.21.15.90:0/2332021069 --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21e00c2ae0 con 0x7f21cc00f2f0 2024-03-11T18:34:52.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:52.264+0000 7f21df7fe640 1 -- 172.21.15.90:0/2332021069 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21e00c2560 con 0x7f21cc00e8d0 2024-03-11T18:34:52.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:52.284+0000 7f21e6099640 1 --2- 172.21.15.90:0/2332021069 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21cc00e8d0 0x7f21cc00ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:34:52.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:52.288+0000 7f21e6099640 1 -- 172.21.15.90:0/2332021069 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21cc00e8d0 msgr2=0x7f21cc00ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-11T18:34:52.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:52.288+0000 7f21e6099640 1 -- 172.21.15.90:0/2332021069 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21cc00e8d0 msgr2=0x7f21cc00ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-11T18:34:52.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:52.288+0000 7f21e6099640 1 --2- 172.21.15.90:0/2332021069 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21cc00e8d0 0x7f21cc00ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-11T18:34:52.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:52.288+0000 7f21e6099640 1 --2- 172.21.15.90:0/2332021069 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21cc00e8d0 0x7f21cc00ecb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-11T18:34:52.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:52.296+0000 7f21e5097640 1 --2- 172.21.15.90:0/2332021069 >> v2:172.21.15.92:3300/0 conn(0x7f21cc00f2f0 0x7f21cc00f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:34:52.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:52.320+0000 7f21e5097640 1 -- 172.21.15.90:0/2332021069 >> v2:172.21.15.92:3300/0 conn(0x7f21cc00f2f0 msgr2=0x7f21cc00f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-11T18:34:52.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:52.320+0000 7f21e5097640 1 -- 172.21.15.90:0/2332021069 >> v2:172.21.15.92:3300/0 conn(0x7f21cc00f2f0 msgr2=0x7f21cc00f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-11T18:34:52.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:52.320+0000 7f21e5097640 1 --2- 172.21.15.90:0/2332021069 >> v2:172.21.15.92:3300/0 conn(0x7f21cc00f2f0 0x7f21cc00f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-11T18:34:52.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:52.324+0000 7f21e5097640 1 --2- 172.21.15.90:0/2332021069 >> v2:172.21.15.92:3300/0 conn(0x7f21cc00f2f0 0x7f21cc00f830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-11T18:34:52.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:52.492+0000 7f21e6099640 1 --2- 172.21.15.90:0/2332021069 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21cc00e8d0 0x7f21cc00ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:34:52.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:52.496+0000 7f21e6099640 1 -- 172.21.15.90:0/2332021069 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21cc00e8d0 msgr2=0x7f21cc00ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-11T18:34:52.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:52.496+0000 7f21e6099640 1 -- 172.21.15.90:0/2332021069 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21cc00e8d0 msgr2=0x7f21cc00ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-11T18:34:52.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:52.496+0000 7f21e6099640 1 --2- 172.21.15.90:0/2332021069 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21cc00e8d0 0x7f21cc00ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-11T18:34:52.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:52.496+0000 7f21e6099640 1 --2- 172.21.15.90:0/2332021069 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21cc00e8d0 0x7f21cc00ecb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-11T18:34:52.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:52.528+0000 7f21e5097640 1 --2- 172.21.15.90:0/2332021069 >> v2:172.21.15.92:3300/0 conn(0x7f21cc00f2f0 0x7f21cc00f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:34:52.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:52.532+0000 7f21e5097640 1 -- 172.21.15.90:0/2332021069 >> v2:172.21.15.92:3300/0 conn(0x7f21cc00f2f0 msgr2=0x7f21cc00f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-11T18:34:52.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:52.532+0000 7f21e5097640 1 -- 172.21.15.90:0/2332021069 >> v2:172.21.15.92:3300/0 conn(0x7f21cc00f2f0 msgr2=0x7f21cc00f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-11T18:34:52.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:52.532+0000 7f21e5097640 1 --2- 172.21.15.90:0/2332021069 >> v2:172.21.15.92:3300/0 conn(0x7f21cc00f2f0 0x7f21cc00f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-11T18:34:52.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:52.532+0000 7f21e5097640 1 --2- 172.21.15.90:0/2332021069 >> v2:172.21.15.92:3300/0 conn(0x7f21cc00f2f0 0x7f21cc00f830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-11T18:34:52.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:52.900+0000 7f21e6099640 1 --2- 172.21.15.90:0/2332021069 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21cc00e8d0 0x7f21cc00ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:34:52.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:52.904+0000 7f21e6099640 1 -- 172.21.15.90:0/2332021069 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21cc00e8d0 msgr2=0x7f21cc00ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-11T18:34:52.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:52.904+0000 7f21e6099640 1 -- 172.21.15.90:0/2332021069 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21cc00e8d0 msgr2=0x7f21cc00ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-11T18:34:52.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:52.904+0000 7f21e6099640 1 --2- 172.21.15.90:0/2332021069 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21cc00e8d0 0x7f21cc00ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-11T18:34:52.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:52.904+0000 7f21e6099640 1 --2- 172.21.15.90:0/2332021069 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21cc00e8d0 0x7f21cc00ecb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-11T18:34:52.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:52.936+0000 7f21e5097640 1 --2- 172.21.15.90:0/2332021069 >> v2:172.21.15.92:3300/0 conn(0x7f21cc00f2f0 0x7f21cc00f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:34:52.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:52.940+0000 7f21e5097640 1 -- 172.21.15.90:0/2332021069 >> v2:172.21.15.92:3300/0 conn(0x7f21cc00f2f0 msgr2=0x7f21cc00f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-11T18:34:52.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:52.940+0000 7f21e5097640 1 -- 172.21.15.90:0/2332021069 >> v2:172.21.15.92:3300/0 conn(0x7f21cc00f2f0 msgr2=0x7f21cc00f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-11T18:34:52.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:52.940+0000 7f21e5097640 1 --2- 172.21.15.90:0/2332021069 >> v2:172.21.15.92:3300/0 conn(0x7f21cc00f2f0 0x7f21cc00f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-11T18:34:52.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:52.940+0000 7f21e5097640 1 --2- 172.21.15.90:0/2332021069 >> v2:172.21.15.92:3300/0 conn(0x7f21cc00f2f0 0x7f21cc00f830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-11T18:34:53.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:53.716+0000 7f21e6099640 1 --2- 172.21.15.90:0/2332021069 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21cc00e8d0 0x7f21cc00ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:34:53.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:53.720+0000 7f21e6099640 1 -- 172.21.15.90:0/2332021069 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21cc00e8d0 msgr2=0x7f21cc00ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-11T18:34:53.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:53.720+0000 7f21e6099640 1 -- 172.21.15.90:0/2332021069 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21cc00e8d0 msgr2=0x7f21cc00ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-11T18:34:53.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:53.720+0000 7f21e6099640 1 --2- 172.21.15.90:0/2332021069 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21cc00e8d0 0x7f21cc00ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-11T18:34:53.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:53.720+0000 7f21e6099640 1 --2- 172.21.15.90:0/2332021069 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21cc00e8d0 0x7f21cc00ecb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-11T18:34:53.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:53.744+0000 7f21e5097640 1 --2- 172.21.15.90:0/2332021069 >> v2:172.21.15.92:3300/0 conn(0x7f21cc00f2f0 0x7f21cc00f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:34:53.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:53.752+0000 7f21e5097640 1 -- 172.21.15.90:0/2332021069 >> v2:172.21.15.92:3300/0 conn(0x7f21cc00f2f0 msgr2=0x7f21cc00f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-11T18:34:53.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:53.752+0000 7f21e5097640 1 -- 172.21.15.90:0/2332021069 >> v2:172.21.15.92:3300/0 conn(0x7f21cc00f2f0 msgr2=0x7f21cc00f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-11T18:34:53.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:53.752+0000 7f21e5097640 1 --2- 172.21.15.90:0/2332021069 >> v2:172.21.15.92:3300/0 conn(0x7f21cc00f2f0 0x7f21cc00f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-11T18:34:53.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:53.752+0000 7f21e5097640 1 --2- 172.21.15.90:0/2332021069 >> v2:172.21.15.92:3300/0 conn(0x7f21cc00f2f0 0x7f21cc00f830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-11T18:34:55.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:55.264+0000 7f21df7fe640 1 -- 172.21.15.90:0/2332021069 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21cc00e8d0 msgr2=0x7f21cc00ecb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T18:34:55.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:55.264+0000 7f21df7fe640 1 --2- 172.21.15.90:0/2332021069 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21cc00e8d0 0x7f21cc00ecb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:34:55.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:55.264+0000 7f21df7fe640 1 -- 172.21.15.90:0/2332021069 >> v2:172.21.15.92:3300/0 conn(0x7f21cc00f2f0 msgr2=0x7f21cc00f830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-11T18:34:55.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:55.264+0000 7f21df7fe640 1 --2- 172.21.15.90:0/2332021069 >> v2:172.21.15.92:3300/0 conn(0x7f21cc00f2f0 0x7f21cc00f830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:34:55.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:55.264+0000 7f21df7fe640 1 -- 172.21.15.90:0/2332021069 >> v1:172.21.15.90:6789/0 conn(0x7f21cc005bf0 legacy=0x7f21cc008660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:34:55.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:55.264+0000 7f21df7fe640 1 --2- 172.21.15.90:0/2332021069 >> v2:172.21.15.90:3300/0 conn(0x7f21cc00c0e0 0x7f21cc018f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:34:55.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:55.264+0000 7f21df7fe640 1 -- 172.21.15.90:0/2332021069 --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21e015bfb0 con 0x7f21cc009260 2024-03-11T18:34:55.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:55.264+0000 7f21df7fe640 1 -- 172.21.15.90:0/2332021069 --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21e00c2ae0 con 0x7f21cc00c0e0 2024-03-11T18:34:55.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:55.344+0000 7f21e5097640 1 --2- 172.21.15.90:0/2332021069 >> v2:172.21.15.90:3300/0 conn(0x7f21cc00c0e0 0x7f21cc018f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:34:55.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:55.476+0000 7f21e5097640 1 -- 172.21.15.90:0/2332021069 >> v2:172.21.15.90:3300/0 conn(0x7f21cc00c0e0 msgr2=0x7f21cc018f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-11T18:34:55.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:55.476+0000 7f21e5097640 1 -- 172.21.15.90:0/2332021069 >> v2:172.21.15.90:3300/0 conn(0x7f21cc00c0e0 msgr2=0x7f21cc018f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-11T18:34:55.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:55.476+0000 7f21e5097640 1 --2- 172.21.15.90:0/2332021069 >> v2:172.21.15.90:3300/0 conn(0x7f21cc00c0e0 0x7f21cc018f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-11T18:34:55.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:55.476+0000 7f21e5097640 1 --2- 172.21.15.90:0/2332021069 >> v2:172.21.15.90:3300/0 conn(0x7f21cc00c0e0 0x7f21cc018f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-11T18:34:55.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:55.788+0000 7f21e5097640 1 --2- 172.21.15.90:0/2332021069 >> v2:172.21.15.90:3300/0 conn(0x7f21cc00c0e0 0x7f21cc018f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:34:55.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:55.800+0000 7f21e5097640 1 -- 172.21.15.90:0/2332021069 >> v2:172.21.15.90:3300/0 conn(0x7f21cc00c0e0 msgr2=0x7f21cc018f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-11T18:34:55.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:55.800+0000 7f21e5097640 1 -- 172.21.15.90:0/2332021069 >> v2:172.21.15.90:3300/0 conn(0x7f21cc00c0e0 msgr2=0x7f21cc018f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-11T18:34:55.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:55.800+0000 7f21e5097640 1 --2- 172.21.15.90:0/2332021069 >> v2:172.21.15.90:3300/0 conn(0x7f21cc00c0e0 0x7f21cc018f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-11T18:34:55.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:55.800+0000 7f21e5097640 1 --2- 172.21.15.90:0/2332021069 >> v2:172.21.15.90:3300/0 conn(0x7f21cc00c0e0 0x7f21cc018f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-11T18:34:56.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.100+0000 7f21e4896640 1 -- 172.21.15.90:0/2332021069 <== mon.1 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3411551891 0 0) 0x7f21e015bfb0 con 0x7f21cc009260 2024-03-11T18:34:56.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.100+0000 7f21e4896640 1 -- 172.21.15.90:0/2332021069 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21c8003530 con 0x7f21cc009260 2024-03-11T18:34:56.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.284+0000 7f21e5097640 1 --2- 172.21.15.90:0/2332021069 >> v2:172.21.15.90:3300/0 conn(0x7f21cc00c0e0 0x7f21cc018f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:34:56.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.304+0000 7f21e4896640 1 -- 172.21.15.90:0/2332021069 <== mon.1 v1:172.21.15.92:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1266540286 0 0) 0x7f21c8003530 con 0x7f21cc009260 2024-03-11T18:34:56.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.304+0000 7f21e4896640 1 -- 172.21.15.90:0/2332021069 >> v2:172.21.15.90:3300/0 conn(0x7f21cc00c0e0 msgr2=0x7f21cc018f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:34:56.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.304+0000 7f21e4896640 1 --2- 172.21.15.90:0/2332021069 >> v2:172.21.15.90:3300/0 conn(0x7f21cc00c0e0 0x7f21cc018f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:34:56.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.304+0000 7f21e4896640 1 -- 172.21.15.90:0/2332021069 --> v1:172.21.15.92:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e0157cc0 con 0x7f21cc009260 2024-03-11T18:34:56.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.304+0000 7f21e4896640 1 -- 172.21.15.90:0/2332021069 <== mon.1 v1:172.21.15.92:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 691735435 0 0) 0x7f21d80099b0 con 0x7f21cc009260 2024-03-11T18:34:56.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.304+0000 7f21e4896640 1 -- 172.21.15.90:0/2332021069 >> v1:172.21.15.92:6789/0 conn(0x7f21cc009260 legacy=0x7f21cc00fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:34:56.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.304+0000 7f21e6099640 1 -- 172.21.15.90:0/2332021069 reap_dead start 2024-03-11T18:34:56.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.304+0000 7f21e4896640 1 --2- 172.21.15.90:0/2332021069 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f21c8003980 0x7f21c8003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:34:56.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.304+0000 7f21e4896640 1 --2- 172.21.15.90:0/2332021069 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21cc00e8d0 0x7f21c80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:34:56.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.304+0000 7f21e4896640 1 --2- 172.21.15.90:0/2332021069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21c8004d10 0x7f21c8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:34:56.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.304+0000 7f21e4896640 1 -- 172.21.15.90:0/2332021069 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e00c2ae0 con 0x7f21c8003980 2024-03-11T18:34:56.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.304+0000 7f21e4896640 1 -- 172.21.15.90:0/2332021069 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c800bb20 con 0x7f21c8004d10 2024-03-11T18:34:56.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.304+0000 7f21e4896640 1 -- 172.21.15.90:0/2332021069 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21c800bce0 con 0x7f21cc00e8d0 2024-03-11T18:34:56.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.328+0000 7f21e6099640 1 --2- 172.21.15.90:0/2332021069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21c8004d10 0x7f21c8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:34:56.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.368+0000 7f21e6099640 1 -- 172.21.15.90:0/2332021069 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21cc00e8d0 msgr2=0x7f21c80047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:34:56.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.368+0000 7f21e6099640 1 --2- 172.21.15.90:0/2332021069 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21cc00e8d0 0x7f21c80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:34:56.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.368+0000 7f21e6099640 1 -- 172.21.15.90:0/2332021069 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f21c8003980 msgr2=0x7f21c8003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:34:56.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.368+0000 7f21e6099640 1 --2- 172.21.15.90:0/2332021069 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f21c8003980 0x7f21c8003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:34:56.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.368+0000 7f21e6099640 1 -- 172.21.15.90:0/2332021069 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c8003530 con 0x7f21c8004d10 2024-03-11T18:34:56.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.376+0000 7f21e6099640 1 --2- 172.21.15.90:0/2332021069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21c8004d10 0x7f21c8005190 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f21e00caa30 tx=0x7f21d001da50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbe31e0048045faf server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:34:56.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.388+0000 7f21e4896640 1 -- 172.21.15.90:0/2332021069 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d000f2d0 con 0x7f21c8004d10 2024-03-11T18:34:56.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.388+0000 7f21e4896640 1 -- 172.21.15.90:0/2332021069 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21d00a8050 con 0x7f21c8004d10 2024-03-11T18:34:56.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.388+0000 7f21e4896640 1 -- 172.21.15.90:0/2332021069 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d000d630 con 0x7f21c8004d10 2024-03-11T18:34:56.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.388+0000 7f21e7afc640 1 -- 172.21.15.90:0/2332021069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21c8004d10 msgr2=0x7f21c8005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:34:56.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.388+0000 7f21e7afc640 1 --2- 172.21.15.90:0/2332021069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21c8004d10 0x7f21c8005190 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f21e00caa30 tx=0x7f21d001da50 comp rx=0 tx=0).stop 2024-03-11T18:34:56.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.388+0000 7f21e7afc640 1 -- 172.21.15.90:0/2332021069 shutdown_connections 2024-03-11T18:34:56.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.388+0000 7f21e7afc640 1 --2- 172.21.15.90:0/2332021069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21c8004d10 0x7f21c8005190 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:34:56.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.388+0000 7f21e7afc640 1 --2- 172.21.15.90:0/2332021069 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21cc00e8d0 0x7f21c80047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:34:56.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.388+0000 7f21e7afc640 1 --2- 172.21.15.90:0/2332021069 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f21c8003980 0x7f21c8003d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:34:56.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.388+0000 7f21e7afc640 1 -- 172.21.15.90:0/2332021069 >> 172.21.15.90:0/2332021069 conn(0x7f21e0070e80 msgr2=0x7f21e0066990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:34:56.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.388+0000 7f21e7afc640 1 -- 172.21.15.90:0/2332021069 shutdown_connections 2024-03-11T18:34:56.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.388+0000 7f21e7afc640 1 -- 172.21.15.90:0/2332021069 wait complete. 2024-03-11T18:34:56.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.392+0000 7f21e7afc640 1 Processor -- start 2024-03-11T18:34:56.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.392+0000 7f21e7afc640 1 -- start start 2024-03-11T18:34:56.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.396+0000 7f21e7afc640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f21c8003980 0x7f21e00cb420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:34:56.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.396+0000 7f21e7afc640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21c8004d10 0x7f21e015ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:34:56.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.396+0000 7f21e7afc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21cc00e8d0 0x7f21e0150aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:34:56.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.396+0000 7f21e7afc640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e015a950 con 0x7f21c8003980 2024-03-11T18:34:56.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.396+0000 7f21e7afc640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e0159d40 con 0x7f21cc00e8d0 2024-03-11T18:34:56.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.396+0000 7f21e7afc640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21e015a020 con 0x7f21c8004d10 2024-03-11T18:34:56.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.396+0000 7f21e6099640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21cc00e8d0 0x7f21e0150aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:34:56.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.396+0000 7f21e6099640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21cc00e8d0 0x7f21e0150aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.90:54236/0 (socket says 172.21.15.90:54236) 2024-03-11T18:34:56.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.396+0000 7f21e6099640 1 -- 172.21.15.90:0/1875335159 learned_addr learned my addr 172.21.15.90:0/1875335159 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:34:56.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.420+0000 7f21e6099640 1 -- 172.21.15.90:0/1875335159 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21c8004d10 msgr2=0x7f21e015ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:34:56.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.420+0000 7f21e6099640 1 --2- 172.21.15.90:0/1875335159 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21c8004d10 0x7f21e015ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:34:56.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.420+0000 7f21e6099640 1 -- 172.21.15.90:0/1875335159 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f21c8003980 msgr2=0x7f21e00cb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:34:56.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.420+0000 7f21e6099640 1 --2- 172.21.15.90:0/1875335159 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f21c8003980 0x7f21e00cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:34:56.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.420+0000 7f21e6099640 1 -- 172.21.15.90:0/1875335159 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e015a300 con 0x7f21cc00e8d0 2024-03-11T18:34:56.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.424+0000 7f21e6099640 1 --2- 172.21.15.90:0/1875335159 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21cc00e8d0 0x7f21e0150aa0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f21d001e2f0 tx=0x7f21c8005800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:34:56.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.428+0000 7f21de7fc640 1 -- 172.21.15.90:0/1875335159 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d000f2d0 con 0x7f21cc00e8d0 2024-03-11T18:34:56.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.432+0000 7f21e7afc640 1 -- 172.21.15.90:0/1875335159 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21e015a5e0 con 0x7f21cc00e8d0 2024-03-11T18:34:56.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.432+0000 7f21e7afc640 1 -- 172.21.15.90:0/1875335159 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21e01f23e0 con 0x7f21cc00e8d0 2024-03-11T18:34:56.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.432+0000 7f21e7afc640 1 -- 172.21.15.90:0/1875335159 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21e004eef0 con 0x7f21cc00e8d0 2024-03-11T18:34:56.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.464+0000 7f21de7fc640 1 -- 172.21.15.90:0/1875335159 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21d000d9f0 con 0x7f21cc00e8d0 2024-03-11T18:34:56.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.464+0000 7f21de7fc640 1 -- 172.21.15.90:0/1875335159 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d000dd60 con 0x7f21cc00e8d0 2024-03-11T18:34:56.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.468+0000 7f21de7fc640 1 -- 172.21.15.90:0/1875335159 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f21d00b3040 con 0x7f21cc00e8d0 2024-03-11T18:34:56.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.468+0000 7f21de7fc640 1 -- 172.21.15.90:0/1875335159 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f21d00b5400 con 0x7f21cc00e8d0 2024-03-11T18:34:56.670 INFO:tasks.ceph.mgr.y.smithi090.stderr:2024-03-11T18:34:56.664+0000 7f7a8cf3c180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-11T18:34:56.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.680+0000 7f21de7fc640 1 -- 172.21.15.90:0/1875335159 <== mon.1 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f21d00b1070 con 0x7f21cc00e8d0 2024-03-11T18:34:56.740 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-03-11T18:34:56.738+0000 7fc564b0c180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-11T18:34:56.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:56.796+0000 7f21e7afc640 1 -- 172.21.15.90:0/1875335159 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f21e00b3490 con 0x7f21cc00e8d0 2024-03-11T18:34:57.136 INFO:tasks.ceph.mgr.y.smithi090.stderr:2024-03-11T18:34:57.132+0000 7f7a8cf3c180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-11T18:34:57.245 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-03-11T18:34:57.242+0000 7fc564b0c180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-11T18:34:57.326 INFO:tasks.ceph.mgr.y.smithi090.stderr:2024-03-11T18:34:57.320+0000 7f7a8cf3c180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-11T18:34:57.438 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-03-11T18:34:57.434+0000 7fc564b0c180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-11T18:34:57.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:57.512+0000 7f21de7fc640 1 -- 172.21.15.90:0/1875335159 <== mon.1 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f21d0009a90 con 0x7f21cc00e8d0 2024-03-11T18:34:57.517 INFO:teuthology.orchestra.run.smithi090.stderr:adjusted tunables profile to default 2024-03-11T18:34:57.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:57.512+0000 7f21e7afc640 1 -- 172.21.15.90:0/1875335159 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21cc00e8d0 msgr2=0x7f21e0150aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:34:57.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:57.512+0000 7f21e7afc640 1 --2- 172.21.15.90:0/1875335159 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21cc00e8d0 0x7f21e0150aa0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f21d001e2f0 tx=0x7f21c8005800 comp rx=0 tx=0).stop 2024-03-11T18:34:57.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:57.512+0000 7f21e7afc640 1 -- 172.21.15.90:0/1875335159 shutdown_connections 2024-03-11T18:34:57.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:57.512+0000 7f21e7afc640 1 --2- 172.21.15.90:0/1875335159 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f21cc00e8d0 0x7f21e0150aa0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:34:57.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:57.512+0000 7f21e7afc640 1 --2- 172.21.15.90:0/1875335159 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f21c8004d10 0x7f21e015ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:34:57.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:57.512+0000 7f21e7afc640 1 --2- 172.21.15.90:0/1875335159 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f21c8003980 0x7f21e00cb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:34:57.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:57.512+0000 7f21e7afc640 1 -- 172.21.15.90:0/1875335159 >> 172.21.15.90:0/1875335159 conn(0x7f21e0070e80 msgr2=0x7f21e0153750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:34:57.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:57.512+0000 7f21e7afc640 1 -- 172.21.15.90:0/1875335159 shutdown_connections 2024-03-11T18:34:57.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:34:57.512+0000 7f21e7afc640 1 -- 172.21.15.90:0/1875335159 wait complete. 2024-03-11T18:34:57.537 INFO:tasks.ceph:check_enable_crimson: False 2024-03-11T18:34:57.537 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-03-11T18:34:57.537 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-11T18:34:57.537 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-03-11T18:34:57.548 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-11T18:34:57.548 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-03-11T18:34:57.604 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-11T18:34:57.604 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-03-11T18:34:57.659 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-11T18:34:57.660 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-03-11T18:34:57.712 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-11T18:34:57.712 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-03-11T18:34:57.726 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-11T18:34:57.726 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-03-11T18:34:57.779 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-11T18:34:57.780 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-03-11T18:34:57.835 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2024-03-11T18:34:57.835 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-03-11T18:34:57.872 INFO:tasks.ceph.mgr.y.smithi090.stderr:2024-03-11T18:34:57.868+0000 7f7a8cf3c180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-11T18:34:57.891 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new 34decff9-7280-4d84-9665-d6af8ea6de84 0 2024-03-11T18:34:57.925 INFO:tasks.ceph.mgr.y.smithi090.stderr:2024-03-11T18:34:57.920+0000 7f7a8cf3c180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-11T18:34:57.976 INFO:tasks.ceph.mgr.y.smithi090.stderr:2024-03-11T18:34:57.972+0000 7f7a8cf3c180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-11T18:34:57.990 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-03-11T18:34:57.986+0000 7fc564b0c180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-11T18:34:58.045 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-03-11T18:34:58.042+0000 7fc564b0c180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-11T18:34:58.052 INFO:tasks.ceph.mgr.y.smithi090.stderr:2024-03-11T18:34:58.044+0000 7f7a8cf3c180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-11T18:34:58.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.050+0000 7f55fee64640 1 Processor -- start 2024-03-11T18:34:58.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.050+0000 7f55fee64640 1 -- start start 2024-03-11T18:34:58.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.050+0000 7f55fee64640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f55f81550c0 0x7f55f80c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:34:58.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.050+0000 7f55fee64640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f55f806fe60 0x7f55f80c2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:34:58.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.050+0000 7f55fee64640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55f80c2ae0 con 0x7f55f81554a0 2024-03-11T18:34:58.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.050+0000 7f55fee64640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55f80c2560 con 0x7f55f806fe60 2024-03-11T18:34:58.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.050+0000 7f55fee64640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f8157cc0 con 0x7f55f81550c0 2024-03-11T18:34:58.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.054+0000 7f55fcc00640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f55f81550c0 0x7f55f80c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:34:58.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.054+0000 7f55fd401640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f55f806fe60 0x7f55f80c2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:34:58.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.054+0000 7f55fcc00640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f55f81550c0 0x7f55f80c8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.92:41822/0 (socket says 172.21.15.92:41822) 2024-03-11T18:34:58.056 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.054+0000 7f55fcc00640 1 -- 172.21.15.92:0/2008579899 learned_addr learned my addr 172.21.15.92:0/2008579899 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-11T18:34:58.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.054+0000 7f55fcc00640 1 -- 172.21.15.92:0/2008579899 >> v2:172.21.15.92:3300/0 conn(0x7f55f806fe60 msgr2=0x7f55f80c2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:34:58.058 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.054+0000 7f55fcc00640 1 --2- 172.21.15.92:0/2008579899 >> v2:172.21.15.92:3300/0 conn(0x7f55f806fe60 0x7f55f80c2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:34:58.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.054+0000 7f55fcc00640 1 -- 172.21.15.92:0/2008579899 >> v1:172.21.15.90:6789/0 conn(0x7f55f81554a0 legacy=0x7f55f815ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:34:58.059 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.054+0000 7f55fcc00640 1 -- 172.21.15.92:0/2008579899 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f80c2800 con 0x7f55f81550c0 2024-03-11T18:34:58.062 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.058+0000 7f55fcc00640 1 --2- 172.21.15.92:0/2008579899 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f55f81550c0 0x7f55f80c8fb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f55ec009870 tx=0x7f55ec01c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7042f0f8b4d51e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:34:58.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.066+0000 7f55f77fe640 1 -- 172.21.15.92:0/2008579899 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec0ac620 con 0x7f55f81550c0 2024-03-11T18:34:58.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.066+0000 7f55f77fe640 1 -- 172.21.15.92:0/2008579899 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55ec0ac780 con 0x7f55f81550c0 2024-03-11T18:34:58.068 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.066+0000 7f55f77fe640 1 -- 172.21.15.92:0/2008579899 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec0acb10 con 0x7f55f81550c0 2024-03-11T18:34:58.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.066+0000 7f55fee64640 1 -- 172.21.15.92:0/2008579899 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f55f81550c0 msgr2=0x7f55f80c8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:34:58.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.066+0000 7f55fee64640 1 --2- 172.21.15.92:0/2008579899 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f55f81550c0 0x7f55f80c8fb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f55ec009870 tx=0x7f55ec01c4d0 comp rx=0 tx=0).stop 2024-03-11T18:34:58.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.066+0000 7f55fee64640 1 -- 172.21.15.92:0/2008579899 shutdown_connections 2024-03-11T18:34:58.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.066+0000 7f55fee64640 1 --2- 172.21.15.92:0/2008579899 >> v2:172.21.15.92:3300/0 conn(0x7f55f806fe60 0x7f55f80c2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:34:58.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.066+0000 7f55fee64640 1 --2- 172.21.15.92:0/2008579899 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f55f81550c0 0x7f55f80c8fb0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:34:58.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.066+0000 7f55fee64640 1 -- 172.21.15.92:0/2008579899 >> 172.21.15.92:0/2008579899 conn(0x7f55f8070e80 msgr2=0x7f55f8066990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:34:58.070 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.066+0000 7f55fee64640 1 -- 172.21.15.92:0/2008579899 shutdown_connections 2024-03-11T18:34:58.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.066+0000 7f55fee64640 1 -- 172.21.15.92:0/2008579899 wait complete. 2024-03-11T18:34:58.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.066+0000 7f55fee64640 1 Processor -- start 2024-03-11T18:34:58.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.066+0000 7f55fee64640 1 -- start start 2024-03-11T18:34:58.071 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.070+0000 7f55fee64640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f55f806fe60 0x7f55f80c6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:34:58.072 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.070+0000 7f55fee64640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55f81550c0 0x7f55f80c5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:34:58.072 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.070+0000 7f55fee64640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f55f81554a0 0x7f55f814b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:34:58.072 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.070+0000 7f55fee64640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f814aca0 con 0x7f55f806fe60 2024-03-11T18:34:58.072 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.070+0000 7f55fee64640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f80c5c00 con 0x7f55f81550c0 2024-03-11T18:34:58.072 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.070+0000 7f55fee64640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f80c5000 con 0x7f55f81554a0 2024-03-11T18:34:58.072 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.070+0000 7f55f7fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55f81550c0 0x7f55f80c5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:34:58.073 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.070+0000 7f55fd401640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f55f81554a0 0x7f55f814b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:34:58.073 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.070+0000 7f55fd401640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f55f81554a0 0x7f55f814b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.92:41834/0 (socket says 172.21.15.92:41834) 2024-03-11T18:34:58.073 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.070+0000 7f55fd401640 1 -- 172.21.15.92:0/3015673635 learned_addr learned my addr 172.21.15.92:0/3015673635 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-11T18:34:58.073 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.070+0000 7f55fcc00640 1 --2- 172.21.15.92:0/3015673635 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f55f806fe60 0x7f55f80c6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:34:58.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.074+0000 7f55fd401640 1 -- 172.21.15.92:0/3015673635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55f81550c0 msgr2=0x7f55f80c5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:34:58.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.074+0000 7f55fd401640 1 --2- 172.21.15.92:0/3015673635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55f81550c0 0x7f55f80c5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:34:58.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.074+0000 7f55fd401640 1 -- 172.21.15.92:0/3015673635 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f55f806fe60 msgr2=0x7f55f80c6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:34:58.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.074+0000 7f55fd401640 1 --2- 172.21.15.92:0/3015673635 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f55f806fe60 0x7f55f80c6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:34:58.076 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.074+0000 7f55fd401640 1 -- 172.21.15.92:0/3015673635 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55ec0094d0 con 0x7f55f81554a0 2024-03-11T18:34:58.078 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.074+0000 7f55fd401640 1 --2- 172.21.15.92:0/3015673635 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f55f81554a0 0x7f55f814b0c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f55f0009cb0 tx=0x7f55f000c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:34:58.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.078+0000 7f55f57fa640 1 -- 172.21.15.92:0/3015673635 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f0009b20 con 0x7f55f81554a0 2024-03-11T18:34:58.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.078+0000 7f55f57fa640 1 -- 172.21.15.92:0/3015673635 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55f000f040 con 0x7f55f81554a0 2024-03-11T18:34:58.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.078+0000 7f55f57fa640 1 -- 172.21.15.92:0/3015673635 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f00146c0 con 0x7f55f81554a0 2024-03-11T18:34:58.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.078+0000 7f55fee64640 1 -- 172.21.15.92:0/3015673635 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55f814a5e0 con 0x7f55f81554a0 2024-03-11T18:34:58.082 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.078+0000 7f55fee64640 1 -- 172.21.15.92:0/3015673635 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f81547b0 con 0x7f55f81554a0 2024-03-11T18:34:58.083 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.078+0000 7f55fee64640 1 -- 172.21.15.92:0/3015673635 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55f814ade0 con 0x7f55f81554a0 2024-03-11T18:34:58.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.086+0000 7f55f57fa640 1 -- 172.21.15.92:0/3015673635 <== mon.2 v2:172.21.15.92:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f55f0014bb0 con 0x7f55f81554a0 2024-03-11T18:34:58.089 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.086+0000 7f55f57fa640 1 -- 172.21.15.92:0/3015673635 <== mon.2 v2:172.21.15.92:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f55f00263f0 con 0x7f55f81554a0 2024-03-11T18:34:58.102 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-03-11T18:34:58.098+0000 7fc564b0c180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-11T18:34:58.107 INFO:tasks.ceph.mgr.y.smithi090.stderr:2024-03-11T18:34:58.100+0000 7f7a8cf3c180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-11T18:34:58.180 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-03-11T18:34:58.178+0000 7fc564b0c180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-11T18:34:58.181 INFO:tasks.ceph.mgr.y.smithi090.stderr:2024-03-11T18:34:58.176+0000 7f7a8cf3c180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-11T18:34:58.232 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-03-11T18:34:58.230+0000 7fc564b0c180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-11T18:34:58.233 INFO:tasks.ceph.mgr.y.smithi090.stderr:2024-03-11T18:34:58.228+0000 7f7a8cf3c180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-11T18:34:58.283 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.282+0000 7f55f57fa640 1 -- 172.21.15.92:0/3015673635 <== mon.2 v2:172.21.15.92:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f55f003bdd0 con 0x7f55f81554a0 2024-03-11T18:34:58.313 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-03-11T18:34:58.310+0000 7fc564b0c180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-11T18:34:58.369 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-03-11T18:34:58.366+0000 7fc564b0c180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-11T18:34:58.373 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.370+0000 7f55fee64640 1 -- 172.21.15.92:0/3015673635 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "osd new", "uuid": "34decff9-7280-4d84-9665-d6af8ea6de84", "id": 0} v 0) v1 -- 0x7f55f80c5270 con 0x7f55f81554a0 2024-03-11T18:34:58.382 INFO:tasks.ceph.mgr.y.smithi090.stderr:2024-03-11T18:34:58.376+0000 7f7a8cf3c180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-11T18:34:58.444 INFO:tasks.ceph.mgr.y.smithi090.stderr:2024-03-11T18:34:58.440+0000 7f7a8cf3c180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-11T18:34:58.522 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-03-11T18:34:58.518+0000 7fc564b0c180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-11T18:34:58.583 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-03-11T18:34:58.582+0000 7fc564b0c180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-11T18:34:58.847 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.842+0000 7f55f57fa640 1 -- 172.21.15.92:0/3015673635 <== mon.2 v2:172.21.15.92:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "34decff9-7280-4d84-9665-d6af8ea6de84", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f55f002a030 con 0x7f55f81554a0 2024-03-11T18:34:58.847 INFO:teuthology.orchestra.run.smithi092.stdout:0 2024-03-11T18:34:58.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.846+0000 7f55fee64640 1 -- 172.21.15.92:0/3015673635 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f55f81554a0 msgr2=0x7f55f814b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:34:58.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.846+0000 7f55fee64640 1 --2- 172.21.15.92:0/3015673635 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f55f81554a0 0x7f55f814b0c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f55f0009cb0 tx=0x7f55f000c490 comp rx=0 tx=0).stop 2024-03-11T18:34:58.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.846+0000 7f55fee64640 1 -- 172.21.15.92:0/3015673635 shutdown_connections 2024-03-11T18:34:58.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.846+0000 7f55fee64640 1 --2- 172.21.15.92:0/3015673635 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f55f81554a0 0x7f55f814b0c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:34:58.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.846+0000 7f55fee64640 1 --2- 172.21.15.92:0/3015673635 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f55f81550c0 0x7f55f80c5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:34:58.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.846+0000 7f55fee64640 1 --2- 172.21.15.92:0/3015673635 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f55f806fe60 0x7f55f80c6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:34:58.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.846+0000 7f55fee64640 1 -- 172.21.15.92:0/3015673635 >> 172.21.15.92:0/3015673635 conn(0x7f55f8070e80 msgr2=0x7f55f8153f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:34:58.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.846+0000 7f55fee64640 1 -- 172.21.15.92:0/3015673635 shutdown_connections 2024-03-11T18:34:58.849 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:58.846+0000 7f55fee64640 1 -- 172.21.15.92:0/3015673635 wait complete. 2024-03-11T18:34:58.871 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new 8ecdce9e-5486-415a-a609-dd16e945440e 1 2024-03-11T18:34:59.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.026+0000 7f38d543c640 1 Processor -- start 2024-03-11T18:34:59.030 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.026+0000 7f38d543c640 1 -- start start 2024-03-11T18:34:59.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.026+0000 7f38d543c640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f38d0151b30 0x7f38d006e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:34:59.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.026+0000 7f38d543c640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f38d0070b00 0x7f38d00c5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:34:59.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.026+0000 7f38d543c640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38d006f430 con 0x7f38d00c4af0 2024-03-11T18:34:59.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.026+0000 7f38d543c640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38d00c65b0 con 0x7f38d0070b00 2024-03-11T18:34:59.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.026+0000 7f38d543c640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d00c6320 con 0x7f38d0151b30 2024-03-11T18:34:59.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.026+0000 7f38ceffd640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f38d0151b30 0x7f38d006e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:34:59.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.030+0000 7f38ceffd640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f38d0151b30 0x7f38d006e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.92:41850/0 (socket says 172.21.15.92:41850) 2024-03-11T18:34:59.031 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.030+0000 7f38ceffd640 1 -- 172.21.15.92:0/1409060847 learned_addr learned my addr 172.21.15.92:0/1409060847 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-11T18:34:59.032 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.030+0000 7f38ce7fc640 1 --2- 172.21.15.92:0/1409060847 >> v2:172.21.15.90:3300/0 conn(0x7f38d0070b00 0x7f38d00c5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:34:59.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.030+0000 7f38ceffd640 1 -- 172.21.15.92:0/1409060847 >> v2:172.21.15.90:3300/0 conn(0x7f38d0070b00 msgr2=0x7f38d00c5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:34:59.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.030+0000 7f38ceffd640 1 --2- 172.21.15.92:0/1409060847 >> v2:172.21.15.90:3300/0 conn(0x7f38d0070b00 0x7f38d00c5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:34:59.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.030+0000 7f38ceffd640 1 -- 172.21.15.92:0/1409060847 >> v1:172.21.15.90:6789/0 conn(0x7f38d00c4af0 legacy=0x7f38d0157c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:34:59.034 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.030+0000 7f38ceffd640 1 -- 172.21.15.92:0/1409060847 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d006e820 con 0x7f38d0151b30 2024-03-11T18:34:59.037 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.034+0000 7f38ceffd640 1 --2- 172.21.15.92:0/1409060847 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f38d0151b30 0x7f38d006e210 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f38c0009870 tx=0x7f38c001c4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5fab04662aed654 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:34:59.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.038+0000 7f38cdffb640 1 -- 172.21.15.92:0/1409060847 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c00aa610 con 0x7f38d0151b30 2024-03-11T18:34:59.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.038+0000 7f38cdffb640 1 -- 172.21.15.92:0/1409060847 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c00a6040 con 0x7f38d0151b30 2024-03-11T18:34:59.041 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.038+0000 7f38cdffb640 1 -- 172.21.15.92:0/1409060847 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c00aa990 con 0x7f38d0151b30 2024-03-11T18:34:59.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.038+0000 7f38d543c640 1 -- 172.21.15.92:0/1409060847 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f38d0151b30 msgr2=0x7f38d006e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:34:59.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.038+0000 7f38d543c640 1 --2- 172.21.15.92:0/1409060847 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f38d0151b30 0x7f38d006e210 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f38c0009870 tx=0x7f38c001c4f0 comp rx=0 tx=0).stop 2024-03-11T18:34:59.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.038+0000 7f38d543c640 1 -- 172.21.15.92:0/1409060847 shutdown_connections 2024-03-11T18:34:59.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.038+0000 7f38d543c640 1 --2- 172.21.15.92:0/1409060847 >> v2:172.21.15.90:3300/0 conn(0x7f38d0070b00 0x7f38d00c5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:34:59.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.038+0000 7f38d543c640 1 --2- 172.21.15.92:0/1409060847 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f38d0151b30 0x7f38d006e210 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:34:59.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.038+0000 7f38d543c640 1 -- 172.21.15.92:0/1409060847 >> 172.21.15.92:0/1409060847 conn(0x7f38d006ac00 msgr2=0x7f38d006b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:34:59.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.038+0000 7f38d543c640 1 -- 172.21.15.92:0/1409060847 shutdown_connections 2024-03-11T18:34:59.042 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.038+0000 7f38d543c640 1 -- 172.21.15.92:0/1409060847 wait complete. 2024-03-11T18:34:59.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.042+0000 7f38d543c640 1 Processor -- start 2024-03-11T18:34:59.043 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.042+0000 7f38d543c640 1 -- start start 2024-03-11T18:34:59.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.042+0000 7f38d543c640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f38d0070b00 0x7f38d0149e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:34:59.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.042+0000 7f38d543c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38d00c4af0 0x7f38d014f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:34:59.044 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.042+0000 7f38d543c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f38d0151b30 0x7f38d00ca4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:34:59.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.042+0000 7f38d543c640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d01f0c80 con 0x7f38d0151b30 2024-03-11T18:34:59.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.042+0000 7f38d543c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d00cb010 con 0x7f38d00c4af0 2024-03-11T18:34:59.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.042+0000 7f38d543c640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d014ddd0 con 0x7f38d0070b00 2024-03-11T18:34:59.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.042+0000 7f38ceffd640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f38d0070b00 0x7f38d0149e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:34:59.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.042+0000 7f38ce7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38d00c4af0 0x7f38d014f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:34:59.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.042+0000 7f38ceffd640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f38d0070b00 0x7f38d0149e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.92:41856/0 (socket says 172.21.15.92:41856) 2024-03-11T18:34:59.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.042+0000 7f38ceffd640 1 -- 172.21.15.92:0/1581206761 learned_addr learned my addr 172.21.15.92:0/1581206761 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-11T18:34:59.045 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.042+0000 7f38cf7fe640 1 --2- 172.21.15.92:0/1581206761 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f38d0151b30 0x7f38d00ca4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:34:59.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.046+0000 7f38ceffd640 1 -- 172.21.15.92:0/1581206761 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38d00c4af0 msgr2=0x7f38d014f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:34:59.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.046+0000 7f38ceffd640 1 --2- 172.21.15.92:0/1581206761 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38d00c4af0 0x7f38d014f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:34:59.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.046+0000 7f38ceffd640 1 -- 172.21.15.92:0/1581206761 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f38d0151b30 msgr2=0x7f38d00ca4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:34:59.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.046+0000 7f38ceffd640 1 --2- 172.21.15.92:0/1581206761 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f38d0151b30 0x7f38d00ca4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:34:59.048 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.046+0000 7f38ceffd640 1 -- 172.21.15.92:0/1581206761 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c00094d0 con 0x7f38d0070b00 2024-03-11T18:34:59.050 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.046+0000 7f38ceffd640 1 --2- 172.21.15.92:0/1581206761 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f38d0070b00 0x7f38d0149e40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f38c001aba0 tx=0x7f38c001ad30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:34:59.054 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.050+0000 7f38b77fe640 1 -- 172.21.15.92:0/1581206761 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c00bd5f0 con 0x7f38d0070b00 2024-03-11T18:34:59.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.050+0000 7f38b77fe640 1 -- 172.21.15.92:0/1581206761 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c00aae90 con 0x7f38d0070b00 2024-03-11T18:34:59.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.050+0000 7f38d543c640 1 -- 172.21.15.92:0/1581206761 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38d01f0e90 con 0x7f38d0070b00 2024-03-11T18:34:59.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.050+0000 7f38b77fe640 1 -- 172.21.15.92:0/1581206761 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c00bd990 con 0x7f38d0070b00 2024-03-11T18:34:59.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.050+0000 7f38d543c640 1 -- 172.21.15.92:0/1581206761 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38d01f1450 con 0x7f38d0070b00 2024-03-11T18:34:59.055 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.054+0000 7f38d543c640 1 -- 172.21.15.92:0/1581206761 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38d004eef0 con 0x7f38d0070b00 2024-03-11T18:34:59.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.170+0000 7f38b77fe640 1 -- 172.21.15.92:0/1581206761 <== mon.2 v2:172.21.15.92:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f38c00059a0 con 0x7f38d0070b00 2024-03-11T18:34:59.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.170+0000 7f38b77fe640 1 -- 172.21.15.92:0/1581206761 <== mon.2 v2:172.21.15.92:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f38c0006800 con 0x7f38d0070b00 2024-03-11T18:34:59.172 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.170+0000 7f38b77fe640 1 -- 172.21.15.92:0/1581206761 <== mon.2 v2:172.21.15.92:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f38c00c7460 con 0x7f38d0070b00 2024-03-11T18:34:59.199 INFO:tasks.ceph.mgr.y.smithi090.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-03-11T18:34:59.199 INFO:tasks.ceph.mgr.y.smithi090.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-03-11T18:34:59.199 INFO:tasks.ceph.mgr.y.smithi090.stderr: from numpy import show_config as show_numpy_config 2024-03-11T18:34:59.208 INFO:tasks.ceph.mgr.x.smithi092.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-03-11T18:34:59.208 INFO:tasks.ceph.mgr.x.smithi092.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-03-11T18:34:59.208 INFO:tasks.ceph.mgr.x.smithi092.stderr: from numpy import show_config as show_numpy_config 2024-03-11T18:34:59.252 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.250+0000 7f38d543c640 1 -- 172.21.15.92:0/1581206761 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "osd new", "uuid": "8ecdce9e-5486-415a-a609-dd16e945440e", "id": 1} v 0) v1 -- 0x7f38d00c8c20 con 0x7f38d0070b00 2024-03-11T18:34:59.253 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-03-11T18:34:59.250+0000 7fc564b0c180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-11T18:34:59.270 INFO:tasks.ceph.mgr.y.smithi090.stderr:2024-03-11T18:34:59.264+0000 7f7a8cf3c180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-11T18:34:59.414 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-03-11T18:34:59.410+0000 7fc564b0c180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-11T18:34:59.475 INFO:tasks.ceph.mgr.y.smithi090.stderr:2024-03-11T18:34:59.468+0000 7f7a8cf3c180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-11T18:34:59.489 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.486+0000 7f38b77fe640 1 -- 172.21.15.92:0/1581206761 <== mon.2 v2:172.21.15.92:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8ecdce9e-5486-415a-a609-dd16e945440e", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f38c0005d40 con 0x7f38d0070b00 2024-03-11T18:34:59.489 INFO:teuthology.orchestra.run.smithi092.stdout:1 2024-03-11T18:34:59.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.486+0000 7f38d543c640 1 -- 172.21.15.92:0/1581206761 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f38d0070b00 msgr2=0x7f38d0149e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:34:59.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.486+0000 7f38d543c640 1 --2- 172.21.15.92:0/1581206761 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f38d0070b00 0x7f38d0149e40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f38c001aba0 tx=0x7f38c001ad30 comp rx=0 tx=0).stop 2024-03-11T18:34:59.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.486+0000 7f38d543c640 1 -- 172.21.15.92:0/1581206761 shutdown_connections 2024-03-11T18:34:59.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.486+0000 7f38d543c640 1 --2- 172.21.15.92:0/1581206761 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f38d0151b30 0x7f38d00ca4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:34:59.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.486+0000 7f38d543c640 1 --2- 172.21.15.92:0/1581206761 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38d00c4af0 0x7f38d014f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:34:59.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.486+0000 7f38d543c640 1 --2- 172.21.15.92:0/1581206761 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f38d0070b00 0x7f38d0149e40 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:34:59.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.486+0000 7f38d543c640 1 -- 172.21.15.92:0/1581206761 >> 172.21.15.92:0/1581206761 conn(0x7f38d006ac00 msgr2=0x7f38d0157c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:34:59.491 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.486+0000 7f38d543c640 1 -- 172.21.15.92:0/1581206761 shutdown_connections 2024-03-11T18:34:59.491 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.486+0000 7f38d543c640 1 -- 172.21.15.92:0/1581206761 wait complete. 2024-03-11T18:34:59.510 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new 737486d4-0e90-4c82-9e79-bf37c812e5ad 2 2024-03-11T18:34:59.585 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-03-11T18:34:59.582+0000 7fc564b0c180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-11T18:34:59.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.626+0000 7fd8150dc640 1 Processor -- start 2024-03-11T18:34:59.629 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.626+0000 7fd8150dc640 1 -- start start 2024-03-11T18:34:59.630 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.626+0000 7fd8150dc640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd8100cc3a0 0x7fd81015a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:34:59.631 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.626+0000 7fd8150dc640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd810159d80 con 0x7fd810153fc0 2024-03-11T18:34:59.631 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.626+0000 7fd8150dc640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8100c6e10 con 0x7fd810066d60 2024-03-11T18:34:59.631 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.626+0000 7fd8150dc640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8100c7990 con 0x7fd8100cc3a0 2024-03-11T18:34:59.631 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.626+0000 7fd807fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd8100cc3a0 0x7fd81015a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:34:59.632 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.626+0000 7fd807fff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd8100cc3a0 0x7fd81015a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46250/0 (socket says 172.21.15.92:46250) 2024-03-11T18:34:59.632 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.626+0000 7fd807fff640 1 -- 172.21.15.92:0/2566577079 learned_addr learned my addr 172.21.15.92:0/2566577079 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-11T18:34:59.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.630+0000 7fd807fff640 1 -- 172.21.15.92:0/2566577079 >> v1:172.21.15.92:6789/0 conn(0x7fd810066d60 legacy=0x7fd810164620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:34:59.633 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.630+0000 7fd807fff640 1 -- 172.21.15.92:0/2566577079 >> v1:172.21.15.90:6789/0 conn(0x7fd810153fc0 legacy=0x7fd81006bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:34:59.634 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.630+0000 7fd807fff640 1 -- 172.21.15.92:0/2566577079 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8100c70f0 con 0x7fd8100cc3a0 2024-03-11T18:34:59.638 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.634+0000 7fd807fff640 1 --2- 172.21.15.92:0/2566577079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8100cc3a0 0x7fd81015a240 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd7f8009870 tx=0x7fd7f801ed10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b20b00a918876fab server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:34:59.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.638+0000 7fd80e59c640 1 -- 172.21.15.92:0/2566577079 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f80a8a10 con 0x7fd8100cc3a0 2024-03-11T18:34:59.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.638+0000 7fd80e59c640 1 -- 172.21.15.92:0/2566577079 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7f80a8b70 con 0x7fd8100cc3a0 2024-03-11T18:34:59.642 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.638+0000 7fd80e59c640 1 -- 172.21.15.92:0/2566577079 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f80b0440 con 0x7fd8100cc3a0 2024-03-11T18:34:59.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.638+0000 7fd8150dc640 1 -- 172.21.15.92:0/2566577079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8100cc3a0 msgr2=0x7fd81015a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:34:59.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.638+0000 7fd8150dc640 1 --2- 172.21.15.92:0/2566577079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8100cc3a0 0x7fd81015a240 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd7f8009870 tx=0x7fd7f801ed10 comp rx=0 tx=0).stop 2024-03-11T18:34:59.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.638+0000 7fd8150dc640 1 -- 172.21.15.92:0/2566577079 shutdown_connections 2024-03-11T18:34:59.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.638+0000 7fd8150dc640 1 --2- 172.21.15.92:0/2566577079 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd8100cc3a0 0x7fd81015a240 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:34:59.643 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.638+0000 7fd8150dc640 1 -- 172.21.15.92:0/2566577079 >> 172.21.15.92:0/2566577079 conn(0x7fd8100cb880 msgr2=0x7fd8100cbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:34:59.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.638+0000 7fd8150dc640 1 -- 172.21.15.92:0/2566577079 shutdown_connections 2024-03-11T18:34:59.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.638+0000 7fd8150dc640 1 -- 172.21.15.92:0/2566577079 wait complete. 2024-03-11T18:34:59.644 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.642+0000 7fd8150dc640 1 Processor -- start 2024-03-11T18:34:59.645 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.642+0000 7fd8150dc640 1 -- start start 2024-03-11T18:34:59.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.642+0000 7fd8150dc640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd810066d60 0x7fd810152b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:34:59.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.642+0000 7fd8150dc640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8100cc3a0 0x7fd8100c3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:34:59.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.642+0000 7fd8150dc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd810153fc0 0x7fd8100c42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:34:59.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.642+0000 7fd8150dc640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8101e4ad0 con 0x7fd8100cc3a0 2024-03-11T18:34:59.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.642+0000 7fd8150dc640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd810153590 con 0x7fd810153fc0 2024-03-11T18:34:59.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.642+0000 7fd8150dc640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8101e3be0 con 0x7fd810066d60 2024-03-11T18:34:59.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.642+0000 7fd80f59e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd810153fc0 0x7fd8100c42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:34:59.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.642+0000 7fd80ed9d640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd810066d60 0x7fd810152b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:34:59.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.642+0000 7fd80f59e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd810153fc0 0x7fd8100c42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46252/0 (socket says 172.21.15.92:46252) 2024-03-11T18:34:59.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.642+0000 7fd80f59e640 1 -- 172.21.15.92:0/916857405 learned_addr learned my addr 172.21.15.92:0/916857405 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-11T18:34:59.646 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.642+0000 7fd807fff640 1 --2- 172.21.15.92:0/916857405 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8100cc3a0 0x7fd8100c3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:34:59.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.646+0000 7fd80f59e640 1 -- 172.21.15.92:0/916857405 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd810066d60 msgr2=0x7fd810152b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:34:59.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.646+0000 7fd80f59e640 1 --2- 172.21.15.92:0/916857405 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd810066d60 0x7fd810152b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:34:59.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.646+0000 7fd80f59e640 1 -- 172.21.15.92:0/916857405 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8100cc3a0 msgr2=0x7fd8100c3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:34:59.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.646+0000 7fd80f59e640 1 --2- 172.21.15.92:0/916857405 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8100cc3a0 0x7fd8100c3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:34:59.649 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.646+0000 7fd80f59e640 1 -- 172.21.15.92:0/916857405 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f80094d0 con 0x7fd810153fc0 2024-03-11T18:34:59.651 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.650+0000 7fd80f59e640 1 --2- 172.21.15.92:0/916857405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd810153fc0 0x7fd8100c42d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd80800c780 tx=0x7fd80800cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:34:59.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.654+0000 7fd8077fe640 1 -- 172.21.15.92:0/916857405 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd808003d90 con 0x7fd810153fc0 2024-03-11T18:34:59.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.654+0000 7fd8077fe640 1 -- 172.21.15.92:0/916857405 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd80800ce30 con 0x7fd810153fc0 2024-03-11T18:34:59.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.654+0000 7fd8077fe640 1 -- 172.21.15.92:0/916857405 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd808005290 con 0x7fd810153fc0 2024-03-11T18:34:59.656 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.654+0000 7fd8150dc640 1 -- 172.21.15.92:0/916857405 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8101e3ec0 con 0x7fd810153fc0 2024-03-11T18:34:59.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.654+0000 7fd8150dc640 1 -- 172.21.15.92:0/916857405 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8101e4480 con 0x7fd810153fc0 2024-03-11T18:34:59.657 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.654+0000 7fd8150dc640 1 -- 172.21.15.92:0/916857405 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd81006a1b0 con 0x7fd810153fc0 2024-03-11T18:34:59.658 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-03-11T18:34:59.654+0000 7fc564b0c180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-11T18:34:59.676 INFO:tasks.ceph.mgr.y.smithi090.stderr:2024-03-11T18:34:59.672+0000 7f7a8cf3c180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-11T18:34:59.748 INFO:tasks.ceph.mgr.y.smithi090.stderr:2024-03-11T18:34:59.740+0000 7f7a8cf3c180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-11T18:34:59.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.858+0000 7fd8077fe640 1 -- 172.21.15.92:0/916857405 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fd8080053f0 con 0x7fd810153fc0 2024-03-11T18:34:59.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.858+0000 7fd8077fe640 1 -- 172.21.15.92:0/916857405 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fd80801b7d0 con 0x7fd810153fc0 2024-03-11T18:34:59.863 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.858+0000 7fd8077fe640 1 -- 172.21.15.92:0/916857405 <== mon.1 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fd808017070 con 0x7fd810153fc0 2024-03-11T18:34:59.888 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-03-11T18:34:59.886+0000 7fc564b0c180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-11T18:34:59.893 INFO:tasks.ceph.mgr.y.smithi090.stderr:2024-03-11T18:34:59.888+0000 7f7a8cf3c180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-11T18:34:59.942 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:34:59.938+0000 7fd8150dc640 1 -- 172.21.15.92:0/916857405 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd new", "uuid": "737486d4-0e90-4c82-9e79-bf37c812e5ad", "id": 2} v 0) v1 -- 0x7fd8101e4c40 con 0x7fd810153fc0 2024-03-11T18:35:00.104 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.102+0000 7fd8077fe640 1 -- 172.21.15.92:0/916857405 <== mon.1 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "737486d4-0e90-4c82-9e79-bf37c812e5ad", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd80801b3b0 con 0x7fd810153fc0 2024-03-11T18:35:00.104 INFO:teuthology.orchestra.run.smithi092.stdout:2 2024-03-11T18:35:00.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.102+0000 7fd8150dc640 1 -- 172.21.15.92:0/916857405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd810153fc0 msgr2=0x7fd8100c42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:00.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.102+0000 7fd8150dc640 1 --2- 172.21.15.92:0/916857405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd810153fc0 0x7fd8100c42d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd80800c780 tx=0x7fd80800cc40 comp rx=0 tx=0).stop 2024-03-11T18:35:00.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.102+0000 7fd8150dc640 1 -- 172.21.15.92:0/916857405 shutdown_connections 2024-03-11T18:35:00.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.102+0000 7fd8150dc640 1 --2- 172.21.15.92:0/916857405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd810153fc0 0x7fd8100c42d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:00.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.102+0000 7fd8150dc640 1 --2- 172.21.15.92:0/916857405 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8100cc3a0 0x7fd8100c3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:00.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.102+0000 7fd8150dc640 1 --2- 172.21.15.92:0/916857405 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd810066d60 0x7fd810152b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:00.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.102+0000 7fd8150dc640 1 -- 172.21.15.92:0/916857405 >> 172.21.15.92:0/916857405 conn(0x7fd8100cb880 msgr2=0x7fd81006c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:35:00.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.102+0000 7fd8150dc640 1 -- 172.21.15.92:0/916857405 shutdown_connections 2024-03-11T18:35:00.106 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.102+0000 7fd8150dc640 1 -- 172.21.15.92:0/916857405 wait complete. 2024-03-11T18:35:00.122 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new 698bd661-bbef-4699-a619-f7bfe23ce12c 3 2024-03-11T18:35:00.232 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.230+0000 7f1ec08f9640 1 Processor -- start 2024-03-11T18:35:00.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.230+0000 7f1ec08f9640 1 -- start start 2024-03-11T18:35:00.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.230+0000 7f1ec08f9640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f1ebc0c6cf0 0x7f1ebc165590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:00.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.230+0000 7f1ec08f9640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ebc0c7d50 con 0x7f1ebc06f400 2024-03-11T18:35:00.233 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.230+0000 7f1ec08f9640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ebc0703f0 con 0x7f1ebc1540c0 2024-03-11T18:35:00.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.230+0000 7f1ec08f9640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ebc15d900 con 0x7f1ebc0c6cf0 2024-03-11T18:35:00.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.230+0000 7f1eba59c640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f1ebc1540c0 0x7f1ebc15d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59722/0 (socket says 172.21.15.92:59722) 2024-03-11T18:35:00.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.230+0000 7f1eba59c640 1 -- 172.21.15.92:0/3339317460 learned_addr learned my addr 172.21.15.92:0/3339317460 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-03-11T18:35:00.234 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.230+0000 7f1ebad9d640 1 --2- 172.21.15.92:0/3339317460 >> v2:172.21.15.90:3300/0 conn(0x7f1ebc0c6cf0 0x7f1ebc165590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:00.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.234+0000 7f1eb959a640 1 -- 172.21.15.92:0/3339317460 <== mon.1 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2770372068 0 0) 0x7f1ebc0703f0 con 0x7f1ebc1540c0 2024-03-11T18:35:00.237 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.234+0000 7f1eb959a640 1 -- 172.21.15.92:0/3339317460 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ea4003530 con 0x7f1ebc1540c0 2024-03-11T18:35:00.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.238+0000 7f1ebad9d640 1 -- 172.21.15.92:0/3339317460 >> v1:172.21.15.92:6789/0 conn(0x7f1ebc1540c0 legacy=0x7f1ebc15d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:00.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.238+0000 7f1ebad9d640 1 -- 172.21.15.92:0/3339317460 >> v1:172.21.15.90:6789/0 conn(0x7f1ebc06f400 legacy=0x7f1ebc0cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:35:00.240 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.238+0000 7f1ebad9d640 1 -- 172.21.15.92:0/3339317460 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ebc06b480 con 0x7f1ebc0c6cf0 2024-03-11T18:35:00.254 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.250+0000 7f1ebad9d640 1 --2- 172.21.15.92:0/3339317460 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1ebc0c6cf0 0x7f1ebc165590 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4009d90 tx=0x7f1eb401c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d6fc8de961b004 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:00.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.262+0000 7f1eb959a640 1 -- 172.21.15.92:0/3339317460 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb40a59c0 con 0x7f1ebc0c6cf0 2024-03-11T18:35:00.264 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.262+0000 7f1eb959a640 1 -- 172.21.15.92:0/3339317460 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb40a5b20 con 0x7f1ebc0c6cf0 2024-03-11T18:35:00.265 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.262+0000 7f1eb959a640 1 -- 172.21.15.92:0/3339317460 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb40ad440 con 0x7f1ebc0c6cf0 2024-03-11T18:35:00.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.262+0000 7f1ec08f9640 1 -- 172.21.15.92:0/3339317460 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1ebc0c6cf0 msgr2=0x7f1ebc165590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:00.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.262+0000 7f1ec08f9640 1 --2- 172.21.15.92:0/3339317460 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1ebc0c6cf0 0x7f1ebc165590 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4009d90 tx=0x7f1eb401c900 comp rx=0 tx=0).stop 2024-03-11T18:35:00.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.262+0000 7f1ec08f9640 1 -- 172.21.15.92:0/3339317460 shutdown_connections 2024-03-11T18:35:00.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.262+0000 7f1ec08f9640 1 --2- 172.21.15.92:0/3339317460 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1ebc0c6cf0 0x7f1ebc165590 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:00.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.262+0000 7f1ec08f9640 1 -- 172.21.15.92:0/3339317460 >> 172.21.15.92:0/3339317460 conn(0x7f1ebc071aa0 msgr2=0x7f1ebc071eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:35:00.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.262+0000 7f1ec08f9640 1 -- 172.21.15.92:0/3339317460 shutdown_connections 2024-03-11T18:35:00.266 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.262+0000 7f1ec08f9640 1 -- 172.21.15.92:0/3339317460 wait complete. 2024-03-11T18:35:00.267 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.262+0000 7f1ec08f9640 1 Processor -- start 2024-03-11T18:35:00.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.262+0000 7f1ec08f9640 1 -- start start 2024-03-11T18:35:00.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.266+0000 7f1ec08f9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ebc06f400 0x7f1ebc149ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:00.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.266+0000 7f1ec08f9640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1ebc0c6cf0 0x7f1ebc0c4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:00.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.266+0000 7f1ec08f9640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1ebc1540c0 0x7f1ebc14f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:00.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.266+0000 7f1ec08f9640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc14f250 con 0x7f1ebc1540c0 2024-03-11T18:35:00.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.266+0000 7f1ec08f9640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc0ca1f0 con 0x7f1ebc06f400 2024-03-11T18:35:00.268 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.266+0000 7f1ec08f9640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc151140 con 0x7f1ebc0c6cf0 2024-03-11T18:35:00.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.266+0000 7f1eb9d9b640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1ebc0c6cf0 0x7f1ebc0c4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:00.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.266+0000 7f1eba59c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ebc06f400 0x7f1ebc149ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:00.269 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.266+0000 7f1eb9d9b640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1ebc0c6cf0 0x7f1ebc0c4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.92:41882/0 (socket says 172.21.15.92:41882) 2024-03-11T18:35:00.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.266+0000 7f1eb9d9b640 1 -- 172.21.15.92:0/2593164069 learned_addr learned my addr 172.21.15.92:0/2593164069 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-11T18:35:00.270 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.266+0000 7f1ebad9d640 1 --2- 172.21.15.92:0/2593164069 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1ebc1540c0 0x7f1ebc14f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:00.272 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.270+0000 7f1eb9d9b640 1 -- 172.21.15.92:0/2593164069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ebc06f400 msgr2=0x7f1ebc149ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:00.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.270+0000 7f1eb9d9b640 1 --2- 172.21.15.92:0/2593164069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ebc06f400 0x7f1ebc149ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:00.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.270+0000 7f1eb9d9b640 1 -- 172.21.15.92:0/2593164069 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1ebc1540c0 msgr2=0x7f1ebc14f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:00.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.270+0000 7f1eb9d9b640 1 --2- 172.21.15.92:0/2593164069 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1ebc1540c0 0x7f1ebc14f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:00.273 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.270+0000 7f1eb9d9b640 1 -- 172.21.15.92:0/2593164069 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb4009a70 con 0x7f1ebc0c6cf0 2024-03-11T18:35:00.275 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.274+0000 7f1eb9d9b640 1 --2- 172.21.15.92:0/2593164069 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1ebc0c6cf0 0x7f1ebc0c4660 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1eb000c490 tx=0x7f1eb0001e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:00.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.274+0000 7f1eaaffd640 1 -- 172.21.15.92:0/2593164069 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb0003600 con 0x7f1ebc0c6cf0 2024-03-11T18:35:00.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.274+0000 7f1eaaffd640 1 -- 172.21.15.92:0/2593164069 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb0003780 con 0x7f1ebc0c6cf0 2024-03-11T18:35:00.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.274+0000 7f1eaaffd640 1 -- 172.21.15.92:0/2593164069 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb0003b70 con 0x7f1ebc0c6cf0 2024-03-11T18:35:00.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.274+0000 7f1ec08f9640 1 -- 172.21.15.92:0/2593164069 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ebc15a600 con 0x7f1ebc0c6cf0 2024-03-11T18:35:00.279 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.278+0000 7f1ec08f9640 1 -- 172.21.15.92:0/2593164069 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ebc14eee0 con 0x7f1ebc0c6cf0 2024-03-11T18:35:00.280 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.278+0000 7f1ec08f9640 1 -- 172.21.15.92:0/2593164069 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ebc15ab20 con 0x7f1ebc0c6cf0 2024-03-11T18:35:00.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.386+0000 7f1eaaffd640 1 -- 172.21.15.92:0/2593164069 <== mon.2 v2:172.21.15.92:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f1eb0003cd0 con 0x7f1ebc0c6cf0 2024-03-11T18:35:00.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.386+0000 7f1eaaffd640 1 -- 172.21.15.92:0/2593164069 <== mon.2 v2:172.21.15.92:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f1eb001b840 con 0x7f1ebc0c6cf0 2024-03-11T18:35:00.389 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.386+0000 7f1eaaffd640 1 -- 172.21.15.92:0/2593164069 <== mon.2 v2:172.21.15.92:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f1ebc15ab20 con 0x7f1ebc0c6cf0 2024-03-11T18:35:00.465 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.462+0000 7f1ec08f9640 1 -- 172.21.15.92:0/2593164069 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "osd new", "uuid": "698bd661-bbef-4699-a619-f7bfe23ce12c", "id": 3} v 0) v1 -- 0x7f1ebc14f3c0 con 0x7f1ebc0c6cf0 2024-03-11T18:35:00.546 INFO:tasks.ceph.mgr.y.smithi090.stderr:2024-03-11T18:35:00.540+0000 7f7a8cf3c180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-11T18:35:00.567 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-03-11T18:35:00.562+0000 7fc564b0c180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-11T18:35:00.624 INFO:tasks.ceph.mgr.y.smithi090.stderr:2024-03-11T18:35:00.616+0000 7f7a8cf3c180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-11T18:35:00.662 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-03-11T18:35:00.658+0000 7fc564b0c180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-11T18:35:00.670 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.666+0000 7f1eaaffd640 1 -- 172.21.15.92:0/2593164069 <== mon.2 v2:172.21.15.92:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "698bd661-bbef-4699-a619-f7bfe23ce12c", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1eb0017070 con 0x7f1ebc0c6cf0 2024-03-11T18:35:00.671 INFO:teuthology.orchestra.run.smithi092.stdout:3 2024-03-11T18:35:00.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.670+0000 7f1ec08f9640 1 -- 172.21.15.92:0/2593164069 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1ebc0c6cf0 msgr2=0x7f1ebc0c4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:00.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.670+0000 7f1ec08f9640 1 --2- 172.21.15.92:0/2593164069 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1ebc0c6cf0 0x7f1ebc0c4660 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1eb000c490 tx=0x7f1eb0001e80 comp rx=0 tx=0).stop 2024-03-11T18:35:00.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.670+0000 7f1ec08f9640 1 -- 172.21.15.92:0/2593164069 shutdown_connections 2024-03-11T18:35:00.673 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.670+0000 7f1ec08f9640 1 --2- 172.21.15.92:0/2593164069 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1ebc1540c0 0x7f1ebc14f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:00.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.670+0000 7f1ec08f9640 1 --2- 172.21.15.92:0/2593164069 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1ebc0c6cf0 0x7f1ebc0c4660 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:00.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.670+0000 7f1ec08f9640 1 --2- 172.21.15.92:0/2593164069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ebc06f400 0x7f1ebc149ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:00.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.670+0000 7f1ec08f9640 1 -- 172.21.15.92:0/2593164069 >> 172.21.15.92:0/2593164069 conn(0x7f1ebc071aa0 msgr2=0x7f1ebc0c5db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:35:00.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.670+0000 7f1ec08f9640 1 -- 172.21.15.92:0/2593164069 shutdown_connections 2024-03-11T18:35:00.674 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.670+0000 7f1ec08f9640 1 -- 172.21.15.92:0/2593164069 wait complete. 2024-03-11T18:35:00.694 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new aba9e31f-73d9-43fe-a10b-2dc5832968b4 4 2024-03-11T18:35:00.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.814+0000 7fbc27cb3640 1 Processor -- start 2024-03-11T18:35:00.817 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.814+0000 7fbc27cb3640 1 -- start start 2024-03-11T18:35:00.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.814+0000 7fbc27cb3640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbc20153fc0 0x7fbc2006bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:00.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.814+0000 7fbc27cb3640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fbc2015a950 0x7fbc2006c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:00.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.814+0000 7fbc27cb3640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc20159d80 con 0x7fbc200cc3a0 2024-03-11T18:35:00.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.814+0000 7fbc27cb3640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc200c7990 con 0x7fbc2015a950 2024-03-11T18:35:00.818 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.814+0000 7fbc27cb3640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc200c6e10 con 0x7fbc20153fc0 2024-03-11T18:35:00.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.814+0000 7fbc25a4f640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbc20153fc0 0x7fbc2006bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:00.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.814+0000 7fbc25a4f640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbc20153fc0 0x7fbc2006bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:46268/0 (socket says 172.21.15.92:46268) 2024-03-11T18:35:00.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.814+0000 7fbc25a4f640 1 -- 172.21.15.92:0/818191768 learned_addr learned my addr 172.21.15.92:0/818191768 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-11T18:35:00.819 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.818+0000 7fbc26250640 1 --2- 172.21.15.92:0/818191768 >> v2:172.21.15.90:3300/0 conn(0x7fbc2015a950 0x7fbc2006c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:00.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.818+0000 7fbc25a4f640 1 -- 172.21.15.92:0/818191768 >> v2:172.21.15.90:3300/0 conn(0x7fbc2015a950 msgr2=0x7fbc2006c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:35:00.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.818+0000 7fbc25a4f640 1 --2- 172.21.15.92:0/818191768 >> v2:172.21.15.90:3300/0 conn(0x7fbc2015a950 0x7fbc2006c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:00.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.818+0000 7fbc25a4f640 1 -- 172.21.15.92:0/818191768 >> v1:172.21.15.90:6789/0 conn(0x7fbc200cc3a0 legacy=0x7fbc2015a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:35:00.821 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.818+0000 7fbc25a4f640 1 -- 172.21.15.92:0/818191768 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc200c70f0 con 0x7fbc20153fc0 2024-03-11T18:35:00.824 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.822+0000 7fbc25a4f640 1 --2- 172.21.15.92:0/818191768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc20153fc0 0x7fbc2006bc10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbc200c7ad0 tx=0x7fbc1c01ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8292beac55ff7332 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:00.826 INFO:tasks.ceph.mgr.y.smithi090.stderr:2024-03-11T18:35:00.820+0000 7f7a8cf3c180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-11T18:35:00.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.826+0000 7fbc24a4d640 1 -- 172.21.15.92:0/818191768 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c0a89d0 con 0x7fbc20153fc0 2024-03-11T18:35:00.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.826+0000 7fbc24a4d640 1 -- 172.21.15.92:0/818191768 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc1c0a8b30 con 0x7fbc20153fc0 2024-03-11T18:35:00.830 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.826+0000 7fbc24a4d640 1 -- 172.21.15.92:0/818191768 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c0b0440 con 0x7fbc20153fc0 2024-03-11T18:35:00.831 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.830+0000 7fbc27cb3640 1 -- 172.21.15.92:0/818191768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc20153fc0 msgr2=0x7fbc2006bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:00.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.830+0000 7fbc27cb3640 1 --2- 172.21.15.92:0/818191768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc20153fc0 0x7fbc2006bc10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbc200c7ad0 tx=0x7fbc1c01ecd0 comp rx=0 tx=0).stop 2024-03-11T18:35:00.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.830+0000 7fbc27cb3640 1 -- 172.21.15.92:0/818191768 shutdown_connections 2024-03-11T18:35:00.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.830+0000 7fbc27cb3640 1 --2- 172.21.15.92:0/818191768 >> v2:172.21.15.90:3300/0 conn(0x7fbc2015a950 0x7fbc2006c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:00.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.830+0000 7fbc27cb3640 1 --2- 172.21.15.92:0/818191768 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc20153fc0 0x7fbc2006bc10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:00.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.830+0000 7fbc27cb3640 1 -- 172.21.15.92:0/818191768 >> 172.21.15.92:0/818191768 conn(0x7fbc200cb880 msgr2=0x7fbc200cbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:35:00.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.830+0000 7fbc27cb3640 1 -- 172.21.15.92:0/818191768 shutdown_connections 2024-03-11T18:35:00.832 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.830+0000 7fbc27cb3640 1 -- 172.21.15.92:0/818191768 wait complete. 2024-03-11T18:35:00.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.830+0000 7fbc27cb3640 1 Processor -- start 2024-03-11T18:35:00.833 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.830+0000 7fbc27cb3640 1 -- start start 2024-03-11T18:35:00.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.830+0000 7fbc27cb3640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc200cc3a0 0x7fbc200c53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:00.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.830+0000 7fbc27cb3640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbc20153fc0 0x7fbc20152210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:00.834 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.830+0000 7fbc27cb3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc2015a950 0x7fbc200c3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:00.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.830+0000 7fbc27cb3640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc20073090 con 0x7fbc200cc3a0 2024-03-11T18:35:00.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.830+0000 7fbc27cb3640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc200c64a0 con 0x7fbc2015a950 2024-03-11T18:35:00.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.830+0000 7fbc27cb3640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc200c6780 con 0x7fbc20153fc0 2024-03-11T18:35:00.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.830+0000 7fbc2524e640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbc20153fc0 0x7fbc20152210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:00.835 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.830+0000 7fbc26250640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc2015a950 0x7fbc200c3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:00.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.834+0000 7fbc2524e640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbc20153fc0 0x7fbc20152210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.92:41894/0 (socket says 172.21.15.92:41894) 2024-03-11T18:35:00.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.834+0000 7fbc2524e640 1 -- 172.21.15.92:0/3178681234 learned_addr learned my addr 172.21.15.92:0/3178681234 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-11T18:35:00.836 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.834+0000 7fbc25a4f640 1 --2- 172.21.15.92:0/3178681234 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc200cc3a0 0x7fbc200c53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:00.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.834+0000 7fbc2524e640 1 -- 172.21.15.92:0/3178681234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc2015a950 msgr2=0x7fbc200c3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:00.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.834+0000 7fbc2524e640 1 --2- 172.21.15.92:0/3178681234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc2015a950 0x7fbc200c3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:00.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.834+0000 7fbc2524e640 1 -- 172.21.15.92:0/3178681234 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc200cc3a0 msgr2=0x7fbc200c53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:00.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.834+0000 7fbc2524e640 1 --2- 172.21.15.92:0/3178681234 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc200cc3a0 0x7fbc200c53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:00.839 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.834+0000 7fbc2524e640 1 -- 172.21.15.92:0/3178681234 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc1c0094d0 con 0x7fbc20153fc0 2024-03-11T18:35:00.841 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.838+0000 7fbc2524e640 1 --2- 172.21.15.92:0/3178681234 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbc20153fc0 0x7fbc20152210 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbc1000c450 tx=0x7fbc1000c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:00.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.842+0000 7fbc167fc640 1 -- 172.21.15.92:0/3178681234 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc10012830 con 0x7fbc20153fc0 2024-03-11T18:35:00.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.842+0000 7fbc167fc640 1 -- 172.21.15.92:0/3178681234 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc100129b0 con 0x7fbc20153fc0 2024-03-11T18:35:00.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.842+0000 7fbc27cb3640 1 -- 172.21.15.92:0/3178681234 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc200726f0 con 0x7fbc20153fc0 2024-03-11T18:35:00.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.842+0000 7fbc167fc640 1 -- 172.21.15.92:0/3178681234 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc10012d60 con 0x7fbc20153fc0 2024-03-11T18:35:00.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.842+0000 7fbc27cb3640 1 -- 172.21.15.92:0/3178681234 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc20072cb0 con 0x7fbc20153fc0 2024-03-11T18:35:00.845 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.842+0000 7fbc27cb3640 1 -- 172.21.15.92:0/3178681234 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc20072e70 con 0x7fbc20153fc0 2024-03-11T18:35:00.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.846+0000 7fbc167fc640 1 -- 172.21.15.92:0/3178681234 <== mon.2 v2:172.21.15.92:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fbc1001c460 con 0x7fbc20153fc0 2024-03-11T18:35:00.848 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.846+0000 7fbc167fc640 1 -- 172.21.15.92:0/3178681234 <== mon.2 v2:172.21.15.92:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7fbc10025c90 con 0x7fbc20153fc0 2024-03-11T18:35:00.874 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-03-11T18:35:00.870+0000 7fc564b0c180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-11T18:35:00.901 INFO:tasks.ceph.mgr.y.smithi090.stderr:2024-03-11T18:35:00.896+0000 7f7a8cf3c180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-11T18:35:00.948 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-03-11T18:35:00.946+0000 7fc564b0c180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-11T18:35:00.953 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:00.950+0000 7fbc167fc640 1 -- 172.21.15.92:0/3178681234 <== mon.2 v2:172.21.15.92:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fbc10025e50 con 0x7fbc20153fc0 2024-03-11T18:35:01.023 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.022+0000 7fbc27cb3640 1 -- 172.21.15.92:0/3178681234 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "osd new", "uuid": "aba9e31f-73d9-43fe-a10b-2dc5832968b4", "id": 4} v 0) v1 -- 0x7fbc200731d0 con 0x7fbc20153fc0 2024-03-11T18:35:01.157 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.154+0000 7fbc167fc640 1 -- 172.21.15.92:0/3178681234 <== mon.2 v2:172.21.15.92:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "aba9e31f-73d9-43fe-a10b-2dc5832968b4", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fbc200731d0 con 0x7fbc20153fc0 2024-03-11T18:35:01.157 INFO:teuthology.orchestra.run.smithi092.stdout:4 2024-03-11T18:35:01.159 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.154+0000 7fbc27cb3640 1 -- 172.21.15.92:0/3178681234 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbc20153fc0 msgr2=0x7fbc20152210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:01.159 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.154+0000 7fbc27cb3640 1 --2- 172.21.15.92:0/3178681234 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbc20153fc0 0x7fbc20152210 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbc1000c450 tx=0x7fbc1000c910 comp rx=0 tx=0).stop 2024-03-11T18:35:01.159 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.154+0000 7fbc27cb3640 1 -- 172.21.15.92:0/3178681234 shutdown_connections 2024-03-11T18:35:01.159 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.154+0000 7fbc27cb3640 1 --2- 172.21.15.92:0/3178681234 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbc2015a950 0x7fbc200c3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:01.159 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.154+0000 7fbc27cb3640 1 --2- 172.21.15.92:0/3178681234 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbc20153fc0 0x7fbc20152210 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:01.159 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.154+0000 7fbc27cb3640 1 --2- 172.21.15.92:0/3178681234 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc200cc3a0 0x7fbc200c53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:01.159 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.154+0000 7fbc27cb3640 1 -- 172.21.15.92:0/3178681234 >> 172.21.15.92:0/3178681234 conn(0x7fbc200cb880 msgr2=0x7fbc200cbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:35:01.159 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.154+0000 7fbc27cb3640 1 -- 172.21.15.92:0/3178681234 shutdown_connections 2024-03-11T18:35:01.159 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.154+0000 7fbc27cb3640 1 -- 172.21.15.92:0/3178681234 wait complete. 2024-03-11T18:35:01.175 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new bdd56766-4b28-4007-8de3-b5b08cf335d3 5 2024-03-11T18:35:01.204 INFO:tasks.ceph.mgr.y.smithi090.stderr:2024-03-11T18:35:01.196+0000 7f7a8cf3c180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-11T18:35:01.248 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-03-11T18:35:01.246+0000 7fc564b0c180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-11T18:35:01.265 INFO:tasks.ceph.mgr.y.smithi090.stderr:2024-03-11T18:35:01.260+0000 7f7a8cf3c180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-11T18:35:01.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.286+0000 7f2296b11640 1 Processor -- start 2024-03-11T18:35:01.288 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.286+0000 7f2296b11640 1 -- start start 2024-03-11T18:35:01.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.286+0000 7f2296b11640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f229006b860 0x7f22900c3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:01.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.286+0000 7f2296b11640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22900c7430 con 0x7f2290151d50 2024-03-11T18:35:01.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.286+0000 7f2296b11640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22900c9aa0 con 0x7f22900c95f0 2024-03-11T18:35:01.289 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.286+0000 7f2296b11640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2290152130 con 0x7f229006b860 2024-03-11T18:35:01.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.286+0000 7f228ffff640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f229006b860 0x7f22900c3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:01.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.286+0000 7f22950ae640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f22900c95f0 0x7f22900c5a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:59730/0 (socket says 172.21.15.92:59730) 2024-03-11T18:35:01.290 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.286+0000 7f22950ae640 1 -- 172.21.15.92:0/1339902443 learned_addr learned my addr 172.21.15.92:0/1339902443 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-03-11T18:35:01.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.290+0000 7f228ffff640 1 -- 172.21.15.92:0/1339902443 >> v1:172.21.15.92:6789/0 conn(0x7f22900c95f0 legacy=0x7f22900c5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:01.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.290+0000 7f228ffff640 1 -- 172.21.15.92:0/1339902443 >> v1:172.21.15.90:6789/0 conn(0x7f2290151d50 legacy=0x7f2290157970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:35:01.292 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.290+0000 7f228ffff640 1 -- 172.21.15.92:0/1339902443 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22900c7a40 con 0x7f229006b860 2024-03-11T18:35:01.296 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.294+0000 7f228ffff640 1 --2- 172.21.15.92:0/1339902443 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f229006b860 0x7f22900c3130 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f22840097f0 tx=0x7f228401ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8296bf76638f0b9a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:01.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.298+0000 7f228f7fe640 1 -- 172.21.15.92:0/1339902443 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22840a8990 con 0x7f229006b860 2024-03-11T18:35:01.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.298+0000 7f228f7fe640 1 -- 172.21.15.92:0/1339902443 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22840a8af0 con 0x7f229006b860 2024-03-11T18:35:01.301 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.298+0000 7f228f7fe640 1 -- 172.21.15.92:0/1339902443 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22840a8e80 con 0x7f229006b860 2024-03-11T18:35:01.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.298+0000 7f2296b11640 1 -- 172.21.15.92:0/1339902443 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f229006b860 msgr2=0x7f22900c3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:01.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.298+0000 7f2296b11640 1 --2- 172.21.15.92:0/1339902443 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f229006b860 0x7f22900c3130 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f22840097f0 tx=0x7f228401ec90 comp rx=0 tx=0).stop 2024-03-11T18:35:01.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.298+0000 7f2296b11640 1 -- 172.21.15.92:0/1339902443 shutdown_connections 2024-03-11T18:35:01.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.298+0000 7f2296b11640 1 --2- 172.21.15.92:0/1339902443 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f229006b860 0x7f22900c3130 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:01.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.298+0000 7f2296b11640 1 -- 172.21.15.92:0/1339902443 >> 172.21.15.92:0/1339902443 conn(0x7f2290071aa0 msgr2=0x7f2290071eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:35:01.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.298+0000 7f2296b11640 1 -- 172.21.15.92:0/1339902443 shutdown_connections 2024-03-11T18:35:01.302 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.298+0000 7f2296b11640 1 -- 172.21.15.92:0/1339902443 wait complete. 2024-03-11T18:35:01.303 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.298+0000 7f2296b11640 1 Processor -- start 2024-03-11T18:35:01.303 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.302+0000 7f2296b11640 1 -- start start 2024-03-11T18:35:01.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.302+0000 7f2296b11640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f229006b860 0x7f2290145720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:01.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.302+0000 7f2296b11640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f22900c95f0 0x7f22900c4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:01.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.302+0000 7f2296b11640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2290151d50 0x7f229013ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:01.304 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.302+0000 7f2296b11640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22900cbbf0 con 0x7f22900c95f0 2024-03-11T18:35:01.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.302+0000 7f2296b11640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2290145c60 con 0x7f2290151d50 2024-03-11T18:35:01.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.302+0000 7f2296b11640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2290143210 con 0x7f229006b860 2024-03-11T18:35:01.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.302+0000 7f22948ad640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f229006b860 0x7f2290145720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:01.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.302+0000 7f22950ae640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2290151d50 0x7f229013ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:01.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.302+0000 7f22948ad640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f229006b860 0x7f2290145720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.92:41916/0 (socket says 172.21.15.92:41916) 2024-03-11T18:35:01.305 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.302+0000 7f22948ad640 1 -- 172.21.15.92:0/708794071 learned_addr learned my addr 172.21.15.92:0/708794071 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-11T18:35:01.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.306+0000 7f22948ad640 1 -- 172.21.15.92:0/708794071 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2290151d50 msgr2=0x7f229013ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:01.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.306+0000 7f22948ad640 1 --2- 172.21.15.92:0/708794071 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2290151d50 0x7f229013ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:01.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.306+0000 7f22948ad640 1 -- 172.21.15.92:0/708794071 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f22900c95f0 msgr2=0x7f22900c4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:01.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.306+0000 7f22948ad640 1 --2- 172.21.15.92:0/708794071 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f22900c95f0 0x7f22900c4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:01.308 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.306+0000 7f22948ad640 1 -- 172.21.15.92:0/708794071 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2284009420 con 0x7f229006b860 2024-03-11T18:35:01.310 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.306+0000 7f22948ad640 1 --2- 172.21.15.92:0/708794071 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f229006b860 0x7f2290145720 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f227c00c490 tx=0x7f227c001e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:01.312 INFO:tasks.ceph.mgr.x.smithi092.stderr:2024-03-11T18:35:01.310+0000 7fc564b0c180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-11T18:35:01.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.310+0000 7f228d7fa640 1 -- 172.21.15.92:0/708794071 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227c003600 con 0x7f229006b860 2024-03-11T18:35:01.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.310+0000 7f228d7fa640 1 -- 172.21.15.92:0/708794071 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f227c003780 con 0x7f229006b860 2024-03-11T18:35:01.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.310+0000 7f228d7fa640 1 -- 172.21.15.92:0/708794071 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227c003b70 con 0x7f229006b860 2024-03-11T18:35:01.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.310+0000 7f2296b11640 1 -- 172.21.15.92:0/708794071 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22901434f0 con 0x7f229006b860 2024-03-11T18:35:01.314 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.310+0000 7f2296b11640 1 -- 172.21.15.92:0/708794071 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2290143ab0 con 0x7f229006b860 2024-03-11T18:35:01.315 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.310+0000 7f2296b11640 1 -- 172.21.15.92:0/708794071 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f229004eef0 con 0x7f229006b860 2024-03-11T18:35:01.316 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.314+0000 7f228d7fa640 1 -- 172.21.15.92:0/708794071 <== mon.2 v2:172.21.15.92:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f227c003cd0 con 0x7f229006b860 2024-03-11T18:35:01.434 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.430+0000 7f228d7fa640 1 -- 172.21.15.92:0/708794071 <== mon.2 v2:172.21.15.92:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f227c01c9c0 con 0x7f229006b860 2024-03-11T18:35:01.435 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.430+0000 7f228d7fa640 1 -- 172.21.15.92:0/708794071 <== mon.2 v2:172.21.15.92:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f227c017070 con 0x7f229006b860 2024-03-11T18:35:01.520 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.518+0000 7f2296b11640 1 -- 172.21.15.92:0/708794071 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "osd new", "uuid": "bdd56766-4b28-4007-8de3-b5b08cf335d3", "id": 5} v 0) v1 -- 0x7f229006f5e0 con 0x7f229006b860 2024-03-11T18:35:01.974 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:01.970+0000 7f228d7fa640 1 -- 172.21.15.92:0/708794071 <== mon.2 v2:172.21.15.92:3301/0 7 ==== mgrmap(e 2) v1 ==== 43246+0+0 (secure 0 0 0) 0x7f227c01c440 con 0x7f229006b860 2024-03-11T18:35:02.430 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.426+0000 7f228d7fa640 1 -- 172.21.15.92:0/708794071 <== mon.2 v2:172.21.15.92:3301/0 8 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bdd56766-4b28-4007-8de3-b5b08cf335d3", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f227c0192c0 con 0x7f229006b860 2024-03-11T18:35:02.430 INFO:teuthology.orchestra.run.smithi092.stdout:5 2024-03-11T18:35:02.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.430+0000 7f2296b11640 1 -- 172.21.15.92:0/708794071 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f229006b860 msgr2=0x7f2290145720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:02.431 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.430+0000 7f2296b11640 1 --2- 172.21.15.92:0/708794071 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f229006b860 0x7f2290145720 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f227c00c490 tx=0x7f227c001e80 comp rx=0 tx=0).stop 2024-03-11T18:35:02.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.430+0000 7f2296b11640 1 -- 172.21.15.92:0/708794071 shutdown_connections 2024-03-11T18:35:02.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.430+0000 7f2296b11640 1 --2- 172.21.15.92:0/708794071 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2290151d50 0x7f229013ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:02.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.430+0000 7f2296b11640 1 --2- 172.21.15.92:0/708794071 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f22900c95f0 0x7f22900c4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:02.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.430+0000 7f2296b11640 1 --2- 172.21.15.92:0/708794071 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f229006b860 0x7f2290145720 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:02.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.430+0000 7f2296b11640 1 -- 172.21.15.92:0/708794071 >> 172.21.15.92:0/708794071 conn(0x7f2290071aa0 msgr2=0x7f229006f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:35:02.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.430+0000 7f2296b11640 1 -- 172.21.15.92:0/708794071 shutdown_connections 2024-03-11T18:35:02.432 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.430+0000 7f2296b11640 1 -- 172.21.15.92:0/708794071 wait complete. 2024-03-11T18:35:02.450 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new 6c580a59-610f-455e-a96b-033299dcc118 6 2024-03-11T18:35:02.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.570+0000 7fe805af5640 1 Processor -- start 2024-03-11T18:35:02.572 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.570+0000 7fe805af5640 1 -- start start 2024-03-11T18:35:02.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.570+0000 7fe805af5640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe800153fe0 0x7fe8000ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:02.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.570+0000 7fe805af5640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fe80006bdb0 0x7fe80006c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:02.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.570+0000 7fe805af5640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe80006d000 con 0x7fe8000cb7d0 2024-03-11T18:35:02.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.570+0000 7fe805af5640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8001543c0 con 0x7fe80006bdb0 2024-03-11T18:35:02.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.570+0000 7fe805af5640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe80015ab90 con 0x7fe800153fe0 2024-03-11T18:35:02.573 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.570+0000 7fe7ff7fe640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe800153fe0 0x7fe8000ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:02.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.574+0000 7fe7ff7fe640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe800153fe0 0x7fe8000ca5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.92:41930/0 (socket says 172.21.15.92:41930) 2024-03-11T18:35:02.576 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.574+0000 7fe7ff7fe640 1 -- 172.21.15.92:0/2073256440 learned_addr learned my addr 172.21.15.92:0/2073256440 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-11T18:35:02.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.574+0000 7fe7ff7fe640 1 -- 172.21.15.92:0/2073256440 >> v2:172.21.15.90:3300/0 conn(0x7fe80006bdb0 msgr2=0x7fe80006c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:35:02.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.574+0000 7fe7ff7fe640 1 --2- 172.21.15.92:0/2073256440 >> v2:172.21.15.90:3300/0 conn(0x7fe80006bdb0 0x7fe80006c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:02.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.574+0000 7fe7ff7fe640 1 -- 172.21.15.92:0/2073256440 >> v1:172.21.15.90:6789/0 conn(0x7fe8000cb7d0 legacy=0x7fe8000cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:35:02.579 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.574+0000 7fe7ff7fe640 1 -- 172.21.15.92:0/2073256440 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe80006c740 con 0x7fe800153fe0 2024-03-11T18:35:02.582 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.578+0000 7fe7ff7fe640 1 --2- 172.21.15.92:0/2073256440 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe800153fe0 0x7fe8000ca5f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8009870 tx=0x7fe7f801c4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bf91ed5b0cada9e server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:02.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.582+0000 7fe7fe7fc640 1 -- 172.21.15.92:0/2073256440 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f80aa610 con 0x7fe800153fe0 2024-03-11T18:35:02.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.582+0000 7fe7fe7fc640 1 -- 172.21.15.92:0/2073256440 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f80a6040 con 0x7fe800153fe0 2024-03-11T18:35:02.586 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.582+0000 7fe7fe7fc640 1 -- 172.21.15.92:0/2073256440 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f80aa990 con 0x7fe800153fe0 2024-03-11T18:35:02.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.586+0000 7fe805af5640 1 -- 172.21.15.92:0/2073256440 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe800153fe0 msgr2=0x7fe8000ca5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:02.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.586+0000 7fe805af5640 1 --2- 172.21.15.92:0/2073256440 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe800153fe0 0x7fe8000ca5f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8009870 tx=0x7fe7f801c4f0 comp rx=0 tx=0).stop 2024-03-11T18:35:02.587 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.586+0000 7fe805af5640 1 -- 172.21.15.92:0/2073256440 shutdown_connections 2024-03-11T18:35:02.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.586+0000 7fe805af5640 1 --2- 172.21.15.92:0/2073256440 >> v2:172.21.15.90:3300/0 conn(0x7fe80006bdb0 0x7fe80006c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:02.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.586+0000 7fe805af5640 1 --2- 172.21.15.92:0/2073256440 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe800153fe0 0x7fe8000ca5f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:02.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.586+0000 7fe805af5640 1 -- 172.21.15.92:0/2073256440 >> 172.21.15.92:0/2073256440 conn(0x7fe800072500 msgr2=0x7fe800072910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:35:02.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.586+0000 7fe805af5640 1 -- 172.21.15.92:0/2073256440 shutdown_connections 2024-03-11T18:35:02.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.586+0000 7fe805af5640 1 -- 172.21.15.92:0/2073256440 wait complete. 2024-03-11T18:35:02.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.586+0000 7fe805af5640 1 Processor -- start 2024-03-11T18:35:02.588 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.586+0000 7fe805af5640 1 -- start start 2024-03-11T18:35:02.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.586+0000 7fe805af5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe80006bdb0 0x7fe8000ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:02.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.586+0000 7fe805af5640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe8000cb7d0 0x7fe8001517d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:02.589 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.586+0000 7fe805af5640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe800153fe0 0x7fe800141460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:02.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.586+0000 7fe805af5640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe80013db10 con 0x7fe800153fe0 2024-03-11T18:35:02.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.586+0000 7fe805af5640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe80015cbf0 con 0x7fe80006bdb0 2024-03-11T18:35:02.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.586+0000 7fe805af5640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8001535e0 con 0x7fe8000cb7d0 2024-03-11T18:35:02.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.586+0000 7fe7feffd640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe8000cb7d0 0x7fe8001517d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:02.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.586+0000 7fe7feffd640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe8000cb7d0 0x7fe8001517d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.92:41932/0 (socket says 172.21.15.92:41932) 2024-03-11T18:35:02.590 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.586+0000 7fe7feffd640 1 -- 172.21.15.92:0/1777938070 learned_addr learned my addr 172.21.15.92:0/1777938070 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-11T18:35:02.593 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.590+0000 7fe7feffd640 1 -- 172.21.15.92:0/1777938070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe80006bdb0 msgr2=0x7fe8000ce6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:02.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.590+0000 7fe7feffd640 1 --2- 172.21.15.92:0/1777938070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe80006bdb0 0x7fe8000ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:02.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.590+0000 7fe7feffd640 1 -- 172.21.15.92:0/1777938070 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe800153fe0 msgr2=0x7fe800141460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:02.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.590+0000 7fe7feffd640 1 --2- 172.21.15.92:0/1777938070 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe800153fe0 0x7fe800141460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:02.594 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.590+0000 7fe7feffd640 1 -- 172.21.15.92:0/1777938070 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f80094d0 con 0x7fe8000cb7d0 2024-03-11T18:35:02.596 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.594+0000 7fe7feffd640 1 --2- 172.21.15.92:0/1777938070 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe8000cb7d0 0x7fe8001517d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe7e800c4c0 tx=0x7fe7e8001e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:02.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.598+0000 7fe7f7fff640 1 -- 172.21.15.92:0/1777938070 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e8003600 con 0x7fe8000cb7d0 2024-03-11T18:35:02.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.598+0000 7fe7f7fff640 1 -- 172.21.15.92:0/1777938070 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7e8003780 con 0x7fe8000cb7d0 2024-03-11T18:35:02.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.598+0000 7fe7f7fff640 1 -- 172.21.15.92:0/1777938070 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e8003b30 con 0x7fe8000cb7d0 2024-03-11T18:35:02.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.598+0000 7fe805af5640 1 -- 172.21.15.92:0/1777938070 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe800142c30 con 0x7fe8000cb7d0 2024-03-11T18:35:02.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.598+0000 7fe805af5640 1 -- 172.21.15.92:0/1777938070 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8001533b0 con 0x7fe8000cb7d0 2024-03-11T18:35:02.602 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.598+0000 7fe805af5640 1 -- 172.21.15.92:0/1777938070 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8000681b0 con 0x7fe8000cb7d0 2024-03-11T18:35:02.612 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.610+0000 7fe7f7fff640 1 -- 172.21.15.92:0/1777938070 <== mon.2 v2:172.21.15.92:3301/0 4 ==== mgrmap(e 2) v1 ==== 43246+0+0 (secure 0 0 0) 0x7fe7e8003c90 con 0x7fe8000cb7d0 2024-03-11T18:35:02.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.730+0000 7fe7f7fff640 1 -- 172.21.15.92:0/1777938070 <== mon.2 v2:172.21.15.92:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fe7e801b3b0 con 0x7fe8000cb7d0 2024-03-11T18:35:02.734 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.730+0000 7fe7f7fff640 1 -- 172.21.15.92:0/1777938070 <== mon.2 v2:172.21.15.92:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fe7e801bb10 con 0x7fe8000cb7d0 2024-03-11T18:35:02.838 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:02.834+0000 7fe805af5640 1 -- 172.21.15.92:0/1777938070 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "osd new", "uuid": "6c580a59-610f-455e-a96b-033299dcc118", "id": 6} v 0) v1 -- 0x7fe80004eef0 con 0x7fe8000cb7d0 2024-03-11T18:35:03.490 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:03.486+0000 7fe7f7fff640 1 -- 172.21.15.92:0/1777938070 <== mon.2 v2:172.21.15.92:3301/0 7 ==== mgrmap(e 3) v1 ==== 86247+0+0 (secure 0 0 0) 0x7fe7e801ad20 con 0x7fe8000cb7d0 2024-03-11T18:35:03.491 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:03.490+0000 7fe7f7fff640 1 --2- 172.21.15.92:0/1777938070 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe7e0066fb0 0x7fe7e00693a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:03.491 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:03.490+0000 7fe7ff7fe640 1 --2- 172.21.15.92:0/1777938070 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe7e0066fb0 0x7fe7e00693a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:03.493 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:03.490+0000 7fe7ff7fe640 1 --2- 172.21.15.92:0/1777938070 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe7e0066fb0 0x7fe7e00693a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe7f80ae000 tx=0x7fe7f8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:03.633 INFO:tasks.ceph.mgr.y.smithi090.stderr:2024-03-11T18:35:03.628+0000 7f7a34389640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-03-11T18:35:03.693 INFO:tasks.ceph.mgr.y.smithi090.stderr:2024-03-11T18:35:03.688+0000 7f7a34389640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-11T18:35:03.750 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:35:03.753 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:35:04.010 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.006+0000 7fe7f7fff640 1 -- 172.21.15.92:0/1777938070 <== mon.2 v2:172.21.15.92:3301/0 8 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6c580a59-610f-455e-a96b-033299dcc118", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe7e8053b70 con 0x7fe8000cb7d0 2024-03-11T18:35:04.011 INFO:teuthology.orchestra.run.smithi092.stdout:6 2024-03-11T18:35:04.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.010+0000 7fe805af5640 1 -- 172.21.15.92:0/1777938070 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe7e0066fb0 msgr2=0x7fe7e00693a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:04.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.010+0000 7fe805af5640 1 --2- 172.21.15.92:0/1777938070 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe7e0066fb0 0x7fe7e00693a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe7f80ae000 tx=0x7fe7f8008040 comp rx=0 tx=0).stop 2024-03-11T18:35:04.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.010+0000 7fe805af5640 1 -- 172.21.15.92:0/1777938070 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe8000cb7d0 msgr2=0x7fe8001517d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:04.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.010+0000 7fe805af5640 1 --2- 172.21.15.92:0/1777938070 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe8000cb7d0 0x7fe8001517d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe7e800c4c0 tx=0x7fe7e8001e80 comp rx=0 tx=0).stop 2024-03-11T18:35:04.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.010+0000 7fe805af5640 1 -- 172.21.15.92:0/1777938070 shutdown_connections 2024-03-11T18:35:04.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.010+0000 7fe805af5640 1 --2- 172.21.15.92:0/1777938070 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe7e0066fb0 0x7fe7e00693a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:04.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.010+0000 7fe805af5640 1 --2- 172.21.15.92:0/1777938070 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe800153fe0 0x7fe800141460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:04.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.010+0000 7fe805af5640 1 --2- 172.21.15.92:0/1777938070 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe8000cb7d0 0x7fe8001517d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:04.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.010+0000 7fe805af5640 1 --2- 172.21.15.92:0/1777938070 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe80006bdb0 0x7fe8000ce6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:04.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.010+0000 7fe805af5640 1 -- 172.21.15.92:0/1777938070 >> 172.21.15.92:0/1777938070 conn(0x7fe800072500 msgr2=0x7fe8001588f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:35:04.015 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.010+0000 7fe805af5640 1 -- 172.21.15.92:0/1777938070 shutdown_connections 2024-03-11T18:35:04.016 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.010+0000 7fe805af5640 1 -- 172.21.15.92:0/1777938070 wait complete. 2024-03-11T18:35:04.035 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster ceph osd new 958184cc-aebf-4c5b-91a0-2a3e38a6b42b 7 2024-03-11T18:35:04.159 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.158+0000 7f792d67d640 1 Processor -- start 2024-03-11T18:35:04.160 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.158+0000 7f792d67d640 1 -- start start 2024-03-11T18:35:04.160 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.158+0000 7f792d67d640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f7928153f10 0x7f792815a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:04.160 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.158+0000 7f792d67d640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f792815abb0 con 0x7f79280afb60 2024-03-11T18:35:04.160 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.158+0000 7f792d67d640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79280c81e0 con 0x7f79280c8e20 2024-03-11T18:35:04.160 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.158+0000 7f792d67d640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f792806c2d0 con 0x7f7928153f10 2024-03-11T18:35:04.161 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.158+0000 7f79267fc640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f79280c8e20 0x7f792815db00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.92:39216/0 (socket says 172.21.15.92:39216) 2024-03-11T18:35:04.161 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.158+0000 7f79267fc640 1 -- 172.21.15.92:0/2081630651 learned_addr learned my addr 172.21.15.92:0/2081630651 (peer_addr_for_me v1:172.21.15.92:0/0) 2024-03-11T18:35:04.181 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.178+0000 7f7925ffb640 1 -- 172.21.15.92:0/2081630651 <== mon.1 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3331784088 0 0) 0x7f79280c81e0 con 0x7f79280c8e20 2024-03-11T18:35:04.181 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.178+0000 7f7925ffb640 1 -- 172.21.15.92:0/2081630651 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7910003530 con 0x7f79280c8e20 2024-03-11T18:35:04.256 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.254+0000 7f7926ffd640 1 --2- 172.21.15.92:0/2081630651 >> v2:172.21.15.90:3300/0 conn(0x7f7928153f10 0x7f792815a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:04.391 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.390+0000 7f7926ffd640 1 -- 172.21.15.92:0/2081630651 >> v1:172.21.15.92:6789/0 conn(0x7f79280c8e20 legacy=0x7f792815db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:04.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.390+0000 7f7926ffd640 1 -- 172.21.15.92:0/2081630651 >> v1:172.21.15.90:6789/0 conn(0x7f79280afb60 legacy=0x7f792806c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:04.392 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.390+0000 7f7926ffd640 1 -- 172.21.15.92:0/2081630651 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f792806bd10 con 0x7f7928153f10 2024-03-11T18:35:04.412 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.410+0000 7f7926ffd640 1 --2- 172.21.15.92:0/2081630651 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7928153f10 0x7f792815a670 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f79180097c0 tx=0x7f791801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd9ca330b6ccb2f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:04.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.442+0000 7f7925ffb640 1 -- 172.21.15.92:0/2081630651 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79180a6040 con 0x7f7928153f10 2024-03-11T18:35:04.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.442+0000 7f7925ffb640 1 -- 172.21.15.92:0/2081630651 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79180aa600 con 0x7f7928153f10 2024-03-11T18:35:04.444 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.442+0000 7f7925ffb640 1 -- 172.21.15.92:0/2081630651 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79180aa990 con 0x7f7928153f10 2024-03-11T18:35:04.445 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.442+0000 7f792d67d640 1 -- 172.21.15.92:0/2081630651 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7928153f10 msgr2=0x7f792815a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:04.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.442+0000 7f792d67d640 1 --2- 172.21.15.92:0/2081630651 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7928153f10 0x7f792815a670 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f79180097c0 tx=0x7f791801c4a0 comp rx=0 tx=0).stop 2024-03-11T18:35:04.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.442+0000 7f792d67d640 1 -- 172.21.15.92:0/2081630651 shutdown_connections 2024-03-11T18:35:04.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.442+0000 7f792d67d640 1 --2- 172.21.15.92:0/2081630651 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7928153f10 0x7f792815a670 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:04.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.442+0000 7f792d67d640 1 -- 172.21.15.92:0/2081630651 >> 172.21.15.92:0/2081630651 conn(0x7f7928070e80 msgr2=0x7f7928066990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:35:04.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.442+0000 7f792d67d640 1 -- 172.21.15.92:0/2081630651 shutdown_connections 2024-03-11T18:35:04.446 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.442+0000 7f792d67d640 1 -- 172.21.15.92:0/2081630651 wait complete. 2024-03-11T18:35:04.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.446+0000 7f792d67d640 1 Processor -- start 2024-03-11T18:35:04.447 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.446+0000 7f792d67d640 1 -- start start 2024-03-11T18:35:04.448 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.446+0000 7f792d67d640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f79280afb60 0x7f7928158f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:04.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.446+0000 7f792d67d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79280c8e20 0x7f7928155830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:04.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.446+0000 7f792d67d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7928153f10 0x7f792814de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:04.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.446+0000 7f792d67d640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7928157be0 con 0x7f7928153f10 2024-03-11T18:35:04.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.446+0000 7f792d67d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7928157ec0 con 0x7f79280c8e20 2024-03-11T18:35:04.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.446+0000 7f792d67d640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79281581a0 con 0x7f79280afb60 2024-03-11T18:35:04.449 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.446+0000 7f79267fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79280c8e20 0x7f7928155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:04.450 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.446+0000 7f79267fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79280c8e20 0x7f7928155830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.92:57100/0 (socket says 172.21.15.92:57100) 2024-03-11T18:35:04.451 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.446+0000 7f79267fc640 1 -- 172.21.15.92:0/2542316870 learned_addr learned my addr 172.21.15.92:0/2542316870 (peer_addr_for_me v2:172.21.15.92:0/0) 2024-03-11T18:35:04.457 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.454+0000 7f79277fe640 1 --2- 172.21.15.92:0/2542316870 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7928153f10 0x7f792814de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:04.459 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.458+0000 7f7926ffd640 1 --2- 172.21.15.92:0/2542316870 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f79280afb60 0x7f7928158f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:04.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.458+0000 7f79267fc640 1 -- 172.21.15.92:0/2542316870 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f79280afb60 msgr2=0x7f7928158f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:04.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.458+0000 7f79267fc640 1 --2- 172.21.15.92:0/2542316870 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f79280afb60 0x7f7928158f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:04.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.458+0000 7f79267fc640 1 -- 172.21.15.92:0/2542316870 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7928153f10 msgr2=0x7f792814de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:04.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.458+0000 7f79267fc640 1 --2- 172.21.15.92:0/2542316870 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7928153f10 0x7f792814de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:04.462 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.458+0000 7f79267fc640 1 -- 172.21.15.92:0/2542316870 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7918009420 con 0x7f79280c8e20 2024-03-11T18:35:04.467 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.466+0000 7f79267fc640 1 --2- 172.21.15.92:0/2542316870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79280c8e20 0x7f7928155830 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f791c00c450 tx=0x7f791c00c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:04.474 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.470+0000 7f790f7fe640 1 -- 172.21.15.92:0/2542316870 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791c005cf0 con 0x7f79280c8e20 2024-03-11T18:35:04.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.470+0000 7f790f7fe640 1 -- 172.21.15.92:0/2542316870 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f791c005e50 con 0x7f79280c8e20 2024-03-11T18:35:04.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.470+0000 7f792d67d640 1 -- 172.21.15.92:0/2542316870 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7928158480 con 0x7f79280c8e20 2024-03-11T18:35:04.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.470+0000 7f790f7fe640 1 -- 172.21.15.92:0/2542316870 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791c007290 con 0x7f79280c8e20 2024-03-11T18:35:04.475 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.470+0000 7f792d67d640 1 -- 172.21.15.92:0/2542316870 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7928158a40 con 0x7f79280c8e20 2024-03-11T18:35:04.476 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.474+0000 7f792d67d640 1 -- 172.21.15.92:0/2542316870 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f792804eef0 con 0x7f79280c8e20 2024-03-11T18:35:04.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.758+0000 7f790f7fe640 1 -- 172.21.15.92:0/2542316870 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 3) v1 ==== 86247+0+0 (secure 0 0 0) 0x7f791c0073f0 con 0x7f79280c8e20 2024-03-11T18:35:04.762 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.758+0000 7f790f7fe640 1 --2- 172.21.15.92:0/2542316870 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f7914066ce0 0x7f79140691a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:04.763 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.762+0000 7f7926ffd640 1 --2- 172.21.15.92:0/2542316870 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f7914066ce0 0x7f79140691a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:04.764 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.762+0000 7f7926ffd640 1 --2- 172.21.15.92:0/2542316870 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f7914066ce0 0x7f79140691a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f79180ae000 tx=0x7f7918008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:04.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.794+0000 7f790f7fe640 1 -- 172.21.15.92:0/2542316870 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f791c054f60 con 0x7f79280c8e20 2024-03-11T18:35:04.796 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.794+0000 7f790f7fe640 1 -- 172.21.15.92:0/2542316870 <== mon.1 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f791c051200 con 0x7f79280c8e20 2024-03-11T18:35:04.948 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.946+0000 7f790f7fe640 1 -- 172.21.15.92:0/2542316870 <== mon.1 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 4) v1 ==== 86353+0+0 (secure 0 0 0) 0x7f791c0515d0 con 0x7f79280c8e20 2024-03-11T18:35:04.969 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:04.966+0000 7f792d67d640 1 -- 172.21.15.92:0/2542316870 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd new", "uuid": "958184cc-aebf-4c5b-91a0-2a3e38a6b42b", "id": 7} v 0) v1 -- 0x7f79280c84f0 con 0x7f79280c8e20 2024-03-11T18:35:05.494 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:05.490+0000 7f790f7fe640 1 -- 172.21.15.92:0/2542316870 <== mon.1 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{"prefix": "osd new", "uuid": "958184cc-aebf-4c5b-91a0-2a3e38a6b42b", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f791c076f90 con 0x7f79280c8e20 2024-03-11T18:35:05.494 INFO:teuthology.orchestra.run.smithi092.stdout:7 2024-03-11T18:35:05.497 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:05.494+0000 7f792d67d640 1 -- 172.21.15.92:0/2542316870 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f7914066ce0 msgr2=0x7f79140691a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:05.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:05.494+0000 7f792d67d640 1 --2- 172.21.15.92:0/2542316870 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f7914066ce0 0x7f79140691a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f79180ae000 tx=0x7f7918008040 comp rx=0 tx=0).stop 2024-03-11T18:35:05.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:05.494+0000 7f792d67d640 1 -- 172.21.15.92:0/2542316870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79280c8e20 msgr2=0x7f7928155830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:05.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:05.494+0000 7f792d67d640 1 --2- 172.21.15.92:0/2542316870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79280c8e20 0x7f7928155830 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f791c00c450 tx=0x7f791c00c910 comp rx=0 tx=0).stop 2024-03-11T18:35:05.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:05.494+0000 7f792d67d640 1 -- 172.21.15.92:0/2542316870 shutdown_connections 2024-03-11T18:35:05.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:05.494+0000 7f792d67d640 1 --2- 172.21.15.92:0/2542316870 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f7914066ce0 0x7f79140691a0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:05.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:05.494+0000 7f792d67d640 1 --2- 172.21.15.92:0/2542316870 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7928153f10 0x7f792814de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:05.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:05.494+0000 7f792d67d640 1 --2- 172.21.15.92:0/2542316870 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79280c8e20 0x7f7928155830 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:05.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:05.494+0000 7f792d67d640 1 --2- 172.21.15.92:0/2542316870 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f79280afb60 0x7f7928158f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:05.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:05.494+0000 7f792d67d640 1 -- 172.21.15.92:0/2542316870 >> 172.21.15.92:0/2542316870 conn(0x7f7928070e80 msgr2=0x7f7928153120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:35:05.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:05.494+0000 7f792d67d640 1 -- 172.21.15.92:0/2542316870 shutdown_connections 2024-03-11T18:35:05.498 INFO:teuthology.orchestra.run.smithi092.stderr:2024-03-11T18:35:05.494+0000 7f792d67d640 1 -- 172.21.15.92:0/2542316870 wait complete. 2024-03-11T18:35:05.518 INFO:tasks.ceph.osd.0:Restarting daemon 2024-03-11T18:35:05.519 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-osd -f --cluster ceph -i 0 2024-03-11T18:35:05.521 INFO:tasks.ceph.osd.0:Started 2024-03-11T18:35:05.521 INFO:tasks.ceph.osd.1:Restarting daemon 2024-03-11T18:35:05.521 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-osd -f --cluster ceph -i 1 2024-03-11T18:35:05.524 INFO:tasks.ceph.osd.1:Started 2024-03-11T18:35:05.524 INFO:tasks.ceph.osd.2:Restarting daemon 2024-03-11T18:35:05.524 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-osd -f --cluster ceph -i 2 2024-03-11T18:35:05.526 INFO:tasks.ceph.osd.2:Started 2024-03-11T18:35:05.526 INFO:tasks.ceph.osd.3:Restarting daemon 2024-03-11T18:35:05.526 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-osd -f --cluster ceph -i 3 2024-03-11T18:35:05.529 INFO:tasks.ceph.osd.3:Started 2024-03-11T18:35:05.530 INFO:tasks.ceph.osd.4:Restarting daemon 2024-03-11T18:35:05.530 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-osd -f --cluster ceph -i 4 2024-03-11T18:35:05.532 INFO:tasks.ceph.osd.4:Started 2024-03-11T18:35:05.533 INFO:tasks.ceph.osd.5:Restarting daemon 2024-03-11T18:35:05.533 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-osd -f --cluster ceph -i 5 2024-03-11T18:35:05.535 INFO:tasks.ceph.osd.5:Started 2024-03-11T18:35:05.535 INFO:tasks.ceph.osd.6:Restarting daemon 2024-03-11T18:35:05.535 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-osd -f --cluster ceph -i 6 2024-03-11T18:35:05.537 INFO:tasks.ceph.osd.6:Started 2024-03-11T18:35:05.537 INFO:tasks.ceph.osd.7:Restarting daemon 2024-03-11T18:35:05.537 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-osd -f --cluster ceph -i 7 2024-03-11T18:35:05.539 INFO:tasks.ceph.osd.7:Started 2024-03-11T18:35:05.540 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T18:35:05.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.672+0000 7fbd6a926640 1 Processor -- start 2024-03-11T18:35:05.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.672+0000 7fbd6a926640 1 -- start start 2024-03-11T18:35:05.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.672+0000 7fbd6a926640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fbd64153cf0 0x7fbd6415d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:05.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.672+0000 7fbd6a926640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbd640c4b60 0x7fbd6415a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:05.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.672+0000 7fbd6a926640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbd640c42f0 0x7fbd640ca370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:05.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.672+0000 7fbd6a926640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd6415e120 con 0x7fbd64153cf0 2024-03-11T18:35:05.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.672+0000 7fbd6a926640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd64159b30 con 0x7fbd640c42f0 2024-03-11T18:35:05.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.672+0000 7fbd6a926640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6415d660 con 0x7fbd640c4b60 2024-03-11T18:35:05.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.672+0000 7fbd637fe640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbd640c4b60 0x7fbd6415a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:05.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.672+0000 7fbd63fff640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fbd64153cf0 0x7fbd6415d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:05.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.672+0000 7fbd68ec3640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbd640c42f0 0x7fbd640ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:05.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.672+0000 7fbd637fe640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbd640c4b60 0x7fbd6415a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.90:48830/0 (socket says 172.21.15.90:48830) 2024-03-11T18:35:05.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.672+0000 7fbd637fe640 1 -- 172.21.15.90:0/3209414025 learned_addr learned my addr 172.21.15.90:0/3209414025 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:35:05.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.676+0000 7fbd637fe640 1 -- 172.21.15.90:0/3209414025 >> v2:172.21.15.92:3300/0 conn(0x7fbd640c42f0 msgr2=0x7fbd640ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:35:05.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.676+0000 7fbd637fe640 1 --2- 172.21.15.90:0/3209414025 >> v2:172.21.15.92:3300/0 conn(0x7fbd640c42f0 0x7fbd640ca370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:05.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.676+0000 7fbd637fe640 1 -- 172.21.15.90:0/3209414025 >> v2:172.21.15.90:3300/0 conn(0x7fbd64153cf0 msgr2=0x7fbd6415d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:35:05.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.676+0000 7fbd637fe640 1 --2- 172.21.15.90:0/3209414025 >> v2:172.21.15.90:3300/0 conn(0x7fbd64153cf0 0x7fbd6415d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:05.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.676+0000 7fbd637fe640 1 -- 172.21.15.90:0/3209414025 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd6415a0b0 con 0x7fbd640c4b60 2024-03-11T18:35:05.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.680+0000 7fbd637fe640 1 --2- 172.21.15.90:0/3209414025 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbd640c4b60 0x7fbd6415a540 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fbd6408caa0 tx=0x7fbd5801c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26fd3e13dbf8976d server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:05.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.684+0000 7fbd62ffd640 1 -- 172.21.15.90:0/3209414025 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd580aa640 con 0x7fbd640c4b60 2024-03-11T18:35:05.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.684+0000 7fbd62ffd640 1 -- 172.21.15.90:0/3209414025 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd580aa7a0 con 0x7fbd640c4b60 2024-03-11T18:35:05.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.684+0000 7fbd62ffd640 1 -- 172.21.15.90:0/3209414025 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd580aab30 con 0x7fbd640c4b60 2024-03-11T18:35:05.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.684+0000 7fbd6a926640 1 -- 172.21.15.90:0/3209414025 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbd640c4b60 msgr2=0x7fbd6415a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:05.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.684+0000 7fbd6a926640 1 --2- 172.21.15.90:0/3209414025 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbd640c4b60 0x7fbd6415a540 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fbd6408caa0 tx=0x7fbd5801c520 comp rx=0 tx=0).stop 2024-03-11T18:35:05.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.684+0000 7fbd6a926640 1 -- 172.21.15.90:0/3209414025 shutdown_connections 2024-03-11T18:35:05.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.684+0000 7fbd6a926640 1 --2- 172.21.15.90:0/3209414025 >> v2:172.21.15.92:3300/0 conn(0x7fbd640c42f0 0x7fbd640ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:05.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.684+0000 7fbd6a926640 1 --2- 172.21.15.90:0/3209414025 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbd640c4b60 0x7fbd6415a540 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:05.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.684+0000 7fbd6a926640 1 --2- 172.21.15.90:0/3209414025 >> v2:172.21.15.90:3300/0 conn(0x7fbd64153cf0 0x7fbd6415d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:05.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.684+0000 7fbd6a926640 1 -- 172.21.15.90:0/3209414025 >> 172.21.15.90:0/3209414025 conn(0x7fbd6406b640 msgr2=0x7fbd6406ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:35:05.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.684+0000 7fbd6a926640 1 -- 172.21.15.90:0/3209414025 shutdown_connections 2024-03-11T18:35:05.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.684+0000 7fbd6a926640 1 -- 172.21.15.90:0/3209414025 wait complete. 2024-03-11T18:35:05.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.684+0000 7fbd6a926640 1 Processor -- start 2024-03-11T18:35:05.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.684+0000 7fbd6a926640 1 -- start start 2024-03-11T18:35:05.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.684+0000 7fbd6a926640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbd640c42f0 0x7fbd640cf380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:05.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.684+0000 7fbd6a926640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd640c4b60 0x7fbd641f2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:05.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.684+0000 7fbd6a926640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd64153cf0 0x7fbd64149bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:05.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.684+0000 7fbd6a926640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd641572f0 con 0x7fbd640c4b60 2024-03-11T18:35:05.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.684+0000 7fbd6a926640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd640c96e0 con 0x7fbd64153cf0 2024-03-11T18:35:05.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.684+0000 7fbd6a926640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd641f6850 con 0x7fbd640c42f0 2024-03-11T18:35:05.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.688+0000 7fbd637fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd640c4b60 0x7fbd641f2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:05.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.688+0000 7fbd68ec3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd64153cf0 0x7fbd64149bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:05.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.688+0000 7fbd637fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd640c4b60 0x7fbd641f2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58990/0 (socket says 172.21.15.90:58990) 2024-03-11T18:35:05.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.688+0000 7fbd637fe640 1 -- 172.21.15.90:0/323341164 learned_addr learned my addr 172.21.15.90:0/323341164 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:35:05.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.692+0000 7fbd637fe640 1 -- 172.21.15.90:0/323341164 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbd640c42f0 msgr2=0x7fbd640cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:05.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.692+0000 7fbd637fe640 1 --2- 172.21.15.90:0/323341164 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbd640c42f0 0x7fbd640cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:05.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.692+0000 7fbd637fe640 1 -- 172.21.15.90:0/323341164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd64153cf0 msgr2=0x7fbd64149bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:05.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.692+0000 7fbd637fe640 1 --2- 172.21.15.90:0/323341164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd64153cf0 0x7fbd64149bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:05.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.692+0000 7fbd637fe640 1 -- 172.21.15.90:0/323341164 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd580094d0 con 0x7fbd640c4b60 2024-03-11T18:35:05.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.692+0000 7fbd637fe640 1 --2- 172.21.15.90:0/323341164 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd640c4b60 0x7fbd641f2880 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbd5801abd0 tx=0x7fbd5801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:05.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.696+0000 7fbd60ff9640 1 -- 172.21.15.90:0/323341164 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd580bd3f0 con 0x7fbd640c4b60 2024-03-11T18:35:05.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.696+0000 7fbd60ff9640 1 -- 172.21.15.90:0/323341164 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd580bd570 con 0x7fbd640c4b60 2024-03-11T18:35:05.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.696+0000 7fbd6a926640 1 -- 172.21.15.90:0/323341164 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd641f6ce0 con 0x7fbd640c4b60 2024-03-11T18:35:05.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.696+0000 7fbd60ff9640 1 -- 172.21.15.90:0/323341164 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd580bd920 con 0x7fbd640c4b60 2024-03-11T18:35:05.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.696+0000 7fbd6a926640 1 -- 172.21.15.90:0/323341164 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd641f5f20 con 0x7fbd640c4b60 2024-03-11T18:35:05.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.696+0000 7fbd6a926640 1 -- 172.21.15.90:0/323341164 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd6404eef0 con 0x7fbd640c4b60 2024-03-11T18:35:05.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.976+0000 7fbd60ff9640 1 -- 172.21.15.90:0/323341164 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 4) v1 ==== 86353+0+0 (secure 0 0 0) 0x7fbd580a8050 con 0x7fbd640c4b60 2024-03-11T18:35:05.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.980+0000 7fbd60ff9640 1 --2- 172.21.15.90:0/323341164 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fbd50066e00 0x7fbd500692c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:05.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.980+0000 7fbd63fff640 1 --2- 172.21.15.90:0/323341164 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fbd50066e00 0x7fbd500692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:05.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.980+0000 7fbd63fff640 1 --2- 172.21.15.90:0/323341164 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fbd50066e00 0x7fbd500692c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c0099c0 tx=0x7fbd5c009480 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:05.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.992+0000 7fbd60ff9640 1 -- 172.21.15.90:0/323341164 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fbd580ef9e0 con 0x7fbd640c4b60 2024-03-11T18:35:05.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:05.992+0000 7fbd60ff9640 1 -- 172.21.15.90:0/323341164 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbd580ebc80 con 0x7fbd640c4b60 2024-03-11T18:35:06.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:06.156+0000 7fbd6a926640 1 -- 172.21.15.90:0/323341164 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbd641f69c0 con 0x7fbd640c4b60 2024-03-11T18:35:06.173 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-03-11T18:35:06.170+0000 7f2e7a8977c0 -1 Falling back to public interface 2024-03-11T18:35:06.181 INFO:tasks.ceph.osd.1.smithi090.stderr:2024-03-11T18:35:06.176+0000 7f6ff91757c0 -1 Falling back to public interface 2024-03-11T18:35:06.181 INFO:tasks.ceph.osd.3.smithi090.stderr:2024-03-11T18:35:06.176+0000 7f36c7d6e7c0 -1 Falling back to public interface 2024-03-11T18:35:06.182 INFO:tasks.ceph.osd.1.smithi090.stderr:2024-03-11T18:35:06.176+0000 7f6ff8135640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6803/0: (98) Address already in use 2024-03-11T18:35:06.188 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-03-11T18:35:06.186+0000 7f0509f677c0 -1 Falling back to public interface 2024-03-11T18:35:06.191 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-03-11T18:35:06.190+0000 7f366f8d47c0 -1 Falling back to public interface 2024-03-11T18:35:06.197 INFO:tasks.ceph.osd.0.smithi090.stderr:2024-03-11T18:35:06.192+0000 7fe9a70857c0 -1 Falling back to public interface 2024-03-11T18:35:06.205 INFO:tasks.ceph.osd.2.smithi090.stderr:2024-03-11T18:35:06.200+0000 7f75521257c0 -1 Falling back to public interface 2024-03-11T18:35:06.221 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-03-11T18:35:06.218+0000 7f3e1f0db7c0 -1 Falling back to public interface 2024-03-11T18:35:06.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:06.240+0000 7fbd60ff9640 1 -- 172.21.15.90:0/323341164 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7fbd580ef340 con 0x7fbd640c4b60 2024-03-11T18:35:06.248 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:35:06.248 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":10,"fsid":"76d56b10-6c29-476a-804b-8482ea90f9e2","created":"2024-03-11T18:34:54.521741+0000","modified":"2024-03-11T18:35:05.213634+0000","last_up_change":"0.000000","last_in_change":"2024-03-11T18:35:05.213634+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"34decff9-7280-4d84-9665-d6af8ea6de84","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8ecdce9e-5486-415a-a609-dd16e945440e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"737486d4-0e90-4c82-9e79-bf37c812e5ad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"698bd661-bbef-4699-a619-f7bfe23ce12c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"aba9e31f-73d9-43fe-a10b-2dc5832968b4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"bdd56766-4b28-4007-8de3-b5b08cf335d3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"6c580a59-610f-455e-a96b-033299dcc118","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"958184cc-aebf-4c5b-91a0-2a3e38a6b42b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T18:35:06.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:06.244+0000 7fbd6a926640 1 -- 172.21.15.90:0/323341164 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fbd50066e00 msgr2=0x7fbd500692c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:06.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:06.244+0000 7fbd6a926640 1 --2- 172.21.15.90:0/323341164 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fbd50066e00 0x7fbd500692c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c0099c0 tx=0x7fbd5c009480 comp rx=0 tx=0).stop 2024-03-11T18:35:06.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:06.244+0000 7fbd6a926640 1 -- 172.21.15.90:0/323341164 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd640c4b60 msgr2=0x7fbd641f2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:06.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:06.244+0000 7fbd6a926640 1 --2- 172.21.15.90:0/323341164 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd640c4b60 0x7fbd641f2880 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbd5801abd0 tx=0x7fbd5801ad60 comp rx=0 tx=0).stop 2024-03-11T18:35:06.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:06.244+0000 7fbd6a926640 1 -- 172.21.15.90:0/323341164 shutdown_connections 2024-03-11T18:35:06.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:06.244+0000 7fbd6a926640 1 --2- 172.21.15.90:0/323341164 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fbd50066e00 0x7fbd500692c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:06.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:06.244+0000 7fbd6a926640 1 --2- 172.21.15.90:0/323341164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbd64153cf0 0x7fbd64149bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:06.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:06.244+0000 7fbd6a926640 1 --2- 172.21.15.90:0/323341164 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd640c4b60 0x7fbd641f2880 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:06.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:06.244+0000 7fbd6a926640 1 --2- 172.21.15.90:0/323341164 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbd640c42f0 0x7fbd640cf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:06.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:06.244+0000 7fbd6a926640 1 -- 172.21.15.90:0/323341164 >> 172.21.15.90:0/323341164 conn(0x7fbd6406b640 msgr2=0x7fbd6415cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:35:06.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:06.244+0000 7fbd6a926640 1 -- 172.21.15.90:0/323341164 shutdown_connections 2024-03-11T18:35:06.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:06.244+0000 7fbd6a926640 1 -- 172.21.15.90:0/323341164 wait complete. 2024-03-11T18:35:06.268 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-03-11T18:35:06.268 INFO:tasks.ceph:Waiting for OSDs to come up 2024-03-11T18:35:07.074 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-11T18:35:07.197 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.192+0000 7f9b58ca3640 1 Processor -- start 2024-03-11T18:35:07.197 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.192+0000 7f9b58ca3640 1 -- start start 2024-03-11T18:35:07.198 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.192+0000 7f9b58ca3640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9b540c9b30 0x7f9b5406b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:07.198 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.192+0000 7f9b58ca3640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f9b540726c0 0x7f9b54151990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:07.198 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.192+0000 7f9b58ca3640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b540c89e0 con 0x7f9b540c5c90 2024-03-11T18:35:07.198 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.192+0000 7f9b58ca3640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b540c9710 con 0x7f9b540726c0 2024-03-11T18:35:07.198 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.192+0000 7f9b58ca3640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b540c2c70 con 0x7f9b540c9b30 2024-03-11T18:35:07.199 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.192+0000 7f9b52d9d640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f9b540726c0 0x7f9b54151990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:07.199 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.192+0000 7f9b5259c640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f9b540c5c90 0x7f9b5406af10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:33040/0 (socket says 172.21.15.90:33040) 2024-03-11T18:35:07.199 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.192+0000 7f9b5259c640 1 -- 172.21.15.90:0/470278643 learned_addr learned my addr 172.21.15.90:0/470278643 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T18:35:07.199 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.192+0000 7f9b51d9b640 1 --2- 172.21.15.90:0/470278643 >> v2:172.21.15.92:3300/0 conn(0x7f9b540c9b30 0x7f9b5406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:07.203 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.196+0000 7f9b52d9d640 1 -- 172.21.15.90:0/470278643 >> v2:172.21.15.92:3300/0 conn(0x7f9b540c9b30 msgr2=0x7f9b5406b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:35:07.203 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.196+0000 7f9b52d9d640 1 --2- 172.21.15.90:0/470278643 >> v2:172.21.15.92:3300/0 conn(0x7f9b540c9b30 0x7f9b5406b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:07.203 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.196+0000 7f9b52d9d640 1 -- 172.21.15.90:0/470278643 >> v1:172.21.15.90:6789/0 conn(0x7f9b540c5c90 legacy=0x7f9b5406af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:07.203 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.196+0000 7f9b52d9d640 1 -- 172.21.15.90:0/470278643 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b540c8ed0 con 0x7f9b540726c0 2024-03-11T18:35:07.207 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.200+0000 7f9b52d9d640 1 --2- 172.21.15.90:0/470278643 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b540726c0 0x7f9b54151990 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9b54155790 tx=0x7f9b4c01f110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c70a1f8bb7356ee server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:07.212 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.204+0000 7f9b5159a640 1 -- 172.21.15.90:0/470278643 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c0af600 con 0x7f9b540726c0 2024-03-11T18:35:07.212 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.204+0000 7f9b5159a640 1 -- 172.21.15.90:0/470278643 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b4c0ab050 con 0x7f9b540726c0 2024-03-11T18:35:07.212 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.204+0000 7f9b5159a640 1 -- 172.21.15.90:0/470278643 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c0af980 con 0x7f9b540726c0 2024-03-11T18:35:07.213 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.208+0000 7f9b58ca3640 1 -- 172.21.15.90:0/470278643 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b540726c0 msgr2=0x7f9b54151990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:07.213 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.208+0000 7f9b58ca3640 1 --2- 172.21.15.90:0/470278643 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b540726c0 0x7f9b54151990 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9b54155790 tx=0x7f9b4c01f110 comp rx=0 tx=0).stop 2024-03-11T18:35:07.213 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.208+0000 7f9b58ca3640 1 -- 172.21.15.90:0/470278643 shutdown_connections 2024-03-11T18:35:07.213 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.208+0000 7f9b58ca3640 1 --2- 172.21.15.90:0/470278643 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b540726c0 0x7f9b54151990 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:07.214 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.208+0000 7f9b58ca3640 1 --2- 172.21.15.90:0/470278643 >> v2:172.21.15.92:3300/0 conn(0x7f9b540c9b30 0x7f9b5406b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:07.214 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.208+0000 7f9b58ca3640 1 -- 172.21.15.90:0/470278643 >> 172.21.15.90:0/470278643 conn(0x7f9b5406bd60 msgr2=0x7f9b5406c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:35:07.214 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.208+0000 7f9b58ca3640 1 -- 172.21.15.90:0/470278643 shutdown_connections 2024-03-11T18:35:07.214 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.208+0000 7f9b58ca3640 1 -- 172.21.15.90:0/470278643 wait complete. 2024-03-11T18:35:07.214 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.208+0000 7f9b58ca3640 1 Processor -- start 2024-03-11T18:35:07.214 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.208+0000 7f9b58ca3640 1 -- start start 2024-03-11T18:35:07.215 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.208+0000 7f9b58ca3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b540726c0 0x7f9b54155d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:07.215 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.208+0000 7f9b58ca3640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9b540c5c90 0x7f9b540cc550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:07.215 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.208+0000 7f9b58ca3640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b540c9b30 0x7f9b540bc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:07.215 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.208+0000 7f9b58ca3640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b54154500 con 0x7f9b540c9b30 2024-03-11T18:35:07.215 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.208+0000 7f9b58ca3640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b541f0610 con 0x7f9b540726c0 2024-03-11T18:35:07.215 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.208+0000 7f9b58ca3640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b541ef760 con 0x7f9b540c5c90 2024-03-11T18:35:07.215 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.208+0000 7f9b52d9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b540c9b30 0x7f9b540bc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:07.216 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.208+0000 7f9b52d9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b540c9b30 0x7f9b540bc5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59008/0 (socket says 172.21.15.90:59008) 2024-03-11T18:35:07.216 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.208+0000 7f9b52d9d640 1 -- 172.21.15.90:0/2167680272 learned_addr learned my addr 172.21.15.90:0/2167680272 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:35:07.216 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.212+0000 7f9b51d9b640 1 --2- 172.21.15.90:0/2167680272 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9b540c5c90 0x7f9b540cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:07.216 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.212+0000 7f9b5259c640 1 --2- 172.21.15.90:0/2167680272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b540726c0 0x7f9b54155d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:07.219 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.212+0000 7f9b52d9d640 1 -- 172.21.15.90:0/2167680272 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9b540c5c90 msgr2=0x7f9b540cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:07.219 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.212+0000 7f9b52d9d640 1 --2- 172.21.15.90:0/2167680272 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9b540c5c90 0x7f9b540cc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:07.219 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.212+0000 7f9b52d9d640 1 -- 172.21.15.90:0/2167680272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b540726c0 msgr2=0x7f9b54155d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:07.219 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.212+0000 7f9b52d9d640 1 --2- 172.21.15.90:0/2167680272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b540726c0 0x7f9b54155d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:07.219 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.212+0000 7f9b52d9d640 1 -- 172.21.15.90:0/2167680272 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b4c009a70 con 0x7f9b540c9b30 2024-03-11T18:35:07.221 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.216+0000 7f9b52d9d640 1 --2- 172.21.15.90:0/2167680272 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b540c9b30 0x7f9b540bc5a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c01bd20 tx=0x7f9b4c01beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:07.225 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.220+0000 7f9b4affd640 1 -- 172.21.15.90:0/2167680272 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c0afca0 con 0x7f9b540c9b30 2024-03-11T18:35:07.225 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.220+0000 7f9b4affd640 1 -- 172.21.15.90:0/2167680272 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b4c0afe00 con 0x7f9b540c9b30 2024-03-11T18:35:07.226 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.220+0000 7f9b4affd640 1 -- 172.21.15.90:0/2167680272 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c0ba660 con 0x7f9b540c9b30 2024-03-11T18:35:07.226 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.220+0000 7f9b58ca3640 1 -- 172.21.15.90:0/2167680272 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b541efa00 con 0x7f9b540c9b30 2024-03-11T18:35:07.226 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.220+0000 7f9b58ca3640 1 -- 172.21.15.90:0/2167680272 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b541effc0 con 0x7f9b540c9b30 2024-03-11T18:35:07.227 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.220+0000 7f9b58ca3640 1 -- 172.21.15.90:0/2167680272 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b540bc2c0 con 0x7f9b540c9b30 2024-03-11T18:35:07.246 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.240+0000 7f9b4affd640 1 -- 172.21.15.90:0/2167680272 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 4) v1 ==== 86353+0+0 (secure 0 0 0) 0x7f9b4c0ba7c0 con 0x7f9b540c9b30 2024-03-11T18:35:07.246 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.240+0000 7f9b4affd640 1 --2- 172.21.15.90:0/2167680272 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f9b34066e00 0x7f9b340692c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:07.246 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.240+0000 7f9b5259c640 1 --2- 172.21.15.90:0/2167680272 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f9b34066e00 0x7f9b340692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:07.247 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.240+0000 7f9b5259c640 1 --2- 172.21.15.90:0/2167680272 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f9b34066e00 0x7f9b340692c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c000f70 tx=0x7f9b3c007e90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:07.578 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.572+0000 7f9b4affd640 1 -- 172.21.15.90:0/2167680272 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f9b4c0f42d0 con 0x7f9b540c9b30 2024-03-11T18:35:07.579 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.572+0000 7f9b4affd640 1 -- 172.21.15.90:0/2167680272 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9b4c0f0570 con 0x7f9b540c9b30 2024-03-11T18:35:07.739 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.732+0000 7f9b58ca3640 1 -- 172.21.15.90:0/2167680272 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9b540bc2c0 con 0x7f9b540c9b30 2024-03-11T18:35:07.765 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.760+0000 7f9b4affd640 1 -- 172.21.15.90:0/2167680272 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f9b540bc2c0 con 0x7f9b540c9b30 2024-03-11T18:35:07.766 INFO:teuthology.misc.health.smithi090.stdout: 2024-03-11T18:35:07.766 INFO:teuthology.misc.health.smithi090.stdout:{"epoch":10,"fsid":"76d56b10-6c29-476a-804b-8482ea90f9e2","created":"2024-03-11T18:34:54.521741+0000","modified":"2024-03-11T18:35:05.213634+0000","last_up_change":"0.000000","last_in_change":"2024-03-11T18:35:05.213634+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"34decff9-7280-4d84-9665-d6af8ea6de84","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8ecdce9e-5486-415a-a609-dd16e945440e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"737486d4-0e90-4c82-9e79-bf37c812e5ad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"698bd661-bbef-4699-a619-f7bfe23ce12c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"aba9e31f-73d9-43fe-a10b-2dc5832968b4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"bdd56766-4b28-4007-8de3-b5b08cf335d3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"6c580a59-610f-455e-a96b-033299dcc118","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"958184cc-aebf-4c5b-91a0-2a3e38a6b42b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T18:35:07.769 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.760+0000 7f9b58ca3640 1 -- 172.21.15.90:0/2167680272 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f9b34066e00 msgr2=0x7f9b340692c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:07.769 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.760+0000 7f9b58ca3640 1 --2- 172.21.15.90:0/2167680272 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f9b34066e00 0x7f9b340692c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c000f70 tx=0x7f9b3c007e90 comp rx=0 tx=0).stop 2024-03-11T18:35:07.769 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.760+0000 7f9b58ca3640 1 -- 172.21.15.90:0/2167680272 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b540c9b30 msgr2=0x7f9b540bc5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:07.769 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.760+0000 7f9b58ca3640 1 --2- 172.21.15.90:0/2167680272 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b540c9b30 0x7f9b540bc5a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c01bd20 tx=0x7f9b4c01beb0 comp rx=0 tx=0).stop 2024-03-11T18:35:07.769 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.760+0000 7f9b58ca3640 1 -- 172.21.15.90:0/2167680272 shutdown_connections 2024-03-11T18:35:07.769 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.760+0000 7f9b58ca3640 1 --2- 172.21.15.90:0/2167680272 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f9b34066e00 0x7f9b340692c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:07.770 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.760+0000 7f9b58ca3640 1 --2- 172.21.15.90:0/2167680272 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b540c9b30 0x7f9b540bc5a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:07.770 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.760+0000 7f9b58ca3640 1 --2- 172.21.15.90:0/2167680272 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9b540c5c90 0x7f9b540cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:07.770 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.760+0000 7f9b58ca3640 1 --2- 172.21.15.90:0/2167680272 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9b540726c0 0x7f9b54155d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:07.770 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.760+0000 7f9b58ca3640 1 -- 172.21.15.90:0/2167680272 >> 172.21.15.90:0/2167680272 conn(0x7f9b5406bd60 msgr2=0x7f9b540c4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:35:07.770 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.760+0000 7f9b58ca3640 1 -- 172.21.15.90:0/2167680272 shutdown_connections 2024-03-11T18:35:07.770 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:07.760+0000 7f9b58ca3640 1 -- 172.21.15.90:0/2167680272 wait complete. 2024-03-11T18:35:07.786 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-11T18:35:09.832 INFO:tasks.ceph.osd.0.smithi090.stderr:2024-03-11T18:35:09.824+0000 7fe9a70857c0 -1 osd.0 0 log_to_monitors true 2024-03-11T18:35:09.919 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-03-11T18:35:09.918+0000 7f0509f677c0 -1 osd.5 0 log_to_monitors true 2024-03-11T18:35:09.955 INFO:tasks.ceph.osd.3.smithi090.stderr:2024-03-11T18:35:09.948+0000 7f36c7d6e7c0 -1 osd.3 0 log_to_monitors true 2024-03-11T18:35:10.013 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-03-11T18:35:10.010+0000 7f2e7a8977c0 -1 osd.4 0 log_to_monitors true 2024-03-11T18:35:10.025 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-03-11T18:35:10.022+0000 7f366f8d47c0 -1 osd.6 0 log_to_monitors true 2024-03-11T18:35:10.035 INFO:tasks.ceph.osd.1.smithi090.stderr:2024-03-11T18:35:10.028+0000 7f6ff91757c0 -1 osd.1 0 log_to_monitors true 2024-03-11T18:35:10.089 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-03-11T18:35:10.086+0000 7f3e1f0db7c0 -1 osd.7 0 log_to_monitors true 2024-03-11T18:35:10.119 INFO:tasks.ceph.osd.2.smithi090.stderr:2024-03-11T18:35:10.112+0000 7f75521257c0 -1 osd.2 0 log_to_monitors true 2024-03-11T18:35:12.106 INFO:tasks.ceph.osd.0.smithi090.stderr:2024-03-11T18:35:12.100+0000 7fe9a302e640 -1 osd.0 0 waiting for initial osdmap 2024-03-11T18:35:12.525 INFO:tasks.ceph.osd.0.smithi090.stderr:2024-03-11T18:35:12.520+0000 7fe99ae63640 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-11T18:35:13.975 INFO:tasks.ceph.osd.3.smithi090.stderr:2024-03-11T18:35:13.968+0000 7f36c4529640 -1 osd.3 0 waiting for initial osdmap 2024-03-11T18:35:13.999 INFO:tasks.ceph.osd.1.smithi090.stderr:2024-03-11T18:35:13.992+0000 7f6ff5930640 -1 osd.1 0 waiting for initial osdmap 2024-03-11T18:35:14.024 INFO:tasks.ceph.osd.2.smithi090.stderr:2024-03-11T18:35:14.016+0000 7f754e8e0640 -1 osd.2 0 waiting for initial osdmap 2024-03-11T18:35:14.033 INFO:tasks.ceph.osd.3.smithi090.stderr:2024-03-11T18:35:14.028+0000 7f36bbb4c640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-11T18:35:14.073 INFO:tasks.ceph.osd.1.smithi090.stderr:2024-03-11T18:35:14.068+0000 7f6fecf53640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-11T18:35:14.093 INFO:tasks.ceph.osd.2.smithi090.stderr:2024-03-11T18:35:14.088+0000 7f7545f03640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-11T18:35:14.520 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-03-11T18:35:14.514+0000 7f366b87d640 -1 osd.6 0 waiting for initial osdmap 2024-03-11T18:35:14.556 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-03-11T18:35:14.554+0000 7f36636b2640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-11T18:35:14.563 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-03-11T18:35:14.562+0000 7f3e1b084640 -1 osd.7 0 waiting for initial osdmap 2024-03-11T18:35:14.565 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-03-11T18:35:14.562+0000 7f2e76840640 -1 osd.4 0 waiting for initial osdmap 2024-03-11T18:35:14.592 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-11T18:35:14.607 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-03-11T18:35:14.606+0000 7f3e12eb9640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-11T18:35:14.608 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-03-11T18:35:14.606+0000 7f0505f10640 -1 osd.5 0 waiting for initial osdmap 2024-03-11T18:35:14.663 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-03-11T18:35:14.662+0000 7f2e6e675640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-11T18:35:14.691 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-03-11T18:35:14.690+0000 7f04fdd45640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-11T18:35:14.720 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:14.716+0000 7f7315e0f640 1 Processor -- start 2024-03-11T18:35:14.721 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:14.716+0000 7f7315e0f640 1 -- start start 2024-03-11T18:35:14.721 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:14.716+0000 7f7315e0f640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f731006c930 0x7f731015dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:14.722 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:14.716+0000 7f7315e0f640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f731006cd10 0x7f7310156eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:14.722 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:14.716+0000 7f7315e0f640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f731015e280 con 0x7f73100c3ed0 2024-03-11T18:35:14.722 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:14.716+0000 7f7315e0f640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f731015a240 con 0x7f731006c930 2024-03-11T18:35:14.722 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:14.716+0000 7f7315e0f640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73100c4360 con 0x7f731006cd10 2024-03-11T18:35:14.723 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:14.716+0000 7f730effd640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f73100c3ed0 0x7f731015a6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.90:44468/0 (socket says 172.21.15.90:44468) 2024-03-11T18:35:14.723 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:14.716+0000 7f730effd640 1 -- 172.21.15.90:0/3821003120 learned_addr learned my addr 172.21.15.90:0/3821003120 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T18:35:14.724 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:14.720+0000 7f730ffff640 1 --2- 172.21.15.90:0/3821003120 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f731006cd10 0x7f7310156eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:14.732 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:14.724+0000 7f730ffff640 1 -- 172.21.15.90:0/3821003120 >> v2:172.21.15.90:3300/0 conn(0x7f731006c930 msgr2=0x7f731015dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:35:14.732 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:14.724+0000 7f730ffff640 1 --2- 172.21.15.90:0/3821003120 >> v2:172.21.15.90:3300/0 conn(0x7f731006c930 0x7f731015dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:14.732 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:14.724+0000 7f730ffff640 1 -- 172.21.15.90:0/3821003120 >> v1:172.21.15.92:6789/0 conn(0x7f73100c3ed0 legacy=0x7f731015a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:14.732 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:14.724+0000 7f730ffff640 1 -- 172.21.15.90:0/3821003120 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73100c7eb0 con 0x7f731006cd10 2024-03-11T18:35:14.737 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:14.732+0000 7f730ffff640 1 --2- 172.21.15.90:0/3821003120 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f731006cd10 0x7f7310156eb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f73080097c0 tx=0x7f730801ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca89cfe6113c2aba server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:14.746 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:14.740+0000 7f730e7fc640 1 -- 172.21.15.90:0/3821003120 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73080a8990 con 0x7f731006cd10 2024-03-11T18:35:14.746 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:14.740+0000 7f730e7fc640 1 -- 172.21.15.90:0/3821003120 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73080a8af0 con 0x7f731006cd10 2024-03-11T18:35:14.747 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:14.740+0000 7f730e7fc640 1 -- 172.21.15.90:0/3821003120 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73080a8e80 con 0x7f731006cd10 2024-03-11T18:35:14.747 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:14.740+0000 7f7315e0f640 1 -- 172.21.15.90:0/3821003120 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f731006cd10 msgr2=0x7f7310156eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:14.747 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:14.740+0000 7f7315e0f640 1 --2- 172.21.15.90:0/3821003120 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f731006cd10 0x7f7310156eb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f73080097c0 tx=0x7f730801ec90 comp rx=0 tx=0).stop 2024-03-11T18:35:14.747 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:14.740+0000 7f7315e0f640 1 -- 172.21.15.90:0/3821003120 shutdown_connections 2024-03-11T18:35:14.747 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:14.740+0000 7f7315e0f640 1 --2- 172.21.15.90:0/3821003120 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f731006cd10 0x7f7310156eb0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:14.747 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:14.740+0000 7f7315e0f640 1 --2- 172.21.15.90:0/3821003120 >> v2:172.21.15.90:3300/0 conn(0x7f731006c930 0x7f731015dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:14.747 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:14.740+0000 7f7315e0f640 1 -- 172.21.15.90:0/3821003120 >> 172.21.15.90:0/3821003120 conn(0x7f73100708b0 msgr2=0x7f7310070cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:35:14.747 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:14.740+0000 7f7315e0f640 1 -- 172.21.15.90:0/3821003120 shutdown_connections 2024-03-11T18:35:14.747 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:14.740+0000 7f7315e0f640 1 -- 172.21.15.90:0/3821003120 wait complete. 2024-03-11T18:35:14.748 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:14.740+0000 7f7315e0f640 1 Processor -- start 2024-03-11T18:35:14.748 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:14.740+0000 7f7315e0f640 1 -- start start 2024-03-11T18:35:14.748 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:14.740+0000 7f7315e0f640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f731006c930 0x7f73100af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:14.748 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:14.740+0000 7f7315e0f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f731006cd10 0x7f73101515b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:14.748 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:14.740+0000 7f7315e0f640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f731015af20 0x7f73100b21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:14.748 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:14.740+0000 7f7315e0f640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73100b1370 con 0x7f731006c930 2024-03-11T18:35:14.748 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:14.740+0000 7f7315e0f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7310158eb0 con 0x7f731006cd10 2024-03-11T18:35:14.748 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:14.740+0000 7f7315e0f640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73100b1d40 con 0x7f731015af20 2024-03-11T18:35:14.749 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:14.744+0000 7f730effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f731006cd10 0x7f73101515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:14.750 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:14.744+0000 7f730ffff640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f731015af20 0x7f73100b21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:14.750 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:14.744+0000 7f730effd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f731006cd10 0x7f73101515b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.90:51060/0 (socket says 172.21.15.90:51060) 2024-03-11T18:35:14.750 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:14.744+0000 7f730effd640 1 -- 172.21.15.90:0/2232897641 learned_addr learned my addr 172.21.15.90:0/2232897641 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:35:14.756 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:14.752+0000 7f730ffff640 1 -- 172.21.15.90:0/2232897641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f731006cd10 msgr2=0x7f73101515b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:14.756 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:14.752+0000 7f730ffff640 1 --2- 172.21.15.90:0/2232897641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f731006cd10 0x7f73101515b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:14.756 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:14.752+0000 7f730ffff640 1 -- 172.21.15.90:0/2232897641 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f731006c930 msgr2=0x7f73100af2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:14.757 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:14.752+0000 7f730ffff640 1 --2- 172.21.15.90:0/2232897641 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f731006c930 0x7f73100af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:14.757 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:14.752+0000 7f730ffff640 1 -- 172.21.15.90:0/2232897641 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7308009420 con 0x7f731015af20 2024-03-11T18:35:14.762 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:14.756+0000 7f730ffff640 1 --2- 172.21.15.90:0/2232897641 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f731015af20 0x7f73100b21d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f73080af000 tx=0x7f730801ba30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:14.774 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:14.768+0000 7f72f7fff640 1 -- 172.21.15.90:0/2232897641 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73080a8930 con 0x7f731015af20 2024-03-11T18:35:14.774 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:14.768+0000 7f72f7fff640 1 -- 172.21.15.90:0/2232897641 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73080b9410 con 0x7f731015af20 2024-03-11T18:35:14.774 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:14.768+0000 7f72f7fff640 1 -- 172.21.15.90:0/2232897641 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73080b97a0 con 0x7f731015af20 2024-03-11T18:35:14.774 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:14.768+0000 7f7315e0f640 1 -- 172.21.15.90:0/2232897641 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73100afa40 con 0x7f731015af20 2024-03-11T18:35:14.774 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:14.768+0000 7f7315e0f640 1 -- 172.21.15.90:0/2232897641 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73101593e0 con 0x7f731015af20 2024-03-11T18:35:14.775 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:14.768+0000 7f7315e0f640 1 -- 172.21.15.90:0/2232897641 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f731004eef0 con 0x7f731015af20 2024-03-11T18:35:15.150 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:15.144+0000 7f72f7fff640 1 -- 172.21.15.90:0/2232897641 <== mon.2 v2:172.21.15.92:3301/0 4 ==== mgrmap(e 4) v1 ==== 86353+0+0 (secure 0 0 0) 0x7f73080b7040 con 0x7f731015af20 2024-03-11T18:35:15.151 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:15.144+0000 7f72f7fff640 1 --2- 172.21.15.90:0/2232897641 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f72fc066e30 0x7f72fc0692f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:15.151 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:15.144+0000 7f730f7fe640 1 --2- 172.21.15.90:0/2232897641 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f72fc066e30 0x7f72fc0692f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:15.152 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:15.148+0000 7f730f7fe640 1 --2- 172.21.15.90:0/2232897641 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f72fc066e30 0x7f72fc0692f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7300009730 tx=0x7f7300006ca0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:15.159 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:15.152+0000 7f72f7fff640 1 -- 172.21.15.90:0/2232897641 <== mon.2 v2:172.21.15.92:3301/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2511+0+0 (secure 0 0 0) 0x7f73080f4300 con 0x7f731015af20 2024-03-11T18:35:15.160 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:15.156+0000 7f72f7fff640 1 -- 172.21.15.90:0/2232897641 <== mon.2 v2:172.21.15.92:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f73080efea0 con 0x7f731015af20 2024-03-11T18:35:15.405 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:15.400+0000 7f7315e0f640 1 -- 172.21.15.90:0/2232897641 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f731006a340 con 0x7f731015af20 2024-03-11T18:35:15.883 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:15.876+0000 7f72f7fff640 1 -- 172.21.15.90:0/2232897641 <== mon.2 v2:172.21.15.92:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+8976 (secure 0 0 0) 0x7f73080f3560 con 0x7f731015af20 2024-03-11T18:35:15.884 INFO:teuthology.misc.health.smithi090.stdout: 2024-03-11T18:35:15.884 INFO:teuthology.misc.health.smithi090.stdout:{"epoch":15,"fsid":"76d56b10-6c29-476a-804b-8482ea90f9e2","created":"2024-03-11T18:34:54.521741+0000","modified":"2024-03-11T18:35:14.518438+0000","last_up_change":"2024-03-11T18:35:14.518438+0000","last_in_change":"2024-03-11T18:35:05.213634+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"34decff9-7280-4d84-9665-d6af8ea6de84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6819","nonce":1755667921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6821","nonce":1755667921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6825","nonce":1755667921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6823","nonce":1755667921}]},"public_addr":"172.21.15.90:6819/1755667921","cluster_addr":"172.21.15.90:6821/1755667921","heartbeat_back_addr":"172.21.15.90:6825/1755667921","heartbeat_front_addr":"172.21.15.90:6823/1755667921","state":["exists","up"]},{"osd":1,"uuid":"8ecdce9e-5486-415a-a609-dd16e945440e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6807","nonce":438682081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6809","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6811","nonce":438682081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6817","nonce":438682081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6815","nonce":438682081}]},"public_addr":"172.21.15.90:6807/438682081","cluster_addr":"172.21.15.90:6811/438682081","heartbeat_back_addr":"172.21.15.90:6817/438682081","heartbeat_front_addr":"172.21.15.90:6815/438682081","state":["exists","up"]},{"osd":2,"uuid":"737486d4-0e90-4c82-9e79-bf37c812e5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6827","nonce":2837704738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6829","nonce":2837704738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6833","nonce":2837704738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6831","nonce":2837704738}]},"public_addr":"172.21.15.90:6827/2837704738","cluster_addr":"172.21.15.90:6829/2837704738","heartbeat_back_addr":"172.21.15.90:6833/2837704738","heartbeat_front_addr":"172.21.15.90:6831/2837704738","state":["exists","up"]},{"osd":3,"uuid":"698bd661-bbef-4699-a619-f7bfe23ce12c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6803","nonce":240987971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6805","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6806","nonce":240987971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6813","nonce":240987971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6810","nonce":240987971}]},"public_addr":"172.21.15.90:6803/240987971","cluster_addr":"172.21.15.90:6806/240987971","heartbeat_back_addr":"172.21.15.90:6813/240987971","heartbeat_front_addr":"172.21.15.90:6810/240987971","state":["exists","up"]},{"osd":4,"uuid":"aba9e31f-73d9-43fe-a10b-2dc5832968b4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"bdd56766-4b28-4007-8de3-b5b08cf335d3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"6c580a59-610f-455e-a96b-033299dcc118","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"958184cc-aebf-4c5b-91a0-2a3e38a6b42b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.816294+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T18:35:15.887 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:15.880+0000 7f7315e0f640 1 -- 172.21.15.90:0/2232897641 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f72fc066e30 msgr2=0x7f72fc0692f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:15.887 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:15.880+0000 7f7315e0f640 1 --2- 172.21.15.90:0/2232897641 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f72fc066e30 0x7f72fc0692f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7300009730 tx=0x7f7300006ca0 comp rx=0 tx=0).stop 2024-03-11T18:35:15.887 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:15.880+0000 7f7315e0f640 1 -- 172.21.15.90:0/2232897641 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f731015af20 msgr2=0x7f73100b21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:15.887 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:15.880+0000 7f7315e0f640 1 --2- 172.21.15.90:0/2232897641 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f731015af20 0x7f73100b21d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f73080af000 tx=0x7f730801ba30 comp rx=0 tx=0).stop 2024-03-11T18:35:15.887 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:15.880+0000 7f7315e0f640 1 -- 172.21.15.90:0/2232897641 shutdown_connections 2024-03-11T18:35:15.887 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:15.880+0000 7f7315e0f640 1 --2- 172.21.15.90:0/2232897641 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f72fc066e30 0x7f72fc0692f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:15.887 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:15.880+0000 7f7315e0f640 1 --2- 172.21.15.90:0/2232897641 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f731015af20 0x7f73100b21d0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:15.887 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:15.880+0000 7f7315e0f640 1 --2- 172.21.15.90:0/2232897641 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f731006cd10 0x7f73101515b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:15.887 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:15.880+0000 7f7315e0f640 1 --2- 172.21.15.90:0/2232897641 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f731006c930 0x7f73100af2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:15.887 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:15.880+0000 7f7315e0f640 1 -- 172.21.15.90:0/2232897641 >> 172.21.15.90:0/2232897641 conn(0x7f73100708b0 msgr2=0x7f73100c2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:35:15.887 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:15.880+0000 7f7315e0f640 1 -- 172.21.15.90:0/2232897641 shutdown_connections 2024-03-11T18:35:15.887 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:15.880+0000 7f7315e0f640 1 -- 172.21.15.90:0/2232897641 wait complete. 2024-03-11T18:35:15.906 DEBUG:teuthology.misc:4 of 8 OSDs are up 2024-03-11T18:35:22.710 DEBUG:teuthology.orchestra.run.smithi090:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-11T18:35:22.825 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:22.820+0000 7fe6446cb640 1 Processor -- start 2024-03-11T18:35:22.825 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:22.820+0000 7fe6446cb640 1 -- start start 2024-03-11T18:35:22.826 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:22.820+0000 7fe6446cb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe63c0c5e70 0x7fe63c0c8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:22.826 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:22.820+0000 7fe6446cb640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fe63c0c71c0 0x7fe63c0cc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:22.826 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:22.820+0000 7fe6446cb640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe63c0cc770 0x7fe63c0c8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:22.826 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:22.820+0000 7fe6446cb640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe63c15aa30 con 0x7fe63c0c71c0 2024-03-11T18:35:22.826 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:22.820+0000 7fe6446cb640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe63c06b3d0 con 0x7fe63c0c5e70 2024-03-11T18:35:22.826 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:22.820+0000 7fe6446cb640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe63c06b670 con 0x7fe63c0cc770 2024-03-11T18:35:22.827 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:22.820+0000 7fe641c66640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fe63c0c71c0 0x7fe63c0cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:22.827 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:22.820+0000 7fe642467640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe63c0c5e70 0x7fe63c0c8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:22.827 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:22.820+0000 7fe642c68640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe63c0cc770 0x7fe63c0c8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:22.827 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:22.820+0000 7fe641c66640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fe63c0c71c0 0x7fe63c0cc230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58890/0 (socket says 172.21.15.90:58890) 2024-03-11T18:35:22.827 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:22.820+0000 7fe641c66640 1 -- 172.21.15.90:0/1461594358 learned_addr learned my addr 172.21.15.90:0/1461594358 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:35:22.831 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:22.824+0000 7fe641c66640 1 -- 172.21.15.90:0/1461594358 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe63c0cc770 msgr2=0x7fe63c0c8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:35:22.832 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:22.824+0000 7fe641c66640 1 --2- 172.21.15.90:0/1461594358 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe63c0cc770 0x7fe63c0c8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:22.832 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:22.824+0000 7fe641c66640 1 -- 172.21.15.90:0/1461594358 >> v2:172.21.15.92:3300/0 conn(0x7fe63c0c5e70 msgr2=0x7fe63c0c8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:35:22.832 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:22.824+0000 7fe641c66640 1 --2- 172.21.15.90:0/1461594358 >> v2:172.21.15.92:3300/0 conn(0x7fe63c0c5e70 0x7fe63c0c8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:22.832 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:22.824+0000 7fe641c66640 1 -- 172.21.15.90:0/1461594358 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe63c15a100 con 0x7fe63c0c71c0 2024-03-11T18:35:22.837 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:22.832+0000 7fe641c66640 1 --2- 172.21.15.90:0/1461594358 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe63c0c71c0 0x7fe63c0cc230 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe63c0cd260 tx=0x7fe63001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45382e7bc207efcc server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:25.228 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:25.220+0000 7fe641465640 1 -- 172.21.15.90:0/1461594358 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6300ac620 con 0x7fe63c0c71c0 2024-03-11T18:35:25.229 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:25.220+0000 7fe641465640 1 -- 172.21.15.90:0/1461594358 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6300ac780 con 0x7fe63c0c71c0 2024-03-11T18:35:25.229 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:25.220+0000 7fe641465640 1 -- 172.21.15.90:0/1461594358 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6300acb10 con 0x7fe63c0c71c0 2024-03-11T18:35:25.229 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:25.224+0000 7fe6446cb640 1 -- 172.21.15.90:0/1461594358 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe63c0c71c0 msgr2=0x7fe63c0cc230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:25.230 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:25.224+0000 7fe6446cb640 1 --2- 172.21.15.90:0/1461594358 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe63c0c71c0 0x7fe63c0cc230 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe63c0cd260 tx=0x7fe63001c4f0 comp rx=0 tx=0).stop 2024-03-11T18:35:25.230 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:25.224+0000 7fe6446cb640 1 -- 172.21.15.90:0/1461594358 shutdown_connections 2024-03-11T18:35:25.230 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:25.224+0000 7fe6446cb640 1 --2- 172.21.15.90:0/1461594358 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe63c0cc770 0x7fe63c0c8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:25.230 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:25.224+0000 7fe6446cb640 1 --2- 172.21.15.90:0/1461594358 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe63c0c71c0 0x7fe63c0cc230 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:25.230 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:25.224+0000 7fe6446cb640 1 --2- 172.21.15.90:0/1461594358 >> v2:172.21.15.92:3300/0 conn(0x7fe63c0c5e70 0x7fe63c0c8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:25.230 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:25.224+0000 7fe6446cb640 1 -- 172.21.15.90:0/1461594358 >> 172.21.15.90:0/1461594358 conn(0x7fe63c06bd60 msgr2=0x7fe63c06c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:35:25.230 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:25.224+0000 7fe6446cb640 1 -- 172.21.15.90:0/1461594358 shutdown_connections 2024-03-11T18:35:25.230 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:25.224+0000 7fe6446cb640 1 -- 172.21.15.90:0/1461594358 wait complete. 2024-03-11T18:35:25.232 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:25.228+0000 7fe6446cb640 1 Processor -- start 2024-03-11T18:35:25.232 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:25.228+0000 7fe6446cb640 1 -- start start 2024-03-11T18:35:25.234 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:25.228+0000 7fe6446cb640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe63c0c5e70 0x7fe63c150a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:25.234 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:25.228+0000 7fe6446cb640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe63c0c71c0 0x7fe63c14c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:25.234 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:25.228+0000 7fe6446cb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe63c0cc770 0x7fe63c0c4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:25.234 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:25.228+0000 7fe6446cb640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe63c1514e0 con 0x7fe63c0c5e70 2024-03-11T18:35:25.234 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:25.228+0000 7fe6446cb640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe63c0c3050 con 0x7fe63c0cc770 2024-03-11T18:35:25.234 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:25.228+0000 7fe6446cb640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe63c1503f0 con 0x7fe63c0c71c0 2024-03-11T18:35:25.235 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:25.228+0000 7fe642c68640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe63c0cc770 0x7fe63c0c4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:25.235 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:25.228+0000 7fe641c66640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe63c0c71c0 0x7fe63c14c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:25.236 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:25.232+0000 7fe642c68640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe63c0cc770 0x7fe63c0c4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.90:36198/0 (socket says 172.21.15.90:36198) 2024-03-11T18:35:25.236 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:25.232+0000 7fe642c68640 1 -- 172.21.15.90:0/2708837621 learned_addr learned my addr 172.21.15.90:0/2708837621 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:35:25.251 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:25.244+0000 7fe642c68640 1 -- 172.21.15.90:0/2708837621 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe63c0c71c0 msgr2=0x7fe63c14c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:25.251 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:25.244+0000 7fe642c68640 1 --2- 172.21.15.90:0/2708837621 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe63c0c71c0 0x7fe63c14c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:25.251 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:25.244+0000 7fe642c68640 1 -- 172.21.15.90:0/2708837621 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe63c0c5e70 msgr2=0x7fe63c150a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:25.251 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:25.244+0000 7fe642c68640 1 --2- 172.21.15.90:0/2708837621 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe63c0c5e70 0x7fe63c150a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:25.251 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:25.244+0000 7fe642c68640 1 -- 172.21.15.90:0/2708837621 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6300094d0 con 0x7fe63c0cc770 2024-03-11T18:35:25.258 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:25.252+0000 7fe642c68640 1 --2- 172.21.15.90:0/2708837621 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe63c0cc770 0x7fe63c0c4c00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe63400efc0 tx=0x7fe63400c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:25.287 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:25.280+0000 7fe62effd640 1 -- 172.21.15.90:0/2708837621 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe634009ba0 con 0x7fe63c0cc770 2024-03-11T18:35:25.287 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:25.280+0000 7fe62effd640 1 -- 172.21.15.90:0/2708837621 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe63400f040 con 0x7fe63c0cc770 2024-03-11T18:35:25.287 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:25.280+0000 7fe62effd640 1 -- 172.21.15.90:0/2708837621 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6340146c0 con 0x7fe63c0cc770 2024-03-11T18:35:25.287 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:25.280+0000 7fe6446cb640 1 -- 172.21.15.90:0/2708837621 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe63c1506d0 con 0x7fe63c0cc770 2024-03-11T18:35:25.287 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:25.280+0000 7fe6446cb640 1 -- 172.21.15.90:0/2708837621 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe63c151a10 con 0x7fe63c0cc770 2024-03-11T18:35:25.287 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:25.280+0000 7fe6446cb640 1 -- 172.21.15.90:0/2708837621 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe63c150860 con 0x7fe63c0cc770 2024-03-11T18:35:25.627 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:25.620+0000 7fe62effd640 1 -- 172.21.15.90:0/2708837621 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 4) v1 ==== 86353+0+0 (secure 0 0 0) 0x7fe634014820 con 0x7fe63c0cc770 2024-03-11T18:35:25.628 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:25.620+0000 7fe62effd640 1 --2- 172.21.15.90:0/2708837621 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe620066e30 0x7fe6200692f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:25.628 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:25.624+0000 7fe642467640 1 --2- 172.21.15.90:0/2708837621 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe620066e30 0x7fe6200692f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:25.629 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:25.624+0000 7fe642467640 1 --2- 172.21.15.90:0/2708837621 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe620066e30 0x7fe6200692f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe638009a60 tx=0x7fe638009520 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:25.640 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:25.632+0000 7fe62effd640 1 -- 172.21.15.90:0/2708837621 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe634056ef0 con 0x7fe63c0cc770 2024-03-11T18:35:25.673 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:25.668+0000 7fe62effd640 1 -- 172.21.15.90:0/2708837621 <== mon.1 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe634052a90 con 0x7fe63c0cc770 2024-03-11T18:35:25.847 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:25.840+0000 7fe6446cb640 1 -- 172.21.15.90:0/2708837621 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe63c04eef0 con 0x7fe63c0cc770 2024-03-11T18:35:31.716 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:31.712+0000 7fe62effd640 1 -- 172.21.15.90:0/2708837621 <== mon.1 v2:172.21.15.92:3300/0 7 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fe634052e60 con 0x7fe63c0cc770 2024-03-11T18:35:32.078 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:32.072+0000 7fe62effd640 1 -- 172.21.15.90:0/2708837621 <== mon.1 v2:172.21.15.92:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12800 (secure 0 0 0) 0x7fe634052a90 con 0x7fe63c0cc770 2024-03-11T18:35:32.079 INFO:teuthology.misc.health.smithi090.stdout: 2024-03-11T18:35:32.079 INFO:teuthology.misc.health.smithi090.stdout:{"epoch":18,"fsid":"76d56b10-6c29-476a-804b-8482ea90f9e2","created":"2024-03-11T18:34:54.521741+0000","modified":"2024-03-11T18:35:18.938013+0000","last_up_change":"2024-03-11T18:35:16.207500+0000","last_in_change":"2024-03-11T18:35:05.213634+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T18:35:16.193842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"34decff9-7280-4d84-9665-d6af8ea6de84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6819","nonce":1755667921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6821","nonce":1755667921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6825","nonce":1755667921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6823","nonce":1755667921}]},"public_addr":"172.21.15.90:6819/1755667921","cluster_addr":"172.21.15.90:6821/1755667921","heartbeat_back_addr":"172.21.15.90:6825/1755667921","heartbeat_front_addr":"172.21.15.90:6823/1755667921","state":["exists","up"]},{"osd":1,"uuid":"8ecdce9e-5486-415a-a609-dd16e945440e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6807","nonce":438682081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6809","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6811","nonce":438682081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6817","nonce":438682081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6815","nonce":438682081}]},"public_addr":"172.21.15.90:6807/438682081","cluster_addr":"172.21.15.90:6811/438682081","heartbeat_back_addr":"172.21.15.90:6817/438682081","heartbeat_front_addr":"172.21.15.90:6815/438682081","state":["exists","up"]},{"osd":2,"uuid":"737486d4-0e90-4c82-9e79-bf37c812e5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6827","nonce":2837704738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6829","nonce":2837704738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6833","nonce":2837704738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6831","nonce":2837704738}]},"public_addr":"172.21.15.90:6827/2837704738","cluster_addr":"172.21.15.90:6829/2837704738","heartbeat_back_addr":"172.21.15.90:6833/2837704738","heartbeat_front_addr":"172.21.15.90:6831/2837704738","state":["exists","up"]},{"osd":3,"uuid":"698bd661-bbef-4699-a619-f7bfe23ce12c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6803","nonce":240987971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6805","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6806","nonce":240987971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6813","nonce":240987971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6810","nonce":240987971}]},"public_addr":"172.21.15.90:6803/240987971","cluster_addr":"172.21.15.90:6806/240987971","heartbeat_back_addr":"172.21.15.90:6813/240987971","heartbeat_front_addr":"172.21.15.90:6810/240987971","state":["exists","up"]},{"osd":4,"uuid":"aba9e31f-73d9-43fe-a10b-2dc5832968b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6801","nonce":502685275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6803","nonce":502685275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6807","nonce":502685275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6805","nonce":502685275}]},"public_addr":"172.21.15.92:6801/502685275","cluster_addr":"172.21.15.92:6803/502685275","heartbeat_back_addr":"172.21.15.92:6807/502685275","heartbeat_front_addr":"172.21.15.92:6805/502685275","state":["exists","up"]},{"osd":5,"uuid":"bdd56766-4b28-4007-8de3-b5b08cf335d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6809","nonce":200142070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6811","nonce":200142070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6815","nonce":200142070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6813","nonce":200142070}]},"public_addr":"172.21.15.92:6809/200142070","cluster_addr":"172.21.15.92:6811/200142070","heartbeat_back_addr":"172.21.15.92:6815/200142070","heartbeat_front_addr":"172.21.15.92:6813/200142070","state":["exists","up"]},{"osd":6,"uuid":"6c580a59-610f-455e-a96b-033299dcc118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6817","nonce":1919010204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6819","nonce":1919010204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6823","nonce":1919010204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6821","nonce":1919010204}]},"public_addr":"172.21.15.92:6817/1919010204","cluster_addr":"172.21.15.92:6819/1919010204","heartbeat_back_addr":"172.21.15.92:6823/1919010204","heartbeat_front_addr":"172.21.15.92:6821/1919010204","state":["exists","up"]},{"osd":7,"uuid":"958184cc-aebf-4c5b-91a0-2a3e38a6b42b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6825","nonce":3475925719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6827","nonce":3475925719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6831","nonce":3475925719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6829","nonce":3475925719}]},"public_addr":"172.21.15.92:6825/3475925719","cluster_addr":"172.21.15.92:6827/3475925719","heartbeat_back_addr":"172.21.15.92:6831/3475925719","heartbeat_front_addr":"172.21.15.92:6829/3475925719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.816294+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.048084+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.137975+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.968246+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.016829+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.931676+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.041673+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.079404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T18:35:32.082 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:32.076+0000 7fe6446cb640 1 -- 172.21.15.90:0/2708837621 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe620066e30 msgr2=0x7fe6200692f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:32.082 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:32.076+0000 7fe6446cb640 1 --2- 172.21.15.90:0/2708837621 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe620066e30 0x7fe6200692f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe638009a60 tx=0x7fe638009520 comp rx=0 tx=0).stop 2024-03-11T18:35:32.082 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:32.076+0000 7fe6446cb640 1 -- 172.21.15.90:0/2708837621 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe63c0cc770 msgr2=0x7fe63c0c4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:32.083 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:32.076+0000 7fe6446cb640 1 --2- 172.21.15.90:0/2708837621 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe63c0cc770 0x7fe63c0c4c00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe63400efc0 tx=0x7fe63400c490 comp rx=0 tx=0).stop 2024-03-11T18:35:32.083 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:32.076+0000 7fe6446cb640 1 -- 172.21.15.90:0/2708837621 shutdown_connections 2024-03-11T18:35:32.083 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:32.076+0000 7fe6446cb640 1 --2- 172.21.15.90:0/2708837621 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe620066e30 0x7fe6200692f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:32.083 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:32.076+0000 7fe6446cb640 1 --2- 172.21.15.90:0/2708837621 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe63c0cc770 0x7fe63c0c4c00 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:32.083 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:32.076+0000 7fe6446cb640 1 --2- 172.21.15.90:0/2708837621 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe63c0c71c0 0x7fe63c14c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:32.083 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:32.076+0000 7fe6446cb640 1 --2- 172.21.15.90:0/2708837621 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe63c0c5e70 0x7fe63c150a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:32.083 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:32.076+0000 7fe6446cb640 1 -- 172.21.15.90:0/2708837621 >> 172.21.15.90:0/2708837621 conn(0x7fe63c06bd60 msgr2=0x7fe63c0caa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:35:32.083 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:32.076+0000 7fe6446cb640 1 -- 172.21.15.90:0/2708837621 shutdown_connections 2024-03-11T18:35:32.083 INFO:teuthology.misc.health.smithi090.stderr:2024-03-11T18:35:32.076+0000 7fe6446cb640 1 -- 172.21.15.90:0/2708837621 wait complete. 2024-03-11T18:35:32.109 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-03-11T18:35:32.110 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-03-11T18:35:32.110 DEBUG:tasks.ceph_manager:running mds.a under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mds.a.log', '--time-stamp=yes', '--vgdb=yes', '--tool=memcheck'] 2024-03-11T18:35:32.110 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-11T18:35:32.110 DEBUG:teuthology.orchestra.run.smithi090:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mds.a.log --time-stamp=yes --vgdb=yes --tool=memcheck ceph-mds -f --cluster ceph -i a 2024-03-11T18:35:32.112 INFO:tasks.ceph.mds.a:Started 2024-03-11T18:35:32.112 DEBUG:tasks.ceph_manager:running mds.c under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mds.c.log', '--time-stamp=yes', '--vgdb=yes', '--tool=memcheck'] 2024-03-11T18:35:32.113 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-11T18:35:32.113 DEBUG:teuthology.orchestra.run.smithi090:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mds.c.log --time-stamp=yes --vgdb=yes --tool=memcheck ceph-mds -f --cluster ceph -i c 2024-03-11T18:35:32.115 INFO:tasks.ceph.mds.c:Started 2024-03-11T18:35:32.115 DEBUG:tasks.ceph_manager:running mds.e under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mds.e.log', '--time-stamp=yes', '--vgdb=yes', '--tool=memcheck'] 2024-03-11T18:35:32.115 INFO:tasks.ceph.mds.e:Restarting daemon 2024-03-11T18:35:32.115 DEBUG:teuthology.orchestra.run.smithi090:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mds.e.log --time-stamp=yes --vgdb=yes --tool=memcheck ceph-mds -f --cluster ceph -i e 2024-03-11T18:35:32.117 INFO:tasks.ceph.mds.e:Started 2024-03-11T18:35:32.117 DEBUG:tasks.ceph_manager:running mds.b under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mds.b.log', '--time-stamp=yes', '--vgdb=yes', '--tool=memcheck'] 2024-03-11T18:35:32.117 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-11T18:35:32.117 DEBUG:teuthology.orchestra.run.smithi092:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mds.b.log --time-stamp=yes --vgdb=yes --tool=memcheck ceph-mds -f --cluster ceph -i b 2024-03-11T18:35:32.120 INFO:tasks.ceph.mds.b:Started 2024-03-11T18:35:32.120 DEBUG:tasks.ceph_manager:running mds.d under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mds.d.log', '--time-stamp=yes', '--vgdb=yes', '--tool=memcheck'] 2024-03-11T18:35:32.120 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-11T18:35:32.120 DEBUG:teuthology.orchestra.run.smithi092:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mds.d.log --time-stamp=yes --vgdb=yes --tool=memcheck ceph-mds -f --cluster ceph -i d 2024-03-11T18:35:32.123 INFO:tasks.ceph.mds.d:Started 2024-03-11T18:35:32.123 DEBUG:tasks.ceph_manager:running mds.f under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mds.f.log', '--time-stamp=yes', '--vgdb=yes', '--tool=memcheck'] 2024-03-11T18:35:32.123 INFO:tasks.ceph.mds.f:Restarting daemon 2024-03-11T18:35:32.123 DEBUG:teuthology.orchestra.run.smithi092:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mds.f.log --time-stamp=yes --vgdb=yes --tool=memcheck ceph-mds -f --cluster ceph -i f 2024-03-11T18:35:32.125 INFO:tasks.ceph.mds.f:Started 2024-03-11T18:35:32.126 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-03-11T18:35:32.126 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T18:35:32.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.304+0000 7f92167ff640 1 Processor -- start 2024-03-11T18:35:32.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.304+0000 7f92167ff640 1 -- start start 2024-03-11T18:35:32.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.304+0000 7f92167ff640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9210151000 0x7f92100c2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:32.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.304+0000 7f92167ff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f92100c5b50 0x7f92100c4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:32.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.304+0000 7f92167ff640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92100c7980 con 0x7f9210066d60 2024-03-11T18:35:32.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.304+0000 7f92167ff640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92100cc820 con 0x7f92100c5b50 2024-03-11T18:35:32.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.304+0000 7f92167ff640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92100caca0 con 0x7f9210151000 2024-03-11T18:35:32.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.308+0000 7f920ffff640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9210151000 0x7f92100c2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:32.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.308+0000 7f920ffff640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9210151000 0x7f92100c2d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.90:36984/0 (socket says 172.21.15.90:36984) 2024-03-11T18:35:32.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.308+0000 7f920ffff640 1 -- 172.21.15.90:0/3065747576 learned_addr learned my addr 172.21.15.90:0/3065747576 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:35:32.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.308+0000 7f920f7fe640 1 --2- 172.21.15.90:0/3065747576 >> v2:172.21.15.92:3300/0 conn(0x7f92100c5b50 0x7f92100c4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:32.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.312+0000 7f920ffff640 1 -- 172.21.15.90:0/3065747576 >> v2:172.21.15.92:3300/0 conn(0x7f92100c5b50 msgr2=0x7f92100c4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:35:32.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.312+0000 7f920ffff640 1 --2- 172.21.15.90:0/3065747576 >> v2:172.21.15.92:3300/0 conn(0x7f92100c5b50 0x7f92100c4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:32.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.312+0000 7f920ffff640 1 -- 172.21.15.90:0/3065747576 >> v1:172.21.15.92:6789/0 conn(0x7f9210066d60 legacy=0x7f92100c6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:32.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.312+0000 7f920ffff640 1 -- 172.21.15.90:0/3065747576 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92100c2ad0 con 0x7f9210151000 2024-03-11T18:35:32.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.316+0000 7f920ffff640 1 --2- 172.21.15.90:0/3065747576 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9210151000 0x7f92100c2d80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9200009870 tx=0x7f920001c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=299694d413c63e server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:32.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.320+0000 7f920effd640 1 -- 172.21.15.90:0/3065747576 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92000ab610 con 0x7f9210151000 2024-03-11T18:35:32.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.320+0000 7f920effd640 1 -- 172.21.15.90:0/3065747576 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92000ab770 con 0x7f9210151000 2024-03-11T18:35:32.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.320+0000 7f920effd640 1 -- 172.21.15.90:0/3065747576 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92000abb00 con 0x7f9210151000 2024-03-11T18:35:32.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.324+0000 7f92167ff640 1 -- 172.21.15.90:0/3065747576 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9210151000 msgr2=0x7f92100c2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:32.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.324+0000 7f92167ff640 1 --2- 172.21.15.90:0/3065747576 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9210151000 0x7f92100c2d80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9200009870 tx=0x7f920001c410 comp rx=0 tx=0).stop 2024-03-11T18:35:32.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.324+0000 7f92167ff640 1 -- 172.21.15.90:0/3065747576 shutdown_connections 2024-03-11T18:35:32.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.324+0000 7f92167ff640 1 --2- 172.21.15.90:0/3065747576 >> v2:172.21.15.92:3300/0 conn(0x7f92100c5b50 0x7f92100c4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:32.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.324+0000 7f92167ff640 1 --2- 172.21.15.90:0/3065747576 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9210151000 0x7f92100c2d80 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:32.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.324+0000 7f92167ff640 1 -- 172.21.15.90:0/3065747576 >> 172.21.15.90:0/3065747576 conn(0x7f92100cb880 msgr2=0x7f92100cbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:35:32.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.324+0000 7f92167ff640 1 -- 172.21.15.90:0/3065747576 shutdown_connections 2024-03-11T18:35:32.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.324+0000 7f92167ff640 1 -- 172.21.15.90:0/3065747576 wait complete. 2024-03-11T18:35:32.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.324+0000 7f92167ff640 1 Processor -- start 2024-03-11T18:35:32.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.324+0000 7f92167ff640 1 -- start start 2024-03-11T18:35:32.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.324+0000 7f92167ff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9210066d60 0x7f92101595a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:32.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.324+0000 7f92167ff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f92100c5b50 0x7f921015c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:32.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.324+0000 7f92167ff640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9210151000 0x7f92101f4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:32.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.324+0000 7f92167ff640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92101f5040 con 0x7f9210066d60 2024-03-11T18:35:32.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.324+0000 7f92167ff640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f921015c540 con 0x7f92100c5b50 2024-03-11T18:35:32.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.324+0000 7f92167ff640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f921015e190 con 0x7f9210151000 2024-03-11T18:35:32.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.328+0000 7f920ffff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9210066d60 0x7f92101595a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:32.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.328+0000 7f9214d9c640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9210151000 0x7f92101f4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:32.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.328+0000 7f920f7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f92100c5b50 0x7f921015c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:32.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.328+0000 7f920ffff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9210066d60 0x7f92101595a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60102/0 (socket says 172.21.15.90:60102) 2024-03-11T18:35:32.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.328+0000 7f920ffff640 1 -- 172.21.15.90:0/591347115 learned_addr learned my addr 172.21.15.90:0/591347115 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:35:32.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.332+0000 7f9214d9c640 1 -- 172.21.15.90:0/591347115 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f92100c5b50 msgr2=0x7f921015c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:32.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.332+0000 7f9214d9c640 1 --2- 172.21.15.90:0/591347115 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f92100c5b50 0x7f921015c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:32.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.332+0000 7f9214d9c640 1 -- 172.21.15.90:0/591347115 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9210066d60 msgr2=0x7f92101595a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:32.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.332+0000 7f9214d9c640 1 --2- 172.21.15.90:0/591347115 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9210066d60 0x7f92101595a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:32.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.332+0000 7f9214d9c640 1 -- 172.21.15.90:0/591347115 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92000094d0 con 0x7f9210151000 2024-03-11T18:35:32.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.336+0000 7f9214d9c640 1 --2- 172.21.15.90:0/591347115 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9210151000 0x7f92101f4b00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f920800c700 tx=0x7f920800cbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:32.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.344+0000 7f920cff9640 1 -- 172.21.15.90:0/591347115 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9208003b80 con 0x7f9210151000 2024-03-11T18:35:32.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.344+0000 7f920cff9640 1 -- 172.21.15.90:0/591347115 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9208003ce0 con 0x7f9210151000 2024-03-11T18:35:32.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.344+0000 7f920cff9640 1 -- 172.21.15.90:0/591347115 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9208005310 con 0x7f9210151000 2024-03-11T18:35:32.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.344+0000 7f92167ff640 1 -- 172.21.15.90:0/591347115 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f921015b640 con 0x7f9210151000 2024-03-11T18:35:32.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.344+0000 7f92167ff640 1 -- 172.21.15.90:0/591347115 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f921015bef0 con 0x7f9210151000 2024-03-11T18:35:32.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.344+0000 7f92167ff640 1 -- 172.21.15.90:0/591347115 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f921015c6d0 con 0x7f9210151000 2024-03-11T18:35:32.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.436+0000 7f920cff9640 1 -- 172.21.15.90:0/591347115 <== mon.2 v2:172.21.15.92:3301/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f9208005470 con 0x7f9210151000 2024-03-11T18:35:32.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.440+0000 7f920cff9640 1 --2- 172.21.15.90:0/591347115 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f91fc066e90 0x7f91fc069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:32.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.440+0000 7f920ffff640 1 --2- 172.21.15.90:0/591347115 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f91fc066e90 0x7f91fc069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:32.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.440+0000 7f920ffff640 1 --2- 172.21.15.90:0/591347115 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f91fc066e90 0x7f91fc069350 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9200009870 tx=0x7f92000b6040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:32.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.812+0000 7f920cff9640 1 -- 172.21.15.90:0/591347115 <== mon.2 v2:172.21.15.92:3301/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9208055be0 con 0x7f9210151000 2024-03-11T18:35:32.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:32.832+0000 7f920cff9640 1 -- 172.21.15.90:0/591347115 <== mon.2 v2:172.21.15.92:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9208051e80 con 0x7f9210151000 2024-03-11T18:35:33.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:33.020+0000 7f92167ff640 1 -- 172.21.15.90:0/591347115 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f921006de90 con 0x7f9210151000 2024-03-11T18:35:33.824 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:35:33.827 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:35:36.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.812+0000 7f920cff9640 1 -- 172.21.15.90:0/591347115 <== mon.2 v2:172.21.15.92:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12800 (secure 0 0 0) 0x7f9208056e60 con 0x7f9210151000 2024-03-11T18:35:36.820 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:35:36.820 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":18,"fsid":"76d56b10-6c29-476a-804b-8482ea90f9e2","created":"2024-03-11T18:34:54.521741+0000","modified":"2024-03-11T18:35:18.938013+0000","last_up_change":"2024-03-11T18:35:16.207500+0000","last_in_change":"2024-03-11T18:35:05.213634+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T18:35:16.193842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"34decff9-7280-4d84-9665-d6af8ea6de84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6819","nonce":1755667921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6821","nonce":1755667921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6825","nonce":1755667921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6823","nonce":1755667921}]},"public_addr":"172.21.15.90:6819/1755667921","cluster_addr":"172.21.15.90:6821/1755667921","heartbeat_back_addr":"172.21.15.90:6825/1755667921","heartbeat_front_addr":"172.21.15.90:6823/1755667921","state":["exists","up"]},{"osd":1,"uuid":"8ecdce9e-5486-415a-a609-dd16e945440e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6807","nonce":438682081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6809","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6811","nonce":438682081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6817","nonce":438682081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6815","nonce":438682081}]},"public_addr":"172.21.15.90:6807/438682081","cluster_addr":"172.21.15.90:6811/438682081","heartbeat_back_addr":"172.21.15.90:6817/438682081","heartbeat_front_addr":"172.21.15.90:6815/438682081","state":["exists","up"]},{"osd":2,"uuid":"737486d4-0e90-4c82-9e79-bf37c812e5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6827","nonce":2837704738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6829","nonce":2837704738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6833","nonce":2837704738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6831","nonce":2837704738}]},"public_addr":"172.21.15.90:6827/2837704738","cluster_addr":"172.21.15.90:6829/2837704738","heartbeat_back_addr":"172.21.15.90:6833/2837704738","heartbeat_front_addr":"172.21.15.90:6831/2837704738","state":["exists","up"]},{"osd":3,"uuid":"698bd661-bbef-4699-a619-f7bfe23ce12c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6803","nonce":240987971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6805","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6806","nonce":240987971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6813","nonce":240987971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6810","nonce":240987971}]},"public_addr":"172.21.15.90:6803/240987971","cluster_addr":"172.21.15.90:6806/240987971","heartbeat_back_addr":"172.21.15.90:6813/240987971","heartbeat_front_addr":"172.21.15.90:6810/240987971","state":["exists","up"]},{"osd":4,"uuid":"aba9e31f-73d9-43fe-a10b-2dc5832968b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6801","nonce":502685275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6803","nonce":502685275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6807","nonce":502685275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6805","nonce":502685275}]},"public_addr":"172.21.15.92:6801/502685275","cluster_addr":"172.21.15.92:6803/502685275","heartbeat_back_addr":"172.21.15.92:6807/502685275","heartbeat_front_addr":"172.21.15.92:6805/502685275","state":["exists","up"]},{"osd":5,"uuid":"bdd56766-4b28-4007-8de3-b5b08cf335d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6809","nonce":200142070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6811","nonce":200142070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6815","nonce":200142070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6813","nonce":200142070}]},"public_addr":"172.21.15.92:6809/200142070","cluster_addr":"172.21.15.92:6811/200142070","heartbeat_back_addr":"172.21.15.92:6815/200142070","heartbeat_front_addr":"172.21.15.92:6813/200142070","state":["exists","up"]},{"osd":6,"uuid":"6c580a59-610f-455e-a96b-033299dcc118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6817","nonce":1919010204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6819","nonce":1919010204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6823","nonce":1919010204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6821","nonce":1919010204}]},"public_addr":"172.21.15.92:6817/1919010204","cluster_addr":"172.21.15.92:6819/1919010204","heartbeat_back_addr":"172.21.15.92:6823/1919010204","heartbeat_front_addr":"172.21.15.92:6821/1919010204","state":["exists","up"]},{"osd":7,"uuid":"958184cc-aebf-4c5b-91a0-2a3e38a6b42b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6825","nonce":3475925719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6827","nonce":3475925719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6831","nonce":3475925719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6829","nonce":3475925719}]},"public_addr":"172.21.15.92:6825/3475925719","cluster_addr":"172.21.15.92:6827/3475925719","heartbeat_back_addr":"172.21.15.92:6831/3475925719","heartbeat_front_addr":"172.21.15.92:6829/3475925719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.816294+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.048084+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.137975+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.968246+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.016829+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.931676+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.041673+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.079404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T18:35:36.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.816+0000 7f92167ff640 1 -- 172.21.15.90:0/591347115 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f91fc066e90 msgr2=0x7f91fc069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:36.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.816+0000 7f92167ff640 1 --2- 172.21.15.90:0/591347115 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f91fc066e90 0x7f91fc069350 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9200009870 tx=0x7f92000b6040 comp rx=0 tx=0).stop 2024-03-11T18:35:36.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.816+0000 7f92167ff640 1 -- 172.21.15.90:0/591347115 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9210151000 msgr2=0x7f92101f4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:36.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.816+0000 7f92167ff640 1 --2- 172.21.15.90:0/591347115 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9210151000 0x7f92101f4b00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f920800c700 tx=0x7f920800cbc0 comp rx=0 tx=0).stop 2024-03-11T18:35:36.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.816+0000 7f92167ff640 1 -- 172.21.15.90:0/591347115 shutdown_connections 2024-03-11T18:35:36.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.816+0000 7f92167ff640 1 --2- 172.21.15.90:0/591347115 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f91fc066e90 0x7f91fc069350 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:36.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.816+0000 7f92167ff640 1 --2- 172.21.15.90:0/591347115 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9210151000 0x7f92101f4b00 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:36.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.816+0000 7f92167ff640 1 --2- 172.21.15.90:0/591347115 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f92100c5b50 0x7f921015c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:36.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.816+0000 7f92167ff640 1 --2- 172.21.15.90:0/591347115 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9210066d60 0x7f92101595a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:36.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.816+0000 7f92167ff640 1 -- 172.21.15.90:0/591347115 >> 172.21.15.90:0/591347115 conn(0x7f92100cb880 msgr2=0x7f92100c36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:35:36.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.816+0000 7f92167ff640 1 -- 172.21.15.90:0/591347115 shutdown_connections 2024-03-11T18:35:36.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.816+0000 7f92167ff640 1 -- 172.21.15.90:0/591347115 wait complete. 2024-03-11T18:35:36.841 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-11T18:35:16.193842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-11T18:35:36.841 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-11T18:35:36.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.940+0000 7f1eee7cd640 1 Processor -- start 2024-03-11T18:35:36.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.940+0000 7f1eee7cd640 1 -- start start 2024-03-11T18:35:36.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.944+0000 7f1eee7cd640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f1ee8151d50 0x7f1ee80c3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:36.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.944+0000 7f1eee7cd640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1ee806b860 0x7f1ee80c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:36.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.944+0000 7f1eee7cd640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f1ee8157970 0x7f1ee80c5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:36.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.944+0000 7f1eee7cd640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ee80c7430 con 0x7f1ee8157970 2024-03-11T18:35:36.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.944+0000 7f1eee7cd640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ee8152130 con 0x7f1ee8151d50 2024-03-11T18:35:36.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.944+0000 7f1eee7cd640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee80c7a40 con 0x7f1ee806b860 2024-03-11T18:35:36.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.944+0000 7f1ee77fe640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1ee806b860 0x7f1ee80c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:36.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.944+0000 7f1eecd6a640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f1ee8157970 0x7f1ee80c5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:36.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.944+0000 7f1ee77fe640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1ee806b860 0x7f1ee80c95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.90:57888/0 (socket says 172.21.15.90:57888) 2024-03-11T18:35:36.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.944+0000 7f1ee77fe640 1 -- 172.21.15.90:0/3687010442 learned_addr learned my addr 172.21.15.90:0/3687010442 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:35:36.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.944+0000 7f1ee7fff640 1 --2- 172.21.15.90:0/3687010442 >> v2:172.21.15.92:3300/0 conn(0x7f1ee8151d50 0x7f1ee80c3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:36.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.948+0000 7f1ee77fe640 1 -- 172.21.15.90:0/3687010442 >> v2:172.21.15.92:3300/0 conn(0x7f1ee8151d50 msgr2=0x7f1ee80c3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:35:36.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.948+0000 7f1ee77fe640 1 --2- 172.21.15.90:0/3687010442 >> v2:172.21.15.92:3300/0 conn(0x7f1ee8151d50 0x7f1ee80c3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:36.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.948+0000 7f1ee77fe640 1 -- 172.21.15.90:0/3687010442 >> v2:172.21.15.90:3300/0 conn(0x7f1ee8157970 msgr2=0x7f1ee80c5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:35:36.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.948+0000 7f1ee77fe640 1 --2- 172.21.15.90:0/3687010442 >> v2:172.21.15.90:3300/0 conn(0x7f1ee8157970 0x7f1ee80c5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:36.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.948+0000 7f1ee77fe640 1 -- 172.21.15.90:0/3687010442 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee80c6e70 con 0x7f1ee806b860 2024-03-11T18:35:36.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.952+0000 7f1ee77fe640 1 --2- 172.21.15.90:0/3687010442 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1ee806b860 0x7f1ee80c95f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1ee80c90a0 tx=0x7f1edc01ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3f9b8b8cfb9dcdf server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:36.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.956+0000 7f1ee6ffd640 1 -- 172.21.15.90:0/3687010442 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1edc0a89f0 con 0x7f1ee806b860 2024-03-11T18:35:36.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.956+0000 7f1ee6ffd640 1 -- 172.21.15.90:0/3687010442 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1edc0a8b50 con 0x7f1ee806b860 2024-03-11T18:35:36.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.956+0000 7f1ee6ffd640 1 -- 172.21.15.90:0/3687010442 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1edc0b0440 con 0x7f1ee806b860 2024-03-11T18:35:36.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.956+0000 7f1eee7cd640 1 -- 172.21.15.90:0/3687010442 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1ee806b860 msgr2=0x7f1ee80c95f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:36.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.956+0000 7f1eee7cd640 1 --2- 172.21.15.90:0/3687010442 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1ee806b860 0x7f1ee80c95f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1ee80c90a0 tx=0x7f1edc01ed30 comp rx=0 tx=0).stop 2024-03-11T18:35:36.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.956+0000 7f1eee7cd640 1 -- 172.21.15.90:0/3687010442 shutdown_connections 2024-03-11T18:35:36.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.956+0000 7f1eee7cd640 1 --2- 172.21.15.90:0/3687010442 >> v2:172.21.15.90:3300/0 conn(0x7f1ee8157970 0x7f1ee80c5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:36.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.956+0000 7f1eee7cd640 1 --2- 172.21.15.90:0/3687010442 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1ee806b860 0x7f1ee80c95f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:36.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.956+0000 7f1eee7cd640 1 --2- 172.21.15.90:0/3687010442 >> v2:172.21.15.92:3300/0 conn(0x7f1ee8151d50 0x7f1ee80c3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:36.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.956+0000 7f1eee7cd640 1 -- 172.21.15.90:0/3687010442 >> 172.21.15.90:0/3687010442 conn(0x7f1ee8071aa0 msgr2=0x7f1ee8071eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:35:36.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.956+0000 7f1eee7cd640 1 -- 172.21.15.90:0/3687010442 shutdown_connections 2024-03-11T18:35:36.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.956+0000 7f1eee7cd640 1 -- 172.21.15.90:0/3687010442 wait complete. 2024-03-11T18:35:36.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.960+0000 7f1eee7cd640 1 Processor -- start 2024-03-11T18:35:36.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.960+0000 7f1eee7cd640 1 -- start start 2024-03-11T18:35:36.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.960+0000 7f1eee7cd640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1ee806b860 0x7f1ee80cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:36.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.960+0000 7f1eee7cd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1ee8151d50 0x7f1ee80cc170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:36.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.960+0000 7f1eee7cd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ee8157970 0x7f1ee8144160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:36.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.960+0000 7f1eee7cd640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee80ca0f0 con 0x7f1ee8151d50 2024-03-11T18:35:36.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.960+0000 7f1eee7cd640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee80ca390 con 0x7f1ee8157970 2024-03-11T18:35:36.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.960+0000 7f1eee7cd640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee81eddb0 con 0x7f1ee806b860 2024-03-11T18:35:36.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.960+0000 7f1ee77fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1ee8151d50 0x7f1ee80cc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:36.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.960+0000 7f1ee7fff640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1ee806b860 0x7f1ee80cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:36.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.960+0000 7f1ee77fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1ee8151d50 0x7f1ee80cc170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51662/0 (socket says 172.21.15.90:51662) 2024-03-11T18:35:36.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.960+0000 7f1ee77fe640 1 -- 172.21.15.90:0/1973578903 learned_addr learned my addr 172.21.15.90:0/1973578903 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:35:36.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.960+0000 7f1eecd6a640 1 --2- 172.21.15.90:0/1973578903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ee8157970 0x7f1ee8144160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:36.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.964+0000 7f1ee77fe640 1 -- 172.21.15.90:0/1973578903 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1ee806b860 msgr2=0x7f1ee80cb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:36.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.964+0000 7f1ee77fe640 1 --2- 172.21.15.90:0/1973578903 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1ee806b860 0x7f1ee80cb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:36.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.964+0000 7f1ee77fe640 1 -- 172.21.15.90:0/1973578903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ee8157970 msgr2=0x7f1ee8144160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:36.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.964+0000 7f1ee77fe640 1 --2- 172.21.15.90:0/1973578903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ee8157970 0x7f1ee8144160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:36.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.964+0000 7f1ee77fe640 1 -- 172.21.15.90:0/1973578903 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1edc0094d0 con 0x7f1ee8151d50 2024-03-11T18:35:36.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.968+0000 7f1ee77fe640 1 --2- 172.21.15.90:0/1973578903 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1ee8151d50 0x7f1ee80cc170 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1edc01d3e0 tx=0x7f1edc01d570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:36.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.972+0000 7f1ee4ff9640 1 -- 172.21.15.90:0/1973578903 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1edc0b0d40 con 0x7f1ee8151d50 2024-03-11T18:35:36.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.972+0000 7f1ee4ff9640 1 -- 172.21.15.90:0/1973578903 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1edc0bb480 con 0x7f1ee8151d50 2024-03-11T18:35:36.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.972+0000 7f1eee7cd640 1 -- 172.21.15.90:0/1973578903 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ee8145fd0 con 0x7f1ee8151d50 2024-03-11T18:35:36.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.972+0000 7f1eee7cd640 1 -- 172.21.15.90:0/1973578903 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ee8142600 con 0x7f1ee8151d50 2024-03-11T18:35:36.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.972+0000 7f1ee4ff9640 1 -- 172.21.15.90:0/1973578903 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1edc0b0ea0 con 0x7f1ee8151d50 2024-03-11T18:35:36.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:36.972+0000 7f1eee7cd640 1 -- 172.21.15.90:0/1973578903 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ee80ca500 con 0x7f1ee8151d50 2024-03-11T18:35:37.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.448+0000 7f1ee4ff9640 1 -- 172.21.15.90:0/1973578903 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f1edc0bb830 con 0x7f1ee8151d50 2024-03-11T18:35:37.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.448+0000 7f1ee4ff9640 1 --2- 172.21.15.90:0/1973578903 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1ed4066e90 0x7f1ed4069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:37.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.448+0000 7f1ee7fff640 1 --2- 172.21.15.90:0/1973578903 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1ed4066e90 0x7f1ed4069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:37.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.448+0000 7f1ee7fff640 1 --2- 172.21.15.90:0/1973578903 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1ed4066e90 0x7f1ed4069350 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8009ae0 tx=0x7f1ed8009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:37.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.476+0000 7f1ee4ff9640 1 -- 172.21.15.90:0/1973578903 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1edc0f43d0 con 0x7f1ee8151d50 2024-03-11T18:35:37.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.476+0000 7f1ee4ff9640 1 -- 172.21.15.90:0/1973578903 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1edc0f0670 con 0x7f1ee8151d50 2024-03-11T18:35:37.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.724+0000 7f1eee7cd640 1 -- 172.21.15.90:0/1973578903 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1ee804eef0 con 0x7f1ee8151d50 2024-03-11T18:35:37.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.756+0000 7f1ee4ff9640 1 -- 172.21.15.90:0/1973578903 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1edc0b5070 con 0x7f1ee8151d50 2024-03-11T18:35:37.763 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 1 2024-03-11T18:35:37.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.760+0000 7f1eee7cd640 1 -- 172.21.15.90:0/1973578903 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1ed4066e90 msgr2=0x7f1ed4069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:37.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.760+0000 7f1eee7cd640 1 --2- 172.21.15.90:0/1973578903 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1ed4066e90 0x7f1ed4069350 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8009ae0 tx=0x7f1ed8009190 comp rx=0 tx=0).stop 2024-03-11T18:35:37.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.760+0000 7f1eee7cd640 1 -- 172.21.15.90:0/1973578903 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1ee8151d50 msgr2=0x7f1ee80cc170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:37.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.760+0000 7f1eee7cd640 1 --2- 172.21.15.90:0/1973578903 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1ee8151d50 0x7f1ee80cc170 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1edc01d3e0 tx=0x7f1edc01d570 comp rx=0 tx=0).stop 2024-03-11T18:35:37.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.760+0000 7f1eee7cd640 1 -- 172.21.15.90:0/1973578903 shutdown_connections 2024-03-11T18:35:37.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.760+0000 7f1eee7cd640 1 --2- 172.21.15.90:0/1973578903 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1ed4066e90 0x7f1ed4069350 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:37.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.760+0000 7f1eee7cd640 1 --2- 172.21.15.90:0/1973578903 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1ee8157970 0x7f1ee8144160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:37.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.760+0000 7f1eee7cd640 1 --2- 172.21.15.90:0/1973578903 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1ee8151d50 0x7f1ee80cc170 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:37.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.760+0000 7f1eee7cd640 1 --2- 172.21.15.90:0/1973578903 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1ee806b860 0x7f1ee80cb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:37.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.760+0000 7f1eee7cd640 1 -- 172.21.15.90:0/1973578903 >> 172.21.15.90:0/1973578903 conn(0x7f1ee8071aa0 msgr2=0x7f1ee814f530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:35:37.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.760+0000 7f1eee7cd640 1 -- 172.21.15.90:0/1973578903 shutdown_connections 2024-03-11T18:35:37.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.760+0000 7f1eee7cd640 1 -- 172.21.15.90:0/1973578903 wait complete. 2024-03-11T18:35:37.785 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T18:35:37.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.884+0000 7f9aef2eb640 1 Processor -- start 2024-03-11T18:35:37.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.884+0000 7f9aef2eb640 1 -- start start 2024-03-11T18:35:37.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.884+0000 7f9aef2eb640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f9ae81550c0 0x7f9ae80c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:37.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.884+0000 7f9aef2eb640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9ae806fe60 0x7f9ae80c2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:37.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.884+0000 7f9aef2eb640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ae80c2ae0 con 0x7f9ae81554a0 2024-03-11T18:35:37.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.884+0000 7f9aef2eb640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ae80c2560 con 0x7f9ae81550c0 2024-03-11T18:35:37.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.884+0000 7f9aef2eb640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae8157cc0 con 0x7f9ae806fe60 2024-03-11T18:35:37.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.884+0000 7f9aed087640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f9ae81550c0 0x7f9ae80c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:37.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.884+0000 7f9aed888640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9ae806fe60 0x7f9ae80c2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:37.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.884+0000 7f9aec886640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f9ae81554a0 0x7f9ae815ba70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.90:35872/0 (socket says 172.21.15.90:35872) 2024-03-11T18:35:37.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.884+0000 7f9aec886640 1 -- 172.21.15.90:0/4117695436 learned_addr learned my addr 172.21.15.90:0/4117695436 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T18:35:37.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.888+0000 7f9aed087640 1 -- 172.21.15.90:0/4117695436 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9ae806fe60 msgr2=0x7f9ae80c2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:35:37.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.888+0000 7f9aed087640 1 --2- 172.21.15.90:0/4117695436 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9ae806fe60 0x7f9ae80c2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:37.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.888+0000 7f9aed087640 1 -- 172.21.15.90:0/4117695436 >> v1:172.21.15.92:6789/0 conn(0x7f9ae81554a0 legacy=0x7f9ae815ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:37.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.888+0000 7f9aed087640 1 -- 172.21.15.90:0/4117695436 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae80c2800 con 0x7f9ae81550c0 2024-03-11T18:35:37.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.892+0000 7f9aed087640 1 --2- 172.21.15.90:0/4117695436 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ae81550c0 0x7f9ae80c8fb0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9ae40097c0 tx=0x7f9ae401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34e4128a7f6af516 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:37.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.896+0000 7f9adffff640 1 -- 172.21.15.90:0/4117695436 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae40ab600 con 0x7f9ae81550c0 2024-03-11T18:35:37.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.896+0000 7f9adffff640 1 -- 172.21.15.90:0/4117695436 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae40a7050 con 0x7f9ae81550c0 2024-03-11T18:35:37.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.896+0000 7f9adffff640 1 -- 172.21.15.90:0/4117695436 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae40ab980 con 0x7f9ae81550c0 2024-03-11T18:35:37.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.896+0000 7f9aef2eb640 1 -- 172.21.15.90:0/4117695436 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ae81550c0 msgr2=0x7f9ae80c8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:37.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.896+0000 7f9aef2eb640 1 --2- 172.21.15.90:0/4117695436 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ae81550c0 0x7f9ae80c8fb0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9ae40097c0 tx=0x7f9ae401c410 comp rx=0 tx=0).stop 2024-03-11T18:35:37.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.896+0000 7f9aef2eb640 1 -- 172.21.15.90:0/4117695436 shutdown_connections 2024-03-11T18:35:37.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.896+0000 7f9aef2eb640 1 --2- 172.21.15.90:0/4117695436 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9ae806fe60 0x7f9ae80c2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:37.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.896+0000 7f9aef2eb640 1 --2- 172.21.15.90:0/4117695436 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ae81550c0 0x7f9ae80c8fb0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:37.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.896+0000 7f9aef2eb640 1 -- 172.21.15.90:0/4117695436 >> 172.21.15.90:0/4117695436 conn(0x7f9ae8070e80 msgr2=0x7f9ae8066990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:35:37.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.896+0000 7f9aef2eb640 1 -- 172.21.15.90:0/4117695436 shutdown_connections 2024-03-11T18:35:37.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.896+0000 7f9aef2eb640 1 -- 172.21.15.90:0/4117695436 wait complete. 2024-03-11T18:35:37.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.900+0000 7f9aef2eb640 1 Processor -- start 2024-03-11T18:35:37.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.900+0000 7f9aef2eb640 1 -- start start 2024-03-11T18:35:37.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.900+0000 7f9aef2eb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ae806fe60 0x7f9ae8156c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:37.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.900+0000 7f9aef2eb640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9ae81550c0 0x7f9ae80b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:37.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.900+0000 7f9aef2eb640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ae81554a0 0x7f9ae81f34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:37.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.900+0000 7f9aef2eb640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae80b0470 con 0x7f9ae81554a0 2024-03-11T18:35:37.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.900+0000 7f9aef2eb640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae80b14d0 con 0x7f9ae806fe60 2024-03-11T18:35:37.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.900+0000 7f9aef2eb640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae80b17b0 con 0x7f9ae81550c0 2024-03-11T18:35:37.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.900+0000 7f9aed888640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ae81554a0 0x7f9ae81f34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:37.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.900+0000 7f9aec886640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9ae81550c0 0x7f9ae80b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:37.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.900+0000 7f9aed087640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ae806fe60 0x7f9ae8156c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:37.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.900+0000 7f9aed888640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ae81554a0 0x7f9ae81f34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51684/0 (socket says 172.21.15.90:51684) 2024-03-11T18:35:37.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.900+0000 7f9aed888640 1 -- 172.21.15.90:0/3285891931 learned_addr learned my addr 172.21.15.90:0/3285891931 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:35:37.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.908+0000 7f9aed888640 1 -- 172.21.15.90:0/3285891931 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9ae81550c0 msgr2=0x7f9ae80b1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:37.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.908+0000 7f9aed888640 1 --2- 172.21.15.90:0/3285891931 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9ae81550c0 0x7f9ae80b1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:37.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.908+0000 7f9aed888640 1 -- 172.21.15.90:0/3285891931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ae806fe60 msgr2=0x7f9ae8156c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:37.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.908+0000 7f9aed888640 1 --2- 172.21.15.90:0/3285891931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ae806fe60 0x7f9ae8156c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:37.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.908+0000 7f9aed888640 1 -- 172.21.15.90:0/3285891931 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae4009420 con 0x7f9ae81554a0 2024-03-11T18:35:37.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.912+0000 7f9aed888640 1 --2- 172.21.15.90:0/3285891931 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ae81554a0 0x7f9ae81f34f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0009cb0 tx=0x7f9ae000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:37.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.916+0000 7f9addffb640 1 -- 172.21.15.90:0/3285891931 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae0009b20 con 0x7f9ae81554a0 2024-03-11T18:35:37.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.916+0000 7f9addffb640 1 -- 172.21.15.90:0/3285891931 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae000f040 con 0x7f9ae81554a0 2024-03-11T18:35:37.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.916+0000 7f9addffb640 1 -- 172.21.15.90:0/3285891931 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae00146c0 con 0x7f9ae81554a0 2024-03-11T18:35:37.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.916+0000 7f9aef2eb640 1 -- 172.21.15.90:0/3285891931 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ae8157800 con 0x7f9ae81554a0 2024-03-11T18:35:37.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.916+0000 7f9aef2eb640 1 -- 172.21.15.90:0/3285891931 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ae8148290 con 0x7f9ae81554a0 2024-03-11T18:35:37.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.916+0000 7f9aef2eb640 1 -- 172.21.15.90:0/3285891931 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ae804eef0 con 0x7f9ae81554a0 2024-03-11T18:35:37.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.952+0000 7f9addffb640 1 -- 172.21.15.90:0/3285891931 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f9ae0014820 con 0x7f9ae81554a0 2024-03-11T18:35:37.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.952+0000 7f9addffb640 1 --2- 172.21.15.90:0/3285891931 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f9acc066e40 0x7f9acc069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:37.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.952+0000 7f9aed087640 1 --2- 172.21.15.90:0/3285891931 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f9acc066e40 0x7f9acc069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:37.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:37.952+0000 7f9aed087640 1 --2- 172.21.15.90:0/3285891931 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f9acc066e40 0x7f9acc069300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9ae40ad000 tx=0x7f9ae4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:38.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:38.268+0000 7f9addffb640 1 -- 172.21.15.90:0/3285891931 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9ae00568a0 con 0x7f9ae81554a0 2024-03-11T18:35:38.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:38.268+0000 7f9addffb640 1 -- 172.21.15.90:0/3285891931 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9ae0052b40 con 0x7f9ae81554a0 2024-03-11T18:35:38.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:38.472+0000 7f9aef2eb640 1 -- 172.21.15.90:0/3285891931 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ae80b18f0 con 0x7f9ae81554a0 2024-03-11T18:35:38.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:38.484+0000 7f9addffb640 1 -- 172.21.15.90:0/3285891931 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7f9ae0018020 con 0x7f9ae81554a0 2024-03-11T18:35:38.490 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:35:38.490 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-03-11T18:35:38.491 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 1 2024-03-11T18:35:38.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:38.488+0000 7f9aef2eb640 1 -- 172.21.15.90:0/3285891931 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f9acc066e40 msgr2=0x7f9acc069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:38.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:38.488+0000 7f9aef2eb640 1 --2- 172.21.15.90:0/3285891931 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f9acc066e40 0x7f9acc069300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9ae40ad000 tx=0x7f9ae4008040 comp rx=0 tx=0).stop 2024-03-11T18:35:38.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:38.488+0000 7f9aef2eb640 1 -- 172.21.15.90:0/3285891931 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ae81554a0 msgr2=0x7f9ae81f34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:38.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:38.488+0000 7f9aef2eb640 1 --2- 172.21.15.90:0/3285891931 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ae81554a0 0x7f9ae81f34f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0009cb0 tx=0x7f9ae000c490 comp rx=0 tx=0).stop 2024-03-11T18:35:38.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:38.488+0000 7f9aef2eb640 1 -- 172.21.15.90:0/3285891931 shutdown_connections 2024-03-11T18:35:38.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:38.488+0000 7f9aef2eb640 1 --2- 172.21.15.90:0/3285891931 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f9acc066e40 0x7f9acc069300 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:38.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:38.488+0000 7f9aef2eb640 1 --2- 172.21.15.90:0/3285891931 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ae81554a0 0x7f9ae81f34f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:38.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:38.488+0000 7f9aef2eb640 1 --2- 172.21.15.90:0/3285891931 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9ae81550c0 0x7f9ae80b1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:38.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:38.488+0000 7f9aef2eb640 1 --2- 172.21.15.90:0/3285891931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ae806fe60 0x7f9ae8156c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:38.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:38.488+0000 7f9aef2eb640 1 -- 172.21.15.90:0/3285891931 >> 172.21.15.90:0/3285891931 conn(0x7f9ae8070e80 msgr2=0x7f9ae80c95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:35:38.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:38.488+0000 7f9aef2eb640 1 -- 172.21.15.90:0/3285891931 shutdown_connections 2024-03-11T18:35:38.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:38.488+0000 7f9aef2eb640 1 -- 172.21.15.90:0/3285891931 wait complete. 2024-03-11T18:35:40.100 INFO:tasks.ceph.mds.c.smithi090.stdout:starting mds.c at 2024-03-11T18:35:40.515 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T18:35:40.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.644+0000 7f8a417fa640 1 Processor -- start 2024-03-11T18:35:40.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.644+0000 7f8a417fa640 1 -- start start 2024-03-11T18:35:40.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.644+0000 7f8a417fa640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f8a3c151b30 0x7f8a3c06e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:40.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.644+0000 7f8a417fa640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a3c06f430 con 0x7f8a3c070b00 2024-03-11T18:35:40.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.644+0000 7f8a417fa640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a3c06e820 con 0x7f8a3c0c4af0 2024-03-11T18:35:40.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.644+0000 7f8a417fa640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a3c0c65b0 con 0x7f8a3c151b30 2024-03-11T18:35:40.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.644+0000 7f8a3affd640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f8a3c151b30 0x7f8a3c06e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:40.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.644+0000 7f8a3affd640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f8a3c151b30 0x7f8a3c06e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51728/0 (socket says 172.21.15.90:51728) 2024-03-11T18:35:40.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.644+0000 7f8a3affd640 1 -- 172.21.15.90:0/2495366038 learned_addr learned my addr 172.21.15.90:0/2495366038 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:35:40.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.648+0000 7f8a3affd640 1 -- 172.21.15.90:0/2495366038 >> v1:172.21.15.92:6789/0 conn(0x7f8a3c0c4af0 legacy=0x7f8a3c157c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:40.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.648+0000 7f8a3affd640 1 -- 172.21.15.90:0/2495366038 >> v1:172.21.15.90:6789/0 conn(0x7f8a3c070b00 legacy=0x7f8a3c0c5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:40.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.648+0000 7f8a3affd640 1 -- 172.21.15.90:0/2495366038 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a3c06eb00 con 0x7f8a3c151b30 2024-03-11T18:35:40.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.652+0000 7f8a3affd640 1 --2- 172.21.15.90:0/2495366038 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a3c151b30 0x7f8a3c06e210 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8a24009870 tx=0x7f8a2401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b16db1066b4e4291 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:40.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.656+0000 7f8a39ffb640 1 -- 172.21.15.90:0/2495366038 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a240a95f0 con 0x7f8a3c151b30 2024-03-11T18:35:40.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.656+0000 7f8a39ffb640 1 -- 172.21.15.90:0/2495366038 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a240a9750 con 0x7f8a3c151b30 2024-03-11T18:35:40.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.656+0000 7f8a39ffb640 1 -- 172.21.15.90:0/2495366038 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a240a9ae0 con 0x7f8a3c151b30 2024-03-11T18:35:40.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.656+0000 7f8a417fa640 1 -- 172.21.15.90:0/2495366038 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a3c151b30 msgr2=0x7f8a3c06e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:40.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.656+0000 7f8a417fa640 1 --2- 172.21.15.90:0/2495366038 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a3c151b30 0x7f8a3c06e210 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8a24009870 tx=0x7f8a2401c530 comp rx=0 tx=0).stop 2024-03-11T18:35:40.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.656+0000 7f8a417fa640 1 -- 172.21.15.90:0/2495366038 shutdown_connections 2024-03-11T18:35:40.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.656+0000 7f8a417fa640 1 --2- 172.21.15.90:0/2495366038 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a3c151b30 0x7f8a3c06e210 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:40.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.656+0000 7f8a417fa640 1 -- 172.21.15.90:0/2495366038 >> 172.21.15.90:0/2495366038 conn(0x7f8a3c06ac00 msgr2=0x7f8a3c06b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:35:40.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.656+0000 7f8a417fa640 1 -- 172.21.15.90:0/2495366038 shutdown_connections 2024-03-11T18:35:40.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.656+0000 7f8a417fa640 1 -- 172.21.15.90:0/2495366038 wait complete. 2024-03-11T18:35:40.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.660+0000 7f8a417fa640 1 Processor -- start 2024-03-11T18:35:40.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.660+0000 7f8a417fa640 1 -- start start 2024-03-11T18:35:40.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.660+0000 7f8a417fa640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a3c070b00 0x7f8a3c0ca4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:40.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.660+0000 7f8a417fa640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a3c0c4af0 0x7f8a3c140c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:40.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.660+0000 7f8a417fa640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8a3c151b30 0x7f8a3c1411b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:40.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.660+0000 7f8a417fa640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3c14db20 con 0x7f8a3c070b00 2024-03-11T18:35:40.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.660+0000 7f8a417fa640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3c14de00 con 0x7f8a3c0c4af0 2024-03-11T18:35:40.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.660+0000 7f8a417fa640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3c14e0e0 con 0x7f8a3c151b30 2024-03-11T18:35:40.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.660+0000 7f8a3affd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a3c070b00 0x7f8a3c0ca4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:40.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.660+0000 7f8a3affd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a3c070b00 0x7f8a3c0ca4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51732/0 (socket says 172.21.15.90:51732) 2024-03-11T18:35:40.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.660+0000 7f8a3affd640 1 -- 172.21.15.90:0/2394685311 learned_addr learned my addr 172.21.15.90:0/2394685311 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:35:40.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.660+0000 7f8a3a7fc640 1 --2- 172.21.15.90:0/2394685311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a3c0c4af0 0x7f8a3c140c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:40.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.660+0000 7f8a3b7fe640 1 --2- 172.21.15.90:0/2394685311 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8a3c151b30 0x7f8a3c1411b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:40.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.664+0000 7f8a3affd640 1 -- 172.21.15.90:0/2394685311 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8a3c151b30 msgr2=0x7f8a3c1411b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:40.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.664+0000 7f8a3affd640 1 --2- 172.21.15.90:0/2394685311 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8a3c151b30 0x7f8a3c1411b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:40.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.664+0000 7f8a3affd640 1 -- 172.21.15.90:0/2394685311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a3c0c4af0 msgr2=0x7f8a3c140c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:40.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.664+0000 7f8a3affd640 1 --2- 172.21.15.90:0/2394685311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a3c0c4af0 0x7f8a3c140c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:40.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.664+0000 7f8a3affd640 1 -- 172.21.15.90:0/2394685311 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a240094d0 con 0x7f8a3c070b00 2024-03-11T18:35:40.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.664+0000 7f8a3affd640 1 --2- 172.21.15.90:0/2394685311 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a3c070b00 0x7f8a3c0ca4d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8a240ab040 tx=0x7f8a2401ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:40.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.668+0000 7f8a337fe640 1 -- 172.21.15.90:0/2394685311 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a240b1800 con 0x7f8a3c070b00 2024-03-11T18:35:40.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.668+0000 7f8a417fa640 1 -- 172.21.15.90:0/2394685311 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a3c14e3c0 con 0x7f8a3c070b00 2024-03-11T18:35:40.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.668+0000 7f8a417fa640 1 -- 172.21.15.90:0/2394685311 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a3c1f4a00 con 0x7f8a3c070b00 2024-03-11T18:35:40.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.668+0000 7f8a417fa640 1 -- 172.21.15.90:0/2394685311 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a3c04eef0 con 0x7f8a3c070b00 2024-03-11T18:35:40.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.688+0000 7f8a337fe640 1 -- 172.21.15.90:0/2394685311 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a240b1960 con 0x7f8a3c070b00 2024-03-11T18:35:40.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.688+0000 7f8a337fe640 1 -- 172.21.15.90:0/2394685311 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a240b1cf0 con 0x7f8a3c070b00 2024-03-11T18:35:40.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.688+0000 7f8a337fe640 1 -- 172.21.15.90:0/2394685311 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f8a240bc450 con 0x7f8a3c070b00 2024-03-11T18:35:40.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.692+0000 7f8a337fe640 1 --2- 172.21.15.90:0/2394685311 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f8a1c066e70 0x7f8a1c069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:40.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.692+0000 7f8a3a7fc640 1 --2- 172.21.15.90:0/2394685311 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f8a1c066e70 0x7f8a1c069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:40.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.692+0000 7f8a3a7fc640 1 --2- 172.21.15.90:0/2394685311 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f8a1c066e70 0x7f8a1c069330 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c071910 tx=0x7f8a2c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:40.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.692+0000 7f8a337fe640 1 -- 172.21.15.90:0/2394685311 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8a240f07f0 con 0x7f8a3c070b00 2024-03-11T18:35:40.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:40.952+0000 7f8a337fe640 1 -- 172.21.15.90:0/2394685311 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8a240f05c0 con 0x7f8a3c070b00 2024-03-11T18:35:41.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:41.184+0000 7f8a417fa640 1 -- 172.21.15.90:0/2394685311 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a3c14e870 con 0x7f8a3c070b00 2024-03-11T18:35:41.268 INFO:tasks.ceph.mds.b.smithi092.stdout:starting mds.b at 2024-03-11T18:35:41.279 INFO:tasks.ceph.mds.d.smithi092.stdout:starting mds.d at 2024-03-11T18:35:41.280 INFO:tasks.ceph.mds.e.smithi090.stdout:starting mds.e at 2024-03-11T18:35:41.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:41.288+0000 7f8a337fe640 1 -- 172.21.15.90:0/2394685311 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7f8a240f0150 con 0x7f8a3c070b00 2024-03-11T18:35:41.297 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:35:41.297 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-03-11T18:35:41.297 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 1 2024-03-11T18:35:41.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:41.292+0000 7f8a417fa640 1 -- 172.21.15.90:0/2394685311 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f8a1c066e70 msgr2=0x7f8a1c069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:41.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:41.292+0000 7f8a417fa640 1 --2- 172.21.15.90:0/2394685311 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f8a1c066e70 0x7f8a1c069330 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c071910 tx=0x7f8a2c009040 comp rx=0 tx=0).stop 2024-03-11T18:35:41.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:41.292+0000 7f8a417fa640 1 -- 172.21.15.90:0/2394685311 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a3c070b00 msgr2=0x7f8a3c0ca4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:41.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:41.292+0000 7f8a417fa640 1 --2- 172.21.15.90:0/2394685311 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a3c070b00 0x7f8a3c0ca4d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8a240ab040 tx=0x7f8a2401ad70 comp rx=0 tx=0).stop 2024-03-11T18:35:41.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:41.292+0000 7f8a417fa640 1 -- 172.21.15.90:0/2394685311 shutdown_connections 2024-03-11T18:35:41.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:41.292+0000 7f8a417fa640 1 --2- 172.21.15.90:0/2394685311 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f8a1c066e70 0x7f8a1c069330 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:41.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:41.292+0000 7f8a417fa640 1 --2- 172.21.15.90:0/2394685311 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8a3c151b30 0x7f8a3c1411b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:41.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:41.292+0000 7f8a417fa640 1 --2- 172.21.15.90:0/2394685311 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8a3c0c4af0 0x7f8a3c140c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:41.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:41.292+0000 7f8a417fa640 1 --2- 172.21.15.90:0/2394685311 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8a3c070b00 0x7f8a3c0ca4d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:41.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:41.292+0000 7f8a417fa640 1 -- 172.21.15.90:0/2394685311 >> 172.21.15.90:0/2394685311 conn(0x7f8a3c06ac00 msgr2=0x7f8a3c14d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:35:41.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:41.292+0000 7f8a417fa640 1 -- 172.21.15.90:0/2394685311 shutdown_connections 2024-03-11T18:35:41.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:41.292+0000 7f8a417fa640 1 -- 172.21.15.90:0/2394685311 wait complete. 2024-03-11T18:35:43.316 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T18:35:43.381 INFO:tasks.ceph.mds.a.smithi090.stdout:starting mds.a at 2024-03-11T18:35:43.388 INFO:tasks.ceph.mds.f.smithi092.stdout:starting mds.f at 2024-03-11T18:35:43.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:43.428+0000 7f8d5f87e640 1 Processor -- start 2024-03-11T18:35:43.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:43.428+0000 7f8d5f87e640 1 -- start start 2024-03-11T18:35:43.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:43.428+0000 7f8d5f87e640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8d581554a0 0x7f8d580c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:43.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:43.428+0000 7f8d5f87e640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8d5806fe60 0x7f8d580c2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:43.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:43.428+0000 7f8d5f87e640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d580c2ae0 con 0x7f8d581550c0 2024-03-11T18:35:43.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:43.428+0000 7f8d5f87e640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d580c2560 con 0x7f8d581554a0 2024-03-11T18:35:43.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:43.428+0000 7f8d5f87e640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d58157cc0 con 0x7f8d5806fe60 2024-03-11T18:35:43.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:43.432+0000 7f8d5ce19640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8d581554a0 0x7f8d580c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:43.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:43.432+0000 7f8d5de1b640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8d5806fe60 0x7f8d580c2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:43.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:43.432+0000 7f8d5d61a640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f8d581550c0 0x7f8d5815ba70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.90:35906/0 (socket says 172.21.15.90:35906) 2024-03-11T18:35:43.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:43.432+0000 7f8d5d61a640 1 -- 172.21.15.90:0/2954549306 learned_addr learned my addr 172.21.15.90:0/2954549306 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T18:35:43.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:43.436+0000 7f8d5de1b640 1 -- 172.21.15.90:0/2954549306 >> v2:172.21.15.92:3300/0 conn(0x7f8d581554a0 msgr2=0x7f8d580c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:35:43.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:43.436+0000 7f8d5de1b640 1 --2- 172.21.15.90:0/2954549306 >> v2:172.21.15.92:3300/0 conn(0x7f8d581554a0 0x7f8d580c8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:43.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:43.436+0000 7f8d5de1b640 1 -- 172.21.15.90:0/2954549306 >> v1:172.21.15.92:6789/0 conn(0x7f8d581550c0 legacy=0x7f8d5815ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:43.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:43.436+0000 7f8d5de1b640 1 -- 172.21.15.90:0/2954549306 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d580c2800 con 0x7f8d5806fe60 2024-03-11T18:35:43.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:43.440+0000 7f8d5de1b640 1 --2- 172.21.15.90:0/2954549306 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8d5806fe60 0x7f8d580c2dd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8d50009d90 tx=0x7f8d5001c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f5035be18b06657 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:47.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:47.320+0000 7f8d4ffff640 1 -- 172.21.15.90:0/2954549306 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d500a5890 con 0x7f8d5806fe60 2024-03-11T18:35:47.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:47.320+0000 7f8d4ffff640 1 -- 172.21.15.90:0/2954549306 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d500a5a10 con 0x7f8d5806fe60 2024-03-11T18:35:47.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:47.320+0000 7f8d4ffff640 1 -- 172.21.15.90:0/2954549306 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d500a5e00 con 0x7f8d5806fe60 2024-03-11T18:35:47.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:47.320+0000 7f8d5f87e640 1 -- 172.21.15.90:0/2954549306 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8d5806fe60 msgr2=0x7f8d580c2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:47.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:47.320+0000 7f8d5f87e640 1 --2- 172.21.15.90:0/2954549306 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8d5806fe60 0x7f8d580c2dd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8d50009d90 tx=0x7f8d5001c8e0 comp rx=0 tx=0).stop 2024-03-11T18:35:47.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:47.320+0000 7f8d5f87e640 1 -- 172.21.15.90:0/2954549306 shutdown_connections 2024-03-11T18:35:47.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:47.320+0000 7f8d5f87e640 1 --2- 172.21.15.90:0/2954549306 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8d5806fe60 0x7f8d580c2dd0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:47.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:47.320+0000 7f8d5f87e640 1 --2- 172.21.15.90:0/2954549306 >> v2:172.21.15.92:3300/0 conn(0x7f8d581554a0 0x7f8d580c8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:47.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:47.320+0000 7f8d5f87e640 1 -- 172.21.15.90:0/2954549306 >> 172.21.15.90:0/2954549306 conn(0x7f8d58070e80 msgr2=0x7f8d58066990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:35:47.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:47.320+0000 7f8d5f87e640 1 -- 172.21.15.90:0/2954549306 shutdown_connections 2024-03-11T18:35:47.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:47.320+0000 7f8d5f87e640 1 -- 172.21.15.90:0/2954549306 wait complete. 2024-03-11T18:35:47.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:47.324+0000 7f8d5f87e640 1 Processor -- start 2024-03-11T18:35:47.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:47.324+0000 7f8d5f87e640 1 -- start start 2024-03-11T18:35:47.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:47.328+0000 7f8d5f87e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d5806fe60 0x7f8d58156b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:47.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:47.328+0000 7f8d5f87e640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8d581550c0 0x7f8d58141c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:47.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:47.328+0000 7f8d5f87e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d581554a0 0x7f8d581421d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:47.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:47.328+0000 7f8d5f87e640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d58147af0 con 0x7f8d581554a0 2024-03-11T18:35:47.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:47.328+0000 7f8d5f87e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5814a760 con 0x7f8d5806fe60 2024-03-11T18:35:47.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:47.328+0000 7f8d5f87e640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5814ac70 con 0x7f8d581550c0 2024-03-11T18:35:47.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:47.328+0000 7f8d5d61a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d5806fe60 0x7f8d58156b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:47.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:47.328+0000 7f8d5d61a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d5806fe60 0x7f8d58156b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.90:45274/0 (socket says 172.21.15.90:45274) 2024-03-11T18:35:47.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:47.328+0000 7f8d5d61a640 1 -- 172.21.15.90:0/550613189 learned_addr learned my addr 172.21.15.90:0/550613189 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:35:47.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:47.340+0000 7f8d5d61a640 1 -- 172.21.15.90:0/550613189 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8d581550c0 msgr2=0x7f8d58141c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:47.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:47.340+0000 7f8d5d61a640 1 --2- 172.21.15.90:0/550613189 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8d581550c0 0x7f8d58141c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:47.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:47.340+0000 7f8d5d61a640 1 -- 172.21.15.90:0/550613189 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d581554a0 msgr2=0x7f8d581421d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:47.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:47.340+0000 7f8d5d61a640 1 --2- 172.21.15.90:0/550613189 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d581554a0 0x7f8d581421d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:47.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:47.340+0000 7f8d5d61a640 1 -- 172.21.15.90:0/550613189 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d50009a70 con 0x7f8d5806fe60 2024-03-11T18:35:47.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:47.344+0000 7f8d5d61a640 1 --2- 172.21.15.90:0/550613189 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d5806fe60 0x7f8d58156b30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8d5400c420 tx=0x7f8d5400c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:47.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:47.356+0000 7f8d4dffb640 1 -- 172.21.15.90:0/550613189 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d54005cf0 con 0x7f8d5806fe60 2024-03-11T18:35:47.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:47.356+0000 7f8d4dffb640 1 -- 172.21.15.90:0/550613189 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d54005e50 con 0x7f8d5806fe60 2024-03-11T18:35:47.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:47.356+0000 7f8d4dffb640 1 -- 172.21.15.90:0/550613189 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d540072f0 con 0x7f8d5806fe60 2024-03-11T18:35:47.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:47.356+0000 7f8d5f87e640 1 -- 172.21.15.90:0/550613189 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d581517a0 con 0x7f8d5806fe60 2024-03-11T18:35:47.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:47.356+0000 7f8d5f87e640 1 -- 172.21.15.90:0/550613189 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d5814a990 con 0x7f8d5806fe60 2024-03-11T18:35:47.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:47.360+0000 7f8d5f87e640 1 -- 172.21.15.90:0/550613189 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d58147c30 con 0x7f8d5806fe60 2024-03-11T18:35:47.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:47.680+0000 7f8d4dffb640 1 -- 172.21.15.90:0/550613189 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f8d54007450 con 0x7f8d5806fe60 2024-03-11T18:35:47.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:47.684+0000 7f8d4dffb640 1 --2- 172.21.15.90:0/550613189 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f8d3c066ec0 0x7f8d3c069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:47.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:47.684+0000 7f8d5ce19640 1 --2- 172.21.15.90:0/550613189 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f8d3c066ec0 0x7f8d3c069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:47.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:47.684+0000 7f8d5ce19640 1 --2- 172.21.15.90:0/550613189 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f8d3c066ec0 0x7f8d3c069380 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8d48006aa0 tx=0x7f8d48006a30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:47.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:47.688+0000 7f8d4dffb640 1 -- 172.21.15.90:0/550613189 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8d540554f0 con 0x7f8d5806fe60 2024-03-11T18:35:47.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:47.716+0000 7f8d4dffb640 1 -- 172.21.15.90:0/550613189 <== mon.1 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8d54051090 con 0x7f8d5806fe60 2024-03-11T18:35:47.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:47.964+0000 7f8d5f87e640 1 -- 172.21.15.90:0/550613189 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8d58147c30 con 0x7f8d5806fe60 2024-03-11T18:35:48.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:48.096+0000 7f8d4dffb640 1 -- 172.21.15.90:0/550613189 <== mon.1 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2111 (secure 0 0 0) 0x7f8d58147c30 con 0x7f8d5806fe60 2024-03-11T18:35:48.104 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:35:48.104 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/641527992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":641527992},{"type":"v1","addr":"172.21.15.90:6839","nonce":641527992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4255,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/3485943375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3485943375},{"type":"v1","addr":"172.21.15.90:6837","nonce":3485943375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-11T18:35:48.104 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 3 2024-03-11T18:35:48.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:48.100+0000 7f8d5f87e640 1 -- 172.21.15.90:0/550613189 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f8d3c066ec0 msgr2=0x7f8d3c069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:48.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:48.100+0000 7f8d5f87e640 1 --2- 172.21.15.90:0/550613189 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f8d3c066ec0 0x7f8d3c069380 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8d48006aa0 tx=0x7f8d48006a30 comp rx=0 tx=0).stop 2024-03-11T18:35:48.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:48.100+0000 7f8d5f87e640 1 -- 172.21.15.90:0/550613189 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d5806fe60 msgr2=0x7f8d58156b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:48.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:48.100+0000 7f8d5f87e640 1 --2- 172.21.15.90:0/550613189 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d5806fe60 0x7f8d58156b30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8d5400c420 tx=0x7f8d5400c8e0 comp rx=0 tx=0).stop 2024-03-11T18:35:48.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:48.100+0000 7f8d5f87e640 1 -- 172.21.15.90:0/550613189 shutdown_connections 2024-03-11T18:35:48.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:48.100+0000 7f8d5f87e640 1 --2- 172.21.15.90:0/550613189 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f8d3c066ec0 0x7f8d3c069380 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:48.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:48.100+0000 7f8d5f87e640 1 --2- 172.21.15.90:0/550613189 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d581554a0 0x7f8d581421d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:48.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:48.100+0000 7f8d5f87e640 1 --2- 172.21.15.90:0/550613189 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8d581550c0 0x7f8d58141c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:48.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:48.100+0000 7f8d5f87e640 1 --2- 172.21.15.90:0/550613189 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8d5806fe60 0x7f8d58156b30 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:48.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:48.100+0000 7f8d5f87e640 1 -- 172.21.15.90:0/550613189 >> 172.21.15.90:0/550613189 conn(0x7f8d58070e80 msgr2=0x7f8d580c9a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:35:48.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:48.100+0000 7f8d5f87e640 1 -- 172.21.15.90:0/550613189 shutdown_connections 2024-03-11T18:35:48.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:48.100+0000 7f8d5f87e640 1 -- 172.21.15.90:0/550613189 wait complete. 2024-03-11T18:35:50.125 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T18:35:50.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.248+0000 7faf8d186640 1 Processor -- start 2024-03-11T18:35:50.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.252+0000 7faf8d186640 1 -- start start 2024-03-11T18:35:50.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.252+0000 7faf8d186640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7faf881575c0 0x7faf8806be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:50.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.252+0000 7faf8d186640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7faf880c7e40 0x7faf880cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:50.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.252+0000 7faf8d186640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7faf8806dab0 0x7faf880cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:50.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.252+0000 7faf8d186640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf8806f3d0 con 0x7faf880c7e40 2024-03-11T18:35:50.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.252+0000 7faf8d186640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf8806d4e0 con 0x7faf8806dab0 2024-03-11T18:35:50.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.252+0000 7faf8d186640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf880c85e0 con 0x7faf881575c0 2024-03-11T18:35:50.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.252+0000 7faf8659c640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7faf880c7e40 0x7faf880cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:50.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.252+0000 7faf8659c640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7faf880c7e40 0x7faf880cd3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38492/0 (socket says 172.21.15.90:38492) 2024-03-11T18:35:50.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.252+0000 7faf8659c640 1 -- 172.21.15.90:0/17632668 learned_addr learned my addr 172.21.15.90:0/17632668 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:35:50.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.256+0000 7faf8659c640 1 -- 172.21.15.90:0/17632668 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7faf881575c0 msgr2=0x7faf8806be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:35:50.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.256+0000 7faf8659c640 1 --2- 172.21.15.90:0/17632668 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7faf881575c0 0x7faf8806be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:50.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.256+0000 7faf8659c640 1 -- 172.21.15.90:0/17632668 >> v2:172.21.15.92:3300/0 conn(0x7faf8806dab0 msgr2=0x7faf880cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:35:50.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.256+0000 7faf8659c640 1 --2- 172.21.15.90:0/17632668 >> v2:172.21.15.92:3300/0 conn(0x7faf8806dab0 0x7faf880cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:50.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.256+0000 7faf8659c640 1 -- 172.21.15.90:0/17632668 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf880700d0 con 0x7faf880c7e40 2024-03-11T18:35:50.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.260+0000 7faf8659c640 1 --2- 172.21.15.90:0/17632668 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf880c7e40 0x7faf880cd3d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7faf8806e280 tx=0x7faf7801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80e9f198fa432f5b server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:50.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.260+0000 7faf85d9b640 1 -- 172.21.15.90:0/17632668 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf780aa610 con 0x7faf880c7e40 2024-03-11T18:35:50.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.260+0000 7faf85d9b640 1 -- 172.21.15.90:0/17632668 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf780a6040 con 0x7faf880c7e40 2024-03-11T18:35:50.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.260+0000 7faf85d9b640 1 -- 172.21.15.90:0/17632668 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf780aa990 con 0x7faf880c7e40 2024-03-11T18:35:50.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.264+0000 7faf8d186640 1 -- 172.21.15.90:0/17632668 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf880c7e40 msgr2=0x7faf880cd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:50.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.264+0000 7faf8d186640 1 --2- 172.21.15.90:0/17632668 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf880c7e40 0x7faf880cd3d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7faf8806e280 tx=0x7faf7801c4f0 comp rx=0 tx=0).stop 2024-03-11T18:35:50.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.264+0000 7faf8d186640 1 -- 172.21.15.90:0/17632668 shutdown_connections 2024-03-11T18:35:50.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.264+0000 7faf8d186640 1 --2- 172.21.15.90:0/17632668 >> v2:172.21.15.92:3300/0 conn(0x7faf8806dab0 0x7faf880cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:50.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.264+0000 7faf8d186640 1 --2- 172.21.15.90:0/17632668 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf880c7e40 0x7faf880cd3d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:50.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.264+0000 7faf8d186640 1 --2- 172.21.15.90:0/17632668 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7faf881575c0 0x7faf8806be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:50.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.264+0000 7faf8d186640 1 -- 172.21.15.90:0/17632668 >> 172.21.15.90:0/17632668 conn(0x7faf88066990 msgr2=0x7faf88066da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:35:50.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.264+0000 7faf8d186640 1 -- 172.21.15.90:0/17632668 shutdown_connections 2024-03-11T18:35:50.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.264+0000 7faf8d186640 1 -- 172.21.15.90:0/17632668 wait complete. 2024-03-11T18:35:50.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.264+0000 7faf8d186640 1 Processor -- start 2024-03-11T18:35:50.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.264+0000 7faf8d186640 1 -- start start 2024-03-11T18:35:50.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.264+0000 7faf8d186640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf8806dab0 0x7faf8814b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:50.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.264+0000 7faf8d186640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7faf880c7e40 0x7faf8814aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:50.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.264+0000 7faf8d186640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf881575c0 0x7faf880707e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:50.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.264+0000 7faf8d186640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8814b400 con 0x7faf8806dab0 2024-03-11T18:35:50.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.264+0000 7faf8d186640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8814ac20 con 0x7faf881575c0 2024-03-11T18:35:50.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.264+0000 7faf8d186640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf880c7760 con 0x7faf880c7e40 2024-03-11T18:35:50.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.264+0000 7faf86d9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf8806dab0 0x7faf8814b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:50.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.264+0000 7faf86d9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf8806dab0 0x7faf8814b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38500/0 (socket says 172.21.15.90:38500) 2024-03-11T18:35:50.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.264+0000 7faf86d9d640 1 -- 172.21.15.90:0/2687932528 learned_addr learned my addr 172.21.15.90:0/2687932528 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:35:50.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.268+0000 7faf86d9d640 1 -- 172.21.15.90:0/2687932528 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7faf880c7e40 msgr2=0x7faf8814aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:50.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.268+0000 7faf86d9d640 1 --2- 172.21.15.90:0/2687932528 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7faf880c7e40 0x7faf8814aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:50.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.268+0000 7faf86d9d640 1 -- 172.21.15.90:0/2687932528 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf881575c0 msgr2=0x7faf880707e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:50.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.268+0000 7faf86d9d640 1 --2- 172.21.15.90:0/2687932528 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf881575c0 0x7faf880707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:50.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.268+0000 7faf86d9d640 1 -- 172.21.15.90:0/2687932528 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf780094d0 con 0x7faf8806dab0 2024-03-11T18:35:50.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.272+0000 7faf86d9d640 1 --2- 172.21.15.90:0/2687932528 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf8806dab0 0x7faf8814b6b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7faf7000b410 tx=0x7faf7000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:50.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.276+0000 7faf7f7fe640 1 -- 172.21.15.90:0/2687932528 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf70011830 con 0x7faf8806dab0 2024-03-11T18:35:50.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.276+0000 7faf7f7fe640 1 -- 172.21.15.90:0/2687932528 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf70011990 con 0x7faf8806dab0 2024-03-11T18:35:50.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.276+0000 7faf8d186640 1 -- 172.21.15.90:0/2687932528 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf88156d40 con 0x7faf8806dab0 2024-03-11T18:35:50.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.276+0000 7faf7f7fe640 1 -- 172.21.15.90:0/2687932528 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf70011d40 con 0x7faf8806dab0 2024-03-11T18:35:50.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.276+0000 7faf8d186640 1 -- 172.21.15.90:0/2687932528 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf880c7a00 con 0x7faf8806dab0 2024-03-11T18:35:50.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.280+0000 7faf8d186640 1 -- 172.21.15.90:0/2687932528 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf8804eef0 con 0x7faf8806dab0 2024-03-11T18:35:50.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.540+0000 7faf7f7fe640 1 -- 172.21.15.90:0/2687932528 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7faf7001a400 con 0x7faf8806dab0 2024-03-11T18:35:50.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.540+0000 7faf7f7fe640 1 --2- 172.21.15.90:0/2687932528 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7faf68066ec0 0x7faf68069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:50.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.540+0000 7faf8659c640 1 --2- 172.21.15.90:0/2687932528 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7faf68066ec0 0x7faf68069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:50.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.544+0000 7faf8659c640 1 --2- 172.21.15.90:0/2687932528 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7faf68066ec0 0x7faf68069380 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faf780ae000 tx=0x7faf78008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:50.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.552+0000 7faf7f7fe640 1 -- 172.21.15.90:0/2687932528 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7faf70056400 con 0x7faf8806dab0 2024-03-11T18:35:50.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.552+0000 7faf7f7fe640 1 -- 172.21.15.90:0/2687932528 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faf70051fa0 con 0x7faf8806dab0 2024-03-11T18:35:50.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.752+0000 7faf8d186640 1 -- 172.21.15.90:0/2687932528 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf881f2930 con 0x7faf8806dab0 2024-03-11T18:35:50.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.780+0000 7faf7f7fe640 1 -- 172.21.15.90:0/2687932528 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+5268 (secure 0 0 0) 0x7faf70015090 con 0x7faf8806dab0 2024-03-11T18:35:50.785 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:35:50.785 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/641527992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":641527992},{"type":"v1","addr":"172.21.15.90:6839","nonce":641527992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4229,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/460417177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":460417177},{"type":"v1","addr":"172.21.15.90:6835","nonce":460417177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/3568438077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":3568438077},{"type":"v1","addr":"172.21.15.92:6835","nonce":3568438077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/2306487548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":2306487548},{"type":"v1","addr":"172.21.15.92:6834","nonce":2306487548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/3209534444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":3209534444},{"type":"v1","addr":"172.21.15.92:6837","nonce":3209534444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":4255,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/3485943375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3485943375},{"type":"v1","addr":"172.21.15.90:6837","nonce":3485943375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-11T18:35:50.786 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 4 2024-03-11T18:35:50.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.780+0000 7faf8d186640 1 -- 172.21.15.90:0/2687932528 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7faf68066ec0 msgr2=0x7faf68069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:50.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.780+0000 7faf8d186640 1 --2- 172.21.15.90:0/2687932528 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7faf68066ec0 0x7faf68069380 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faf780ae000 tx=0x7faf78008040 comp rx=0 tx=0).stop 2024-03-11T18:35:50.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.780+0000 7faf8d186640 1 -- 172.21.15.90:0/2687932528 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf8806dab0 msgr2=0x7faf8814b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:50.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.780+0000 7faf8d186640 1 --2- 172.21.15.90:0/2687932528 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf8806dab0 0x7faf8814b6b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7faf7000b410 tx=0x7faf7000b8d0 comp rx=0 tx=0).stop 2024-03-11T18:35:50.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.780+0000 7faf8d186640 1 -- 172.21.15.90:0/2687932528 shutdown_connections 2024-03-11T18:35:50.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.780+0000 7faf8d186640 1 --2- 172.21.15.90:0/2687932528 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7faf68066ec0 0x7faf68069380 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:50.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.780+0000 7faf8d186640 1 --2- 172.21.15.90:0/2687932528 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7faf881575c0 0x7faf880707e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:50.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.780+0000 7faf8d186640 1 --2- 172.21.15.90:0/2687932528 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7faf880c7e40 0x7faf8814aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:50.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.780+0000 7faf8d186640 1 --2- 172.21.15.90:0/2687932528 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf8806dab0 0x7faf8814b6b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:50.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.780+0000 7faf8d186640 1 -- 172.21.15.90:0/2687932528 >> 172.21.15.90:0/2687932528 conn(0x7faf88066990 msgr2=0x7faf880cc840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:35:50.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.780+0000 7faf8d186640 1 -- 172.21.15.90:0/2687932528 shutdown_connections 2024-03-11T18:35:50.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.780+0000 7faf8d186640 1 -- 172.21.15.90:0/2687932528 wait complete. 2024-03-11T18:35:50.807 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T18:35:50.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.924+0000 7fa7677ce640 1 Processor -- start 2024-03-11T18:35:50.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.924+0000 7fa7677ce640 1 -- start start 2024-03-11T18:35:50.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.928+0000 7fa7677ce640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fa760151b30 0x7fa76006e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:50.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.928+0000 7fa7677ce640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa760070b00 0x7fa7600c5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:50.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.928+0000 7fa7677ce640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa7600c4af0 0x7fa7600c4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:50.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.928+0000 7fa7677ce640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7600c65b0 con 0x7fa760151b30 2024-03-11T18:35:50.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.928+0000 7fa7677ce640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa76006f430 con 0x7fa7600c4af0 2024-03-11T18:35:50.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.928+0000 7fa7677ce640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7600c6320 con 0x7fa760070b00 2024-03-11T18:35:50.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.928+0000 7fa76556a640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fa760151b30 0x7fa76006e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:50.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.928+0000 7fa76556a640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fa760151b30 0x7fa76006e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38512/0 (socket says 172.21.15.90:38512) 2024-03-11T18:35:50.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.928+0000 7fa76556a640 1 -- 172.21.15.90:0/1599082041 learned_addr learned my addr 172.21.15.90:0/1599082041 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:35:50.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.928+0000 7fa764d69640 1 --2- 172.21.15.90:0/1599082041 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa760070b00 0x7fa7600c5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:50.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.928+0000 7fa765d6b640 1 --2- 172.21.15.90:0/1599082041 >> v2:172.21.15.92:3300/0 conn(0x7fa7600c4af0 0x7fa7600c4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:50.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.932+0000 7fa76556a640 1 -- 172.21.15.90:0/1599082041 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa760070b00 msgr2=0x7fa7600c5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:35:50.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.932+0000 7fa76556a640 1 --2- 172.21.15.90:0/1599082041 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa760070b00 0x7fa7600c5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:50.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.932+0000 7fa76556a640 1 -- 172.21.15.90:0/1599082041 >> v2:172.21.15.92:3300/0 conn(0x7fa7600c4af0 msgr2=0x7fa7600c4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:35:50.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.932+0000 7fa76556a640 1 --2- 172.21.15.90:0/1599082041 >> v2:172.21.15.92:3300/0 conn(0x7fa7600c4af0 0x7fa7600c4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:50.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.932+0000 7fa76556a640 1 -- 172.21.15.90:0/1599082041 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa76006e820 con 0x7fa760151b30 2024-03-11T18:35:50.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.936+0000 7fa76556a640 1 --2- 172.21.15.90:0/1599082041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa760151b30 0x7fa76006e210 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa76006b840 tx=0x7fa75c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2de6825039514d81 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:50.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.940+0000 7fa757fff640 1 -- 172.21.15.90:0/1599082041 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75c0aa610 con 0x7fa760151b30 2024-03-11T18:35:50.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.940+0000 7fa757fff640 1 -- 172.21.15.90:0/1599082041 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa75c0a6040 con 0x7fa760151b30 2024-03-11T18:35:50.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.940+0000 7fa757fff640 1 -- 172.21.15.90:0/1599082041 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75c0aa990 con 0x7fa760151b30 2024-03-11T18:35:50.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.940+0000 7fa7677ce640 1 -- 172.21.15.90:0/1599082041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa760151b30 msgr2=0x7fa76006e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:50.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.940+0000 7fa7677ce640 1 --2- 172.21.15.90:0/1599082041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa760151b30 0x7fa76006e210 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa76006b840 tx=0x7fa75c01c4f0 comp rx=0 tx=0).stop 2024-03-11T18:35:50.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.940+0000 7fa7677ce640 1 -- 172.21.15.90:0/1599082041 shutdown_connections 2024-03-11T18:35:50.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.940+0000 7fa7677ce640 1 --2- 172.21.15.90:0/1599082041 >> v2:172.21.15.92:3300/0 conn(0x7fa7600c4af0 0x7fa7600c4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:50.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.940+0000 7fa7677ce640 1 --2- 172.21.15.90:0/1599082041 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa760070b00 0x7fa7600c5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:50.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.940+0000 7fa7677ce640 1 --2- 172.21.15.90:0/1599082041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa760151b30 0x7fa76006e210 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:50.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.940+0000 7fa7677ce640 1 -- 172.21.15.90:0/1599082041 >> 172.21.15.90:0/1599082041 conn(0x7fa76006ac00 msgr2=0x7fa76006b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:35:50.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.940+0000 7fa7677ce640 1 -- 172.21.15.90:0/1599082041 shutdown_connections 2024-03-11T18:35:50.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.940+0000 7fa7677ce640 1 -- 172.21.15.90:0/1599082041 wait complete. 2024-03-11T18:35:50.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.944+0000 7fa7677ce640 1 Processor -- start 2024-03-11T18:35:50.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.944+0000 7fa7677ce640 1 -- start start 2024-03-11T18:35:50.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.944+0000 7fa7677ce640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa760070b00 0x7fa7600c27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:50.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.944+0000 7fa7677ce640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7600c4af0 0x7fa760156a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:50.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.944+0000 7fa7677ce640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa760151b30 0x7fa760152460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:50.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.944+0000 7fa7677ce640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7600c2d20 con 0x7fa7600c4af0 2024-03-11T18:35:50.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.944+0000 7fa7677ce640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7601ef0a0 con 0x7fa760070b00 2024-03-11T18:35:50.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.944+0000 7fa7677ce640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7601552b0 con 0x7fa760151b30 2024-03-11T18:35:50.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.944+0000 7fa764d69640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7600c4af0 0x7fa760156a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:50.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.944+0000 7fa764d69640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7600c4af0 0x7fa760156a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38520/0 (socket says 172.21.15.90:38520) 2024-03-11T18:35:50.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.944+0000 7fa764d69640 1 -- 172.21.15.90:0/3401781569 learned_addr learned my addr 172.21.15.90:0/3401781569 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:35:50.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.944+0000 7fa76556a640 1 --2- 172.21.15.90:0/3401781569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa760070b00 0x7fa7600c27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:50.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.944+0000 7fa765d6b640 1 --2- 172.21.15.90:0/3401781569 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa760151b30 0x7fa760152460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:50.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.948+0000 7fa764d69640 1 -- 172.21.15.90:0/3401781569 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa760151b30 msgr2=0x7fa760152460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:50.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.948+0000 7fa764d69640 1 --2- 172.21.15.90:0/3401781569 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa760151b30 0x7fa760152460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:50.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.948+0000 7fa764d69640 1 -- 172.21.15.90:0/3401781569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa760070b00 msgr2=0x7fa7600c27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:50.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.948+0000 7fa764d69640 1 --2- 172.21.15.90:0/3401781569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa760070b00 0x7fa7600c27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:50.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.948+0000 7fa764d69640 1 -- 172.21.15.90:0/3401781569 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa75c0094d0 con 0x7fa7600c4af0 2024-03-11T18:35:50.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.952+0000 7fa764d69640 1 --2- 172.21.15.90:0/3401781569 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7600c4af0 0x7fa760156a70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa75000d800 tx=0x7fa75000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:50.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.956+0000 7fa755ffb640 1 -- 172.21.15.90:0/3401781569 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa750004020 con 0x7fa7600c4af0 2024-03-11T18:35:50.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.956+0000 7fa755ffb640 1 -- 172.21.15.90:0/3401781569 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa750004180 con 0x7fa7600c4af0 2024-03-11T18:35:50.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.956+0000 7fa7677ce640 1 -- 172.21.15.90:0/3401781569 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7601ee1d0 con 0x7fa7600c4af0 2024-03-11T18:35:50.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.956+0000 7fa755ffb640 1 -- 172.21.15.90:0/3401781569 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa750004330 con 0x7fa7600c4af0 2024-03-11T18:35:50.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.956+0000 7fa7677ce640 1 -- 172.21.15.90:0/3401781569 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7601ee770 con 0x7fa7600c4af0 2024-03-11T18:35:50.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:50.956+0000 7fa7677ce640 1 -- 172.21.15.90:0/3401781569 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa76004eef0 con 0x7fa7600c4af0 2024-03-11T18:35:51.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.244+0000 7fa755ffb640 1 -- 172.21.15.90:0/3401781569 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fa7500096a0 con 0x7fa7600c4af0 2024-03-11T18:35:51.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.244+0000 7fa755ffb640 1 --2- 172.21.15.90:0/3401781569 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fa744066ec0 0x7fa744069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:51.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.244+0000 7fa76556a640 1 --2- 172.21.15.90:0/3401781569 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fa744066ec0 0x7fa744069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:51.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.248+0000 7fa76556a640 1 --2- 172.21.15.90:0/3401781569 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fa744066ec0 0x7fa744069380 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa75c0ae000 tx=0x7fa75c0b5040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:51.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.284+0000 7fa755ffb640 1 -- 172.21.15.90:0/3401781569 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa750004490 con 0x7fa7600c4af0 2024-03-11T18:35:51.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.288+0000 7fa755ffb640 1 -- 172.21.15.90:0/3401781569 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa750052330 con 0x7fa7600c4af0 2024-03-11T18:35:51.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.464+0000 7fa7677ce640 1 -- 172.21.15.90:0/3401781569 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa760072400 con 0x7fa7600c4af0 2024-03-11T18:35:51.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.608+0000 7fa755ffb640 1 -- 172.21.15.90:0/3401781569 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12800 (secure 0 0 0) 0x7fa750018090 con 0x7fa7600c4af0 2024-03-11T18:35:51.615 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:35:51.615 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":18,"fsid":"76d56b10-6c29-476a-804b-8482ea90f9e2","created":"2024-03-11T18:34:54.521741+0000","modified":"2024-03-11T18:35:18.938013+0000","last_up_change":"2024-03-11T18:35:16.207500+0000","last_in_change":"2024-03-11T18:35:05.213634+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T18:35:16.193842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"34decff9-7280-4d84-9665-d6af8ea6de84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6819","nonce":1755667921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6821","nonce":1755667921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6825","nonce":1755667921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6823","nonce":1755667921}]},"public_addr":"172.21.15.90:6819/1755667921","cluster_addr":"172.21.15.90:6821/1755667921","heartbeat_back_addr":"172.21.15.90:6825/1755667921","heartbeat_front_addr":"172.21.15.90:6823/1755667921","state":["exists","up"]},{"osd":1,"uuid":"8ecdce9e-5486-415a-a609-dd16e945440e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6807","nonce":438682081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6809","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6811","nonce":438682081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6817","nonce":438682081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6815","nonce":438682081}]},"public_addr":"172.21.15.90:6807/438682081","cluster_addr":"172.21.15.90:6811/438682081","heartbeat_back_addr":"172.21.15.90:6817/438682081","heartbeat_front_addr":"172.21.15.90:6815/438682081","state":["exists","up"]},{"osd":2,"uuid":"737486d4-0e90-4c82-9e79-bf37c812e5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6827","nonce":2837704738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6829","nonce":2837704738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6833","nonce":2837704738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6831","nonce":2837704738}]},"public_addr":"172.21.15.90:6827/2837704738","cluster_addr":"172.21.15.90:6829/2837704738","heartbeat_back_addr":"172.21.15.90:6833/2837704738","heartbeat_front_addr":"172.21.15.90:6831/2837704738","state":["exists","up"]},{"osd":3,"uuid":"698bd661-bbef-4699-a619-f7bfe23ce12c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6803","nonce":240987971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6805","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6806","nonce":240987971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6813","nonce":240987971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6810","nonce":240987971}]},"public_addr":"172.21.15.90:6803/240987971","cluster_addr":"172.21.15.90:6806/240987971","heartbeat_back_addr":"172.21.15.90:6813/240987971","heartbeat_front_addr":"172.21.15.90:6810/240987971","state":["exists","up"]},{"osd":4,"uuid":"aba9e31f-73d9-43fe-a10b-2dc5832968b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6801","nonce":502685275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6803","nonce":502685275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6807","nonce":502685275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6805","nonce":502685275}]},"public_addr":"172.21.15.92:6801/502685275","cluster_addr":"172.21.15.92:6803/502685275","heartbeat_back_addr":"172.21.15.92:6807/502685275","heartbeat_front_addr":"172.21.15.92:6805/502685275","state":["exists","up"]},{"osd":5,"uuid":"bdd56766-4b28-4007-8de3-b5b08cf335d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6809","nonce":200142070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6811","nonce":200142070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6815","nonce":200142070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6813","nonce":200142070}]},"public_addr":"172.21.15.92:6809/200142070","cluster_addr":"172.21.15.92:6811/200142070","heartbeat_back_addr":"172.21.15.92:6815/200142070","heartbeat_front_addr":"172.21.15.92:6813/200142070","state":["exists","up"]},{"osd":6,"uuid":"6c580a59-610f-455e-a96b-033299dcc118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6817","nonce":1919010204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6819","nonce":1919010204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6823","nonce":1919010204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6821","nonce":1919010204}]},"public_addr":"172.21.15.92:6817/1919010204","cluster_addr":"172.21.15.92:6819/1919010204","heartbeat_back_addr":"172.21.15.92:6823/1919010204","heartbeat_front_addr":"172.21.15.92:6821/1919010204","state":["exists","up"]},{"osd":7,"uuid":"958184cc-aebf-4c5b-91a0-2a3e38a6b42b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6825","nonce":3475925719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6827","nonce":3475925719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6831","nonce":3475925719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6829","nonce":3475925719}]},"public_addr":"172.21.15.92:6825/3475925719","cluster_addr":"172.21.15.92:6827/3475925719","heartbeat_back_addr":"172.21.15.92:6831/3475925719","heartbeat_front_addr":"172.21.15.92:6829/3475925719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.816294+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.048084+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.137975+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.968246+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.016829+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.931676+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.041673+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.079404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T18:35:51.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.612+0000 7fa7677ce640 1 -- 172.21.15.90:0/3401781569 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fa744066ec0 msgr2=0x7fa744069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:51.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.612+0000 7fa7677ce640 1 --2- 172.21.15.90:0/3401781569 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fa744066ec0 0x7fa744069380 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa75c0ae000 tx=0x7fa75c0b5040 comp rx=0 tx=0).stop 2024-03-11T18:35:51.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.612+0000 7fa7677ce640 1 -- 172.21.15.90:0/3401781569 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7600c4af0 msgr2=0x7fa760156a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:51.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.612+0000 7fa7677ce640 1 --2- 172.21.15.90:0/3401781569 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7600c4af0 0x7fa760156a70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa75000d800 tx=0x7fa75000dcc0 comp rx=0 tx=0).stop 2024-03-11T18:35:51.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.612+0000 7fa7677ce640 1 -- 172.21.15.90:0/3401781569 shutdown_connections 2024-03-11T18:35:51.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.612+0000 7fa7677ce640 1 --2- 172.21.15.90:0/3401781569 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fa744066ec0 0x7fa744069380 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:51.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.612+0000 7fa7677ce640 1 --2- 172.21.15.90:0/3401781569 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa760151b30 0x7fa760152460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:51.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.612+0000 7fa7677ce640 1 --2- 172.21.15.90:0/3401781569 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7600c4af0 0x7fa760156a70 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:51.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.612+0000 7fa7677ce640 1 --2- 172.21.15.90:0/3401781569 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa760070b00 0x7fa7600c27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:51.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.612+0000 7fa7677ce640 1 -- 172.21.15.90:0/3401781569 >> 172.21.15.90:0/3401781569 conn(0x7fa76006ac00 msgr2=0x7fa7600c7560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:35:51.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.612+0000 7fa7677ce640 1 -- 172.21.15.90:0/3401781569 shutdown_connections 2024-03-11T18:35:51.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.612+0000 7fa7677ce640 1 -- 172.21.15.90:0/3401781569 wait complete. 2024-03-11T18:35:51.636 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-11T18:35:16.193842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-11T18:35:51.637 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-11T18:35:51.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.756+0000 7ff9c2a0f640 1 Processor -- start 2024-03-11T18:35:51.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.756+0000 7ff9c2a0f640 1 -- start start 2024-03-11T18:35:51.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.760+0000 7ff9c2a0f640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff9bc15da50 0x7ff9bc0caf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:51.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.760+0000 7ff9c2a0f640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9bc0cb5b0 con 0x7ff9bc1541c0 2024-03-11T18:35:51.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.760+0000 7ff9c2a0f640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9bc159e10 con 0x7ff9bc0cfda0 2024-03-11T18:35:51.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.760+0000 7ff9c2a0f640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9bc0b5410 con 0x7ff9bc15da50 2024-03-11T18:35:51.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.760+0000 7ff9bbfff640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7ff9bc1541c0 0x7ff9bc0ca180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:51654/0 (socket says 172.21.15.90:51654) 2024-03-11T18:35:51.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.760+0000 7ff9bbfff640 1 -- 172.21.15.90:0/1720776892 learned_addr learned my addr 172.21.15.90:0/1720776892 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T18:35:51.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.760+0000 7ff9c0fac640 1 --2- 172.21.15.90:0/1720776892 >> v2:172.21.15.92:3300/0 conn(0x7ff9bc15da50 0x7ff9bc0caf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:51.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.764+0000 7ff9baffd640 1 -- 172.21.15.90:0/1720776892 <== mon.0 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 96466882 0 0) 0x7ff9bc0cb5b0 con 0x7ff9bc1541c0 2024-03-11T18:35:51.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.764+0000 7ff9baffd640 1 -- 172.21.15.90:0/1720776892 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9a4003530 con 0x7ff9bc1541c0 2024-03-11T18:35:51.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.768+0000 7ff9baffd640 1 -- 172.21.15.90:0/1720776892 <== mon.0 v1:172.21.15.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1865550673 0 0) 0x7ff9a4003530 con 0x7ff9bc1541c0 2024-03-11T18:35:51.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.768+0000 7ff9baffd640 1 -- 172.21.15.90:0/1720776892 >> v2:172.21.15.92:3300/0 conn(0x7ff9bc15da50 msgr2=0x7ff9bc0caf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:35:51.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.768+0000 7ff9baffd640 1 --2- 172.21.15.90:0/1720776892 >> v2:172.21.15.92:3300/0 conn(0x7ff9bc15da50 0x7ff9bc0caf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:51.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.768+0000 7ff9baffd640 1 -- 172.21.15.90:0/1720776892 >> v1:172.21.15.92:6789/0 conn(0x7ff9bc0cfda0 legacy=0x7ff9bc15d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:51.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.768+0000 7ff9baffd640 1 -- 172.21.15.90:0/1720776892 --> v1:172.21.15.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9bc15a0f0 con 0x7ff9bc1541c0 2024-03-11T18:35:51.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.768+0000 7ff9baffd640 1 -- 172.21.15.90:0/1720776892 <== mon.0 v1:172.21.15.90:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 691735435 0 0) 0x7ff9b4001680 con 0x7ff9bc1541c0 2024-03-11T18:35:51.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.768+0000 7ff9baffd640 1 -- 172.21.15.90:0/1720776892 >> v1:172.21.15.90:6789/0 conn(0x7ff9bc1541c0 legacy=0x7ff9bc0ca180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:51.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.768+0000 7ff9baffd640 1 --2- 172.21.15.90:0/1720776892 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9a40039d0 0x7ff9a4003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:51.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.768+0000 7ff9baffd640 1 --2- 172.21.15.90:0/1720776892 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff9a40047d0 0x7ff9a4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:51.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.768+0000 7ff9baffd640 1 --2- 172.21.15.90:0/1720776892 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9a4005190 0x7ff9a400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:51.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.768+0000 7ff9baffd640 1 -- 172.21.15.90:0/1720776892 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a400c040 con 0x7ff9a4005190 2024-03-11T18:35:51.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.768+0000 7ff9baffd640 1 -- 172.21.15.90:0/1720776892 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a400c1b0 con 0x7ff9a40039d0 2024-03-11T18:35:51.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.768+0000 7ff9baffd640 1 -- 172.21.15.90:0/1720776892 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a400c370 con 0x7ff9a40047d0 2024-03-11T18:35:51.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.768+0000 7ff9bb7fe640 1 --2- 172.21.15.90:0/1720776892 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff9a40047d0 0x7ff9a4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:51.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.768+0000 7ff9c0fac640 1 --2- 172.21.15.90:0/1720776892 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9a4005190 0x7ff9a400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:51.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.772+0000 7ff9c0fac640 1 -- 172.21.15.90:0/1720776892 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff9a40047d0 msgr2=0x7ff9a4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:35:51.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.772+0000 7ff9c0fac640 1 --2- 172.21.15.90:0/1720776892 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff9a40047d0 0x7ff9a4004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:51.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.772+0000 7ff9c0fac640 1 -- 172.21.15.90:0/1720776892 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9a40039d0 msgr2=0x7ff9a4003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:35:51.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.772+0000 7ff9c0fac640 1 --2- 172.21.15.90:0/1720776892 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9a40039d0 0x7ff9a4003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:51.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.772+0000 7ff9c0fac640 1 -- 172.21.15.90:0/1720776892 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a4003530 con 0x7ff9a4005190 2024-03-11T18:35:51.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.776+0000 7ff9c0fac640 1 -- 172.21.15.90:0/1720776892 reap_dead start 2024-03-11T18:35:51.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.776+0000 7ff9c0fac640 1 --2- 172.21.15.90:0/1720776892 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9a4005190 0x7ff9a400b990 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac009ae0 tx=0x7ff9ac01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b97796e7d1ec34 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:51.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.780+0000 7ff9baffd640 1 -- 172.21.15.90:0/1720776892 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac004020 con 0x7ff9a4005190 2024-03-11T18:35:51.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.780+0000 7ff9baffd640 1 -- 172.21.15.90:0/1720776892 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9ac004180 con 0x7ff9a4005190 2024-03-11T18:35:51.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.780+0000 7ff9baffd640 1 -- 172.21.15.90:0/1720776892 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac0a56a0 con 0x7ff9a4005190 2024-03-11T18:35:51.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.780+0000 7ff9c2a0f640 1 -- 172.21.15.90:0/1720776892 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9a4005190 msgr2=0x7ff9a400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:51.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.780+0000 7ff9c2a0f640 1 --2- 172.21.15.90:0/1720776892 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9a4005190 0x7ff9a400b990 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac009ae0 tx=0x7ff9ac01cab0 comp rx=0 tx=0).stop 2024-03-11T18:35:51.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.780+0000 7ff9c2a0f640 1 -- 172.21.15.90:0/1720776892 shutdown_connections 2024-03-11T18:35:51.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.780+0000 7ff9c2a0f640 1 --2- 172.21.15.90:0/1720776892 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9a4005190 0x7ff9a400b990 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:51.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.780+0000 7ff9c2a0f640 1 -- 172.21.15.90:0/1720776892 >> 172.21.15.90:0/1720776892 conn(0x7ff9bc06b620 msgr2=0x7ff9bc06ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:35:51.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.780+0000 7ff9c2a0f640 1 -- 172.21.15.90:0/1720776892 shutdown_connections 2024-03-11T18:35:51.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.780+0000 7ff9c2a0f640 1 -- 172.21.15.90:0/1720776892 wait complete. 2024-03-11T18:35:51.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.784+0000 7ff9c2a0f640 1 Processor -- start 2024-03-11T18:35:51.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.784+0000 7ff9c2a0f640 1 -- start start 2024-03-11T18:35:51.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.784+0000 7ff9c2a0f640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9a4005190 0x7ff9bc14da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:51.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.784+0000 7ff9c2a0f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9bc159560 0x7ff9bc15ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:51.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.784+0000 7ff9c2a0f640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff9bc0c6eb0 0x7ff9bc0c7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:51.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.784+0000 7ff9c2a0f640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9bc1f25a0 con 0x7ff9a4005190 2024-03-11T18:35:51.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.784+0000 7ff9c2a0f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9bc0bb820 con 0x7ff9bc159560 2024-03-11T18:35:51.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.784+0000 7ff9c2a0f640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9bc0bbb00 con 0x7ff9bc0c6eb0 2024-03-11T18:35:51.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.788+0000 7ff9bb7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9bc159560 0x7ff9bc15ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:51.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.788+0000 7ff9bb7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9bc159560 0x7ff9bc15ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.90:45358/0 (socket says 172.21.15.90:45358) 2024-03-11T18:35:51.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.788+0000 7ff9bb7fe640 1 -- 172.21.15.90:0/702127051 learned_addr learned my addr 172.21.15.90:0/702127051 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:35:51.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.792+0000 7ff9c0fac640 1 --2- 172.21.15.90:0/702127051 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff9bc0c6eb0 0x7ff9bc0c7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:51.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.796+0000 7ff9bb7fe640 1 -- 172.21.15.90:0/702127051 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff9bc0c6eb0 msgr2=0x7ff9bc0c7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:51.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.796+0000 7ff9bb7fe640 1 --2- 172.21.15.90:0/702127051 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff9bc0c6eb0 0x7ff9bc0c7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:51.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.796+0000 7ff9bb7fe640 1 -- 172.21.15.90:0/702127051 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9a4005190 msgr2=0x7ff9bc14da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:51.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.796+0000 7ff9bb7fe640 1 --2- 172.21.15.90:0/702127051 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9a4005190 0x7ff9bc14da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:51.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.796+0000 7ff9bb7fe640 1 -- 172.21.15.90:0/702127051 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9bc0bbde0 con 0x7ff9bc159560 2024-03-11T18:35:51.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.800+0000 7ff9bb7fe640 1 --2- 172.21.15.90:0/702127051 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9bc159560 0x7ff9bc15ae80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff9b000f960 tx=0x7ff9b000fe20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:51.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.804+0000 7ff9b8ff9640 1 -- 172.21.15.90:0/702127051 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b0003b90 con 0x7ff9bc159560 2024-03-11T18:35:51.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.804+0000 7ff9c2a0f640 1 -- 172.21.15.90:0/702127051 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9bc0bc0c0 con 0x7ff9bc159560 2024-03-11T18:35:51.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.804+0000 7ff9c2a0f640 1 -- 172.21.15.90:0/702127051 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9bc14df50 con 0x7ff9bc159560 2024-03-11T18:35:51.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.808+0000 7ff9c2a0f640 1 -- 172.21.15.90:0/702127051 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9bc04eef0 con 0x7ff9bc159560 2024-03-11T18:35:51.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.808+0000 7ff9b8ff9640 1 -- 172.21.15.90:0/702127051 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9b0003cf0 con 0x7ff9bc159560 2024-03-11T18:35:51.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:51.808+0000 7ff9b8ff9640 1 -- 172.21.15.90:0/702127051 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b000d9f0 con 0x7ff9bc159560 2024-03-11T18:35:52.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.128+0000 7ff9b8ff9640 1 -- 172.21.15.90:0/702127051 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7ff9b000db50 con 0x7ff9bc159560 2024-03-11T18:35:52.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.128+0000 7ff9b8ff9640 1 --2- 172.21.15.90:0/702127051 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7ff9a8066e40 0x7ff9a8069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:52.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.128+0000 7ff9bbfff640 1 --2- 172.21.15.90:0/702127051 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7ff9a8066e40 0x7ff9a8069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:52.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.128+0000 7ff9bbfff640 1 --2- 172.21.15.90:0/702127051 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7ff9a8066e40 0x7ff9a8069300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4000fc0 tx=0x7ff9b4001570 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:52.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.136+0000 7ff9b8ff9640 1 -- 172.21.15.90:0/702127051 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff9b00569f0 con 0x7ff9bc159560 2024-03-11T18:35:52.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.164+0000 7ff9b8ff9640 1 -- 172.21.15.90:0/702127051 <== mon.1 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff9b0052590 con 0x7ff9bc159560 2024-03-11T18:35:52.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.340+0000 7ff9c2a0f640 1 -- 172.21.15.90:0/702127051 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff9bc1f26e0 con 0x7ff9bc159560 2024-03-11T18:35:52.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.564+0000 7ff9b8ff9640 1 -- 172.21.15.90:0/702127051 <== mon.1 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff9bc1f26e0 con 0x7ff9bc159560 2024-03-11T18:35:52.569 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 1 2024-03-11T18:35:52.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.564+0000 7ff9c2a0f640 1 -- 172.21.15.90:0/702127051 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7ff9a8066e40 msgr2=0x7ff9a8069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:52.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.564+0000 7ff9c2a0f640 1 --2- 172.21.15.90:0/702127051 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7ff9a8066e40 0x7ff9a8069300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4000fc0 tx=0x7ff9b4001570 comp rx=0 tx=0).stop 2024-03-11T18:35:52.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.564+0000 7ff9c2a0f640 1 -- 172.21.15.90:0/702127051 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9bc159560 msgr2=0x7ff9bc15ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:52.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.564+0000 7ff9c2a0f640 1 --2- 172.21.15.90:0/702127051 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9bc159560 0x7ff9bc15ae80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff9b000f960 tx=0x7ff9b000fe20 comp rx=0 tx=0).stop 2024-03-11T18:35:52.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.564+0000 7ff9c2a0f640 1 -- 172.21.15.90:0/702127051 shutdown_connections 2024-03-11T18:35:52.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.564+0000 7ff9c2a0f640 1 --2- 172.21.15.90:0/702127051 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7ff9a8066e40 0x7ff9a8069300 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:52.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.564+0000 7ff9c2a0f640 1 --2- 172.21.15.90:0/702127051 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff9bc0c6eb0 0x7ff9bc0c7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:52.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.564+0000 7ff9c2a0f640 1 --2- 172.21.15.90:0/702127051 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff9bc159560 0x7ff9bc15ae80 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:52.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.564+0000 7ff9c2a0f640 1 --2- 172.21.15.90:0/702127051 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9a4005190 0x7ff9bc14da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:52.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.564+0000 7ff9c2a0f640 1 -- 172.21.15.90:0/702127051 >> 172.21.15.90:0/702127051 conn(0x7ff9bc06b620 msgr2=0x7ff9bc0c7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:35:52.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.564+0000 7ff9c2a0f640 1 -- 172.21.15.90:0/702127051 shutdown_connections 2024-03-11T18:35:52.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.564+0000 7ff9c2a0f640 1 -- 172.21.15.90:0/702127051 wait complete. 2024-03-11T18:35:52.593 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-11T18:35:52.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.708+0000 7f9484687640 1 Processor -- start 2024-03-11T18:35:52.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.708+0000 7f9484687640 1 -- start start 2024-03-11T18:35:52.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.712+0000 7f9484687640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f947c151b30 0x7f947c06e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:52.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.712+0000 7f9484687640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f947c06f430 con 0x7f947c070b00 2024-03-11T18:35:52.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.712+0000 7f9484687640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f947c06e820 con 0x7f947c0c4af0 2024-03-11T18:35:52.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.712+0000 7f9484687640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f947c0c65b0 con 0x7f947c151b30 2024-03-11T18:35:52.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.712+0000 7f9481c22640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f947c070b00 0x7f947c0c5de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:51670/0 (socket says 172.21.15.90:51670) 2024-03-11T18:35:52.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.712+0000 7f9481c22640 1 -- 172.21.15.90:0/2328196032 learned_addr learned my addr 172.21.15.90:0/2328196032 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T18:35:52.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.712+0000 7f9482423640 1 --2- 172.21.15.90:0/2328196032 >> v2:172.21.15.92:3300/0 conn(0x7f947c151b30 0x7f947c06e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:52.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.716+0000 7f9481421640 1 -- 172.21.15.90:0/2328196032 <== mon.0 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3363904432 0 0) 0x7f947c06f430 con 0x7f947c070b00 2024-03-11T18:35:52.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.716+0000 7f9481421640 1 -- 172.21.15.90:0/2328196032 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9468003530 con 0x7f947c070b00 2024-03-11T18:35:52.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.720+0000 7f9481421640 1 -- 172.21.15.90:0/2328196032 <== mon.0 v1:172.21.15.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3979344493 0 0) 0x7f9468003530 con 0x7f947c070b00 2024-03-11T18:35:52.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.720+0000 7f9481421640 1 -- 172.21.15.90:0/2328196032 >> v2:172.21.15.92:3300/0 conn(0x7f947c151b30 msgr2=0x7f947c06e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:35:52.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.720+0000 7f9481421640 1 --2- 172.21.15.90:0/2328196032 >> v2:172.21.15.92:3300/0 conn(0x7f947c151b30 0x7f947c06e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:52.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.720+0000 7f9481421640 1 -- 172.21.15.90:0/2328196032 >> v1:172.21.15.92:6789/0 conn(0x7f947c0c4af0 legacy=0x7f947c157c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:52.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.720+0000 7f9481421640 1 -- 172.21.15.90:0/2328196032 --> v1:172.21.15.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f947c06eb00 con 0x7f947c070b00 2024-03-11T18:35:52.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.720+0000 7f9481421640 1 -- 172.21.15.90:0/2328196032 <== mon.0 v1:172.21.15.90:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 691735435 0 0) 0x7f946c001680 con 0x7f947c070b00 2024-03-11T18:35:52.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.720+0000 7f9481421640 1 -- 172.21.15.90:0/2328196032 >> v1:172.21.15.90:6789/0 conn(0x7f947c070b00 legacy=0x7f947c0c5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:52.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.720+0000 7f9481421640 1 --2- 172.21.15.90:0/2328196032 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f94680039d0 0x7f9468003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:52.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.720+0000 7f9481421640 1 --2- 172.21.15.90:0/2328196032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94680047d0 0x7f9468004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:52.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.720+0000 7f9481421640 1 --2- 172.21.15.90:0/2328196032 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9468005190 0x7f946800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:52.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.720+0000 7f9481421640 1 -- 172.21.15.90:0/2328196032 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946800c060 con 0x7f9468005190 2024-03-11T18:35:52.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.720+0000 7f9481421640 1 -- 172.21.15.90:0/2328196032 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946800c1d0 con 0x7f94680047d0 2024-03-11T18:35:52.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.720+0000 7f9481421640 1 -- 172.21.15.90:0/2328196032 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f946800c390 con 0x7f94680039d0 2024-03-11T18:35:52.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.720+0000 7f9482423640 1 --2- 172.21.15.90:0/2328196032 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f94680039d0 0x7f9468003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:52.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.720+0000 7f9482c24640 1 --2- 172.21.15.90:0/2328196032 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9468005190 0x7f946800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:52.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.724+0000 7f9482c24640 1 -- 172.21.15.90:0/2328196032 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f94680039d0 msgr2=0x7f9468003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:35:52.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.724+0000 7f9482c24640 1 --2- 172.21.15.90:0/2328196032 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f94680039d0 0x7f9468003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:52.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.724+0000 7f9482c24640 1 -- 172.21.15.90:0/2328196032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94680047d0 msgr2=0x7f9468004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:35:52.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.724+0000 7f9482c24640 1 --2- 172.21.15.90:0/2328196032 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94680047d0 0x7f9468004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:52.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.724+0000 7f9482c24640 1 -- 172.21.15.90:0/2328196032 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9468003530 con 0x7f9468005190 2024-03-11T18:35:52.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.728+0000 7f9482c24640 1 -- 172.21.15.90:0/2328196032 reap_dead start 2024-03-11T18:35:52.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.728+0000 7f9482c24640 1 --2- 172.21.15.90:0/2328196032 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9468005190 0x7f946800b990 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9474001e70 tx=0x7f947401c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96f93e42aeeaaec6 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:52.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.732+0000 7f9481421640 1 -- 172.21.15.90:0/2328196032 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9474003b80 con 0x7f9468005190 2024-03-11T18:35:52.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.732+0000 7f9481421640 1 -- 172.21.15.90:0/2328196032 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9474003d20 con 0x7f9468005190 2024-03-11T18:35:52.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.732+0000 7f9481421640 1 -- 172.21.15.90:0/2328196032 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9474005260 con 0x7f9468005190 2024-03-11T18:35:52.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.732+0000 7f9484687640 1 -- 172.21.15.90:0/2328196032 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9468005190 msgr2=0x7f946800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:52.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.732+0000 7f9484687640 1 --2- 172.21.15.90:0/2328196032 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9468005190 0x7f946800b990 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9474001e70 tx=0x7f947401c780 comp rx=0 tx=0).stop 2024-03-11T18:35:52.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.732+0000 7f9484687640 1 -- 172.21.15.90:0/2328196032 shutdown_connections 2024-03-11T18:35:52.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.732+0000 7f9484687640 1 --2- 172.21.15.90:0/2328196032 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9468005190 0x7f946800b990 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:52.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.732+0000 7f9484687640 1 -- 172.21.15.90:0/2328196032 >> 172.21.15.90:0/2328196032 conn(0x7f947c06ac00 msgr2=0x7f947c06b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:35:52.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.732+0000 7f9484687640 1 -- 172.21.15.90:0/2328196032 shutdown_connections 2024-03-11T18:35:52.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.732+0000 7f9484687640 1 -- 172.21.15.90:0/2328196032 wait complete. 2024-03-11T18:35:52.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.732+0000 7f9484687640 1 Processor -- start 2024-03-11T18:35:52.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.732+0000 7f9484687640 1 -- start start 2024-03-11T18:35:52.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.736+0000 7f9484687640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9468005190 0x7f947c148380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:52.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.736+0000 7f9484687640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f947c156830 0x7f947c1ed350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:52.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.736+0000 7f9484687640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f947c158b30 0x7f947c14a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:52.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.736+0000 7f9484687640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947c1575c0 con 0x7f9468005190 2024-03-11T18:35:52.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.736+0000 7f9484687640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947c1488c0 con 0x7f947c158b30 2024-03-11T18:35:52.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.736+0000 7f9484687640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f947c1ec940 con 0x7f947c156830 2024-03-11T18:35:52.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.736+0000 7f9482423640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9468005190 0x7f947c148380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:52.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.736+0000 7f9482423640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9468005190 0x7f947c148380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38540/0 (socket says 172.21.15.90:38540) 2024-03-11T18:35:52.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.736+0000 7f9482423640 1 -- 172.21.15.90:0/1247471851 learned_addr learned my addr 172.21.15.90:0/1247471851 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:35:52.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.736+0000 7f9482c24640 1 --2- 172.21.15.90:0/1247471851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f947c158b30 0x7f947c14a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:52.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.736+0000 7f9481c22640 1 --2- 172.21.15.90:0/1247471851 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f947c156830 0x7f947c1ed350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:52.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.740+0000 7f9482423640 1 -- 172.21.15.90:0/1247471851 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f947c156830 msgr2=0x7f947c1ed350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:52.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.740+0000 7f9482423640 1 --2- 172.21.15.90:0/1247471851 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f947c156830 0x7f947c1ed350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:52.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.740+0000 7f9482423640 1 -- 172.21.15.90:0/1247471851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f947c158b30 msgr2=0x7f947c14a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:52.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.740+0000 7f9482423640 1 --2- 172.21.15.90:0/1247471851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f947c158b30 0x7f947c14a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:52.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.740+0000 7f9482423640 1 -- 172.21.15.90:0/1247471851 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f947c1ed890 con 0x7f9468005190 2024-03-11T18:35:52.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.740+0000 7f9482423640 1 --2- 172.21.15.90:0/1247471851 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9468005190 0x7f947c148380 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9478011990 tx=0x7f9478011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:52.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.744+0000 7f9472ffd640 1 -- 172.21.15.90:0/1247471851 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947800fad0 con 0x7f9468005190 2024-03-11T18:35:52.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.744+0000 7f9472ffd640 1 -- 172.21.15.90:0/1247471851 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f947800fc30 con 0x7f9468005190 2024-03-11T18:35:52.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.744+0000 7f9472ffd640 1 -- 172.21.15.90:0/1247471851 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9478014440 con 0x7f9468005190 2024-03-11T18:35:52.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.744+0000 7f9484687640 1 -- 172.21.15.90:0/1247471851 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f947c156d20 con 0x7f9468005190 2024-03-11T18:35:52.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.744+0000 7f9484687640 1 -- 172.21.15.90:0/1247471851 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f947c1572e0 con 0x7f9468005190 2024-03-11T18:35:52.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.748+0000 7f9484687640 1 -- 172.21.15.90:0/1247471851 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f947c04eef0 con 0x7f9468005190 2024-03-11T18:35:52.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.768+0000 7f9472ffd640 1 -- 172.21.15.90:0/1247471851 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f947801d400 con 0x7f9468005190 2024-03-11T18:35:52.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.768+0000 7f9472ffd640 1 --2- 172.21.15.90:0/1247471851 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f9460066ec0 0x7f9460069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:52.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.768+0000 7f9481c22640 1 --2- 172.21.15.90:0/1247471851 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f9460066ec0 0x7f9460069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:52.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:52.768+0000 7f9481c22640 1 --2- 172.21.15.90:0/1247471851 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f9460066ec0 0x7f9460069380 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f947c1475a0 tx=0x7f946c007be0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:53.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.036+0000 7f9472ffd640 1 -- 172.21.15.90:0/1247471851 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9478056260 con 0x7f9468005190 2024-03-11T18:35:53.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.036+0000 7f9472ffd640 1 -- 172.21.15.90:0/1247471851 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9478052500 con 0x7f9468005190 2024-03-11T18:35:53.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.204+0000 7f9484687640 1 -- 172.21.15.90:0/1247471851 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f947c14a9d0 con 0x7f9468005190 2024-03-11T18:35:53.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.220+0000 7f9472ffd640 1 -- 172.21.15.90:0/1247471851 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) v1 ==== 84+0+65 (secure 0 0 0) 0x7f9478055bc0 con 0x7f9468005190 2024-03-11T18:35:53.227 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:35:53.227 INFO:teuthology.orchestra.run.smithi090.stdout:[ 2024-03-11T18:35:53.227 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-03-11T18:35:53.227 INFO:teuthology.orchestra.run.smithi090.stdout: "poolnum": 1, 2024-03-11T18:35:53.227 INFO:teuthology.orchestra.run.smithi090.stdout: "poolname": ".mgr" 2024-03-11T18:35:53.227 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-03-11T18:35:53.228 INFO:teuthology.orchestra.run.smithi090.stdout:] 2024-03-11T18:35:53.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.224+0000 7f9484687640 1 -- 172.21.15.90:0/1247471851 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f9460066ec0 msgr2=0x7f9460069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:53.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.224+0000 7f9484687640 1 --2- 172.21.15.90:0/1247471851 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f9460066ec0 0x7f9460069380 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f947c1475a0 tx=0x7f946c007be0 comp rx=0 tx=0).stop 2024-03-11T18:35:53.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.224+0000 7f9484687640 1 -- 172.21.15.90:0/1247471851 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9468005190 msgr2=0x7f947c148380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:53.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.224+0000 7f9484687640 1 --2- 172.21.15.90:0/1247471851 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9468005190 0x7f947c148380 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9478011990 tx=0x7f9478011e50 comp rx=0 tx=0).stop 2024-03-11T18:35:53.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.224+0000 7f9484687640 1 -- 172.21.15.90:0/1247471851 shutdown_connections 2024-03-11T18:35:53.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.224+0000 7f9484687640 1 --2- 172.21.15.90:0/1247471851 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f9460066ec0 0x7f9460069380 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:53.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.224+0000 7f9484687640 1 --2- 172.21.15.90:0/1247471851 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f947c158b30 0x7f947c14a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:53.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.224+0000 7f9484687640 1 --2- 172.21.15.90:0/1247471851 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f947c156830 0x7f947c1ed350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:53.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.224+0000 7f9484687640 1 --2- 172.21.15.90:0/1247471851 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9468005190 0x7f947c148380 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:53.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.224+0000 7f9484687640 1 -- 172.21.15.90:0/1247471851 >> 172.21.15.90:0/1247471851 conn(0x7f947c06ac00 msgr2=0x7f947c157b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:35:53.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.224+0000 7f9484687640 1 -- 172.21.15.90:0/1247471851 shutdown_connections 2024-03-11T18:35:53.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.224+0000 7f9484687640 1 -- 172.21.15.90:0/1247471851 wait complete. 2024-03-11T18:35:53.247 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-11T18:35:53.247 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-11T18:35:53.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.368+0000 7fce64442640 1 Processor -- start 2024-03-11T18:35:53.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.368+0000 7fce64442640 1 -- start start 2024-03-11T18:35:53.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.368+0000 7fce64442640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fce5c06bd10 0x7fce5c06c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:53.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.368+0000 7fce64442640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fce5c15a830 0x7fce5c1644c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:53.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.368+0000 7fce64442640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce5c070370 con 0x7fce5c06dbd0 2024-03-11T18:35:53.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.368+0000 7fce64442640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce5c0c4540 con 0x7fce5c06bd10 2024-03-11T18:35:53.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.368+0000 7fce64442640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce5c070180 con 0x7fce5c15a830 2024-03-11T18:35:53.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.368+0000 7fce621de640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fce5c06bd10 0x7fce5c06c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:53.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.368+0000 7fce629df640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fce5c15a830 0x7fce5c1644c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:53.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.372+0000 7fce621de640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fce5c06bd10 0x7fce5c06c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.90:45398/0 (socket says 172.21.15.90:45398) 2024-03-11T18:35:53.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.372+0000 7fce621de640 1 -- 172.21.15.90:0/1818007320 learned_addr learned my addr 172.21.15.90:0/1818007320 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:35:53.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.376+0000 7fce629df640 1 -- 172.21.15.90:0/1818007320 >> v2:172.21.15.92:3300/0 conn(0x7fce5c06bd10 msgr2=0x7fce5c06c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:35:53.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.376+0000 7fce629df640 1 --2- 172.21.15.90:0/1818007320 >> v2:172.21.15.92:3300/0 conn(0x7fce5c06bd10 0x7fce5c06c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:53.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.376+0000 7fce629df640 1 -- 172.21.15.90:0/1818007320 >> v1:172.21.15.92:6789/0 conn(0x7fce5c06dbd0 legacy=0x7fce5c15a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:53.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.376+0000 7fce629df640 1 -- 172.21.15.90:0/1818007320 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce5c159c20 con 0x7fce5c15a830 2024-03-11T18:35:53.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.384+0000 7fce629df640 1 --2- 172.21.15.90:0/1818007320 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fce5c15a830 0x7fce5c1644c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fce54009d90 tx=0x7fce5401c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e3c3ee29199bde1 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:53.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.392+0000 7fce611dc640 1 -- 172.21.15.90:0/1818007320 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce540a5970 con 0x7fce5c15a830 2024-03-11T18:35:53.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.392+0000 7fce611dc640 1 -- 172.21.15.90:0/1818007320 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce540a5ad0 con 0x7fce5c15a830 2024-03-11T18:35:53.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.392+0000 7fce611dc640 1 -- 172.21.15.90:0/1818007320 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce540a5e60 con 0x7fce5c15a830 2024-03-11T18:35:53.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.392+0000 7fce64442640 1 -- 172.21.15.90:0/1818007320 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fce5c15a830 msgr2=0x7fce5c1644c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:53.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.392+0000 7fce64442640 1 --2- 172.21.15.90:0/1818007320 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fce5c15a830 0x7fce5c1644c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fce54009d90 tx=0x7fce5401c8b0 comp rx=0 tx=0).stop 2024-03-11T18:35:53.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.392+0000 7fce64442640 1 -- 172.21.15.90:0/1818007320 shutdown_connections 2024-03-11T18:35:53.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.392+0000 7fce64442640 1 --2- 172.21.15.90:0/1818007320 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fce5c15a830 0x7fce5c1644c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:53.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.392+0000 7fce64442640 1 --2- 172.21.15.90:0/1818007320 >> v2:172.21.15.92:3300/0 conn(0x7fce5c06bd10 0x7fce5c06c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:53.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.392+0000 7fce64442640 1 -- 172.21.15.90:0/1818007320 >> 172.21.15.90:0/1818007320 conn(0x7fce5c06b030 msgr2=0x7fce5c066990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:35:53.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.392+0000 7fce64442640 1 -- 172.21.15.90:0/1818007320 shutdown_connections 2024-03-11T18:35:53.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.392+0000 7fce64442640 1 -- 172.21.15.90:0/1818007320 wait complete. 2024-03-11T18:35:53.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.396+0000 7fce64442640 1 Processor -- start 2024-03-11T18:35:53.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.396+0000 7fce64442640 1 -- start start 2024-03-11T18:35:53.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.396+0000 7fce64442640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce5c06bd10 0x7fce5c163fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:53.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.396+0000 7fce64442640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fce5c06dbd0 0x7fce5c0ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:53.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.396+0000 7fce64442640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce5c15a830 0x7fce5c0addd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:53.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.396+0000 7fce64442640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5c0ae4a0 con 0x7fce5c15a830 2024-03-11T18:35:53.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.396+0000 7fce64442640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5c0cd330 con 0x7fce5c06bd10 2024-03-11T18:35:53.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.396+0000 7fce64442640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce5c14f0e0 con 0x7fce5c06dbd0 2024-03-11T18:35:53.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.396+0000 7fce629df640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce5c15a830 0x7fce5c0addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:53.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.396+0000 7fce629df640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce5c15a830 0x7fce5c0addd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38544/0 (socket says 172.21.15.90:38544) 2024-03-11T18:35:53.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.396+0000 7fce629df640 1 -- 172.21.15.90:0/1030776989 learned_addr learned my addr 172.21.15.90:0/1030776989 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:35:53.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.396+0000 7fce621de640 1 --2- 172.21.15.90:0/1030776989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce5c06bd10 0x7fce5c163fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:53.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.400+0000 7fce619dd640 1 --2- 172.21.15.90:0/1030776989 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fce5c06dbd0 0x7fce5c0ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:53.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.400+0000 7fce629df640 1 -- 172.21.15.90:0/1030776989 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fce5c06dbd0 msgr2=0x7fce5c0ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:53.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.400+0000 7fce629df640 1 --2- 172.21.15.90:0/1030776989 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fce5c06dbd0 0x7fce5c0ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:53.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.400+0000 7fce629df640 1 -- 172.21.15.90:0/1030776989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce5c06bd10 msgr2=0x7fce5c163fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:53.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.400+0000 7fce629df640 1 --2- 172.21.15.90:0/1030776989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce5c06bd10 0x7fce5c163fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:53.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.400+0000 7fce629df640 1 -- 172.21.15.90:0/1030776989 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce54009a70 con 0x7fce5c15a830 2024-03-11T18:35:53.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.404+0000 7fce629df640 1 --2- 172.21.15.90:0/1030776989 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce5c15a830 0x7fce5c0addd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fce5401a4e0 tx=0x7fce54019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:53.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.408+0000 7fce52ffd640 1 -- 172.21.15.90:0/1030776989 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce540aed00 con 0x7fce5c15a830 2024-03-11T18:35:53.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.408+0000 7fce52ffd640 1 -- 172.21.15.90:0/1030776989 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce540aee60 con 0x7fce5c15a830 2024-03-11T18:35:53.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.408+0000 7fce64442640 1 -- 172.21.15.90:0/1030776989 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce5c1f2ea0 con 0x7fce5c15a830 2024-03-11T18:35:53.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.408+0000 7fce52ffd640 1 -- 172.21.15.90:0/1030776989 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce540b7630 con 0x7fce5c15a830 2024-03-11T18:35:53.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.408+0000 7fce64442640 1 -- 172.21.15.90:0/1030776989 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce5c0ccd70 con 0x7fce5c15a830 2024-03-11T18:35:53.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.408+0000 7fce64442640 1 -- 172.21.15.90:0/1030776989 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce5c159090 con 0x7fce5c15a830 2024-03-11T18:35:53.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.716+0000 7fce52ffd640 1 -- 172.21.15.90:0/1030776989 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fce540b4050 con 0x7fce5c15a830 2024-03-11T18:35:53.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.716+0000 7fce52ffd640 1 --2- 172.21.15.90:0/1030776989 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fce40066ec0 0x7fce40069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:53.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.716+0000 7fce621de640 1 --2- 172.21.15.90:0/1030776989 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fce40066ec0 0x7fce40069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:53.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.716+0000 7fce621de640 1 --2- 172.21.15.90:0/1030776989 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fce40066ec0 0x7fce40069380 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fce58009b50 tx=0x7fce58009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:53.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.744+0000 7fce52ffd640 1 -- 172.21.15.90:0/1030776989 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fce540f0eb0 con 0x7fce5c15a830 2024-03-11T18:35:53.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.744+0000 7fce52ffd640 1 -- 172.21.15.90:0/1030776989 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fce540ed150 con 0x7fce5c15a830 2024-03-11T18:35:53.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:53.916+0000 7fce64442640 1 -- 172.21.15.90:0/1030776989 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fce5c04efd0 con 0x7fce5c15a830 2024-03-11T18:35:55.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.132+0000 7fce52ffd640 1 -- 172.21.15.90:0/1030776989 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) v1 ==== 138+0+0 (secure 0 0 0) 0x7fce540c70b0 con 0x7fce5c15a830 2024-03-11T18:35:55.140 INFO:teuthology.orchestra.run.smithi090.stderr:pool 'cephfs_metadata' created 2024-03-11T18:35:55.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.136+0000 7fce64442640 1 -- 172.21.15.90:0/1030776989 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fce40066ec0 msgr2=0x7fce40069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:55.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.136+0000 7fce64442640 1 --2- 172.21.15.90:0/1030776989 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fce40066ec0 0x7fce40069380 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fce58009b50 tx=0x7fce58009240 comp rx=0 tx=0).stop 2024-03-11T18:35:55.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.136+0000 7fce64442640 1 -- 172.21.15.90:0/1030776989 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce5c15a830 msgr2=0x7fce5c0addd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:55.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.136+0000 7fce64442640 1 --2- 172.21.15.90:0/1030776989 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce5c15a830 0x7fce5c0addd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fce5401a4e0 tx=0x7fce54019650 comp rx=0 tx=0).stop 2024-03-11T18:35:55.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.136+0000 7fce64442640 1 -- 172.21.15.90:0/1030776989 shutdown_connections 2024-03-11T18:35:55.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.136+0000 7fce64442640 1 --2- 172.21.15.90:0/1030776989 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fce40066ec0 0x7fce40069380 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:55.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.136+0000 7fce64442640 1 --2- 172.21.15.90:0/1030776989 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce5c15a830 0x7fce5c0addd0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:55.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.136+0000 7fce64442640 1 --2- 172.21.15.90:0/1030776989 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fce5c06dbd0 0x7fce5c0ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:55.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.136+0000 7fce64442640 1 --2- 172.21.15.90:0/1030776989 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce5c06bd10 0x7fce5c163fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:55.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.136+0000 7fce64442640 1 -- 172.21.15.90:0/1030776989 >> 172.21.15.90:0/1030776989 conn(0x7fce5c06b030 msgr2=0x7fce5c0c8160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:35:55.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.136+0000 7fce64442640 1 -- 172.21.15.90:0/1030776989 shutdown_connections 2024-03-11T18:35:55.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.136+0000 7fce64442640 1 -- 172.21.15.90:0/1030776989 wait complete. 2024-03-11T18:35:55.164 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-03-11T18:35:55.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.288+0000 7fda09613640 1 Processor -- start 2024-03-11T18:35:55.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.288+0000 7fda09613640 1 -- start start 2024-03-11T18:35:55.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.288+0000 7fda09613640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fda04070b00 0x7fda0406e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:55.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.288+0000 7fda09613640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fda040c4af0 0x7fda040c4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:55.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.288+0000 7fda09613640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda0406f430 con 0x7fda04151b30 2024-03-11T18:35:55.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.288+0000 7fda09613640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda040c65b0 con 0x7fda04070b00 2024-03-11T18:35:55.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.288+0000 7fda09613640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda0406e820 con 0x7fda040c4af0 2024-03-11T18:35:55.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.288+0000 7fda02ffd640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7fda04151b30 0x7fda040c5de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:37448/0 (socket says 172.21.15.90:37448) 2024-03-11T18:35:55.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.288+0000 7fda02ffd640 1 -- 172.21.15.90:0/3107274630 learned_addr learned my addr 172.21.15.90:0/3107274630 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T18:35:55.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.292+0000 7fda037fe640 1 --2- 172.21.15.90:0/3107274630 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fda040c4af0 0x7fda040c4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:55.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.292+0000 7fda01ffb640 1 -- 172.21.15.90:0/3107274630 <== mon.0 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3005646990 0 0) 0x7fda0406f430 con 0x7fda04151b30 2024-03-11T18:35:55.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.292+0000 7fda01ffb640 1 -- 172.21.15.90:0/3107274630 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9ec003530 con 0x7fda04151b30 2024-03-11T18:35:55.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.300+0000 7fda01ffb640 1 -- 172.21.15.90:0/3107274630 <== mon.0 v1:172.21.15.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 790657594 0 0) 0x7fd9ec003530 con 0x7fda04151b30 2024-03-11T18:35:55.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.300+0000 7fda01ffb640 1 -- 172.21.15.90:0/3107274630 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fda040c4af0 msgr2=0x7fda040c4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:35:55.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.300+0000 7fda01ffb640 1 --2- 172.21.15.90:0/3107274630 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fda040c4af0 0x7fda040c4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:55.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.300+0000 7fda01ffb640 1 -- 172.21.15.90:0/3107274630 >> v2:172.21.15.92:3300/0 conn(0x7fda04070b00 msgr2=0x7fda0406e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:35:55.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.300+0000 7fda01ffb640 1 --2- 172.21.15.90:0/3107274630 >> v2:172.21.15.92:3300/0 conn(0x7fda04070b00 0x7fda0406e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:55.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.300+0000 7fda01ffb640 1 -- 172.21.15.90:0/3107274630 --> v1:172.21.15.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda0406eb00 con 0x7fda04151b30 2024-03-11T18:35:55.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.300+0000 7fda01ffb640 1 -- 172.21.15.90:0/3107274630 <== mon.0 v1:172.21.15.90:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 691735435 0 0) 0x7fd9f4001680 con 0x7fda04151b30 2024-03-11T18:35:55.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.300+0000 7fda01ffb640 1 -- 172.21.15.90:0/3107274630 >> v1:172.21.15.90:6789/0 conn(0x7fda04151b30 legacy=0x7fda040c5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:55.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.300+0000 7fda01ffb640 1 --2- 172.21.15.90:0/3107274630 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fda040c4af0 0x7fd9ec003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:55.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.300+0000 7fda01ffb640 1 --2- 172.21.15.90:0/3107274630 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9ec003f50 0x7fd9ec0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:55.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.300+0000 7fda01ffb640 1 --2- 172.21.15.90:0/3107274630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9ec004d10 0x7fd9ec005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:55.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.300+0000 7fda01ffb640 1 -- 172.21.15.90:0/3107274630 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda0406e820 con 0x7fd9ec003f50 2024-03-11T18:35:55.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.300+0000 7fda01ffb640 1 -- 172.21.15.90:0/3107274630 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec00baa0 con 0x7fd9ec004d10 2024-03-11T18:35:55.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.300+0000 7fda01ffb640 1 -- 172.21.15.90:0/3107274630 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec00bc60 con 0x7fda040c4af0 2024-03-11T18:35:55.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.304+0000 7fda027fc640 1 --2- 172.21.15.90:0/3107274630 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9ec003f50 0x7fd9ec0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:55.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.308+0000 7fda027fc640 1 -- 172.21.15.90:0/3107274630 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fda040c4af0 msgr2=0x7fd9ec003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:35:55.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.308+0000 7fda027fc640 1 --2- 172.21.15.90:0/3107274630 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fda040c4af0 0x7fd9ec003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:55.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.308+0000 7fda027fc640 1 -- 172.21.15.90:0/3107274630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9ec004d10 msgr2=0x7fd9ec005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:35:55.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.308+0000 7fda027fc640 1 --2- 172.21.15.90:0/3107274630 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd9ec004d10 0x7fd9ec005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:55.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.308+0000 7fda027fc640 1 -- 172.21.15.90:0/3107274630 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9ec003530 con 0x7fd9ec003f50 2024-03-11T18:35:55.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.312+0000 7fda027fc640 1 --2- 172.21.15.90:0/3107274630 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9ec003f50 0x7fd9ec0047d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fda0406e750 tx=0x7fd9f801c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c2732d9725d929 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:55.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.320+0000 7fda01ffb640 1 -- 172.21.15.90:0/3107274630 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f80ab600 con 0x7fd9ec003f50 2024-03-11T18:35:55.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.320+0000 7fda01ffb640 1 -- 172.21.15.90:0/3107274630 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f80a7050 con 0x7fd9ec003f50 2024-03-11T18:35:55.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.320+0000 7fda01ffb640 1 -- 172.21.15.90:0/3107274630 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f80aba00 con 0x7fd9ec003f50 2024-03-11T18:35:55.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.320+0000 7fda09613640 1 -- 172.21.15.90:0/3107274630 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9ec003f50 msgr2=0x7fd9ec0047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:55.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.320+0000 7fda09613640 1 --2- 172.21.15.90:0/3107274630 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd9ec003f50 0x7fd9ec0047d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fda0406e750 tx=0x7fd9f801c360 comp rx=0 tx=0).stop 2024-03-11T18:35:55.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.320+0000 7fda037fe640 1 -- 172.21.15.90:0/3107274630 reap_dead start 2024-03-11T18:35:55.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.320+0000 7fda09613640 1 -- 172.21.15.90:0/3107274630 shutdown_connections 2024-03-11T18:35:55.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.320+0000 7fda09613640 1 -- 172.21.15.90:0/3107274630 >> 172.21.15.90:0/3107274630 conn(0x7fda0406ac00 msgr2=0x7fda0406b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:35:55.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.320+0000 7fda09613640 1 -- 172.21.15.90:0/3107274630 shutdown_connections 2024-03-11T18:35:55.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.320+0000 7fda09613640 1 -- 172.21.15.90:0/3107274630 wait complete. 2024-03-11T18:35:55.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.320+0000 7fda09613640 1 Processor -- start 2024-03-11T18:35:55.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.320+0000 7fda09613640 1 -- start start 2024-03-11T18:35:55.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.324+0000 7fda09613640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda041543f0 0x7fda040cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:55.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.324+0000 7fda09613640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda040c9220 0x7fda040cbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:55.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.324+0000 7fda09613640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fda041547d0 0x7fda041469b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:55.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.324+0000 7fda09613640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda040c9660 con 0x7fda040c9220 2024-03-11T18:35:55.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.324+0000 7fda09613640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda04158260 con 0x7fda041543f0 2024-03-11T18:35:55.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.324+0000 7fda09613640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda040c2740 con 0x7fda041547d0 2024-03-11T18:35:55.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.324+0000 7fda027fc640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda040c9220 0x7fda040cbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:55.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.324+0000 7fda027fc640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda040c9220 0x7fda040cbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54106/0 (socket says 172.21.15.90:54106) 2024-03-11T18:35:55.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.324+0000 7fda027fc640 1 -- 172.21.15.90:0/1846658780 learned_addr learned my addr 172.21.15.90:0/1846658780 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:35:55.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.324+0000 7fda037fe640 1 --2- 172.21.15.90:0/1846658780 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fda041547d0 0x7fda041469b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:55.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.328+0000 7fda027fc640 1 -- 172.21.15.90:0/1846658780 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fda041547d0 msgr2=0x7fda041469b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:55.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.328+0000 7fda027fc640 1 --2- 172.21.15.90:0/1846658780 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fda041547d0 0x7fda041469b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:55.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.328+0000 7fda027fc640 1 -- 172.21.15.90:0/1846658780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda041543f0 msgr2=0x7fda040cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:55.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.328+0000 7fda027fc640 1 --2- 172.21.15.90:0/1846658780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda041543f0 0x7fda040cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:55.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.328+0000 7fda027fc640 1 -- 172.21.15.90:0/1846658780 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda041ee230 con 0x7fda040c9220 2024-03-11T18:35:55.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.328+0000 7fda027fc640 1 --2- 172.21.15.90:0/1846658780 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda040c9220 0x7fda040cbce0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8000c10 tx=0x7fd9ec005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:55.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.332+0000 7fd9eb7fe640 1 -- 172.21.15.90:0/1846658780 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8003b10 con 0x7fda040c9220 2024-03-11T18:35:55.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.332+0000 7fd9eb7fe640 1 -- 172.21.15.90:0/1846658780 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f8003c70 con 0x7fda040c9220 2024-03-11T18:35:55.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.332+0000 7fd9eb7fe640 1 -- 172.21.15.90:0/1846658780 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f80b5410 con 0x7fda040c9220 2024-03-11T18:35:55.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.336+0000 7fda09613640 1 -- 172.21.15.90:0/1846658780 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda04157fe0 con 0x7fda040c9220 2024-03-11T18:35:55.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.336+0000 7fda09613640 1 -- 172.21.15.90:0/1846658780 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda041edec0 con 0x7fda040c9220 2024-03-11T18:35:55.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.336+0000 7fda09613640 1 -- 172.21.15.90:0/1846658780 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda041edcf0 con 0x7fda040c9220 2024-03-11T18:35:55.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.620+0000 7fd9eb7fe640 1 -- 172.21.15.90:0/1846658780 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fd9f80b2050 con 0x7fda040c9220 2024-03-11T18:35:55.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.620+0000 7fd9eb7fe640 1 --2- 172.21.15.90:0/1846658780 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fd9f0066e90 0x7fd9f0069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:55.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.620+0000 7fda02ffd640 1 --2- 172.21.15.90:0/1846658780 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fd9f0066e90 0x7fd9f0069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:55.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.620+0000 7fda02ffd640 1 --2- 172.21.15.90:0/1846658780 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fd9f0066e90 0x7fd9f0069350 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd9f400af00 tx=0x7fd9f40015d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:55.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.660+0000 7fd9eb7fe640 1 -- 172.21.15.90:0/1846658780 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fd9f80efdc0 con 0x7fda040c9220 2024-03-11T18:35:55.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.680+0000 7fd9eb7fe640 1 -- 172.21.15.90:0/1846658780 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd9f80efb90 con 0x7fda040c9220 2024-03-11T18:35:55.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:55.860+0000 7fda09613640 1 -- 172.21.15.90:0/1846658780 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7fda04068030 con 0x7fda040c9220 2024-03-11T18:35:56.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.232+0000 7fd9eb7fe640 1 -- 172.21.15.90:0/1846658780 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v20) v1 ==== 171+0+0 (secure 0 0 0) 0x7fd9f80b0020 con 0x7fda040c9220 2024-03-11T18:35:56.240 INFO:teuthology.orchestra.run.smithi090.stderr:pool 'cephfs_data' created 2024-03-11T18:35:56.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.236+0000 7fda09613640 1 -- 172.21.15.90:0/1846658780 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fd9f0066e90 msgr2=0x7fd9f0069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:56.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.236+0000 7fda09613640 1 --2- 172.21.15.90:0/1846658780 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fd9f0066e90 0x7fd9f0069350 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd9f400af00 tx=0x7fd9f40015d0 comp rx=0 tx=0).stop 2024-03-11T18:35:56.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.236+0000 7fda09613640 1 -- 172.21.15.90:0/1846658780 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda040c9220 msgr2=0x7fda040cbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:56.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.236+0000 7fda09613640 1 --2- 172.21.15.90:0/1846658780 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda040c9220 0x7fda040cbce0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8000c10 tx=0x7fd9ec005800 comp rx=0 tx=0).stop 2024-03-11T18:35:56.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.236+0000 7fda09613640 1 -- 172.21.15.90:0/1846658780 shutdown_connections 2024-03-11T18:35:56.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.236+0000 7fda09613640 1 --2- 172.21.15.90:0/1846658780 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fd9f0066e90 0x7fd9f0069350 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:56.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.236+0000 7fda09613640 1 --2- 172.21.15.90:0/1846658780 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fda041547d0 0x7fda041469b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:56.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.236+0000 7fda09613640 1 --2- 172.21.15.90:0/1846658780 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda040c9220 0x7fda040cbce0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:56.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.236+0000 7fda09613640 1 --2- 172.21.15.90:0/1846658780 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fda041543f0 0x7fda040cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:56.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.236+0000 7fda09613640 1 -- 172.21.15.90:0/1846658780 >> 172.21.15.90:0/1846658780 conn(0x7fda0406ac00 msgr2=0x7fda040711e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:35:56.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.236+0000 7fda09613640 1 -- 172.21.15.90:0/1846658780 shutdown_connections 2024-03-11T18:35:56.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.236+0000 7fda09613640 1 -- 172.21.15.90:0/1846658780 wait complete. 2024-03-11T18:35:56.269 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-11T18:35:56.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.400+0000 7f046f2ea640 1 Processor -- start 2024-03-11T18:35:56.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.400+0000 7f046f2ea640 1 -- start start 2024-03-11T18:35:56.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.400+0000 7f046f2ea640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f04681598c0 0x7f046806e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:56.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.400+0000 7f046f2ea640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0468070b00 con 0x7f04680c3d30 2024-03-11T18:35:56.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.400+0000 7f046f2ea640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0468071df0 con 0x7f046806f220 2024-03-11T18:35:56.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.400+0000 7f046f2ea640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f046806eef0 con 0x7f04681598c0 2024-03-11T18:35:56.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.400+0000 7f046d086640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f04681598c0 0x7f046806e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:56.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.400+0000 7f046d887640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f04680c3d30 0x7f046814fe20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:37460/0 (socket says 172.21.15.90:37460) 2024-03-11T18:35:56.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.400+0000 7f046d887640 1 -- 172.21.15.90:0/2914416204 learned_addr learned my addr 172.21.15.90:0/2914416204 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T18:35:56.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.404+0000 7f046d086640 1 -- 172.21.15.90:0/2914416204 >> v1:172.21.15.92:6789/0 conn(0x7f046806f220 legacy=0x7f0468159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:35:56.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.404+0000 7f046d086640 1 -- 172.21.15.90:0/2914416204 >> v1:172.21.15.90:6789/0 conn(0x7f04680c3d30 legacy=0x7f046814fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:56.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.404+0000 7f046d086640 1 -- 172.21.15.90:0/2914416204 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f046815a3b0 con 0x7f04681598c0 2024-03-11T18:35:56.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.412+0000 7f046d086640 1 --2- 172.21.15.90:0/2914416204 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04681598c0 0x7f046806e9b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f04600097c0 tx=0x7f046001ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=913df1e8aeb4473b server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:56.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.412+0000 7f0467fff640 1 -- 172.21.15.90:0/2914416204 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04600a8950 con 0x7f04681598c0 2024-03-11T18:35:56.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.416+0000 7f0467fff640 1 -- 172.21.15.90:0/2914416204 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04600a8ab0 con 0x7f04681598c0 2024-03-11T18:35:56.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.416+0000 7f0467fff640 1 -- 172.21.15.90:0/2914416204 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04600a8e40 con 0x7f04681598c0 2024-03-11T18:35:56.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.416+0000 7f046f2ea640 1 -- 172.21.15.90:0/2914416204 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04681598c0 msgr2=0x7f046806e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:56.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.416+0000 7f046f2ea640 1 --2- 172.21.15.90:0/2914416204 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04681598c0 0x7f046806e9b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f04600097c0 tx=0x7f046001ec50 comp rx=0 tx=0).stop 2024-03-11T18:35:56.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.416+0000 7f046f2ea640 1 -- 172.21.15.90:0/2914416204 shutdown_connections 2024-03-11T18:35:56.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.416+0000 7f046f2ea640 1 --2- 172.21.15.90:0/2914416204 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04681598c0 0x7f046806e9b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:56.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.416+0000 7f046f2ea640 1 -- 172.21.15.90:0/2914416204 >> 172.21.15.90:0/2914416204 conn(0x7f046806ac00 msgr2=0x7f046806b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:35:56.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.416+0000 7f046f2ea640 1 -- 172.21.15.90:0/2914416204 shutdown_connections 2024-03-11T18:35:56.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.416+0000 7f046f2ea640 1 -- 172.21.15.90:0/2914416204 wait complete. 2024-03-11T18:35:56.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.416+0000 7f046f2ea640 1 Processor -- start 2024-03-11T18:35:56.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.416+0000 7f046f2ea640 1 -- start start 2024-03-11T18:35:56.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.416+0000 7f046f2ea640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f046806f220 0x7f0468157c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:56.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.416+0000 7f046f2ea640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04680c3d30 0x7f04681f2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:56.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.416+0000 7f046f2ea640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f04681598c0 0x7f04681485e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:56.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.416+0000 7f046f2ea640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04681f39b0 con 0x7f04680c3d30 2024-03-11T18:35:56.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.416+0000 7f046f2ea640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04681493f0 con 0x7f046806f220 2024-03-11T18:35:56.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.416+0000 7f046f2ea640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04681f3710 con 0x7f04681598c0 2024-03-11T18:35:56.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.420+0000 7f046d086640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f046806f220 0x7f0468157c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:56.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.424+0000 7f046d086640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f046806f220 0x7f0468157c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.90:34530/0 (socket says 172.21.15.90:34530) 2024-03-11T18:35:56.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.424+0000 7f046d086640 1 -- 172.21.15.90:0/3220722651 learned_addr learned my addr 172.21.15.90:0/3220722651 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:35:56.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.452+0000 7f046c885640 1 --2- 172.21.15.90:0/3220722651 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04680c3d30 0x7f04681f2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:56.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.456+0000 7f046d086640 1 -- 172.21.15.90:0/3220722651 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f04681598c0 msgr2=0x7f04681485e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:56.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.456+0000 7f046d086640 1 --2- 172.21.15.90:0/3220722651 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f04681598c0 0x7f04681485e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:56.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.456+0000 7f046d086640 1 -- 172.21.15.90:0/3220722651 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04680c3d30 msgr2=0x7f04681f2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:56.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.456+0000 7f046d086640 1 --2- 172.21.15.90:0/3220722651 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04680c3d30 0x7f04681f2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:56.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.456+0000 7f046d086640 1 -- 172.21.15.90:0/3220722651 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0460009420 con 0x7f046806f220 2024-03-11T18:35:56.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.468+0000 7f046d086640 1 --2- 172.21.15.90:0/3220722651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f046806f220 0x7f0468157c40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f046001b860 tx=0x7f046001b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:56.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.504+0000 7f0465ffb640 1 -- 172.21.15.90:0/3220722651 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04600a8950 con 0x7f046806f220 2024-03-11T18:35:56.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.504+0000 7f0465ffb640 1 -- 172.21.15.90:0/3220722651 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04600b9410 con 0x7f046806f220 2024-03-11T18:35:56.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.504+0000 7f0465ffb640 1 -- 172.21.15.90:0/3220722651 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04600b97a0 con 0x7f046806f220 2024-03-11T18:35:56.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.504+0000 7f046f2ea640 1 -- 172.21.15.90:0/3220722651 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0468147d20 con 0x7f046806f220 2024-03-11T18:35:56.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.504+0000 7f046f2ea640 1 -- 172.21.15.90:0/3220722651 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0468158180 con 0x7f046806f220 2024-03-11T18:35:56.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.508+0000 7f046f2ea640 1 -- 172.21.15.90:0/3220722651 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0468158f60 con 0x7f046806f220 2024-03-11T18:35:56.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.560+0000 7f0465ffb640 1 -- 172.21.15.90:0/3220722651 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f04600b7040 con 0x7f046806f220 2024-03-11T18:35:56.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.560+0000 7f0465ffb640 1 --2- 172.21.15.90:0/3220722651 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0454066e90 0x7f0454069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:56.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.560+0000 7f046c885640 1 --2- 172.21.15.90:0/3220722651 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0454066e90 0x7f0454069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:56.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.560+0000 7f046c885640 1 --2- 172.21.15.90:0/3220722651 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0454066e90 0x7f0454069350 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f04680c4930 tx=0x7f045c00d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:56.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.568+0000 7f0465ffb640 1 -- 172.21.15.90:0/3220722651 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f04600f3e60 con 0x7f046806f220 2024-03-11T18:35:56.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:56.896+0000 7f0465ffb640 1 -- 172.21.15.90:0/3220722651 <== mon.1 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f04600efa00 con 0x7f046806f220 2024-03-11T18:35:57.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:57.104+0000 7f046f2ea640 1 -- 172.21.15.90:0/3220722651 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f0468158f60 con 0x7f046806f220 2024-03-11T18:35:59.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.116+0000 7f0465ffb640 1 -- 172.21.15.90:0/3220722651 <== mon.1 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v5) v1 ==== 172+0+0 (secure 0 0 0) 0x7f0468158f60 con 0x7f046806f220 2024-03-11T18:35:59.121 INFO:teuthology.orchestra.run.smithi090.stderr:new fs with metadata pool 2 and data pool 3 2024-03-11T18:35:59.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.116+0000 7f046f2ea640 1 -- 172.21.15.90:0/3220722651 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0454066e90 msgr2=0x7f0454069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:59.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.116+0000 7f046f2ea640 1 --2- 172.21.15.90:0/3220722651 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0454066e90 0x7f0454069350 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f04680c4930 tx=0x7f045c00d040 comp rx=0 tx=0).stop 2024-03-11T18:35:59.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.116+0000 7f046f2ea640 1 -- 172.21.15.90:0/3220722651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f046806f220 msgr2=0x7f0468157c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:59.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.116+0000 7f046f2ea640 1 --2- 172.21.15.90:0/3220722651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f046806f220 0x7f0468157c40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f046001b860 tx=0x7f046001b9f0 comp rx=0 tx=0).stop 2024-03-11T18:35:59.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.120+0000 7f046f2ea640 1 -- 172.21.15.90:0/3220722651 shutdown_connections 2024-03-11T18:35:59.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.120+0000 7f046f2ea640 1 --2- 172.21.15.90:0/3220722651 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0454066e90 0x7f0454069350 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:59.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.120+0000 7f046f2ea640 1 --2- 172.21.15.90:0/3220722651 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f04681598c0 0x7f04681485e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:59.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.120+0000 7f046f2ea640 1 --2- 172.21.15.90:0/3220722651 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04680c3d30 0x7f04681f2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:59.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.120+0000 7f046f2ea640 1 --2- 172.21.15.90:0/3220722651 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f046806f220 0x7f0468157c40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:59.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.120+0000 7f046f2ea640 1 -- 172.21.15.90:0/3220722651 >> 172.21.15.90:0/3220722651 conn(0x7f046806ac00 msgr2=0x7f04680c4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:35:59.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.120+0000 7f046f2ea640 1 -- 172.21.15.90:0/3220722651 shutdown_connections 2024-03-11T18:35:59.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.120+0000 7f046f2ea640 1 -- 172.21.15.90:0/3220722651 wait complete. 2024-03-11T18:35:59.157 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-03-11T18:35:59.157 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-03-11T18:35:59.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.280+0000 7fb0da706640 1 Processor -- start 2024-03-11T18:35:59.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.280+0000 7fb0da706640 1 -- start start 2024-03-11T18:35:59.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.280+0000 7fb0da706640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb0d40c8e80 0x7fb0d40c7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:59.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.280+0000 7fb0da706640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0d415a370 con 0x7fb0d4153e00 2024-03-11T18:35:59.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.280+0000 7fb0da706640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0d40c93c0 con 0x7fb0d406ad00 2024-03-11T18:35:59.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.280+0000 7fb0da706640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0d406cb70 con 0x7fb0d40c8e80 2024-03-11T18:35:59.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.284+0000 7fb0d3fff640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7fb0d4153e00 0x7fb0d40c8620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:37462/0 (socket says 172.21.15.90:37462) 2024-03-11T18:35:59.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.284+0000 7fb0d3fff640 1 -- 172.21.15.90:0/1525423548 learned_addr learned my addr 172.21.15.90:0/1525423548 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T18:35:59.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.284+0000 7fb0d8ca3640 1 --2- 172.21.15.90:0/1525423548 >> v2:172.21.15.92:3300/0 conn(0x7fb0d40c8e80 0x7fb0d40c7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:59.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.284+0000 7fb0d2ffd640 1 -- 172.21.15.90:0/1525423548 <== mon.0 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1630898798 0 0) 0x7fb0d415a370 con 0x7fb0d4153e00 2024-03-11T18:35:59.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.284+0000 7fb0d2ffd640 1 -- 172.21.15.90:0/1525423548 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0bc003530 con 0x7fb0d4153e00 2024-03-11T18:35:59.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.288+0000 7fb0d2ffd640 1 -- 172.21.15.90:0/1525423548 <== mon.0 v1:172.21.15.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3161950873 0 0) 0x7fb0bc003530 con 0x7fb0d4153e00 2024-03-11T18:35:59.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.288+0000 7fb0d2ffd640 1 -- 172.21.15.90:0/1525423548 >> v2:172.21.15.92:3300/0 conn(0x7fb0d40c8e80 msgr2=0x7fb0d40c7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:35:59.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.288+0000 7fb0d2ffd640 1 --2- 172.21.15.90:0/1525423548 >> v2:172.21.15.92:3300/0 conn(0x7fb0d40c8e80 0x7fb0d40c7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:59.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.288+0000 7fb0d2ffd640 1 -- 172.21.15.90:0/1525423548 >> v1:172.21.15.92:6789/0 conn(0x7fb0d406ad00 legacy=0x7fb0d4159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:59.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.288+0000 7fb0d2ffd640 1 -- 172.21.15.90:0/1525423548 --> v1:172.21.15.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d40c96a0 con 0x7fb0d4153e00 2024-03-11T18:35:59.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.288+0000 7fb0d2ffd640 1 -- 172.21.15.90:0/1525423548 <== mon.0 v1:172.21.15.90:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 691735435 0 0) 0x7fb0cc001680 con 0x7fb0d4153e00 2024-03-11T18:35:59.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.288+0000 7fb0d2ffd640 1 -- 172.21.15.90:0/1525423548 >> v1:172.21.15.90:6789/0 conn(0x7fb0d4153e00 legacy=0x7fb0d40c8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:59.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.288+0000 7fb0d2ffd640 1 --2- 172.21.15.90:0/1525423548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0bc0039d0 0x7fb0bc003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:59.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.288+0000 7fb0d2ffd640 1 --2- 172.21.15.90:0/1525423548 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0bc0047d0 0x7fb0bc004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:59.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.288+0000 7fb0d2ffd640 1 --2- 172.21.15.90:0/1525423548 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb0bc005190 0x7fb0bc00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:59.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.288+0000 7fb0d2ffd640 1 -- 172.21.15.90:0/1525423548 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0bc00c040 con 0x7fb0bc0047d0 2024-03-11T18:35:59.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.288+0000 7fb0d2ffd640 1 -- 172.21.15.90:0/1525423548 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0bc00c1b0 con 0x7fb0bc0039d0 2024-03-11T18:35:59.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.288+0000 7fb0d2ffd640 1 -- 172.21.15.90:0/1525423548 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0bc00c370 con 0x7fb0bc005190 2024-03-11T18:35:59.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.292+0000 7fb0d8ca3640 1 --2- 172.21.15.90:0/1525423548 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb0bc005190 0x7fb0bc00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:59.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.292+0000 7fb0d37fe640 1 --2- 172.21.15.90:0/1525423548 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0bc0047d0 0x7fb0bc004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:59.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.296+0000 7fb0d3fff640 1 --2- 172.21.15.90:0/1525423548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0bc0039d0 0x7fb0bc003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:59.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.296+0000 7fb0d37fe640 1 -- 172.21.15.90:0/1525423548 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb0bc005190 msgr2=0x7fb0bc00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:35:59.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.296+0000 7fb0d37fe640 1 --2- 172.21.15.90:0/1525423548 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb0bc005190 0x7fb0bc00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:59.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.296+0000 7fb0d37fe640 1 -- 172.21.15.90:0/1525423548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0bc0039d0 msgr2=0x7fb0bc003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:35:59.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.296+0000 7fb0d37fe640 1 --2- 172.21.15.90:0/1525423548 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0bc0039d0 0x7fb0bc003df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:59.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.296+0000 7fb0d37fe640 1 -- 172.21.15.90:0/1525423548 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0bc003530 con 0x7fb0bc0047d0 2024-03-11T18:35:59.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.296+0000 7fb0d8ca3640 1 -- 172.21.15.90:0/1525423548 reap_dead start 2024-03-11T18:35:59.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.300+0000 7fb0d37fe640 1 --2- 172.21.15.90:0/1525423548 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0bc0047d0 0x7fb0bc004c50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8001780 tx=0x7fb0c801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd37c7a8dd4e94d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:59.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.304+0000 7fb0d2ffd640 1 -- 172.21.15.90:0/1525423548 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c80052d0 con 0x7fb0bc0047d0 2024-03-11T18:35:59.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.304+0000 7fb0d2ffd640 1 -- 172.21.15.90:0/1525423548 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0c80a7050 con 0x7fb0bc0047d0 2024-03-11T18:35:59.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.304+0000 7fb0d2ffd640 1 -- 172.21.15.90:0/1525423548 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c8003840 con 0x7fb0bc0047d0 2024-03-11T18:35:59.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.304+0000 7fb0da706640 1 -- 172.21.15.90:0/1525423548 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0bc0047d0 msgr2=0x7fb0bc004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:59.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.304+0000 7fb0da706640 1 --2- 172.21.15.90:0/1525423548 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0bc0047d0 0x7fb0bc004c50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8001780 tx=0x7fb0c801c400 comp rx=0 tx=0).stop 2024-03-11T18:35:59.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.304+0000 7fb0da706640 1 -- 172.21.15.90:0/1525423548 shutdown_connections 2024-03-11T18:35:59.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.304+0000 7fb0da706640 1 --2- 172.21.15.90:0/1525423548 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0bc0047d0 0x7fb0bc004c50 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:59.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.304+0000 7fb0da706640 1 -- 172.21.15.90:0/1525423548 >> 172.21.15.90:0/1525423548 conn(0x7fb0d406dad0 msgr2=0x7fb0d406dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:35:59.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.304+0000 7fb0da706640 1 -- 172.21.15.90:0/1525423548 shutdown_connections 2024-03-11T18:35:59.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.304+0000 7fb0da706640 1 -- 172.21.15.90:0/1525423548 wait complete. 2024-03-11T18:35:59.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.304+0000 7fb0da706640 1 Processor -- start 2024-03-11T18:35:59.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.304+0000 7fb0da706640 1 -- start start 2024-03-11T18:35:59.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.304+0000 7fb0da706640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0bc0047d0 0x7fb0d41595f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:59.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.304+0000 7fb0da706640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb0d415cd20 0x7fb0d414b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:59.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.304+0000 7fb0da706640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0d414bd10 0x7fb0d4152170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:59.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.304+0000 7fb0da706640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d41f2b70 con 0x7fb0d414bd10 2024-03-11T18:35:59.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.304+0000 7fb0da706640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d4157860 con 0x7fb0bc0047d0 2024-03-11T18:35:59.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.304+0000 7fb0da706640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d4157550 con 0x7fb0d415cd20 2024-03-11T18:35:59.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.308+0000 7fb0d8ca3640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0d414bd10 0x7fb0d4152170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:59.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.308+0000 7fb0d8ca3640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0d414bd10 0x7fb0d4152170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54122/0 (socket says 172.21.15.90:54122) 2024-03-11T18:35:59.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.308+0000 7fb0d8ca3640 1 -- 172.21.15.90:0/2584743044 learned_addr learned my addr 172.21.15.90:0/2584743044 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:35:59.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.308+0000 7fb0d3fff640 1 --2- 172.21.15.90:0/2584743044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0bc0047d0 0x7fb0d41595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:59.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.308+0000 7fb0d37fe640 1 --2- 172.21.15.90:0/2584743044 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb0d415cd20 0x7fb0d414b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:59.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.312+0000 7fb0d8ca3640 1 -- 172.21.15.90:0/2584743044 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb0d415cd20 msgr2=0x7fb0d414b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:59.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.312+0000 7fb0d8ca3640 1 --2- 172.21.15.90:0/2584743044 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb0d415cd20 0x7fb0d414b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:59.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.312+0000 7fb0d8ca3640 1 -- 172.21.15.90:0/2584743044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0bc0047d0 msgr2=0x7fb0d41595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:35:59.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.312+0000 7fb0d8ca3640 1 --2- 172.21.15.90:0/2584743044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0bc0047d0 0x7fb0d41595f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:35:59.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.312+0000 7fb0d8ca3640 1 -- 172.21.15.90:0/2584743044 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d4158270 con 0x7fb0d414bd10 2024-03-11T18:35:59.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.316+0000 7fb0d8ca3640 1 --2- 172.21.15.90:0/2584743044 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0d414bd10 0x7fb0d4152170 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb0d40c7890 tx=0x7fb0c400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:59.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.320+0000 7fb0d0ff9640 1 -- 172.21.15.90:0/2584743044 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c400e040 con 0x7fb0d414bd10 2024-03-11T18:35:59.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.320+0000 7fb0d0ff9640 1 -- 172.21.15.90:0/2584743044 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0c400f040 con 0x7fb0d414bd10 2024-03-11T18:35:59.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.320+0000 7fb0d0ff9640 1 -- 172.21.15.90:0/2584743044 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c4015430 con 0x7fb0d414bd10 2024-03-11T18:35:59.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.320+0000 7fb0da706640 1 -- 172.21.15.90:0/2584743044 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0d41f2dc0 con 0x7fb0d414bd10 2024-03-11T18:35:59.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.320+0000 7fb0da706640 1 -- 172.21.15.90:0/2584743044 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0d41f3380 con 0x7fb0d414bd10 2024-03-11T18:35:59.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.320+0000 7fb0da706640 1 -- 172.21.15.90:0/2584743044 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0d404efd0 con 0x7fb0d414bd10 2024-03-11T18:35:59.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.340+0000 7fb0d0ff9640 1 -- 172.21.15.90:0/2584743044 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fb0c4015590 con 0x7fb0d414bd10 2024-03-11T18:35:59.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.340+0000 7fb0d0ff9640 1 --2- 172.21.15.90:0/2584743044 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fb0c0066ec0 0x7fb0c0069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:35:59.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.340+0000 7fb0d3fff640 1 --2- 172.21.15.90:0/2584743044 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fb0c0066ec0 0x7fb0c0069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:35:59.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.340+0000 7fb0d3fff640 1 --2- 172.21.15.90:0/2584743044 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fb0c0066ec0 0x7fb0c0069380 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc001000 tx=0x7fb0cc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:35:59.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.600+0000 7fb0d0ff9640 1 -- 172.21.15.90:0/2584743044 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb0c4056b30 con 0x7fb0d414bd10 2024-03-11T18:35:59.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.600+0000 7fb0d0ff9640 1 -- 172.21.15.90:0/2584743044 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb0c4052dd0 con 0x7fb0d414bd10 2024-03-11T18:35:59.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:35:59.780+0000 7fb0da706640 1 -- 172.21.15.90:0/2584743044 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7fb0d40c78c0 con 0x7fb0d414bd10 2024-03-11T18:36:01.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.016+0000 7fb0d0ff9640 1 -- 172.21.15.90:0/2584743044 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v24) v1 ==== 157+0+0 (secure 0 0 0) 0x7fb0c4056490 con 0x7fb0d414bd10 2024-03-11T18:36:01.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.020+0000 7fb0da706640 1 -- 172.21.15.90:0/2584743044 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fb0c0066ec0 msgr2=0x7fb0c0069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:01.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.020+0000 7fb0da706640 1 --2- 172.21.15.90:0/2584743044 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fb0c0066ec0 0x7fb0c0069380 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc001000 tx=0x7fb0cc009040 comp rx=0 tx=0).stop 2024-03-11T18:36:01.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.020+0000 7fb0da706640 1 -- 172.21.15.90:0/2584743044 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0d414bd10 msgr2=0x7fb0d4152170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:01.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.020+0000 7fb0da706640 1 --2- 172.21.15.90:0/2584743044 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0d414bd10 0x7fb0d4152170 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb0d40c7890 tx=0x7fb0c400f580 comp rx=0 tx=0).stop 2024-03-11T18:36:01.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.020+0000 7fb0da706640 1 -- 172.21.15.90:0/2584743044 shutdown_connections 2024-03-11T18:36:01.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.020+0000 7fb0da706640 1 --2- 172.21.15.90:0/2584743044 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fb0c0066ec0 0x7fb0c0069380 secure :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc001000 tx=0x7fb0cc009040 comp rx=0 tx=0).stop 2024-03-11T18:36:01.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.020+0000 7fb0da706640 1 --2- 172.21.15.90:0/2584743044 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0d414bd10 0x7fb0d4152170 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:01.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.020+0000 7fb0da706640 1 --2- 172.21.15.90:0/2584743044 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb0d415cd20 0x7fb0d414b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:01.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.020+0000 7fb0da706640 1 --2- 172.21.15.90:0/2584743044 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0bc0047d0 0x7fb0d41595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:01.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.020+0000 7fb0da706640 1 -- 172.21.15.90:0/2584743044 >> 172.21.15.90:0/2584743044 conn(0x7fb0d406dad0 msgr2=0x7fb0d406f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:01.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.020+0000 7fb0da706640 1 -- 172.21.15.90:0/2584743044 shutdown_connections 2024-03-11T18:36:01.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.020+0000 7fb0da706640 1 -- 172.21.15.90:0/2584743044 wait complete. 2024-03-11T18:36:01.049 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-03-11T18:36:01.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.184+0000 7f165dc6b640 1 Processor -- start 2024-03-11T18:36:01.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.184+0000 7f165dc6b640 1 -- start start 2024-03-11T18:36:01.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.184+0000 7f165dc6b640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f1658150a90 0x7f1658157220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:01.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.184+0000 7f165dc6b640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f16580cc280 0x7f16580cc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:01.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.184+0000 7f165dc6b640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16580c5e60 con 0x7f16580c65b0 2024-03-11T18:36:01.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.184+0000 7f165dc6b640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1658156a60 con 0x7f1658150a90 2024-03-11T18:36:01.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.184+0000 7f165dc6b640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16580c55c0 con 0x7f16580cc280 2024-03-11T18:36:01.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.184+0000 7f16577fe640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f1658150a90 0x7f1658157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:01.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.188+0000 7f16577fe640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f1658150a90 0x7f1658157220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54132/0 (socket says 172.21.15.90:54132) 2024-03-11T18:36:01.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.188+0000 7f16577fe640 1 -- 172.21.15.90:0/4033035114 learned_addr learned my addr 172.21.15.90:0/4033035114 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:01.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.188+0000 7f1657fff640 1 --2- 172.21.15.90:0/4033035114 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f16580cc280 0x7f16580cc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:01.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.188+0000 7f16577fe640 1 -- 172.21.15.90:0/4033035114 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f16580cc280 msgr2=0x7f16580cc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:01.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.188+0000 7f16577fe640 1 --2- 172.21.15.90:0/4033035114 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f16580cc280 0x7f16580cc6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:01.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.188+0000 7f16577fe640 1 -- 172.21.15.90:0/4033035114 >> v1:172.21.15.92:6789/0 conn(0x7f16580c65b0 legacy=0x7f16580c37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:01.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.188+0000 7f16577fe640 1 -- 172.21.15.90:0/4033035114 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16580c58a0 con 0x7f1658150a90 2024-03-11T18:36:01.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.192+0000 7f16577fe640 1 --2- 172.21.15.90:0/4033035114 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1658150a90 0x7f1658157220 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1648009870 tx=0x7f164801e770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c820ba4c9ba8f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:01.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.196+0000 7f16567fc640 1 -- 172.21.15.90:0/4033035114 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16480ad660 con 0x7f1658150a90 2024-03-11T18:36:01.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.196+0000 7f16567fc640 1 -- 172.21.15.90:0/4033035114 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16480ad7c0 con 0x7f1658150a90 2024-03-11T18:36:01.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.196+0000 7f16567fc640 1 -- 172.21.15.90:0/4033035114 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16480adb50 con 0x7f1658150a90 2024-03-11T18:36:01.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.196+0000 7f165dc6b640 1 -- 172.21.15.90:0/4033035114 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1658150a90 msgr2=0x7f1658157220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:01.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.196+0000 7f165dc6b640 1 --2- 172.21.15.90:0/4033035114 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1658150a90 0x7f1658157220 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1648009870 tx=0x7f164801e770 comp rx=0 tx=0).stop 2024-03-11T18:36:01.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.196+0000 7f165dc6b640 1 -- 172.21.15.90:0/4033035114 shutdown_connections 2024-03-11T18:36:01.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.196+0000 7f165dc6b640 1 --2- 172.21.15.90:0/4033035114 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f16580cc280 0x7f16580cc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:01.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.196+0000 7f165dc6b640 1 --2- 172.21.15.90:0/4033035114 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1658150a90 0x7f1658157220 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:01.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.196+0000 7f165dc6b640 1 -- 172.21.15.90:0/4033035114 >> 172.21.15.90:0/4033035114 conn(0x7f1658071aa0 msgr2=0x7f1658071eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:01.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.196+0000 7f165dc6b640 1 -- 172.21.15.90:0/4033035114 shutdown_connections 2024-03-11T18:36:01.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.196+0000 7f165dc6b640 1 -- 172.21.15.90:0/4033035114 wait complete. 2024-03-11T18:36:01.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.200+0000 7f165dc6b640 1 Processor -- start 2024-03-11T18:36:01.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.200+0000 7f165dc6b640 1 -- start start 2024-03-11T18:36:01.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.200+0000 7f165dc6b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16580c65b0 0x7f16581f6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:01.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.200+0000 7f165dc6b640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f16580cc280 0x7f16581562a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:01.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.200+0000 7f165dc6b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1658150a90 0x7f165813eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:01.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.200+0000 7f165dc6b640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f165813fe40 con 0x7f1658150a90 2024-03-11T18:36:01.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.200+0000 7f165dc6b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16581f7080 con 0x7f16580c65b0 2024-03-11T18:36:01.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.200+0000 7f165dc6b640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16581f67e0 con 0x7f16580cc280 2024-03-11T18:36:01.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.200+0000 7f1657fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1658150a90 0x7f165813eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:01.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.200+0000 7f1657fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1658150a90 0x7f165813eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54134/0 (socket says 172.21.15.90:54134) 2024-03-11T18:36:01.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.200+0000 7f1657fff640 1 -- 172.21.15.90:0/3159027046 learned_addr learned my addr 172.21.15.90:0/3159027046 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:01.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.200+0000 7f16577fe640 1 --2- 172.21.15.90:0/3159027046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16580c65b0 0x7f16581f6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:01.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.204+0000 7f1656ffd640 1 --2- 172.21.15.90:0/3159027046 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f16580cc280 0x7f16581562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:01.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.204+0000 7f1657fff640 1 -- 172.21.15.90:0/3159027046 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f16580cc280 msgr2=0x7f16581562a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:01.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.204+0000 7f1657fff640 1 --2- 172.21.15.90:0/3159027046 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f16580cc280 0x7f16581562a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:01.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.204+0000 7f1657fff640 1 -- 172.21.15.90:0/3159027046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16580c65b0 msgr2=0x7f16581f6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:01.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.204+0000 7f1657fff640 1 --2- 172.21.15.90:0/3159027046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16580c65b0 0x7f16581f6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:01.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.204+0000 7f1657fff640 1 -- 172.21.15.90:0/3159027046 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16480094d0 con 0x7f1658150a90 2024-03-11T18:36:01.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.208+0000 7f1657fff640 1 --2- 172.21.15.90:0/3159027046 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1658150a90 0x7f165813eff0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1650007e40 tx=0x7f165000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:01.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.212+0000 7f163ffff640 1 -- 172.21.15.90:0/3159027046 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1650014620 con 0x7f1658150a90 2024-03-11T18:36:01.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.212+0000 7f163ffff640 1 -- 172.21.15.90:0/3159027046 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f165000ee90 con 0x7f1658150a90 2024-03-11T18:36:01.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.212+0000 7f163ffff640 1 -- 172.21.15.90:0/3159027046 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16500149e0 con 0x7f1658150a90 2024-03-11T18:36:01.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.212+0000 7f165dc6b640 1 -- 172.21.15.90:0/3159027046 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16581f6ac0 con 0x7f1658150a90 2024-03-11T18:36:01.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.212+0000 7f165dc6b640 1 -- 172.21.15.90:0/3159027046 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f165813e7f0 con 0x7f1658150a90 2024-03-11T18:36:01.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.212+0000 7f165dc6b640 1 -- 172.21.15.90:0/3159027046 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f165804eef0 con 0x7f1658150a90 2024-03-11T18:36:01.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.232+0000 7f163ffff640 1 -- 172.21.15.90:0/3159027046 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f1650014b40 con 0x7f1658150a90 2024-03-11T18:36:01.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.232+0000 7f163ffff640 1 --2- 172.21.15.90:0/3159027046 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1644066e90 0x7f1644069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:01.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.232+0000 7f16577fe640 1 --2- 172.21.15.90:0/3159027046 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1644066e90 0x7f1644069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:01.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.232+0000 7f16577fe640 1 --2- 172.21.15.90:0/3159027046 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1644066e90 0x7f1644069350 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f16480b1000 tx=0x7f16480a9000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:01.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.236+0000 7f163ffff640 1 -- 172.21.15.90:0/3159027046 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6007+0+0 (secure 0 0 0) 0x7f1650056dc0 con 0x7f1658150a90 2024-03-11T18:36:01.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.516+0000 7f163ffff640 1 -- 172.21.15.90:0/3159027046 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1650053060 con 0x7f1658150a90 2024-03-11T18:36:01.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:01.684+0000 7f165dc6b640 1 -- 172.21.15.90:0/3159027046 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f1658070010 con 0x7f1658150a90 2024-03-11T18:36:03.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.300+0000 7f163ffff640 1 -- 172.21.15.90:0/3159027046 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v26) v1 ==== 228+0+0 (secure 0 0 0) 0x7f1650056720 con 0x7f1658150a90 2024-03-11T18:36:03.307 INFO:teuthology.orchestra.run.smithi090.stderr:pool 'cephfs_data_ec' created 2024-03-11T18:36:03.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.304+0000 7f165dc6b640 1 -- 172.21.15.90:0/3159027046 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1644066e90 msgr2=0x7f1644069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:03.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.304+0000 7f165dc6b640 1 --2- 172.21.15.90:0/3159027046 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1644066e90 0x7f1644069350 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f16480b1000 tx=0x7f16480a9000 comp rx=0 tx=0).stop 2024-03-11T18:36:03.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.304+0000 7f165dc6b640 1 -- 172.21.15.90:0/3159027046 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1658150a90 msgr2=0x7f165813eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:03.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.304+0000 7f165dc6b640 1 --2- 172.21.15.90:0/3159027046 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1658150a90 0x7f165813eff0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1650007e40 tx=0x7f165000e4b0 comp rx=0 tx=0).stop 2024-03-11T18:36:03.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.304+0000 7f165dc6b640 1 -- 172.21.15.90:0/3159027046 shutdown_connections 2024-03-11T18:36:03.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.304+0000 7f165dc6b640 1 --2- 172.21.15.90:0/3159027046 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1644066e90 0x7f1644069350 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:03.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.304+0000 7f165dc6b640 1 --2- 172.21.15.90:0/3159027046 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1658150a90 0x7f165813eff0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:03.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.304+0000 7f165dc6b640 1 --2- 172.21.15.90:0/3159027046 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f16580cc280 0x7f16581562a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:03.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.304+0000 7f165dc6b640 1 --2- 172.21.15.90:0/3159027046 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16580c65b0 0x7f16581f6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:03.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.304+0000 7f165dc6b640 1 -- 172.21.15.90:0/3159027046 >> 172.21.15.90:0/3159027046 conn(0x7f1658071aa0 msgr2=0x7f16580c3360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:03.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.304+0000 7f165dc6b640 1 -- 172.21.15.90:0/3159027046 shutdown_connections 2024-03-11T18:36:03.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.304+0000 7f165dc6b640 1 -- 172.21.15.90:0/3159027046 wait complete. 2024-03-11T18:36:03.330 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-03-11T18:36:03.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.456+0000 7fe787c76640 1 Processor -- start 2024-03-11T18:36:03.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.456+0000 7fe787c76640 1 -- start start 2024-03-11T18:36:03.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.456+0000 7fe787c76640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fe78006ca50 0x7fe7800c5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:03.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.456+0000 7fe787c76640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe7800c6040 0x7fe78015b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:03.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.456+0000 7fe787c76640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7800c7550 con 0x7fe780151ba0 2024-03-11T18:36:03.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.456+0000 7fe787c76640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7800c8c60 con 0x7fe78006ca50 2024-03-11T18:36:03.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.456+0000 7fe787c76640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7800c86a0 con 0x7fe7800c6040 2024-03-11T18:36:03.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.456+0000 7fe785211640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fe78006ca50 0x7fe7800c5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:03.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.456+0000 7fe785211640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fe78006ca50 0x7fe7800c5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54140/0 (socket says 172.21.15.90:54140) 2024-03-11T18:36:03.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.456+0000 7fe785211640 1 -- 172.21.15.90:0/3042910641 learned_addr learned my addr 172.21.15.90:0/3042910641 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:03.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.460+0000 7fe785211640 1 -- 172.21.15.90:0/3042910641 >> v2:172.21.15.92:3300/0 conn(0x7fe7800c6040 msgr2=0x7fe78015b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:03.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.460+0000 7fe785211640 1 --2- 172.21.15.90:0/3042910641 >> v2:172.21.15.92:3300/0 conn(0x7fe7800c6040 0x7fe78015b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:03.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.460+0000 7fe785211640 1 -- 172.21.15.90:0/3042910641 >> v1:172.21.15.92:6789/0 conn(0x7fe780151ba0 legacy=0x7fe78006f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:03.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.460+0000 7fe785211640 1 -- 172.21.15.90:0/3042910641 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7800c8980 con 0x7fe78006ca50 2024-03-11T18:36:03.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.464+0000 7fe785211640 1 --2- 172.21.15.90:0/3042910641 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe78006ca50 0x7fe7800c5b00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe78006ce90 tx=0x7fe77001ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=335f5355a1338f35 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:03.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.472+0000 7fe784a10640 1 -- 172.21.15.90:0/3042910641 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7700a89d0 con 0x7fe78006ca50 2024-03-11T18:36:03.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.472+0000 7fe784a10640 1 -- 172.21.15.90:0/3042910641 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7700a8b30 con 0x7fe78006ca50 2024-03-11T18:36:03.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.472+0000 7fe784a10640 1 -- 172.21.15.90:0/3042910641 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7700b0440 con 0x7fe78006ca50 2024-03-11T18:36:03.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.472+0000 7fe787c76640 1 -- 172.21.15.90:0/3042910641 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe78006ca50 msgr2=0x7fe7800c5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:03.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.472+0000 7fe787c76640 1 --2- 172.21.15.90:0/3042910641 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe78006ca50 0x7fe7800c5b00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe78006ce90 tx=0x7fe77001ecd0 comp rx=0 tx=0).stop 2024-03-11T18:36:03.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.476+0000 7fe787c76640 1 -- 172.21.15.90:0/3042910641 shutdown_connections 2024-03-11T18:36:03.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.476+0000 7fe787c76640 1 --2- 172.21.15.90:0/3042910641 >> v2:172.21.15.92:3300/0 conn(0x7fe7800c6040 0x7fe78015b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:03.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.476+0000 7fe787c76640 1 --2- 172.21.15.90:0/3042910641 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe78006ca50 0x7fe7800c5b00 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:03.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.476+0000 7fe787c76640 1 -- 172.21.15.90:0/3042910641 >> 172.21.15.90:0/3042910641 conn(0x7fe78006dad0 msgr2=0x7fe78006dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:03.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.476+0000 7fe787c76640 1 -- 172.21.15.90:0/3042910641 shutdown_connections 2024-03-11T18:36:03.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.476+0000 7fe787c76640 1 -- 172.21.15.90:0/3042910641 wait complete. 2024-03-11T18:36:03.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.476+0000 7fe787c76640 1 Processor -- start 2024-03-11T18:36:03.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.476+0000 7fe787c76640 1 -- start start 2024-03-11T18:36:03.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.476+0000 7fe787c76640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe78006ca50 0x7fe780153400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:03.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.476+0000 7fe787c76640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7800c6040 0x7fe78014bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:03.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.476+0000 7fe787c76640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe780151ba0 0x7fe7801562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:03.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.476+0000 7fe787c76640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7801ed7a0 con 0x7fe78006ca50 2024-03-11T18:36:03.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.476+0000 7fe787c76640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe780154dc0 con 0x7fe7800c6040 2024-03-11T18:36:03.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.476+0000 7fe787c76640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7801550a0 con 0x7fe780151ba0 2024-03-11T18:36:03.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.476+0000 7fe785a12640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe78006ca50 0x7fe780153400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:03.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.476+0000 7fe785a12640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe78006ca50 0x7fe780153400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54150/0 (socket says 172.21.15.90:54150) 2024-03-11T18:36:03.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.476+0000 7fe785a12640 1 -- 172.21.15.90:0/1701428618 learned_addr learned my addr 172.21.15.90:0/1701428618 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:03.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.480+0000 7fe786213640 1 --2- 172.21.15.90:0/1701428618 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe780151ba0 0x7fe7801562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:03.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.480+0000 7fe785a12640 1 -- 172.21.15.90:0/1701428618 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe780151ba0 msgr2=0x7fe7801562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:03.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.480+0000 7fe785a12640 1 --2- 172.21.15.90:0/1701428618 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe780151ba0 0x7fe7801562c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:03.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.480+0000 7fe785a12640 1 -- 172.21.15.90:0/1701428618 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7800c6040 msgr2=0x7fe78014bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:03.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.480+0000 7fe785a12640 1 --2- 172.21.15.90:0/1701428618 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7800c6040 0x7fe78014bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:03.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.480+0000 7fe785a12640 1 -- 172.21.15.90:0/1701428618 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7700094d0 con 0x7fe78006ca50 2024-03-11T18:36:03.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.484+0000 7fe785a12640 1 --2- 172.21.15.90:0/1701428618 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe78006ca50 0x7fe780153400 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe77c00b3e0 tx=0x7fe77c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:03.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.492+0000 7fe7767fc640 1 -- 172.21.15.90:0/1701428618 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77c011830 con 0x7fe78006ca50 2024-03-11T18:36:03.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.492+0000 7fe7767fc640 1 -- 172.21.15.90:0/1701428618 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe77c0119b0 con 0x7fe78006ca50 2024-03-11T18:36:03.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.492+0000 7fe787c76640 1 -- 172.21.15.90:0/1701428618 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7801eda60 con 0x7fe78006ca50 2024-03-11T18:36:03.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.492+0000 7fe7767fc640 1 -- 172.21.15.90:0/1701428618 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77c011d60 con 0x7fe78006ca50 2024-03-11T18:36:03.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.492+0000 7fe787c76640 1 -- 172.21.15.90:0/1701428618 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7801ee020 con 0x7fe78006ca50 2024-03-11T18:36:03.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.492+0000 7fe787c76640 1 -- 172.21.15.90:0/1701428618 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe780074340 con 0x7fe78006ca50 2024-03-11T18:36:03.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.512+0000 7fe7767fc640 1 -- 172.21.15.90:0/1701428618 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fe77c01a400 con 0x7fe78006ca50 2024-03-11T18:36:03.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.512+0000 7fe7767fc640 1 --2- 172.21.15.90:0/1701428618 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe764066ec0 0x7fe764069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:03.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.512+0000 7fe785211640 1 --2- 172.21.15.90:0/1701428618 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe764066ec0 0x7fe764069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:03.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.512+0000 7fe785211640 1 --2- 172.21.15.90:0/1701428618 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe764066ec0 0x7fe764069380 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe7700b2000 tx=0x7fe770004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:03.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.516+0000 7fe7767fc640 1 -- 172.21.15.90:0/1701428618 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fe77c056370 con 0x7fe78006ca50 2024-03-11T18:36:03.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.788+0000 7fe7767fc640 1 -- 172.21.15.90:0/1701428618 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe77c051f10 con 0x7fe78006ca50 2024-03-11T18:36:03.918 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:36:03.921 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:36:03.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:03.976+0000 7fe787c76640 1 -- 172.21.15.90:0/1701428618 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fe78006c550 con 0x7fe78006ca50 2024-03-11T18:36:04.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.612+0000 7fe7767fc640 1 -- 172.21.15.90:0/1701428618 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v27) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe77c015070 con 0x7fe78006ca50 2024-03-11T18:36:04.619 INFO:teuthology.orchestra.run.smithi090.stderr:set pool 4 allow_ec_overwrites to true 2024-03-11T18:36:04.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.620+0000 7fe787c76640 1 -- 172.21.15.90:0/1701428618 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe764066ec0 msgr2=0x7fe764069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:04.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.620+0000 7fe787c76640 1 --2- 172.21.15.90:0/1701428618 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe764066ec0 0x7fe764069380 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe7700b2000 tx=0x7fe770004740 comp rx=0 tx=0).stop 2024-03-11T18:36:04.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.620+0000 7fe787c76640 1 -- 172.21.15.90:0/1701428618 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe78006ca50 msgr2=0x7fe780153400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:04.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.620+0000 7fe787c76640 1 --2- 172.21.15.90:0/1701428618 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe78006ca50 0x7fe780153400 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe77c00b3e0 tx=0x7fe77c00b8a0 comp rx=0 tx=0).stop 2024-03-11T18:36:04.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.620+0000 7fe787c76640 1 -- 172.21.15.90:0/1701428618 shutdown_connections 2024-03-11T18:36:04.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.620+0000 7fe787c76640 1 --2- 172.21.15.90:0/1701428618 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe764066ec0 0x7fe764069380 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:04.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.620+0000 7fe787c76640 1 --2- 172.21.15.90:0/1701428618 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe780151ba0 0x7fe7801562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:04.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.620+0000 7fe787c76640 1 --2- 172.21.15.90:0/1701428618 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe7800c6040 0x7fe78014bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:04.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.620+0000 7fe787c76640 1 --2- 172.21.15.90:0/1701428618 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe78006ca50 0x7fe780153400 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:04.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.620+0000 7fe787c76640 1 -- 172.21.15.90:0/1701428618 >> 172.21.15.90:0/1701428618 conn(0x7fe78006dad0 msgr2=0x7fe78014b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:04.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.620+0000 7fe787c76640 1 -- 172.21.15.90:0/1701428618 shutdown_connections 2024-03-11T18:36:04.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.620+0000 7fe787c76640 1 -- 172.21.15.90:0/1701428618 wait complete. 2024-03-11T18:36:04.648 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-03-11T18:36:04.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.820+0000 7fb0d30cc640 1 Processor -- start 2024-03-11T18:36:04.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.820+0000 7fb0d30cc640 1 -- start start 2024-03-11T18:36:04.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.824+0000 7fb0d30cc640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fb0cc150a90 0x7fb0cc157220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:04.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.824+0000 7fb0d30cc640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb0cc0c65b0 0x7fb0cc0c37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:04.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.824+0000 7fb0d30cc640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb0cc0c3d30 0x7fb0cc0cc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:04.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.824+0000 7fb0d30cc640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0cc0c5e60 con 0x7fb0cc150a90 2024-03-11T18:36:04.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.824+0000 7fb0d30cc640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0cc156a60 con 0x7fb0cc0c65b0 2024-03-11T18:36:04.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.824+0000 7fb0d30cc640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0cc0c55c0 con 0x7fb0cc0c3d30 2024-03-11T18:36:04.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.824+0000 7fb0d1669640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb0cc0c3d30 0x7fb0cc0cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:04.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.824+0000 7fb0d1669640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb0cc0c3d30 0x7fb0cc0cc280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.90:51320/0 (socket says 172.21.15.90:51320) 2024-03-11T18:36:04.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.824+0000 7fb0d1669640 1 -- 172.21.15.90:0/547376645 learned_addr learned my addr 172.21.15.90:0/547376645 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:04.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.832+0000 7fb0d1669640 1 -- 172.21.15.90:0/547376645 >> v2:172.21.15.92:3300/0 conn(0x7fb0cc0c65b0 msgr2=0x7fb0cc0c37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:04.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.832+0000 7fb0d1669640 1 --2- 172.21.15.90:0/547376645 >> v2:172.21.15.92:3300/0 conn(0x7fb0cc0c65b0 0x7fb0cc0c37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:04.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.832+0000 7fb0d1669640 1 -- 172.21.15.90:0/547376645 >> v2:172.21.15.90:3300/0 conn(0x7fb0cc150a90 msgr2=0x7fb0cc157220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:04.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.832+0000 7fb0d1669640 1 --2- 172.21.15.90:0/547376645 >> v2:172.21.15.90:3300/0 conn(0x7fb0cc150a90 0x7fb0cc157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:04.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.832+0000 7fb0d1669640 1 -- 172.21.15.90:0/547376645 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0cc0c58a0 con 0x7fb0cc0c3d30 2024-03-11T18:36:04.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.836+0000 7fb0d1669640 1 --2- 172.21.15.90:0/547376645 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb0cc0c3d30 0x7fb0cc0cc280 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc0c9e30 tx=0x7fb0c401c9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c44f7ee2ecebeed3 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:04.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.844+0000 7fb0cb7fe640 1 -- 172.21.15.90:0/547376645 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c40a5560 con 0x7fb0cc0c3d30 2024-03-11T18:36:04.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.844+0000 7fb0cb7fe640 1 -- 172.21.15.90:0/547376645 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0c40a56c0 con 0x7fb0cc0c3d30 2024-03-11T18:36:04.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.844+0000 7fb0cb7fe640 1 -- 172.21.15.90:0/547376645 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c40a5a50 con 0x7fb0cc0c3d30 2024-03-11T18:36:04.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.848+0000 7fb0d30cc640 1 -- 172.21.15.90:0/547376645 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb0cc0c3d30 msgr2=0x7fb0cc0cc280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:04.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.848+0000 7fb0d30cc640 1 --2- 172.21.15.90:0/547376645 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb0cc0c3d30 0x7fb0cc0cc280 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc0c9e30 tx=0x7fb0c401c9f0 comp rx=0 tx=0).stop 2024-03-11T18:36:04.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.848+0000 7fb0d30cc640 1 -- 172.21.15.90:0/547376645 shutdown_connections 2024-03-11T18:36:04.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.848+0000 7fb0d30cc640 1 --2- 172.21.15.90:0/547376645 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb0cc0c3d30 0x7fb0cc0cc280 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:04.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.848+0000 7fb0d30cc640 1 --2- 172.21.15.90:0/547376645 >> v2:172.21.15.92:3300/0 conn(0x7fb0cc0c65b0 0x7fb0cc0c37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:04.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.848+0000 7fb0d30cc640 1 --2- 172.21.15.90:0/547376645 >> v2:172.21.15.90:3300/0 conn(0x7fb0cc150a90 0x7fb0cc157220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:04.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.848+0000 7fb0d30cc640 1 -- 172.21.15.90:0/547376645 >> 172.21.15.90:0/547376645 conn(0x7fb0cc071aa0 msgr2=0x7fb0cc071eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:04.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.848+0000 7fb0d30cc640 1 -- 172.21.15.90:0/547376645 shutdown_connections 2024-03-11T18:36:04.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.848+0000 7fb0d30cc640 1 -- 172.21.15.90:0/547376645 wait complete. 2024-03-11T18:36:04.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.848+0000 7fb0d30cc640 1 Processor -- start 2024-03-11T18:36:04.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.848+0000 7fb0d30cc640 1 -- start start 2024-03-11T18:36:04.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.848+0000 7fb0d30cc640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0cc0c3d30 0x7fb0cc157ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:04.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.848+0000 7fb0d30cc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0cc0c65b0 0x7fb0cc146dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:04.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.848+0000 7fb0d30cc640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb0cc150a90 0x7fb0cc141cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:04.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.848+0000 7fb0d30cc640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0cc159ab0 con 0x7fb0cc0c3d30 2024-03-11T18:36:04.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.848+0000 7fb0d30cc640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0cc06bfe0 con 0x7fb0cc0c65b0 2024-03-11T18:36:04.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.848+0000 7fb0d30cc640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0cc158c50 con 0x7fb0cc150a90 2024-03-11T18:36:04.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.852+0000 7fb0cbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0cc0c65b0 0x7fb0cc146dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:04.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.852+0000 7fb0cbfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0cc0c65b0 0x7fb0cc146dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.90:50324/0 (socket says 172.21.15.90:50324) 2024-03-11T18:36:04.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.852+0000 7fb0cbfff640 1 -- 172.21.15.90:0/969923891 learned_addr learned my addr 172.21.15.90:0/969923891 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:04.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.852+0000 7fb0d1669640 1 --2- 172.21.15.90:0/969923891 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb0cc150a90 0x7fb0cc141cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:04.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.864+0000 7fb0d1669640 1 -- 172.21.15.90:0/969923891 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0cc0c65b0 msgr2=0x7fb0cc146dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:04.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.864+0000 7fb0d1669640 1 --2- 172.21.15.90:0/969923891 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0cc0c65b0 0x7fb0cc146dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:04.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.864+0000 7fb0d1669640 1 -- 172.21.15.90:0/969923891 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0cc0c3d30 msgr2=0x7fb0cc157ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:04.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.864+0000 7fb0d1669640 1 --2- 172.21.15.90:0/969923891 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0cc0c3d30 0x7fb0cc157ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:04.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.864+0000 7fb0d1669640 1 -- 172.21.15.90:0/969923891 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c4009b30 con 0x7fb0cc150a90 2024-03-11T18:36:04.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.868+0000 7fb0d1669640 1 --2- 172.21.15.90:0/969923891 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb0cc150a90 0x7fb0cc141cb0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb0c40ab000 tx=0x7fb0c4019790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:04.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.880+0000 7fb0c97fa640 1 -- 172.21.15.90:0/969923891 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c40b8670 con 0x7fb0cc150a90 2024-03-11T18:36:04.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.880+0000 7fb0c97fa640 1 -- 172.21.15.90:0/969923891 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0c40b87d0 con 0x7fb0cc150a90 2024-03-11T18:36:04.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.880+0000 7fb0c97fa640 1 -- 172.21.15.90:0/969923891 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c40b8b80 con 0x7fb0cc150a90 2024-03-11T18:36:04.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.880+0000 7fb0d30cc640 1 -- 172.21.15.90:0/969923891 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0cc158f30 con 0x7fb0cc150a90 2024-03-11T18:36:04.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.880+0000 7fb0d30cc640 1 -- 172.21.15.90:0/969923891 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0cc06b990 con 0x7fb0cc150a90 2024-03-11T18:36:04.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:04.880+0000 7fb0d30cc640 1 -- 172.21.15.90:0/969923891 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0cc04eef0 con 0x7fb0cc150a90 2024-03-11T18:36:05.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:05.220+0000 7fb0c97fa640 1 -- 172.21.15.90:0/969923891 <== mon.2 v2:172.21.15.92:3301/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fb0c40b8ce0 con 0x7fb0cc150a90 2024-03-11T18:36:05.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:05.220+0000 7fb0c97fa640 1 --2- 172.21.15.90:0/969923891 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fb0b8066ec0 0x7fb0b8069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:05.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:05.220+0000 7fb0d0e68640 1 --2- 172.21.15.90:0/969923891 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fb0b8066ec0 0x7fb0b8069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:05.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:05.224+0000 7fb0d0e68640 1 --2- 172.21.15.90:0/969923891 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fb0b8066ec0 0x7fb0b8069380 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc009730 tx=0x7fb0bc006ca0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:05.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:05.224+0000 7fb0c97fa640 1 -- 172.21.15.90:0/969923891 <== mon.2 v2:172.21.15.92:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fb0c40f1aa0 con 0x7fb0cc150a90 2024-03-11T18:36:05.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:05.236+0000 7fb0c97fa640 1 -- 172.21.15.90:0/969923891 <== mon.2 v2:172.21.15.92:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb0c40ed640 con 0x7fb0cc150a90 2024-03-11T18:36:05.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:05.448+0000 7fb0d30cc640 1 -- 172.21.15.90:0/969923891 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7fb0cc0c4150 con 0x7fb0cc150a90 2024-03-11T18:36:06.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.660+0000 7fb0c97fa640 1 -- 172.21.15.90:0/969923891 <== mon.2 v2:172.21.15.92:3301/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) v1 ==== 137+0+0 (secure 0 0 0) 0x7fb0c4009510 con 0x7fb0cc150a90 2024-03-11T18:36:06.666 INFO:teuthology.orchestra.run.smithi090.stderr:added data pool 4 to fsmap 2024-03-11T18:36:06.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.660+0000 7fb0d30cc640 1 -- 172.21.15.90:0/969923891 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fb0b8066ec0 msgr2=0x7fb0b8069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:06.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.660+0000 7fb0d30cc640 1 --2- 172.21.15.90:0/969923891 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fb0b8066ec0 0x7fb0b8069380 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc009730 tx=0x7fb0bc006ca0 comp rx=0 tx=0).stop 2024-03-11T18:36:06.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.660+0000 7fb0d30cc640 1 -- 172.21.15.90:0/969923891 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb0cc150a90 msgr2=0x7fb0cc141cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:06.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.660+0000 7fb0d30cc640 1 --2- 172.21.15.90:0/969923891 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb0cc150a90 0x7fb0cc141cb0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb0c40ab000 tx=0x7fb0c4019790 comp rx=0 tx=0).stop 2024-03-11T18:36:06.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.664+0000 7fb0d30cc640 1 -- 172.21.15.90:0/969923891 shutdown_connections 2024-03-11T18:36:06.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.664+0000 7fb0d30cc640 1 --2- 172.21.15.90:0/969923891 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fb0b8066ec0 0x7fb0b8069380 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:06.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.664+0000 7fb0d30cc640 1 --2- 172.21.15.90:0/969923891 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb0cc150a90 0x7fb0cc141cb0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:06.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.664+0000 7fb0d30cc640 1 --2- 172.21.15.90:0/969923891 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb0cc0c65b0 0x7fb0cc146dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:06.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.664+0000 7fb0d30cc640 1 --2- 172.21.15.90:0/969923891 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0cc0c3d30 0x7fb0cc157ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:06.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.664+0000 7fb0d30cc640 1 -- 172.21.15.90:0/969923891 >> 172.21.15.90:0/969923891 conn(0x7fb0cc071aa0 msgr2=0x7fb0cc14b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:06.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.664+0000 7fb0d30cc640 1 -- 172.21.15.90:0/969923891 shutdown_connections 2024-03-11T18:36:06.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.664+0000 7fb0d30cc640 1 -- 172.21.15.90:0/969923891 wait complete. 2024-03-11T18:36:06.691 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T18:36:06.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.808+0000 7f6adc066640 1 Processor -- start 2024-03-11T18:36:06.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.808+0000 7f6adc066640 1 -- start start 2024-03-11T18:36:06.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.812+0000 7f6adc066640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6ad41509f0 0x7f6ad4157390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:06.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.812+0000 7f6adc066640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f6ad4150dd0 0x7f6ad40c28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:06.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.812+0000 7f6adc066640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6ad4070b00 0x7f6ad40c4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:06.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.812+0000 7f6adc066640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ad40c5de0 con 0x7f6ad4150dd0 2024-03-11T18:36:06.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.812+0000 7f6adc066640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ad4156910 con 0x7f6ad41509f0 2024-03-11T18:36:06.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.812+0000 7f6adc066640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad41578d0 con 0x7f6ad4070b00 2024-03-11T18:36:06.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.812+0000 7f6ad9601640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f6ad4150dd0 0x7f6ad40c28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:06.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.812+0000 7f6ad9601640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f6ad4150dd0 0x7f6ad40c28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55410/0 (socket says 172.21.15.90:55410) 2024-03-11T18:36:06.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.812+0000 7f6ad9601640 1 -- 172.21.15.90:0/1387075159 learned_addr learned my addr 172.21.15.90:0/1387075159 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:06.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.812+0000 7f6ada603640 1 --2- 172.21.15.90:0/1387075159 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6ad4070b00 0x7f6ad40c4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:06.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.812+0000 7f6ad9e02640 1 --2- 172.21.15.90:0/1387075159 >> v2:172.21.15.92:3300/0 conn(0x7f6ad41509f0 0x7f6ad4157390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:06.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.816+0000 7f6ad9601640 1 -- 172.21.15.90:0/1387075159 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6ad4070b00 msgr2=0x7f6ad40c4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:06.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.816+0000 7f6ad9601640 1 --2- 172.21.15.90:0/1387075159 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6ad4070b00 0x7f6ad40c4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:06.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.816+0000 7f6ad9601640 1 -- 172.21.15.90:0/1387075159 >> v2:172.21.15.92:3300/0 conn(0x7f6ad41509f0 msgr2=0x7f6ad4157390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:06.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.816+0000 7f6ad9601640 1 --2- 172.21.15.90:0/1387075159 >> v2:172.21.15.92:3300/0 conn(0x7f6ad41509f0 0x7f6ad4157390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:06.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.816+0000 7f6ad9601640 1 -- 172.21.15.90:0/1387075159 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad40c6030 con 0x7f6ad4150dd0 2024-03-11T18:36:06.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.816+0000 7f6ad9601640 1 --2- 172.21.15.90:0/1387075159 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ad4150dd0 0x7f6ad40c28e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6ad40c8b00 tx=0x7f6ac40213e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d9252888c896d98 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:06.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.824+0000 7f6ad8e00640 1 -- 172.21.15.90:0/1387075159 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac40aaea0 con 0x7f6ad4150dd0 2024-03-11T18:36:06.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.824+0000 7f6ad8e00640 1 -- 172.21.15.90:0/1387075159 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ac40ad050 con 0x7f6ad4150dd0 2024-03-11T18:36:06.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.824+0000 7f6ad8e00640 1 -- 172.21.15.90:0/1387075159 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac40b1820 con 0x7f6ad4150dd0 2024-03-11T18:36:06.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.824+0000 7f6adc066640 1 -- 172.21.15.90:0/1387075159 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ad4150dd0 msgr2=0x7f6ad40c28e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:06.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.824+0000 7f6adc066640 1 --2- 172.21.15.90:0/1387075159 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ad4150dd0 0x7f6ad40c28e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6ad40c8b00 tx=0x7f6ac40213e0 comp rx=0 tx=0).stop 2024-03-11T18:36:06.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.824+0000 7f6adc066640 1 -- 172.21.15.90:0/1387075159 shutdown_connections 2024-03-11T18:36:06.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.824+0000 7f6adc066640 1 --2- 172.21.15.90:0/1387075159 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6ad4070b00 0x7f6ad40c4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:06.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.824+0000 7f6adc066640 1 --2- 172.21.15.90:0/1387075159 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ad4150dd0 0x7f6ad40c28e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:06.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.824+0000 7f6adc066640 1 --2- 172.21.15.90:0/1387075159 >> v2:172.21.15.92:3300/0 conn(0x7f6ad41509f0 0x7f6ad4157390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:06.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.824+0000 7f6adc066640 1 -- 172.21.15.90:0/1387075159 >> 172.21.15.90:0/1387075159 conn(0x7f6ad406ac00 msgr2=0x7f6ad406b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:06.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.824+0000 7f6adc066640 1 -- 172.21.15.90:0/1387075159 shutdown_connections 2024-03-11T18:36:06.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.824+0000 7f6adc066640 1 -- 172.21.15.90:0/1387075159 wait complete. 2024-03-11T18:36:06.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.828+0000 7f6adc066640 1 Processor -- start 2024-03-11T18:36:06.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.828+0000 7f6adc066640 1 -- start start 2024-03-11T18:36:06.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.828+0000 7f6adc066640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ad4070b00 0x7f6ad4154bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:06.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.828+0000 7f6adc066640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6ad41509f0 0x7f6ad4156110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:06.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.828+0000 7f6adc066640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ad4150dd0 0x7f6ad4146530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:06.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.828+0000 7f6adc066640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad4155110 con 0x7f6ad4070b00 2024-03-11T18:36:06.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.828+0000 7f6adc066640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad413ee20 con 0x7f6ad4150dd0 2024-03-11T18:36:06.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.828+0000 7f6adc066640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad4155c80 con 0x7f6ad41509f0 2024-03-11T18:36:06.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.828+0000 7f6ad9e02640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ad4070b00 0x7f6ad4154bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:06.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.828+0000 7f6ad9e02640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ad4070b00 0x7f6ad4154bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55420/0 (socket says 172.21.15.90:55420) 2024-03-11T18:36:06.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.828+0000 7f6ad9e02640 1 -- 172.21.15.90:0/2648995437 learned_addr learned my addr 172.21.15.90:0/2648995437 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:06.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.828+0000 7f6ada603640 1 --2- 172.21.15.90:0/2648995437 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ad4150dd0 0x7f6ad4146530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:06.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.828+0000 7f6ad9601640 1 --2- 172.21.15.90:0/2648995437 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6ad41509f0 0x7f6ad4156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:06.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.832+0000 7f6ad9e02640 1 -- 172.21.15.90:0/2648995437 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6ad41509f0 msgr2=0x7f6ad4156110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:06.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.832+0000 7f6ad9e02640 1 --2- 172.21.15.90:0/2648995437 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6ad41509f0 0x7f6ad4156110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:06.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.832+0000 7f6ad9e02640 1 -- 172.21.15.90:0/2648995437 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ad4150dd0 msgr2=0x7f6ad4146530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:06.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.832+0000 7f6ad9e02640 1 --2- 172.21.15.90:0/2648995437 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ad4150dd0 0x7f6ad4146530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:06.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.832+0000 7f6ad9e02640 1 -- 172.21.15.90:0/2648995437 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac400b770 con 0x7f6ad4070b00 2024-03-11T18:36:06.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.836+0000 7f6ad9e02640 1 --2- 172.21.15.90:0/2648995437 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ad4070b00 0x7f6ad4154bd0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6ad000d800 tx=0x7f6ad000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:06.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.840+0000 7f6aca7fc640 1 -- 172.21.15.90:0/2648995437 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad000b6c0 con 0x7f6ad4070b00 2024-03-11T18:36:06.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.840+0000 7f6adc066640 1 -- 172.21.15.90:0/2648995437 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ad414a7e0 con 0x7f6ad4070b00 2024-03-11T18:36:06.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.840+0000 7f6adc066640 1 -- 172.21.15.90:0/2648995437 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ad4155630 con 0x7f6ad4070b00 2024-03-11T18:36:06.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.840+0000 7f6adc066640 1 -- 172.21.15.90:0/2648995437 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ad404eef0 con 0x7f6ad4070b00 2024-03-11T18:36:06.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.840+0000 7f6aca7fc640 1 -- 172.21.15.90:0/2648995437 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ad000b820 con 0x7f6ad4070b00 2024-03-11T18:36:06.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:06.840+0000 7f6aca7fc640 1 -- 172.21.15.90:0/2648995437 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad000bb90 con 0x7f6ad4070b00 2024-03-11T18:36:07.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.132+0000 7f6aca7fc640 1 -- 172.21.15.90:0/2648995437 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f6ad000bcf0 con 0x7f6ad4070b00 2024-03-11T18:36:07.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.132+0000 7f6aca7fc640 1 --2- 172.21.15.90:0/2648995437 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f6ab8066e40 0x7f6ab8069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:07.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.132+0000 7f6ad9601640 1 --2- 172.21.15.90:0/2648995437 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f6ab8066e40 0x7f6ab8069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:07.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.132+0000 7f6ad9601640 1 --2- 172.21.15.90:0/2648995437 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f6ab8066e40 0x7f6ab8069300 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6ac40b3000 tx=0x7f6ac4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:07.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.168+0000 7f6aca7fc640 1 -- 172.21.15.90:0/2648995437 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f6ad0015e20 con 0x7f6ad4070b00 2024-03-11T18:36:07.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.168+0000 7f6aca7fc640 1 -- 172.21.15.90:0/2648995437 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6ad0052830 con 0x7f6ad4070b00 2024-03-11T18:36:07.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.376+0000 7f6adc066640 1 -- 172.21.15.90:0/2648995437 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6ad40721d0 con 0x7f6ad4070b00 2024-03-11T18:36:07.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.412+0000 7f6aca7fc640 1 -- 172.21.15.90:0/2648995437 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+6508 (secure 0 0 0) 0x7f6ad0055ef0 con 0x7f6ad4070b00 2024-03-11T18:36:07.420 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:36:07.420 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/641527992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":641527992},{"type":"v1","addr":"172.21.15.90:6839","nonce":641527992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4229,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/460417177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":460417177},{"type":"v1","addr":"172.21.15.90:6835","nonce":460417177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/3568438077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":3568438077},{"type":"v1","addr":"172.21.15.92:6835","nonce":3568438077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/2306487548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":2306487548},{"type":"v1","addr":"172.21.15.92:6834","nonce":2306487548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/3209534444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":3209534444},{"type":"v1","addr":"172.21.15.92:6837","nonce":3209534444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-11T18:35:57.363570+0000","modified":"2024-03-11T18:36:05.539381+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4255":{"gid":4255,"name":"e","rank":0,"incarnation":5,"state":"up:active","state_seq":6,"addr":"172.21.15.90:6837/3485943375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3485943375},{"type":"v1","addr":"172.21.15.90:6837","nonce":3485943375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-11T18:36:07.421 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 7 2024-03-11T18:36:07.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.416+0000 7f6adc066640 1 -- 172.21.15.90:0/2648995437 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f6ab8066e40 msgr2=0x7f6ab8069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:07.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.416+0000 7f6adc066640 1 --2- 172.21.15.90:0/2648995437 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f6ab8066e40 0x7f6ab8069300 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6ac40b3000 tx=0x7f6ac4008040 comp rx=0 tx=0).stop 2024-03-11T18:36:07.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.416+0000 7f6adc066640 1 -- 172.21.15.90:0/2648995437 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ad4070b00 msgr2=0x7f6ad4154bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:07.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.416+0000 7f6adc066640 1 --2- 172.21.15.90:0/2648995437 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ad4070b00 0x7f6ad4154bd0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6ad000d800 tx=0x7f6ad000dcc0 comp rx=0 tx=0).stop 2024-03-11T18:36:07.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.416+0000 7f6adc066640 1 -- 172.21.15.90:0/2648995437 shutdown_connections 2024-03-11T18:36:07.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.416+0000 7f6adc066640 1 --2- 172.21.15.90:0/2648995437 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f6ab8066e40 0x7f6ab8069300 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:07.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.416+0000 7f6adc066640 1 --2- 172.21.15.90:0/2648995437 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6ad4150dd0 0x7f6ad4146530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:07.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.416+0000 7f6adc066640 1 --2- 172.21.15.90:0/2648995437 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6ad41509f0 0x7f6ad4156110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:07.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.416+0000 7f6adc066640 1 --2- 172.21.15.90:0/2648995437 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ad4070b00 0x7f6ad4154bd0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:07.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.416+0000 7f6adc066640 1 -- 172.21.15.90:0/2648995437 >> 172.21.15.90:0/2648995437 conn(0x7f6ad406ac00 msgr2=0x7f6ad4145410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:07.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.416+0000 7f6adc066640 1 -- 172.21.15.90:0/2648995437 shutdown_connections 2024-03-11T18:36:07.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.416+0000 7f6adc066640 1 -- 172.21.15.90:0/2648995437 wait complete. 2024-03-11T18:36:07.441 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T18:36:07.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.564+0000 7fed08f07640 1 Processor -- start 2024-03-11T18:36:07.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.564+0000 7fed08f07640 1 -- start start 2024-03-11T18:36:07.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.564+0000 7fed08f07640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fed041598c0 0x7fed0406e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:07.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.564+0000 7fed08f07640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fed0406f220 0x7fed04159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:07.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.564+0000 7fed08f07640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed04070b00 con 0x7fed0415a1e0 2024-03-11T18:36:07.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.564+0000 7fed08f07640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed0406eef0 con 0x7fed0406f220 2024-03-11T18:36:07.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.564+0000 7fed08f07640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed04071df0 con 0x7fed041598c0 2024-03-11T18:36:07.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.596+0000 7fed01d9b640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fed0406f220 0x7fed04159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:07.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.596+0000 7fed01d9b640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fed0406f220 0x7fed04159ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55430/0 (socket says 172.21.15.90:55430) 2024-03-11T18:36:07.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.596+0000 7fed01d9b640 1 -- 172.21.15.90:0/810667752 learned_addr learned my addr 172.21.15.90:0/810667752 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:07.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.600+0000 7fed01d9b640 1 -- 172.21.15.90:0/810667752 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fed041598c0 msgr2=0x7fed0406e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:07.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.600+0000 7fed01d9b640 1 --2- 172.21.15.90:0/810667752 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fed041598c0 0x7fed0406e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:07.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.600+0000 7fed01d9b640 1 -- 172.21.15.90:0/810667752 >> v1:172.21.15.92:6789/0 conn(0x7fed0415a1e0 legacy=0x7fed040c3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:07.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.600+0000 7fed01d9b640 1 -- 172.21.15.90:0/810667752 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed04070d50 con 0x7fed0406f220 2024-03-11T18:36:07.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.608+0000 7fed01d9b640 1 --2- 172.21.15.90:0/810667752 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed0406f220 0x7fed04159ca0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fecf4009870 tx=0x7fecf4020f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d248d4168d983a31 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:07.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.612+0000 7fed0159a640 1 -- 172.21.15.90:0/810667752 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf40aabe0 con 0x7fed0406f220 2024-03-11T18:36:07.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.612+0000 7fed0159a640 1 -- 172.21.15.90:0/810667752 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecf40aad40 con 0x7fed0406f220 2024-03-11T18:36:07.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.612+0000 7fed0159a640 1 -- 172.21.15.90:0/810667752 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf40b2660 con 0x7fed0406f220 2024-03-11T18:36:07.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.612+0000 7fed08f07640 1 -- 172.21.15.90:0/810667752 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed0406f220 msgr2=0x7fed04159ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:07.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.612+0000 7fed08f07640 1 --2- 172.21.15.90:0/810667752 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed0406f220 0x7fed04159ca0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fecf4009870 tx=0x7fecf4020f20 comp rx=0 tx=0).stop 2024-03-11T18:36:07.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.612+0000 7fed08f07640 1 -- 172.21.15.90:0/810667752 shutdown_connections 2024-03-11T18:36:07.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.612+0000 7fed08f07640 1 --2- 172.21.15.90:0/810667752 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed0406f220 0x7fed04159ca0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:07.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.612+0000 7fed08f07640 1 --2- 172.21.15.90:0/810667752 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fed041598c0 0x7fed0406e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:07.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.612+0000 7fed08f07640 1 -- 172.21.15.90:0/810667752 >> 172.21.15.90:0/810667752 conn(0x7fed0406ac00 msgr2=0x7fed0406b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:07.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.616+0000 7fed08f07640 1 -- 172.21.15.90:0/810667752 shutdown_connections 2024-03-11T18:36:07.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.616+0000 7fed08f07640 1 -- 172.21.15.90:0/810667752 wait complete. 2024-03-11T18:36:07.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.616+0000 7fed08f07640 1 Processor -- start 2024-03-11T18:36:07.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.616+0000 7fed08f07640 1 -- start start 2024-03-11T18:36:07.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.616+0000 7fed08f07640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed0406f220 0x7fed04154fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:07.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.616+0000 7fed08f07640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed041598c0 0x7fed04158dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:07.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.616+0000 7fed08f07640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fed041534e0 0x7fed041f3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:07.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.616+0000 7fed08f07640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed041f3250 con 0x7fed041598c0 2024-03-11T18:36:07.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.616+0000 7fed08f07640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed041e19c0 con 0x7fed0406f220 2024-03-11T18:36:07.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.616+0000 7fed08f07640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed04155510 con 0x7fed041534e0 2024-03-11T18:36:07.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.616+0000 7fed01d9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed041598c0 0x7fed04158dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:07.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.616+0000 7fed01d9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed041598c0 0x7fed04158dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55434/0 (socket says 172.21.15.90:55434) 2024-03-11T18:36:07.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.616+0000 7fed01d9b640 1 -- 172.21.15.90:0/3551607321 learned_addr learned my addr 172.21.15.90:0/3551607321 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:07.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.620+0000 7fed01d9b640 1 -- 172.21.15.90:0/3551607321 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fed041534e0 msgr2=0x7fed041f3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:07.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.620+0000 7fed01d9b640 1 --2- 172.21.15.90:0/3551607321 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fed041534e0 0x7fed041f3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:07.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.620+0000 7fed01d9b640 1 -- 172.21.15.90:0/3551607321 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed0406f220 msgr2=0x7fed04154fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:07.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.620+0000 7fed01d9b640 1 --2- 172.21.15.90:0/3551607321 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed0406f220 0x7fed04154fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:07.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.620+0000 7fed01d9b640 1 -- 172.21.15.90:0/3551607321 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecf40094d0 con 0x7fed041598c0 2024-03-11T18:36:07.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.624+0000 7fed01d9b640 1 --2- 172.21.15.90:0/3551607321 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed041598c0 0x7fed04158dc0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fecf40b5040 tx=0x7fecf401dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:07.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.624+0000 7fecfaffd640 1 -- 172.21.15.90:0/3551607321 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf40bb610 con 0x7fed041598c0 2024-03-11T18:36:07.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.624+0000 7fed08f07640 1 -- 172.21.15.90:0/3551607321 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed041f2e90 con 0x7fed041598c0 2024-03-11T18:36:07.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.624+0000 7fed08f07640 1 -- 172.21.15.90:0/3551607321 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed041e1f10 con 0x7fed041598c0 2024-03-11T18:36:07.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.628+0000 7fed08f07640 1 -- 172.21.15.90:0/3551607321 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed041f3390 con 0x7fed041598c0 2024-03-11T18:36:07.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.628+0000 7fecfaffd640 1 -- 172.21.15.90:0/3551607321 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecf40bb770 con 0x7fed041598c0 2024-03-11T18:36:07.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.628+0000 7fecfaffd640 1 -- 172.21.15.90:0/3551607321 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf40bbb20 con 0x7fed041598c0 2024-03-11T18:36:07.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.908+0000 7fecfaffd640 1 -- 172.21.15.90:0/3551607321 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fecf40c43b0 con 0x7fed041598c0 2024-03-11T18:36:07.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.912+0000 7fecfaffd640 1 --2- 172.21.15.90:0/3551607321 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fece4066e40 0x7fece4069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:07.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.912+0000 7fed0259c640 1 --2- 172.21.15.90:0/3551607321 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fece4066e40 0x7fece4069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:07.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.912+0000 7fed0259c640 1 --2- 172.21.15.90:0/3551607321 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fece4066e40 0x7fece4069300 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fecec009730 tx=0x7fecec006ca0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:07.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.968+0000 7fecfaffd640 1 -- 172.21.15.90:0/3551607321 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fecf40c6e30 con 0x7fed041598c0 2024-03-11T18:36:07.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:07.968+0000 7fecfaffd640 1 -- 172.21.15.90:0/3551607321 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fecf40ef4a0 con 0x7fed041598c0 2024-03-11T18:36:08.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.136+0000 7fed08f07640 1 -- 172.21.15.90:0/3551607321 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fed041f3390 con 0x7fed041598c0 2024-03-11T18:36:08.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.188+0000 7fecfaffd640 1 -- 172.21.15.90:0/3551607321 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18430 (secure 0 0 0) 0x7fed041f3390 con 0x7fed041598c0 2024-03-11T18:36:08.194 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:36:08.194 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":28,"fsid":"76d56b10-6c29-476a-804b-8482ea90f9e2","created":"2024-03-11T18:34:54.521741+0000","modified":"2024-03-11T18:36:05.526044+0000","last_up_change":"2024-03-11T18:35:16.207500+0000","last_in_change":"2024-03-11T18:35:05.213634+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T18:35:16.193842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T18:35:54.001469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T18:35:55.917321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T18:36:02.210943+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"34decff9-7280-4d84-9665-d6af8ea6de84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6819","nonce":1755667921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6821","nonce":1755667921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6825","nonce":1755667921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6823","nonce":1755667921}]},"public_addr":"172.21.15.90:6819/1755667921","cluster_addr":"172.21.15.90:6821/1755667921","heartbeat_back_addr":"172.21.15.90:6825/1755667921","heartbeat_front_addr":"172.21.15.90:6823/1755667921","state":["exists","up"]},{"osd":1,"uuid":"8ecdce9e-5486-415a-a609-dd16e945440e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6807","nonce":438682081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6809","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6811","nonce":438682081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6817","nonce":438682081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6815","nonce":438682081}]},"public_addr":"172.21.15.90:6807/438682081","cluster_addr":"172.21.15.90:6811/438682081","heartbeat_back_addr":"172.21.15.90:6817/438682081","heartbeat_front_addr":"172.21.15.90:6815/438682081","state":["exists","up"]},{"osd":2,"uuid":"737486d4-0e90-4c82-9e79-bf37c812e5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6827","nonce":2837704738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6829","nonce":2837704738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6833","nonce":2837704738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6831","nonce":2837704738}]},"public_addr":"172.21.15.90:6827/2837704738","cluster_addr":"172.21.15.90:6829/2837704738","heartbeat_back_addr":"172.21.15.90:6833/2837704738","heartbeat_front_addr":"172.21.15.90:6831/2837704738","state":["exists","up"]},{"osd":3,"uuid":"698bd661-bbef-4699-a619-f7bfe23ce12c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6803","nonce":240987971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6805","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6806","nonce":240987971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6813","nonce":240987971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6810","nonce":240987971}]},"public_addr":"172.21.15.90:6803/240987971","cluster_addr":"172.21.15.90:6806/240987971","heartbeat_back_addr":"172.21.15.90:6813/240987971","heartbeat_front_addr":"172.21.15.90:6810/240987971","state":["exists","up"]},{"osd":4,"uuid":"aba9e31f-73d9-43fe-a10b-2dc5832968b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6801","nonce":502685275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6803","nonce":502685275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6807","nonce":502685275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6805","nonce":502685275}]},"public_addr":"172.21.15.92:6801/502685275","cluster_addr":"172.21.15.92:6803/502685275","heartbeat_back_addr":"172.21.15.92:6807/502685275","heartbeat_front_addr":"172.21.15.92:6805/502685275","state":["exists","up"]},{"osd":5,"uuid":"bdd56766-4b28-4007-8de3-b5b08cf335d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6809","nonce":200142070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6811","nonce":200142070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6815","nonce":200142070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6813","nonce":200142070}]},"public_addr":"172.21.15.92:6809/200142070","cluster_addr":"172.21.15.92:6811/200142070","heartbeat_back_addr":"172.21.15.92:6815/200142070","heartbeat_front_addr":"172.21.15.92:6813/200142070","state":["exists","up"]},{"osd":6,"uuid":"6c580a59-610f-455e-a96b-033299dcc118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6817","nonce":1919010204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6819","nonce":1919010204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6823","nonce":1919010204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6821","nonce":1919010204}]},"public_addr":"172.21.15.92:6817/1919010204","cluster_addr":"172.21.15.92:6819/1919010204","heartbeat_back_addr":"172.21.15.92:6823/1919010204","heartbeat_front_addr":"172.21.15.92:6821/1919010204","state":["exists","up"]},{"osd":7,"uuid":"958184cc-aebf-4c5b-91a0-2a3e38a6b42b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6825","nonce":3475925719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6827","nonce":3475925719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6831","nonce":3475925719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6829","nonce":3475925719}]},"public_addr":"172.21.15.92:6825/3475925719","cluster_addr":"172.21.15.92:6827/3475925719","heartbeat_back_addr":"172.21.15.92:6831/3475925719","heartbeat_front_addr":"172.21.15.92:6829/3475925719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.816294+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.048084+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.137975+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.968246+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.016829+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.931676+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.041673+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.079404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T18:36:08.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.188+0000 7fed08f07640 1 -- 172.21.15.90:0/3551607321 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fece4066e40 msgr2=0x7fece4069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:08.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.188+0000 7fed08f07640 1 --2- 172.21.15.90:0/3551607321 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fece4066e40 0x7fece4069300 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fecec009730 tx=0x7fecec006ca0 comp rx=0 tx=0).stop 2024-03-11T18:36:08.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.188+0000 7fed08f07640 1 -- 172.21.15.90:0/3551607321 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed041598c0 msgr2=0x7fed04158dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:08.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.188+0000 7fed08f07640 1 --2- 172.21.15.90:0/3551607321 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed041598c0 0x7fed04158dc0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fecf40b5040 tx=0x7fecf401dcc0 comp rx=0 tx=0).stop 2024-03-11T18:36:08.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.188+0000 7fed08f07640 1 -- 172.21.15.90:0/3551607321 shutdown_connections 2024-03-11T18:36:08.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.188+0000 7fed08f07640 1 --2- 172.21.15.90:0/3551607321 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fece4066e40 0x7fece4069300 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:08.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.188+0000 7fed08f07640 1 --2- 172.21.15.90:0/3551607321 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fed041534e0 0x7fed041f3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:08.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.188+0000 7fed08f07640 1 --2- 172.21.15.90:0/3551607321 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed041598c0 0x7fed04158dc0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:08.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.188+0000 7fed08f07640 1 --2- 172.21.15.90:0/3551607321 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fed0406f220 0x7fed04154fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:08.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.188+0000 7fed08f07640 1 -- 172.21.15.90:0/3551607321 >> 172.21.15.90:0/3551607321 conn(0x7fed0406ac00 msgr2=0x7fed040c5ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:08.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.192+0000 7fed08f07640 1 -- 172.21.15.90:0/3551607321 shutdown_connections 2024-03-11T18:36:08.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.192+0000 7fed08f07640 1 -- 172.21.15.90:0/3551607321 wait complete. 2024-03-11T18:36:08.213 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T18:36:08.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.320+0000 7f4cd831a640 1 Processor -- start 2024-03-11T18:36:08.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.320+0000 7f4cd831a640 1 -- start start 2024-03-11T18:36:08.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.324+0000 7f4cd831a640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4cd0070b00 0x7f4cd006e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:08.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.324+0000 7f4cd831a640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4cd00c4af0 0x7f4cd00c4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:08.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.324+0000 7f4cd831a640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cd006f430 con 0x7f4cd0151b30 2024-03-11T18:36:08.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.324+0000 7f4cd831a640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cd00c65b0 con 0x7f4cd00c4af0 2024-03-11T18:36:08.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.324+0000 7f4cd831a640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd006e820 con 0x7f4cd0070b00 2024-03-11T18:36:08.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.324+0000 7f4cd68b7640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4cd00c4af0 0x7f4cd00c4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:08.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.324+0000 7f4cd58b5640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4cd0070b00 0x7f4cd006e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:08.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.324+0000 7f4cd68b7640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4cd00c4af0 0x7f4cd00c4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.90:50372/0 (socket says 172.21.15.90:50372) 2024-03-11T18:36:08.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.324+0000 7f4cd68b7640 1 -- 172.21.15.90:0/515862977 learned_addr learned my addr 172.21.15.90:0/515862977 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:08.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.332+0000 7f4cd58b5640 1 -- 172.21.15.90:0/515862977 >> v2:172.21.15.92:3300/0 conn(0x7f4cd00c4af0 msgr2=0x7f4cd00c4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:08.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.332+0000 7f4cd58b5640 1 --2- 172.21.15.90:0/515862977 >> v2:172.21.15.92:3300/0 conn(0x7f4cd00c4af0 0x7f4cd00c4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:08.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.332+0000 7f4cd58b5640 1 -- 172.21.15.90:0/515862977 >> v1:172.21.15.92:6789/0 conn(0x7f4cd0151b30 legacy=0x7f4cd00c5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:08.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.332+0000 7f4cd58b5640 1 -- 172.21.15.90:0/515862977 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cd006eb00 con 0x7f4cd0070b00 2024-03-11T18:36:08.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.336+0000 7f4cd58b5640 1 --2- 172.21.15.90:0/515862977 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4cd0070b00 0x7f4cd006e210 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4cc00097c0 tx=0x7f4cc001c430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d86c3b3bf1a7071 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:08.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.344+0000 7f4cd50b4640 1 -- 172.21.15.90:0/515862977 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc00ab5f0 con 0x7f4cd0070b00 2024-03-11T18:36:08.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.344+0000 7f4cd50b4640 1 -- 172.21.15.90:0/515862977 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cc00ab750 con 0x7f4cd0070b00 2024-03-11T18:36:08.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.344+0000 7f4cd50b4640 1 -- 172.21.15.90:0/515862977 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc00abae0 con 0x7f4cd0070b00 2024-03-11T18:36:08.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.348+0000 7f4cd831a640 1 -- 172.21.15.90:0/515862977 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4cd0070b00 msgr2=0x7f4cd006e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:08.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.348+0000 7f4cd831a640 1 --2- 172.21.15.90:0/515862977 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4cd0070b00 0x7f4cd006e210 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4cc00097c0 tx=0x7f4cc001c430 comp rx=0 tx=0).stop 2024-03-11T18:36:08.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.348+0000 7f4cd831a640 1 -- 172.21.15.90:0/515862977 shutdown_connections 2024-03-11T18:36:08.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.348+0000 7f4cd831a640 1 --2- 172.21.15.90:0/515862977 >> v2:172.21.15.92:3300/0 conn(0x7f4cd00c4af0 0x7f4cd00c4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:08.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.348+0000 7f4cd831a640 1 --2- 172.21.15.90:0/515862977 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4cd0070b00 0x7f4cd006e210 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:08.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.348+0000 7f4cd831a640 1 -- 172.21.15.90:0/515862977 >> 172.21.15.90:0/515862977 conn(0x7f4cd006ac00 msgr2=0x7f4cd006b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:08.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.348+0000 7f4cd831a640 1 -- 172.21.15.90:0/515862977 shutdown_connections 2024-03-11T18:36:08.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.348+0000 7f4cd831a640 1 -- 172.21.15.90:0/515862977 wait complete. 2024-03-11T18:36:08.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.348+0000 7f4cd831a640 1 Processor -- start 2024-03-11T18:36:08.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.348+0000 7f4cd831a640 1 -- start start 2024-03-11T18:36:08.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.348+0000 7f4cd831a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4cd0070b00 0x7f4cd01f4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:08.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.348+0000 7f4cd831a640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4cd00c4af0 0x7f4cd0145e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:08.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.348+0000 7f4cd831a640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4cd0151b30 0x7f4cd0144f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:08.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.348+0000 7f4cd831a640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd01f0500 con 0x7f4cd0151b30 2024-03-11T18:36:08.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.348+0000 7f4cd831a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd01f49c0 con 0x7f4cd0070b00 2024-03-11T18:36:08.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.348+0000 7f4cd831a640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd0146380 con 0x7f4cd00c4af0 2024-03-11T18:36:08.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.348+0000 7f4cd68b7640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4cd0151b30 0x7f4cd0144f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:08.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.352+0000 7f4cd68b7640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4cd0151b30 0x7f4cd0144f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55444/0 (socket says 172.21.15.90:55444) 2024-03-11T18:36:08.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.352+0000 7f4cd68b7640 1 -- 172.21.15.90:0/2408842779 learned_addr learned my addr 172.21.15.90:0/2408842779 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:08.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.352+0000 7f4cd60b6640 1 --2- 172.21.15.90:0/2408842779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4cd0070b00 0x7f4cd01f4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:08.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.352+0000 7f4cd58b5640 1 --2- 172.21.15.90:0/2408842779 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4cd00c4af0 0x7f4cd0145e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:08.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.356+0000 7f4cd68b7640 1 -- 172.21.15.90:0/2408842779 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4cd00c4af0 msgr2=0x7f4cd0145e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:08.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.356+0000 7f4cd68b7640 1 --2- 172.21.15.90:0/2408842779 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4cd00c4af0 0x7f4cd0145e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:08.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.356+0000 7f4cd68b7640 1 -- 172.21.15.90:0/2408842779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4cd0070b00 msgr2=0x7f4cd01f4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:08.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.356+0000 7f4cd68b7640 1 --2- 172.21.15.90:0/2408842779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4cd0070b00 0x7f4cd01f4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:08.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.356+0000 7f4cd68b7640 1 -- 172.21.15.90:0/2408842779 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc0009420 con 0x7f4cd0151b30 2024-03-11T18:36:08.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.356+0000 7f4cd68b7640 1 --2- 172.21.15.90:0/2408842779 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4cd0151b30 0x7f4cd0144f80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8009d90 tx=0x7f4cc800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:08.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.360+0000 7f4cc6ffd640 1 -- 172.21.15.90:0/2408842779 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc800ccc0 con 0x7f4cd0151b30 2024-03-11T18:36:08.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.360+0000 7f4cc6ffd640 1 -- 172.21.15.90:0/2408842779 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cc800f040 con 0x7f4cd0151b30 2024-03-11T18:36:08.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.360+0000 7f4cc6ffd640 1 -- 172.21.15.90:0/2408842779 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc80039a0 con 0x7f4cd0151b30 2024-03-11T18:36:08.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.360+0000 7f4cd831a640 1 -- 172.21.15.90:0/2408842779 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cd014f960 con 0x7f4cd0151b30 2024-03-11T18:36:08.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.360+0000 7f4cd831a640 1 -- 172.21.15.90:0/2408842779 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cd014fc00 con 0x7f4cd0151b30 2024-03-11T18:36:08.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.364+0000 7f4cd831a640 1 -- 172.21.15.90:0/2408842779 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cd004eef0 con 0x7f4cd0151b30 2024-03-11T18:36:08.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.384+0000 7f4cc6ffd640 1 -- 172.21.15.90:0/2408842779 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f4cc8003b00 con 0x7f4cd0151b30 2024-03-11T18:36:08.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.384+0000 7f4cc6ffd640 1 --2- 172.21.15.90:0/2408842779 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f4cb4066e90 0x7f4cb4069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:08.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.384+0000 7f4cd60b6640 1 --2- 172.21.15.90:0/2408842779 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f4cb4066e90 0x7f4cb4069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:08.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.384+0000 7f4cd60b6640 1 --2- 172.21.15.90:0/2408842779 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f4cb4066e90 0x7f4cb4069350 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc007fd0 tx=0x7f4ccc007e40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:08.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.388+0000 7f4cc6ffd640 1 -- 172.21.15.90:0/2408842779 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4cc8056f80 con 0x7f4cd0151b30 2024-03-11T18:36:08.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.668+0000 7f4cc6ffd640 1 -- 172.21.15.90:0/2408842779 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4cc8052b20 con 0x7f4cd0151b30 2024-03-11T18:36:08.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.840+0000 7f4cd831a640 1 -- 172.21.15.90:0/2408842779 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4cd014cb40 con 0x7f4cd0151b30 2024-03-11T18:36:08.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.892+0000 7f4cc6ffd640 1 -- 172.21.15.90:0/2408842779 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18430 (secure 0 0 0) 0x7f4cc80561e0 con 0x7f4cd0151b30 2024-03-11T18:36:08.899 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:36:08.899 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":28,"fsid":"76d56b10-6c29-476a-804b-8482ea90f9e2","created":"2024-03-11T18:34:54.521741+0000","modified":"2024-03-11T18:36:05.526044+0000","last_up_change":"2024-03-11T18:35:16.207500+0000","last_in_change":"2024-03-11T18:35:05.213634+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T18:35:16.193842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T18:35:54.001469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T18:35:55.917321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T18:36:02.210943+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"34decff9-7280-4d84-9665-d6af8ea6de84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6819","nonce":1755667921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6821","nonce":1755667921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6825","nonce":1755667921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6823","nonce":1755667921}]},"public_addr":"172.21.15.90:6819/1755667921","cluster_addr":"172.21.15.90:6821/1755667921","heartbeat_back_addr":"172.21.15.90:6825/1755667921","heartbeat_front_addr":"172.21.15.90:6823/1755667921","state":["exists","up"]},{"osd":1,"uuid":"8ecdce9e-5486-415a-a609-dd16e945440e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6807","nonce":438682081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6809","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6811","nonce":438682081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6817","nonce":438682081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6815","nonce":438682081}]},"public_addr":"172.21.15.90:6807/438682081","cluster_addr":"172.21.15.90:6811/438682081","heartbeat_back_addr":"172.21.15.90:6817/438682081","heartbeat_front_addr":"172.21.15.90:6815/438682081","state":["exists","up"]},{"osd":2,"uuid":"737486d4-0e90-4c82-9e79-bf37c812e5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6827","nonce":2837704738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6829","nonce":2837704738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6833","nonce":2837704738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6831","nonce":2837704738}]},"public_addr":"172.21.15.90:6827/2837704738","cluster_addr":"172.21.15.90:6829/2837704738","heartbeat_back_addr":"172.21.15.90:6833/2837704738","heartbeat_front_addr":"172.21.15.90:6831/2837704738","state":["exists","up"]},{"osd":3,"uuid":"698bd661-bbef-4699-a619-f7bfe23ce12c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6803","nonce":240987971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6805","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6806","nonce":240987971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6813","nonce":240987971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6810","nonce":240987971}]},"public_addr":"172.21.15.90:6803/240987971","cluster_addr":"172.21.15.90:6806/240987971","heartbeat_back_addr":"172.21.15.90:6813/240987971","heartbeat_front_addr":"172.21.15.90:6810/240987971","state":["exists","up"]},{"osd":4,"uuid":"aba9e31f-73d9-43fe-a10b-2dc5832968b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6801","nonce":502685275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6803","nonce":502685275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6807","nonce":502685275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6805","nonce":502685275}]},"public_addr":"172.21.15.92:6801/502685275","cluster_addr":"172.21.15.92:6803/502685275","heartbeat_back_addr":"172.21.15.92:6807/502685275","heartbeat_front_addr":"172.21.15.92:6805/502685275","state":["exists","up"]},{"osd":5,"uuid":"bdd56766-4b28-4007-8de3-b5b08cf335d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6809","nonce":200142070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6811","nonce":200142070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6815","nonce":200142070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6813","nonce":200142070}]},"public_addr":"172.21.15.92:6809/200142070","cluster_addr":"172.21.15.92:6811/200142070","heartbeat_back_addr":"172.21.15.92:6815/200142070","heartbeat_front_addr":"172.21.15.92:6813/200142070","state":["exists","up"]},{"osd":6,"uuid":"6c580a59-610f-455e-a96b-033299dcc118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6817","nonce":1919010204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6819","nonce":1919010204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6823","nonce":1919010204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6821","nonce":1919010204}]},"public_addr":"172.21.15.92:6817/1919010204","cluster_addr":"172.21.15.92:6819/1919010204","heartbeat_back_addr":"172.21.15.92:6823/1919010204","heartbeat_front_addr":"172.21.15.92:6821/1919010204","state":["exists","up"]},{"osd":7,"uuid":"958184cc-aebf-4c5b-91a0-2a3e38a6b42b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6825","nonce":3475925719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6827","nonce":3475925719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6831","nonce":3475925719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6829","nonce":3475925719}]},"public_addr":"172.21.15.92:6825/3475925719","cluster_addr":"172.21.15.92:6827/3475925719","heartbeat_back_addr":"172.21.15.92:6831/3475925719","heartbeat_front_addr":"172.21.15.92:6829/3475925719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.816294+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.048084+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.137975+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.968246+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.016829+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.931676+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.041673+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.079404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T18:36:08.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.896+0000 7f4cd831a640 1 -- 172.21.15.90:0/2408842779 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f4cb4066e90 msgr2=0x7f4cb4069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:08.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.896+0000 7f4cd831a640 1 --2- 172.21.15.90:0/2408842779 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f4cb4066e90 0x7f4cb4069350 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc007fd0 tx=0x7f4ccc007e40 comp rx=0 tx=0).stop 2024-03-11T18:36:08.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.896+0000 7f4cd831a640 1 -- 172.21.15.90:0/2408842779 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4cd0151b30 msgr2=0x7f4cd0144f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:08.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.896+0000 7f4cd831a640 1 --2- 172.21.15.90:0/2408842779 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4cd0151b30 0x7f4cd0144f80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8009d90 tx=0x7f4cc800c490 comp rx=0 tx=0).stop 2024-03-11T18:36:08.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.896+0000 7f4cd831a640 1 -- 172.21.15.90:0/2408842779 shutdown_connections 2024-03-11T18:36:08.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.896+0000 7f4cd831a640 1 --2- 172.21.15.90:0/2408842779 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f4cb4066e90 0x7f4cb4069350 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:08.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.896+0000 7f4cd831a640 1 --2- 172.21.15.90:0/2408842779 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4cd0151b30 0x7f4cd0144f80 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:08.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.896+0000 7f4cd831a640 1 --2- 172.21.15.90:0/2408842779 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4cd00c4af0 0x7f4cd0145e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:08.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.896+0000 7f4cd831a640 1 --2- 172.21.15.90:0/2408842779 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4cd0070b00 0x7f4cd01f4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:08.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.896+0000 7f4cd831a640 1 -- 172.21.15.90:0/2408842779 >> 172.21.15.90:0/2408842779 conn(0x7f4cd006ac00 msgr2=0x7f4cd006afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:08.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.896+0000 7f4cd831a640 1 -- 172.21.15.90:0/2408842779 shutdown_connections 2024-03-11T18:36:08.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:08.896+0000 7f4cd831a640 1 -- 172.21.15.90:0/2408842779 wait complete. 2024-03-11T18:36:08.937 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T18:36:09.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.052+0000 7f504a49b640 1 Processor -- start 2024-03-11T18:36:09.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.052+0000 7f504a49b640 1 -- start start 2024-03-11T18:36:09.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.052+0000 7f504a49b640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f5044150dd0 0x7f50440c28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:09.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.052+0000 7f504a49b640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f5044070b00 0x7f50440c4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:09.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.052+0000 7f504a49b640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50440c5de0 con 0x7f50441509f0 2024-03-11T18:36:09.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.052+0000 7f504a49b640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5044156910 con 0x7f5044150dd0 2024-03-11T18:36:09.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.052+0000 7f504a49b640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50440c6030 con 0x7f5044070b00 2024-03-11T18:36:09.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.052+0000 7f50437fe640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f5044150dd0 0x7f50440c28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:09.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.052+0000 7f50437fe640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f5044150dd0 0x7f50440c28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55456/0 (socket says 172.21.15.90:55456) 2024-03-11T18:36:09.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.052+0000 7f50437fe640 1 -- 172.21.15.90:0/222231210 learned_addr learned my addr 172.21.15.90:0/222231210 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:09.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.052+0000 7f5048a38640 1 --2- 172.21.15.90:0/222231210 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f5044070b00 0x7f50440c4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:09.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.056+0000 7f50437fe640 1 -- 172.21.15.90:0/222231210 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f5044070b00 msgr2=0x7f50440c4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:09.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.056+0000 7f50437fe640 1 --2- 172.21.15.90:0/222231210 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f5044070b00 0x7f50440c4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:09.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.056+0000 7f50437fe640 1 -- 172.21.15.90:0/222231210 >> v1:172.21.15.90:6789/0 conn(0x7f50441509f0 legacy=0x7f5044157390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:09.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.056+0000 7f50437fe640 1 -- 172.21.15.90:0/222231210 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50440c6310 con 0x7f5044150dd0 2024-03-11T18:36:09.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.060+0000 7f50437fe640 1 --2- 172.21.15.90:0/222231210 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5044150dd0 0x7f50440c28e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5038009870 tx=0x7f503801ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a1becc7d63d263f server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:09.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.068+0000 7f5042ffd640 1 -- 172.21.15.90:0/222231210 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50380a8a50 con 0x7f5044150dd0 2024-03-11T18:36:09.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.068+0000 7f5042ffd640 1 -- 172.21.15.90:0/222231210 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50380a8bb0 con 0x7f5044150dd0 2024-03-11T18:36:09.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.068+0000 7f5042ffd640 1 -- 172.21.15.90:0/222231210 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50380b0440 con 0x7f5044150dd0 2024-03-11T18:36:09.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.068+0000 7f504a49b640 1 -- 172.21.15.90:0/222231210 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5044150dd0 msgr2=0x7f50440c28e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:09.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.068+0000 7f504a49b640 1 --2- 172.21.15.90:0/222231210 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5044150dd0 0x7f50440c28e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5038009870 tx=0x7f503801ed50 comp rx=0 tx=0).stop 2024-03-11T18:36:09.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.068+0000 7f504a49b640 1 -- 172.21.15.90:0/222231210 shutdown_connections 2024-03-11T18:36:09.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.068+0000 7f504a49b640 1 --2- 172.21.15.90:0/222231210 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f5044070b00 0x7f50440c4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:09.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.068+0000 7f504a49b640 1 --2- 172.21.15.90:0/222231210 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5044150dd0 0x7f50440c28e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:09.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.068+0000 7f504a49b640 1 -- 172.21.15.90:0/222231210 >> 172.21.15.90:0/222231210 conn(0x7f504406ac00 msgr2=0x7f504406b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:09.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.068+0000 7f504a49b640 1 -- 172.21.15.90:0/222231210 shutdown_connections 2024-03-11T18:36:09.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.068+0000 7f504a49b640 1 -- 172.21.15.90:0/222231210 wait complete. 2024-03-11T18:36:09.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.068+0000 7f504a49b640 1 Processor -- start 2024-03-11T18:36:09.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.068+0000 7f504a49b640 1 -- start start 2024-03-11T18:36:09.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.068+0000 7f504a49b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5044070b00 0x7f50441560e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:09.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.072+0000 7f504a49b640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f50441509f0 0x7f504413f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:09.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.072+0000 7f504a49b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50441407c0 0x7f504413f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:09.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.072+0000 7f504a49b640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50440c29a0 con 0x7f5044070b00 2024-03-11T18:36:09.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.072+0000 7f504a49b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f504413c9c0 con 0x7f50441407c0 2024-03-11T18:36:09.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.072+0000 7f504a49b640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f504413ebc0 con 0x7f50441509f0 2024-03-11T18:36:09.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.072+0000 7f5043fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5044070b00 0x7f50441560e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:09.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.072+0000 7f5043fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5044070b00 0x7f50441560e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55470/0 (socket says 172.21.15.90:55470) 2024-03-11T18:36:09.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.072+0000 7f5043fff640 1 -- 172.21.15.90:0/3201449856 learned_addr learned my addr 172.21.15.90:0/3201449856 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:09.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.072+0000 7f50437fe640 1 --2- 172.21.15.90:0/3201449856 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f50441509f0 0x7f504413f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:09.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.072+0000 7f5048a38640 1 --2- 172.21.15.90:0/3201449856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50441407c0 0x7f504413f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:09.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.076+0000 7f5043fff640 1 -- 172.21.15.90:0/3201449856 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f50441509f0 msgr2=0x7f504413f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:09.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.076+0000 7f5043fff640 1 --2- 172.21.15.90:0/3201449856 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f50441509f0 0x7f504413f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:09.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.076+0000 7f5043fff640 1 -- 172.21.15.90:0/3201449856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50441407c0 msgr2=0x7f504413f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:09.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.076+0000 7f5043fff640 1 --2- 172.21.15.90:0/3201449856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50441407c0 0x7f504413f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:09.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.076+0000 7f5043fff640 1 -- 172.21.15.90:0/3201449856 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50380094d0 con 0x7f5044070b00 2024-03-11T18:36:09.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.076+0000 7f5043fff640 1 --2- 172.21.15.90:0/3201449856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5044070b00 0x7f50441560e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f503400c370 tx=0x7f503400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:09.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.080+0000 7f5040ff9640 1 -- 172.21.15.90:0/3201449856 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50340052d0 con 0x7f5044070b00 2024-03-11T18:36:09.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.084+0000 7f5040ff9640 1 -- 172.21.15.90:0/3201449856 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5034005430 con 0x7f5044070b00 2024-03-11T18:36:09.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.084+0000 7f504a49b640 1 -- 172.21.15.90:0/3201449856 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f504413a800 con 0x7f5044070b00 2024-03-11T18:36:09.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.084+0000 7f5040ff9640 1 -- 172.21.15.90:0/3201449856 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5034007260 con 0x7f5044070b00 2024-03-11T18:36:09.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.084+0000 7f504a49b640 1 -- 172.21.15.90:0/3201449856 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f504413ada0 con 0x7f5044070b00 2024-03-11T18:36:09.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.084+0000 7f504a49b640 1 -- 172.21.15.90:0/3201449856 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f504404eef0 con 0x7f5044070b00 2024-03-11T18:36:09.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.380+0000 7f5040ff9640 1 -- 172.21.15.90:0/3201449856 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f50340073c0 con 0x7f5044070b00 2024-03-11T18:36:09.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.380+0000 7f5040ff9640 1 --2- 172.21.15.90:0/3201449856 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f5030066e90 0x7f5030069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:09.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.380+0000 7f50437fe640 1 --2- 172.21.15.90:0/3201449856 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f5030066e90 0x7f5030069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:09.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.380+0000 7f50437fe640 1 --2- 172.21.15.90:0/3201449856 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f5030066e90 0x7f5030069350 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f50380b2000 tx=0x7f50380aa070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:09.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.408+0000 7f5040ff9640 1 -- 172.21.15.90:0/3201449856 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5034007670 con 0x7f5044070b00 2024-03-11T18:36:09.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.412+0000 7f5040ff9640 1 -- 172.21.15.90:0/3201449856 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f50340510a0 con 0x7f5044070b00 2024-03-11T18:36:09.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.616+0000 7f504a49b640 1 -- 172.21.15.90:0/3201449856 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f504406e730 con 0x7f5044070b00 2024-03-11T18:36:09.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.632+0000 7f5040ff9640 1 -- 172.21.15.90:0/3201449856 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+6508 (secure 0 0 0) 0x7f5034020090 con 0x7f5044070b00 2024-03-11T18:36:09.639 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:36:09.639 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/641527992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":641527992},{"type":"v1","addr":"172.21.15.90:6839","nonce":641527992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4229,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/460417177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":460417177},{"type":"v1","addr":"172.21.15.90:6835","nonce":460417177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/3568438077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":3568438077},{"type":"v1","addr":"172.21.15.92:6835","nonce":3568438077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/2306487548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":2306487548},{"type":"v1","addr":"172.21.15.92:6834","nonce":2306487548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":4253,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6837/3209534444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":3209534444},{"type":"v1","addr":"172.21.15.92:6837","nonce":3209534444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-11T18:35:57.363570+0000","modified":"2024-03-11T18:36:05.539381+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4255":{"gid":4255,"name":"e","rank":0,"incarnation":5,"state":"up:active","state_seq":6,"addr":"172.21.15.90:6837/3485943375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3485943375},{"type":"v1","addr":"172.21.15.90:6837","nonce":3485943375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-11T18:36:09.640 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 7 2024-03-11T18:36:09.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.636+0000 7f504a49b640 1 -- 172.21.15.90:0/3201449856 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f5030066e90 msgr2=0x7f5030069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:09.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.636+0000 7f504a49b640 1 --2- 172.21.15.90:0/3201449856 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f5030066e90 0x7f5030069350 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f50380b2000 tx=0x7f50380aa070 comp rx=0 tx=0).stop 2024-03-11T18:36:09.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.636+0000 7f504a49b640 1 -- 172.21.15.90:0/3201449856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5044070b00 msgr2=0x7f50441560e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:09.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.636+0000 7f504a49b640 1 --2- 172.21.15.90:0/3201449856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5044070b00 0x7f50441560e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f503400c370 tx=0x7f503400c830 comp rx=0 tx=0).stop 2024-03-11T18:36:09.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.636+0000 7f504a49b640 1 -- 172.21.15.90:0/3201449856 shutdown_connections 2024-03-11T18:36:09.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.636+0000 7f504a49b640 1 --2- 172.21.15.90:0/3201449856 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f5030066e90 0x7f5030069350 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:09.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.636+0000 7f504a49b640 1 --2- 172.21.15.90:0/3201449856 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f50441407c0 0x7f504413f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:09.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.636+0000 7f504a49b640 1 --2- 172.21.15.90:0/3201449856 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f50441509f0 0x7f504413f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:09.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.636+0000 7f504a49b640 1 --2- 172.21.15.90:0/3201449856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5044070b00 0x7f50441560e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:09.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.636+0000 7f504a49b640 1 -- 172.21.15.90:0/3201449856 >> 172.21.15.90:0/3201449856 conn(0x7f504406ac00 msgr2=0x7f5044070f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:09.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.636+0000 7f504a49b640 1 -- 172.21.15.90:0/3201449856 shutdown_connections 2024-03-11T18:36:09.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.636+0000 7f504a49b640 1 -- 172.21.15.90:0/3201449856 wait complete. 2024-03-11T18:36:09.659 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T18:36:09.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.772+0000 7f4489495640 1 Processor -- start 2024-03-11T18:36:09.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.772+0000 7f4489495640 1 -- start start 2024-03-11T18:36:09.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.776+0000 7f4489495640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f4484070b00 0x7f448406e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:09.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.776+0000 7f4489495640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f44840c4af0 0x7f44840c4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:09.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.776+0000 7f4489495640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f448406f430 con 0x7f4484151b30 2024-03-11T18:36:09.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.776+0000 7f4489495640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44840c65b0 con 0x7f4484070b00 2024-03-11T18:36:09.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.776+0000 7f4489495640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f448406e820 con 0x7f44840c4af0 2024-03-11T18:36:09.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.776+0000 7f44827fc640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f4484070b00 0x7f448406e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:09.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.776+0000 7f44827fc640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f4484070b00 0x7f448406e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55486/0 (socket says 172.21.15.90:55486) 2024-03-11T18:36:09.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.776+0000 7f44827fc640 1 -- 172.21.15.90:0/2200891401 learned_addr learned my addr 172.21.15.90:0/2200891401 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:09.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.776+0000 7f44837fe640 1 --2- 172.21.15.90:0/2200891401 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f44840c4af0 0x7f44840c4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:09.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.780+0000 7f44827fc640 1 -- 172.21.15.90:0/2200891401 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f44840c4af0 msgr2=0x7f44840c4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:09.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.780+0000 7f44827fc640 1 --2- 172.21.15.90:0/2200891401 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f44840c4af0 0x7f44840c4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:09.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.780+0000 7f44827fc640 1 -- 172.21.15.90:0/2200891401 >> v1:172.21.15.92:6789/0 conn(0x7f4484151b30 legacy=0x7f44840c5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:09.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.780+0000 7f44827fc640 1 -- 172.21.15.90:0/2200891401 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f448406eb00 con 0x7f4484070b00 2024-03-11T18:36:09.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.784+0000 7f44827fc640 1 --2- 172.21.15.90:0/2200891401 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4484070b00 0x7f448406e210 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4478009870 tx=0x7f447801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=627f098d7a9dd8ae server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:09.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.788+0000 7f4481ffb640 1 -- 172.21.15.90:0/2200891401 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44780a95f0 con 0x7f4484070b00 2024-03-11T18:36:09.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.788+0000 7f4481ffb640 1 -- 172.21.15.90:0/2200891401 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44780a9750 con 0x7f4484070b00 2024-03-11T18:36:09.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.788+0000 7f4481ffb640 1 -- 172.21.15.90:0/2200891401 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44780a9ae0 con 0x7f4484070b00 2024-03-11T18:36:09.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.788+0000 7f4489495640 1 -- 172.21.15.90:0/2200891401 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4484070b00 msgr2=0x7f448406e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:09.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.788+0000 7f4489495640 1 --2- 172.21.15.90:0/2200891401 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4484070b00 0x7f448406e210 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4478009870 tx=0x7f447801c530 comp rx=0 tx=0).stop 2024-03-11T18:36:09.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.788+0000 7f4489495640 1 -- 172.21.15.90:0/2200891401 shutdown_connections 2024-03-11T18:36:09.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.788+0000 7f4489495640 1 --2- 172.21.15.90:0/2200891401 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f44840c4af0 0x7f44840c4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:09.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.788+0000 7f4489495640 1 --2- 172.21.15.90:0/2200891401 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4484070b00 0x7f448406e210 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:09.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.788+0000 7f4489495640 1 -- 172.21.15.90:0/2200891401 >> 172.21.15.90:0/2200891401 conn(0x7f448406ac00 msgr2=0x7f448406b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:09.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.788+0000 7f4489495640 1 -- 172.21.15.90:0/2200891401 shutdown_connections 2024-03-11T18:36:09.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.788+0000 7f4489495640 1 -- 172.21.15.90:0/2200891401 wait complete. 2024-03-11T18:36:09.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.792+0000 7f4489495640 1 Processor -- start 2024-03-11T18:36:09.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.792+0000 7f4489495640 1 -- start start 2024-03-11T18:36:09.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.792+0000 7f4489495640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4484070b00 0x7f4484147870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:09.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.792+0000 7f4489495640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44840c4af0 0x7f44840ca4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:09.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.792+0000 7f4489495640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4484151b30 0x7f44841f0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:09.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.792+0000 7f4489495640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448413fb30 con 0x7f4484151b30 2024-03-11T18:36:09.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.792+0000 7f4489495640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448413efb0 con 0x7f44840c4af0 2024-03-11T18:36:09.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.792+0000 7f4489495640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f448413f570 con 0x7f4484070b00 2024-03-11T18:36:09.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.792+0000 7f44827fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44840c4af0 0x7f44840ca4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:09.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.792+0000 7f4482ffd640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4484070b00 0x7f4484147870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:09.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.792+0000 7f44827fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44840c4af0 0x7f44840ca4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.90:50388/0 (socket says 172.21.15.90:50388) 2024-03-11T18:36:09.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.792+0000 7f44827fc640 1 -- 172.21.15.90:0/4084658017 learned_addr learned my addr 172.21.15.90:0/4084658017 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:09.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.804+0000 7f44827fc640 1 -- 172.21.15.90:0/4084658017 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4484070b00 msgr2=0x7f4484147870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:09.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.804+0000 7f44827fc640 1 --2- 172.21.15.90:0/4084658017 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4484070b00 0x7f4484147870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:09.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.804+0000 7f44827fc640 1 -- 172.21.15.90:0/4084658017 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4484151b30 msgr2=0x7f44841f0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:09.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.804+0000 7f44827fc640 1 --2- 172.21.15.90:0/4084658017 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4484151b30 0x7f44841f0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:09.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.804+0000 7f44827fc640 1 -- 172.21.15.90:0/4084658017 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44780094d0 con 0x7f44840c4af0 2024-03-11T18:36:09.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.808+0000 7f44827fc640 1 --2- 172.21.15.90:0/4084658017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44840c4af0 0x7f44840ca4d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f447801a160 tx=0x7f44780192d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:09.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.820+0000 7f446b7fe640 1 -- 172.21.15.90:0/4084658017 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44780a9e90 con 0x7f44840c4af0 2024-03-11T18:36:09.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.820+0000 7f446b7fe640 1 -- 172.21.15.90:0/4084658017 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44780bb3f0 con 0x7f44840c4af0 2024-03-11T18:36:09.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.820+0000 7f4489495640 1 -- 172.21.15.90:0/4084658017 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f448413f850 con 0x7f44840c4af0 2024-03-11T18:36:09.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.820+0000 7f446b7fe640 1 -- 172.21.15.90:0/4084658017 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44780bb780 con 0x7f44840c4af0 2024-03-11T18:36:09.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.820+0000 7f4489495640 1 -- 172.21.15.90:0/4084658017 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4484149ac0 con 0x7f44840c4af0 2024-03-11T18:36:09.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:09.820+0000 7f4489495640 1 -- 172.21.15.90:0/4084658017 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4484149640 con 0x7f44840c4af0 2024-03-11T18:36:10.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:10.156+0000 7f446b7fe640 1 -- 172.21.15.90:0/4084658017 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f44780c4430 con 0x7f44840c4af0 2024-03-11T18:36:10.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:10.156+0000 7f446b7fe640 1 --2- 172.21.15.90:0/4084658017 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f4470066e90 0x7f4470069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:10.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:10.156+0000 7f4482ffd640 1 --2- 172.21.15.90:0/4084658017 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f4470066e90 0x7f4470069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:10.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:10.160+0000 7f4482ffd640 1 --2- 172.21.15.90:0/4084658017 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f4470066e90 0x7f4470069350 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4474001810 tx=0x7f4474009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:10.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:10.164+0000 7f446b7fe640 1 -- 172.21.15.90:0/4084658017 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f44780efec0 con 0x7f44840c4af0 2024-03-11T18:36:10.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:10.184+0000 7f446b7fe640 1 -- 172.21.15.90:0/4084658017 <== mon.1 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f44780eba60 con 0x7f44840c4af0 2024-03-11T18:36:10.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:10.352+0000 7f4489495640 1 -- 172.21.15.90:0/4084658017 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4484149640 con 0x7f44840c4af0 2024-03-11T18:36:10.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:10.644+0000 7f446b7fe640 1 -- 172.21.15.90:0/4084658017 <== mon.1 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18430 (secure 0 0 0) 0x7f4484149640 con 0x7f44840c4af0 2024-03-11T18:36:10.652 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:36:10.652 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":28,"fsid":"76d56b10-6c29-476a-804b-8482ea90f9e2","created":"2024-03-11T18:34:54.521741+0000","modified":"2024-03-11T18:36:05.526044+0000","last_up_change":"2024-03-11T18:35:16.207500+0000","last_in_change":"2024-03-11T18:35:05.213634+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T18:35:16.193842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T18:35:54.001469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T18:35:55.917321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T18:36:02.210943+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"34decff9-7280-4d84-9665-d6af8ea6de84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6819","nonce":1755667921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6821","nonce":1755667921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6825","nonce":1755667921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6823","nonce":1755667921}]},"public_addr":"172.21.15.90:6819/1755667921","cluster_addr":"172.21.15.90:6821/1755667921","heartbeat_back_addr":"172.21.15.90:6825/1755667921","heartbeat_front_addr":"172.21.15.90:6823/1755667921","state":["exists","up"]},{"osd":1,"uuid":"8ecdce9e-5486-415a-a609-dd16e945440e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6807","nonce":438682081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6809","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6811","nonce":438682081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6817","nonce":438682081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6815","nonce":438682081}]},"public_addr":"172.21.15.90:6807/438682081","cluster_addr":"172.21.15.90:6811/438682081","heartbeat_back_addr":"172.21.15.90:6817/438682081","heartbeat_front_addr":"172.21.15.90:6815/438682081","state":["exists","up"]},{"osd":2,"uuid":"737486d4-0e90-4c82-9e79-bf37c812e5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6827","nonce":2837704738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6829","nonce":2837704738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6833","nonce":2837704738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6831","nonce":2837704738}]},"public_addr":"172.21.15.90:6827/2837704738","cluster_addr":"172.21.15.90:6829/2837704738","heartbeat_back_addr":"172.21.15.90:6833/2837704738","heartbeat_front_addr":"172.21.15.90:6831/2837704738","state":["exists","up"]},{"osd":3,"uuid":"698bd661-bbef-4699-a619-f7bfe23ce12c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6803","nonce":240987971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6805","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6806","nonce":240987971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6813","nonce":240987971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6810","nonce":240987971}]},"public_addr":"172.21.15.90:6803/240987971","cluster_addr":"172.21.15.90:6806/240987971","heartbeat_back_addr":"172.21.15.90:6813/240987971","heartbeat_front_addr":"172.21.15.90:6810/240987971","state":["exists","up"]},{"osd":4,"uuid":"aba9e31f-73d9-43fe-a10b-2dc5832968b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6801","nonce":502685275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6803","nonce":502685275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6807","nonce":502685275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6805","nonce":502685275}]},"public_addr":"172.21.15.92:6801/502685275","cluster_addr":"172.21.15.92:6803/502685275","heartbeat_back_addr":"172.21.15.92:6807/502685275","heartbeat_front_addr":"172.21.15.92:6805/502685275","state":["exists","up"]},{"osd":5,"uuid":"bdd56766-4b28-4007-8de3-b5b08cf335d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6809","nonce":200142070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6811","nonce":200142070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6815","nonce":200142070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6813","nonce":200142070}]},"public_addr":"172.21.15.92:6809/200142070","cluster_addr":"172.21.15.92:6811/200142070","heartbeat_back_addr":"172.21.15.92:6815/200142070","heartbeat_front_addr":"172.21.15.92:6813/200142070","state":["exists","up"]},{"osd":6,"uuid":"6c580a59-610f-455e-a96b-033299dcc118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6817","nonce":1919010204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6819","nonce":1919010204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6823","nonce":1919010204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6821","nonce":1919010204}]},"public_addr":"172.21.15.92:6817/1919010204","cluster_addr":"172.21.15.92:6819/1919010204","heartbeat_back_addr":"172.21.15.92:6823/1919010204","heartbeat_front_addr":"172.21.15.92:6821/1919010204","state":["exists","up"]},{"osd":7,"uuid":"958184cc-aebf-4c5b-91a0-2a3e38a6b42b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6825","nonce":3475925719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6827","nonce":3475925719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6831","nonce":3475925719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6829","nonce":3475925719}]},"public_addr":"172.21.15.92:6825/3475925719","cluster_addr":"172.21.15.92:6827/3475925719","heartbeat_back_addr":"172.21.15.92:6831/3475925719","heartbeat_front_addr":"172.21.15.92:6829/3475925719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.816294+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.048084+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.137975+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.968246+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.016829+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.931676+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.041673+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.079404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T18:36:10.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:10.648+0000 7f4489495640 1 -- 172.21.15.90:0/4084658017 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f4470066e90 msgr2=0x7f4470069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:10.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:10.648+0000 7f4489495640 1 --2- 172.21.15.90:0/4084658017 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f4470066e90 0x7f4470069350 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4474001810 tx=0x7f4474009040 comp rx=0 tx=0).stop 2024-03-11T18:36:10.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:10.648+0000 7f4489495640 1 -- 172.21.15.90:0/4084658017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44840c4af0 msgr2=0x7f44840ca4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:10.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:10.648+0000 7f4489495640 1 --2- 172.21.15.90:0/4084658017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44840c4af0 0x7f44840ca4d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f447801a160 tx=0x7f44780192d0 comp rx=0 tx=0).stop 2024-03-11T18:36:10.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:10.648+0000 7f4489495640 1 -- 172.21.15.90:0/4084658017 shutdown_connections 2024-03-11T18:36:10.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:10.648+0000 7f4489495640 1 --2- 172.21.15.90:0/4084658017 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f4470066e90 0x7f4470069350 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:10.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:10.648+0000 7f4489495640 1 --2- 172.21.15.90:0/4084658017 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4484151b30 0x7f44841f0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:10.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:10.648+0000 7f4489495640 1 --2- 172.21.15.90:0/4084658017 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f44840c4af0 0x7f44840ca4d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:10.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:10.648+0000 7f4489495640 1 --2- 172.21.15.90:0/4084658017 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4484070b00 0x7f4484147870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:10.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:10.648+0000 7f4489495640 1 -- 172.21.15.90:0/4084658017 >> 172.21.15.90:0/4084658017 conn(0x7f448406ac00 msgr2=0x7f4484150480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:10.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:10.648+0000 7f4489495640 1 -- 172.21.15.90:0/4084658017 shutdown_connections 2024-03-11T18:36:10.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:10.648+0000 7f4489495640 1 -- 172.21.15.90:0/4084658017 wait complete. 2024-03-11T18:36:11.280 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-11T18:35:57.363570+0000', 'modified': '2024-03-11T18:36:05.539381+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4255}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4255': {'gid': 4255, 'name': 'e', 'rank': 0, 'incarnation': 5, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.90:6837/3485943375', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.90:6836', 'nonce': 3485943375}, {'type': 'v1', 'addr': '172.21.15.90:6837', 'nonce': 3485943375}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2024-03-11T18:36:11.280 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-11T18:36:11.280 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-03-11T18:36:11.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.404+0000 7f3750ecd640 1 Processor -- start 2024-03-11T18:36:11.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.404+0000 7f3750ecd640 1 -- start start 2024-03-11T18:36:11.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.404+0000 7f3750ecd640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f374c0cc280 0x7f374c0cc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:11.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.408+0000 7f3750ecd640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f374c0c5e60 con 0x7f374c0c65b0 2024-03-11T18:36:11.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.408+0000 7f3750ecd640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f374c156a60 con 0x7f374c150a90 2024-03-11T18:36:11.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.408+0000 7f3750ecd640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f374c0c55c0 con 0x7f374c0cc280 2024-03-11T18:36:11.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.408+0000 7f3741d9b640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f374c0c65b0 0x7f374c0c37f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:36074/0 (socket says 172.21.15.90:36074) 2024-03-11T18:36:11.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.408+0000 7f3741d9b640 1 -- 172.21.15.90:0/3200551877 learned_addr learned my addr 172.21.15.90:0/3200551877 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T18:36:11.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.408+0000 7f374ad9d640 1 --2- 172.21.15.90:0/3200551877 >> v2:172.21.15.92:3300/0 conn(0x7f374c0cc280 0x7f374c0cc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:11.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.408+0000 7f3749d9b640 1 -- 172.21.15.90:0/3200551877 <== mon.0 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3175485932 0 0) 0x7f374c0c5e60 con 0x7f374c0c65b0 2024-03-11T18:36:11.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.408+0000 7f3749d9b640 1 -- 172.21.15.90:0/3200551877 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3738005740 con 0x7f374c0c65b0 2024-03-11T18:36:11.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.412+0000 7f3749d9b640 1 -- 172.21.15.90:0/3200551877 <== mon.0 v1:172.21.15.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3985472966 0 0) 0x7f3738005740 con 0x7f374c0c65b0 2024-03-11T18:36:11.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.412+0000 7f3749d9b640 1 -- 172.21.15.90:0/3200551877 >> v2:172.21.15.92:3300/0 conn(0x7f374c0cc280 msgr2=0x7f374c0cc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:11.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.412+0000 7f3749d9b640 1 --2- 172.21.15.90:0/3200551877 >> v2:172.21.15.92:3300/0 conn(0x7f374c0cc280 0x7f374c0cc6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:11.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.412+0000 7f3749d9b640 1 -- 172.21.15.90:0/3200551877 >> v1:172.21.15.92:6789/0 conn(0x7f374c150a90 legacy=0x7f374c157220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:11.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.412+0000 7f3749d9b640 1 -- 172.21.15.90:0/3200551877 --> v1:172.21.15.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f374c0c58a0 con 0x7f374c0c65b0 2024-03-11T18:36:11.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.412+0000 7f3749d9b640 1 -- 172.21.15.90:0/3200551877 <== mon.0 v1:172.21.15.90:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 691735435 0 0) 0x7f3734001680 con 0x7f374c0c65b0 2024-03-11T18:36:11.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.412+0000 7f3749d9b640 1 -- 172.21.15.90:0/3200551877 >> v1:172.21.15.90:6789/0 conn(0x7f374c0c65b0 legacy=0x7f374c0c37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:11.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.412+0000 7f3749d9b640 1 --2- 172.21.15.90:0/3200551877 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3738005b90 0x7f3738005f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:11.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.412+0000 7f3749d9b640 1 --2- 172.21.15.90:0/3200551877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37380069e0 0x7f3738006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:11.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.412+0000 7f3749d9b640 1 --2- 172.21.15.90:0/3200551877 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f37380073a0 0x7f373800dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:11.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.412+0000 7f3749d9b640 1 -- 172.21.15.90:0/3200551877 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374c0c55c0 con 0x7f3738005b90 2024-03-11T18:36:11.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.412+0000 7f3749d9b640 1 -- 172.21.15.90:0/3200551877 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373800e220 con 0x7f37380069e0 2024-03-11T18:36:11.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.412+0000 7f3749d9b640 1 -- 172.21.15.90:0/3200551877 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f373800e3e0 con 0x7f37380073a0 2024-03-11T18:36:11.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.412+0000 7f374a59c640 1 --2- 172.21.15.90:0/3200551877 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3738005b90 0x7f3738005f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:11.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.416+0000 7f374ad9d640 1 --2- 172.21.15.90:0/3200551877 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f37380073a0 0x7f373800dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:11.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.420+0000 7f374a59c640 1 -- 172.21.15.90:0/3200551877 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f37380073a0 msgr2=0x7f373800dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:11.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.420+0000 7f374a59c640 1 --2- 172.21.15.90:0/3200551877 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f37380073a0 0x7f373800dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:11.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.420+0000 7f374a59c640 1 -- 172.21.15.90:0/3200551877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37380069e0 msgr2=0x7f3738006e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:11.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.420+0000 7f374a59c640 1 --2- 172.21.15.90:0/3200551877 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f37380069e0 0x7f3738006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:11.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.420+0000 7f374a59c640 1 -- 172.21.15.90:0/3200551877 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3738005740 con 0x7f3738005b90 2024-03-11T18:36:11.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.420+0000 7f374ad9d640 1 -- 172.21.15.90:0/3200551877 reap_dead start 2024-03-11T18:36:11.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.424+0000 7f374a59c640 1 --2- 172.21.15.90:0/3200551877 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3738005b90 0x7f3738005f90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f373c001650 tx=0x7f373c01c2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18980a50693a87a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:11.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.428+0000 7f3749d9b640 1 -- 172.21.15.90:0/3200551877 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373c005270 con 0x7f3738005b90 2024-03-11T18:36:11.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.428+0000 7f3749d9b640 1 -- 172.21.15.90:0/3200551877 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f373c0a7070 con 0x7f3738005b90 2024-03-11T18:36:11.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.428+0000 7f3749d9b640 1 -- 172.21.15.90:0/3200551877 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373c003620 con 0x7f3738005b90 2024-03-11T18:36:11.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.428+0000 7f3750ecd640 1 -- 172.21.15.90:0/3200551877 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3738005b90 msgr2=0x7f3738005f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:11.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.428+0000 7f3750ecd640 1 --2- 172.21.15.90:0/3200551877 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3738005b90 0x7f3738005f90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f373c001650 tx=0x7f373c01c2e0 comp rx=0 tx=0).stop 2024-03-11T18:36:11.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.428+0000 7f3750ecd640 1 -- 172.21.15.90:0/3200551877 shutdown_connections 2024-03-11T18:36:11.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.428+0000 7f3750ecd640 1 --2- 172.21.15.90:0/3200551877 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3738005b90 0x7f3738005f90 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:11.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.428+0000 7f3750ecd640 1 -- 172.21.15.90:0/3200551877 >> 172.21.15.90:0/3200551877 conn(0x7f374c071aa0 msgr2=0x7f374c071eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:11.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.428+0000 7f3750ecd640 1 -- 172.21.15.90:0/3200551877 shutdown_connections 2024-03-11T18:36:11.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.428+0000 7f3750ecd640 1 -- 172.21.15.90:0/3200551877 wait complete. 2024-03-11T18:36:11.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.428+0000 7f3750ecd640 1 Processor -- start 2024-03-11T18:36:11.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.428+0000 7f3750ecd640 1 -- start start 2024-03-11T18:36:11.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.428+0000 7f3750ecd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3738005b90 0x7f374c14afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:11.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.428+0000 7f3750ecd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f374c1f1aa0 0x7f374c06c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:11.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.428+0000 7f3750ecd640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f374c147510 0x7f374c13eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:11.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.428+0000 7f3750ecd640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374c14bec0 con 0x7f374c1f1aa0 2024-03-11T18:36:11.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.428+0000 7f3750ecd640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374c14b520 con 0x7f3738005b90 2024-03-11T18:36:11.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.428+0000 7f3750ecd640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f374c1f2070 con 0x7f374c147510 2024-03-11T18:36:11.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.428+0000 7f3741d9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f374c1f1aa0 0x7f374c06c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:11.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.432+0000 7f3741d9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f374c1f1aa0 0x7f374c06c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55508/0 (socket says 172.21.15.90:55508) 2024-03-11T18:36:11.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.432+0000 7f3741d9b640 1 -- 172.21.15.90:0/381523617 learned_addr learned my addr 172.21.15.90:0/381523617 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:11.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.432+0000 7f374a59c640 1 --2- 172.21.15.90:0/381523617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3738005b90 0x7f374c14afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:11.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.432+0000 7f374ad9d640 1 --2- 172.21.15.90:0/381523617 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f374c147510 0x7f374c13eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:11.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.432+0000 7f3741d9b640 1 -- 172.21.15.90:0/381523617 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f374c147510 msgr2=0x7f374c13eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:11.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.432+0000 7f3741d9b640 1 --2- 172.21.15.90:0/381523617 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f374c147510 0x7f374c13eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:11.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.432+0000 7f3741d9b640 1 -- 172.21.15.90:0/381523617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3738005b90 msgr2=0x7f374c14afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:11.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.432+0000 7f3741d9b640 1 --2- 172.21.15.90:0/381523617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3738005b90 0x7f374c14afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:11.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.432+0000 7f3741d9b640 1 -- 172.21.15.90:0/381523617 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f374c1f2350 con 0x7f374c1f1aa0 2024-03-11T18:36:11.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.436+0000 7f3741d9b640 1 --2- 172.21.15.90:0/381523617 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f374c1f1aa0 0x7f374c06c3c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f373400c680 tx=0x7f373400cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:11.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.440+0000 7f37437fe640 1 -- 172.21.15.90:0/381523617 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37340072d0 con 0x7f374c1f1aa0 2024-03-11T18:36:11.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.440+0000 7f37437fe640 1 -- 172.21.15.90:0/381523617 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3734007430 con 0x7f374c1f1aa0 2024-03-11T18:36:11.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.440+0000 7f37437fe640 1 -- 172.21.15.90:0/381523617 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37340047f0 con 0x7f374c1f1aa0 2024-03-11T18:36:11.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.440+0000 7f3750ecd640 1 -- 172.21.15.90:0/381523617 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f374c1f2630 con 0x7f374c1f1aa0 2024-03-11T18:36:11.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.440+0000 7f3750ecd640 1 -- 172.21.15.90:0/381523617 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f374c146d10 con 0x7f374c1f1aa0 2024-03-11T18:36:11.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.444+0000 7f3754036000 1 Processor -- start 2024-03-11T18:36:11.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.444+0000 7f3754036000 1 -- start start 2024-03-11T18:36:11.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.444+0000 7f3754036000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a8f74e4f70 0x55a8f74ab350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:11.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.444+0000 7f3754036000 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a8f74ab890 0x55a8f74e6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:11.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.444+0000 7f3754036000 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55a8f74e6bc0 0x55a8f74e6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:11.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.444+0000 7f3754036000 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8f736dd00 con 0x55a8f74ab890 2024-03-11T18:36:11.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.444+0000 7f3754036000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8f72ff430 con 0x55a8f74e4f70 2024-03-11T18:36:11.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.444+0000 7f3754036000 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a8f72fc960 con 0x55a8f74e6bc0 2024-03-11T18:36:11.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.444+0000 7f3741d9b640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55a8f74e6bc0 0x55a8f74e6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:11.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.444+0000 7f374ad9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a8f74e4f70 0x55a8f74ab350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:11.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.448+0000 7f3741d9b640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55a8f74e6bc0 0x55a8f74e6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.90:51458/0 (socket says 172.21.15.90:51458) 2024-03-11T18:36:11.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.448+0000 7f3741d9b640 1 -- 172.21.15.90:0/3478898699 learned_addr learned my addr 172.21.15.90:0/3478898699 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:11.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.452+0000 7f374ad9d640 1 -- 172.21.15.90:0/3478898699 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55a8f74e6bc0 msgr2=0x55a8f74e6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:11.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.452+0000 7f374ad9d640 1 --2- 172.21.15.90:0/3478898699 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55a8f74e6bc0 0x55a8f74e6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:11.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.452+0000 7f374ad9d640 1 -- 172.21.15.90:0/3478898699 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a8f74ab890 msgr2=0x55a8f74e6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:11.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.452+0000 7f374ad9d640 1 --2- 172.21.15.90:0/3478898699 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a8f74ab890 0x55a8f74e6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:11.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.452+0000 7f374ad9d640 1 -- 172.21.15.90:0/3478898699 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8f6fee890 con 0x55a8f74e4f70 2024-03-11T18:36:11.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.456+0000 7f374ad9d640 1 --2- 172.21.15.90:0/3478898699 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a8f74e4f70 0x55a8f74ab350 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f374c06d700 tx=0x7f374400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2ec47a000f74f27 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:11.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.460+0000 7f37437fe640 1 -- 172.21.15.90:0/381523617 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f3734004ad0 con 0x7f374c1f1aa0 2024-03-11T18:36:11.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.460+0000 7f37437fe640 1 --2- 172.21.15.90:0/381523617 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f372c066e90 0x7f372c069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:11.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.460+0000 7f374a59c640 1 --2- 172.21.15.90:0/381523617 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f372c066e90 0x7f372c069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:11.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.460+0000 7f374a59c640 1 --2- 172.21.15.90:0/381523617 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f372c066e90 0x7f372c069350 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f373c0aa000 tx=0x7f373c005040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:11.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.464+0000 7f37437fe640 1 -- 172.21.15.90:0/381523617 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3734019260 con 0x7f374c1f1aa0 2024-03-11T18:36:11.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.468+0000 7f372bfff640 1 -- 172.21.15.90:0/3478898699 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374400e040 con 0x55a8f74e4f70 2024-03-11T18:36:11.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.468+0000 7f372bfff640 1 -- 172.21.15.90:0/3478898699 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f374400f040 con 0x55a8f74e4f70 2024-03-11T18:36:11.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.468+0000 7f372bfff640 1 -- 172.21.15.90:0/3478898699 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3744003da0 con 0x55a8f74e4f70 2024-03-11T18:36:11.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.468+0000 7f3754036000 1 -- 172.21.15.90:0/3478898699 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a8f74e4f70 msgr2=0x55a8f74ab350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:11.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.468+0000 7f3754036000 1 --2- 172.21.15.90:0/3478898699 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a8f74e4f70 0x55a8f74ab350 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f374c06d700 tx=0x7f374400b040 comp rx=0 tx=0).stop 2024-03-11T18:36:11.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.468+0000 7f3754036000 1 -- 172.21.15.90:0/3478898699 shutdown_connections 2024-03-11T18:36:11.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.468+0000 7f3754036000 1 --2- 172.21.15.90:0/3478898699 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55a8f74e6bc0 0x55a8f74e6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:11.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.468+0000 7f3754036000 1 --2- 172.21.15.90:0/3478898699 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a8f74ab890 0x55a8f74e6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:11.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.468+0000 7f3754036000 1 --2- 172.21.15.90:0/3478898699 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a8f74e4f70 0x55a8f74ab350 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:11.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.468+0000 7f3754036000 1 -- 172.21.15.90:0/3478898699 >> 172.21.15.90:0/3478898699 conn(0x55a8f74abff0 msgr2=0x55a8f74e4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:11.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.468+0000 7f3754036000 1 -- 172.21.15.90:0/3478898699 shutdown_connections 2024-03-11T18:36:11.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.468+0000 7f3754036000 1 -- 172.21.15.90:0/3478898699 wait complete. 2024-03-11T18:36:11.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.468+0000 7f3754036000 1 -- start start 2024-03-11T18:36:11.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.468+0000 7f3754036000 1 Processor -- start 2024-03-11T18:36:11.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.468+0000 7f3754036000 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55a8f74ab890 0x55a8f74f1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:11.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.468+0000 7f3754036000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a8f74e4f70 0x55a8f74f20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:11.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.468+0000 7f3754036000 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a8f74e6bc0 0x55a8f74aa720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:11.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.468+0000 7f3754036000 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8f72f8140 con 0x55a8f74e6bc0 2024-03-11T18:36:11.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.468+0000 7f3754036000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8f7292630 con 0x55a8f74e4f70 2024-03-11T18:36:11.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.468+0000 7f3754036000 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a8f72922d0 con 0x55a8f74ab890 2024-03-11T18:36:11.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.468+0000 7f3754036000 10 client.0 _ll_register_callbacks cb 0x55a8f73a81c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T18:36:11.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.472+0000 7f374ad9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a8f74e6bc0 0x55a8f74aa720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:11.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.472+0000 7f374ad9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a8f74e6bc0 0x55a8f74aa720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55516/0 (socket says 172.21.15.90:55516) 2024-03-11T18:36:11.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.472+0000 7f374ad9d640 1 -- 172.21.15.90:0/16529371 learned_addr learned my addr 172.21.15.90:0/16529371 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:11.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.476+0000 7f374ad9d640 1 -- 172.21.15.90:0/16529371 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55a8f74ab890 msgr2=0x55a8f74f1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:11.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.476+0000 7f374ad9d640 1 --2- 172.21.15.90:0/16529371 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55a8f74ab890 0x55a8f74f1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:11.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.476+0000 7f374ad9d640 1 -- 172.21.15.90:0/16529371 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a8f74e4f70 msgr2=0x55a8f74f20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:11.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.476+0000 7f374ad9d640 1 --2- 172.21.15.90:0/16529371 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a8f74e4f70 0x55a8f74f20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:11.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.476+0000 7f374ad9d640 1 -- 172.21.15.90:0/16529371 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3744009c50 con 0x55a8f74e6bc0 2024-03-11T18:36:11.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.476+0000 7f374ad9d640 1 --2- 172.21.15.90:0/16529371 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a8f74e6bc0 0x55a8f74aa720 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3744000f90 tx=0x7f3744004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=372bbee75556614f server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:11.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.476+0000 7f37297fa640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-03-11T18:36:11.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.480+0000 7f37297fa640 1 -- 172.21.15.90:0/16529371 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374400e040 con 0x55a8f74e6bc0 2024-03-11T18:36:11.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.480+0000 7f37297fa640 1 -- 172.21.15.90:0/16529371 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3744004450 con 0x55a8f74e6bc0 2024-03-11T18:36:11.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.480+0000 7f3754036000 1 -- 172.21.15.90:0/16529371 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a8f72ab850 con 0x55a8f74e6bc0 2024-03-11T18:36:11.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.484+0000 7f37297fa640 1 -- 172.21.15.90:0/16529371 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3744004650 con 0x55a8f74e6bc0 2024-03-11T18:36:11.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.484+0000 7f3754036000 10 client.4314 fetch_fsmap learned FSMap version 7 2024-03-11T18:36:11.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.484+0000 7f3754036000 1 -- 172.21.15.90:0/16529371 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55a8f72b7e70 con 0x55a8f74e6bc0 2024-03-11T18:36:11.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.488+0000 7f37297fa640 1 -- 172.21.15.90:0/16529371 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a8f72ab850 con 0x55a8f74e6bc0 2024-03-11T18:36:11.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.492+0000 7f37297fa640 1 -- 172.21.15.90:0/16529371 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 7) v1 ==== 4452+0+0 (secure 0 0 0) 0x7f3744014040 con 0x55a8f74e6bc0 2024-03-11T18:36:11.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.492+0000 7f3754036000 10 client.4314 fetch_fsmap finished waiting for FSMap version 7 2024-03-11T18:36:11.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.492+0000 7f3754036000 10 client.4314 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-11T18:36:11.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.492+0000 7f3754036000 20 client.4314 populate_metadata read hostname 'smithi090' 2024-03-11T18:36:11.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.492+0000 7f3754036000 1 --2- 172.21.15.90:0/16529371 >> [v2:172.21.15.90:6836/3485943375,v1:172.21.15.90:6837/3485943375] conn(0x55a8f74eff60 0x55a8f74f0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:11.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.492+0000 7f3754036000 4 client.4314 mds_command: new command op to 4255 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T18:36:11.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.492+0000 7f3754036000 1 -- 172.21.15.90:0/16529371 --> [v2:172.21.15.90:6836/3485943375,v1:172.21.15.90:6837/3485943375] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a8f72f4f20 con 0x55a8f74eff60 2024-03-11T18:36:11.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.500+0000 7f374a59c640 1 --2- 172.21.15.90:0/16529371 >> [v2:172.21.15.90:6836/3485943375,v1:172.21.15.90:6837/3485943375] conn(0x55a8f74eff60 0x55a8f74f0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:11.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.616+0000 7f37297fa640 10 client.4314 ms_handle_connect on v2:172.21.15.90:6836/3485943375 2024-03-11T18:36:11.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.616+0000 7f374a59c640 1 --2- 172.21.15.90:0/16529371 >> [v2:172.21.15.90:6836/3485943375,v1:172.21.15.90:6837/3485943375] conn(0x55a8f74eff60 0x55a8f74f0340 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=124b7cf9eac662af server_cookie=3ce450d7ef5ea832 in_seq=0 out_seq=0 2024-03-11T18:36:11.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.844+0000 7f37297fa640 1 -- 172.21.15.90:0/16529371 <== mds.0 v2:172.21.15.90:6836/3485943375 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a8f72f4f20 con 0x55a8f74eff60 2024-03-11T18:36:11.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.844+0000 7f37297fa640 10 client.4314 handle_command_reply: tid=0 2024-03-11T18:36:11.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.844+0000 7f3754036000 1 client.4314 shutdown 2024-03-11T18:36:11.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.844+0000 7f3754036000 1 -- 172.21.15.90:0/16529371 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a8f74e6bc0 msgr2=0x55a8f74aa720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:11.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.844+0000 7f3754036000 1 --2- 172.21.15.90:0/16529371 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a8f74e6bc0 0x55a8f74aa720 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3744000f90 tx=0x7f3744004a50 comp rx=0 tx=0).stop 2024-03-11T18:36:11.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.844+0000 7f3754036000 1 -- 172.21.15.90:0/16529371 shutdown_connections 2024-03-11T18:36:11.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.844+0000 7f3754036000 1 --2- 172.21.15.90:0/16529371 >> [v2:172.21.15.90:6836/3485943375,v1:172.21.15.90:6837/3485943375] conn(0x55a8f74eff60 0x55a8f74f0340 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:11.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.844+0000 7f3754036000 1 --2- 172.21.15.90:0/16529371 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a8f74e6bc0 0x55a8f74aa720 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:11.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.844+0000 7f3754036000 1 --2- 172.21.15.90:0/16529371 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a8f74e4f70 0x55a8f74f20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:11.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.844+0000 7f37297fa640 0 client.4314 ms_handle_reset on v2:172.21.15.90:6836/3485943375 2024-03-11T18:36:11.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.844+0000 7f3754036000 1 --2- 172.21.15.90:0/16529371 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55a8f74ab890 0x55a8f74f1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:11.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.844+0000 7f3754036000 1 -- 172.21.15.90:0/16529371 >> 172.21.15.90:0/16529371 conn(0x55a8f74abff0 msgr2=0x55a8f74e4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:11.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.844+0000 7f3754036000 1 -- 172.21.15.90:0/16529371 shutdown_connections 2024-03-11T18:36:11.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.844+0000 7f3754036000 1 -- 172.21.15.90:0/16529371 wait complete. 2024-03-11T18:36:11.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.844+0000 7f3754036000 20 client.4314 trim_cache size 0 max 16384 2024-03-11T18:36:11.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.880+0000 7f3754036000 1 Processor -- start 2024-03-11T18:36:11.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.880+0000 7f3754036000 1 -- start start 2024-03-11T18:36:11.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.880+0000 7f3754036000 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55a8f74ab890 0x55a8f74e7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:11.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.880+0000 7f3754036000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a8f74e4f70 0x55a8f74e5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:11.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.880+0000 7f3754036000 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a8f74e6bc0 0x55a8f750c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:11.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.880+0000 7f3754036000 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8f6faf940 con 0x55a8f74e6bc0 2024-03-11T18:36:11.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.880+0000 7f3754036000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8f72f5950 con 0x55a8f74e4f70 2024-03-11T18:36:11.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.880+0000 7f3754036000 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a8f7297f10 con 0x55a8f74ab890 2024-03-11T18:36:11.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.884+0000 7f3741d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a8f74e4f70 0x55a8f74e5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:11.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.884+0000 7f374a59c640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55a8f74ab890 0x55a8f74e7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:11.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.884+0000 7f3741d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a8f74e4f70 0x55a8f74e5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.90:50460/0 (socket says 172.21.15.90:50460) 2024-03-11T18:36:11.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.884+0000 7f3741d9b640 1 -- 172.21.15.90:0/357565274 learned_addr learned my addr 172.21.15.90:0/357565274 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:11.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.888+0000 7f374ad9d640 1 --2- 172.21.15.90:0/357565274 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a8f74e6bc0 0x55a8f750c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:11.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.888+0000 7f374ad9d640 1 -- 172.21.15.90:0/357565274 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55a8f74ab890 msgr2=0x55a8f74e7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:11.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.888+0000 7f374ad9d640 1 --2- 172.21.15.90:0/357565274 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55a8f74ab890 0x55a8f74e7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:11.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.888+0000 7f374ad9d640 1 -- 172.21.15.90:0/357565274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a8f74e4f70 msgr2=0x55a8f74e5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:11.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.888+0000 7f374ad9d640 1 --2- 172.21.15.90:0/357565274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a8f74e4f70 0x55a8f74e5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:11.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.888+0000 7f374ad9d640 1 -- 172.21.15.90:0/357565274 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8f729de10 con 0x55a8f74e6bc0 2024-03-11T18:36:11.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.892+0000 7f374ad9d640 1 --2- 172.21.15.90:0/357565274 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a8f74e6bc0 0x55a8f750c740 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3744009c20 tx=0x7f3744021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8c34052925e7fe server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:11.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.896+0000 7f372b7fe640 1 -- 172.21.15.90:0/357565274 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374400e040 con 0x55a8f74e6bc0 2024-03-11T18:36:11.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.896+0000 7f372b7fe640 1 -- 172.21.15.90:0/357565274 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3744004250 con 0x55a8f74e6bc0 2024-03-11T18:36:11.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.896+0000 7f372b7fe640 1 -- 172.21.15.90:0/357565274 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3744004600 con 0x55a8f74e6bc0 2024-03-11T18:36:11.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.900+0000 7f3754036000 1 -- 172.21.15.90:0/357565274 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a8f74e6bc0 msgr2=0x55a8f750c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:11.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.900+0000 7f3754036000 1 --2- 172.21.15.90:0/357565274 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a8f74e6bc0 0x55a8f750c740 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3744009c20 tx=0x7f3744021000 comp rx=0 tx=0).stop 2024-03-11T18:36:11.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.900+0000 7f3754036000 1 -- 172.21.15.90:0/357565274 shutdown_connections 2024-03-11T18:36:11.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.900+0000 7f3754036000 1 --2- 172.21.15.90:0/357565274 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a8f74e6bc0 0x55a8f750c740 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:11.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.900+0000 7f3754036000 1 --2- 172.21.15.90:0/357565274 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a8f74e4f70 0x55a8f74e5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:11.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.900+0000 7f3754036000 1 --2- 172.21.15.90:0/357565274 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55a8f74ab890 0x55a8f74e7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:11.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.900+0000 7f3754036000 1 -- 172.21.15.90:0/357565274 >> 172.21.15.90:0/357565274 conn(0x55a8f74abff0 msgr2=0x55a8f74e5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:11.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.900+0000 7f3754036000 1 -- 172.21.15.90:0/357565274 shutdown_connections 2024-03-11T18:36:11.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.900+0000 7f3754036000 1 -- 172.21.15.90:0/357565274 wait complete. 2024-03-11T18:36:11.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.900+0000 7f3754036000 1 -- start start 2024-03-11T18:36:11.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.900+0000 7f3754036000 1 Processor -- start 2024-03-11T18:36:11.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.900+0000 7f3754036000 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a8f74ab890 0x55a8f754fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:11.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.900+0000 7f3754036000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a8f74e4f70 0x55a8f7550230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:11.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.900+0000 7f3754036000 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55a8f74e6bc0 0x55a8f7546740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:11.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.900+0000 7f3754036000 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8f72f5540 con 0x55a8f74ab890 2024-03-11T18:36:11.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.900+0000 7f3754036000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8f72c2c80 con 0x55a8f74e4f70 2024-03-11T18:36:11.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.900+0000 7f3754036000 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a8f6fef1b0 con 0x55a8f74e6bc0 2024-03-11T18:36:11.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.900+0000 7f3754036000 10 client.0 _ll_register_callbacks cb 0x55a8f72fa2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T18:36:11.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.900+0000 7f374a59c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a8f74ab890 0x55a8f754fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:11.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.900+0000 7f374a59c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a8f74ab890 0x55a8f754fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55532/0 (socket says 172.21.15.90:55532) 2024-03-11T18:36:11.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.900+0000 7f374a59c640 1 -- 172.21.15.90:0/667695654 learned_addr learned my addr 172.21.15.90:0/667695654 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:11.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.904+0000 7f3741d9b640 1 --2- 172.21.15.90:0/667695654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a8f74e4f70 0x55a8f7550230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:11.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.904+0000 7f374ad9d640 1 --2- 172.21.15.90:0/667695654 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55a8f74e6bc0 0x55a8f7546740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:11.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.904+0000 7f374a59c640 1 -- 172.21.15.90:0/667695654 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55a8f74e6bc0 msgr2=0x55a8f7546740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:11.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.904+0000 7f374a59c640 1 --2- 172.21.15.90:0/667695654 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55a8f74e6bc0 0x55a8f7546740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:11.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.904+0000 7f374a59c640 1 -- 172.21.15.90:0/667695654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a8f74e4f70 msgr2=0x55a8f7550230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:11.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.904+0000 7f374a59c640 1 --2- 172.21.15.90:0/667695654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a8f74e4f70 0x55a8f7550230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:11.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.904+0000 7f374a59c640 1 -- 172.21.15.90:0/667695654 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a8f72b7e70 con 0x55a8f74ab890 2024-03-11T18:36:11.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.908+0000 7f374a59c640 1 --2- 172.21.15.90:0/667695654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a8f74ab890 0x55a8f754fcf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f373c01c110 tx=0x7f373c019080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2612115bc01b814c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:11.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.908+0000 7f37297fa640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-03-11T18:36:11.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.908+0000 7f37297fa640 1 -- 172.21.15.90:0/667695654 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373c0c0040 con 0x55a8f74ab890 2024-03-11T18:36:11.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.912+0000 7f3754036000 1 -- 172.21.15.90:0/667695654 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a8f727de80 con 0x55a8f74ab890 2024-03-11T18:36:11.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.912+0000 7f37297fa640 1 -- 172.21.15.90:0/667695654 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f373c0cfe20 con 0x55a8f74ab890 2024-03-11T18:36:11.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.912+0000 7f37297fa640 1 -- 172.21.15.90:0/667695654 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f373c003a70 con 0x55a8f74ab890 2024-03-11T18:36:11.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.912+0000 7f3754036000 10 client.4320 fetch_fsmap learned FSMap version 7 2024-03-11T18:36:11.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.912+0000 7f3754036000 1 -- 172.21.15.90:0/667695654 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55a8f6feecc0 con 0x55a8f74ab890 2024-03-11T18:36:11.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.920+0000 7f37297fa640 1 -- 172.21.15.90:0/667695654 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a8f727de80 con 0x55a8f74ab890 2024-03-11T18:36:11.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.920+0000 7f37297fa640 1 -- 172.21.15.90:0/667695654 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 7) v1 ==== 4452+0+0 (secure 0 0 0) 0x7f373c0b6070 con 0x55a8f74ab890 2024-03-11T18:36:11.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.920+0000 7f3754036000 10 client.4320 fetch_fsmap finished waiting for FSMap version 7 2024-03-11T18:36:11.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.920+0000 7f3754036000 10 client.4320 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-11T18:36:11.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.920+0000 7f3754036000 20 client.4320 populate_metadata read hostname 'smithi090' 2024-03-11T18:36:11.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.920+0000 7f3754036000 1 --2- 172.21.15.90:0/667695654 >> [v2:172.21.15.90:6836/3485943375,v1:172.21.15.90:6837/3485943375] conn(0x55a8f754c7f0 0x55a8f754cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:11.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.920+0000 7f3754036000 4 client.4320 mds_command: new command op to 4255 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-11T18:36:11.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.920+0000 7f3754036000 1 -- 172.21.15.90:0/667695654 --> [v2:172.21.15.90:6836/3485943375,v1:172.21.15.90:6837/3485943375] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a8f72c0fa0 con 0x55a8f754c7f0 2024-03-11T18:36:11.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.920+0000 7f3741d9b640 1 --2- 172.21.15.90:0/667695654 >> [v2:172.21.15.90:6836/3485943375,v1:172.21.15.90:6837/3485943375] conn(0x55a8f754c7f0 0x55a8f754cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:11.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.924+0000 7f3741d9b640 1 --2- 172.21.15.90:0/667695654 >> [v2:172.21.15.90:6836/3485943375,v1:172.21.15.90:6837/3485943375] conn(0x55a8f754c7f0 0x55a8f754cbd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57c8bc7d187bd3c6 server_cookie=9fb61ed3dc6dd4a6 in_seq=0 out_seq=0 2024-03-11T18:36:11.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.924+0000 7f37297fa640 10 client.4320 ms_handle_connect on v2:172.21.15.90:6836/3485943375 2024-03-11T18:36:11.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.944+0000 7f37297fa640 1 -- 172.21.15.90:0/667695654 <== mds.0 v2:172.21.15.90:6836/3485943375 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+318 (crc 0 0 0) 0x55a8f72c0fa0 con 0x55a8f754c7f0 2024-03-11T18:36:11.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.944+0000 7f37297fa640 10 client.4320 handle_command_reply: tid=0 2024-03-11T18:36:11.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.944+0000 7f3754036000 1 client.4320 shutdown 2024-03-11T18:36:11.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.944+0000 7f3754036000 1 -- 172.21.15.90:0/667695654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a8f74ab890 msgr2=0x55a8f754fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:11.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.944+0000 7f3754036000 1 --2- 172.21.15.90:0/667695654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a8f74ab890 0x55a8f754fcf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f373c01c110 tx=0x7f373c019080 comp rx=0 tx=0).stop 2024-03-11T18:36:11.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.944+0000 7f3754036000 1 -- 172.21.15.90:0/667695654 shutdown_connections 2024-03-11T18:36:11.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.944+0000 7f3754036000 1 --2- 172.21.15.90:0/667695654 >> [v2:172.21.15.90:6836/3485943375,v1:172.21.15.90:6837/3485943375] conn(0x55a8f754c7f0 0x55a8f754cbd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:11.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.944+0000 7f3754036000 1 --2- 172.21.15.90:0/667695654 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55a8f74e6bc0 0x55a8f7546740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:11.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.944+0000 7f3754036000 1 --2- 172.21.15.90:0/667695654 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55a8f74e4f70 0x55a8f7550230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:11.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.944+0000 7f3754036000 1 --2- 172.21.15.90:0/667695654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a8f74ab890 0x55a8f754fcf0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:11.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.944+0000 7f37297fa640 0 client.4320 ms_handle_reset on v2:172.21.15.90:6836/3485943375 2024-03-11T18:36:11.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.944+0000 7f3754036000 1 -- 172.21.15.90:0/667695654 >> 172.21.15.90:0/667695654 conn(0x55a8f74abff0 msgr2=0x55a8f74e5be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:11.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.944+0000 7f3754036000 1 -- 172.21.15.90:0/667695654 shutdown_connections 2024-03-11T18:36:11.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.944+0000 7f3754036000 1 -- 172.21.15.90:0/667695654 wait complete. 2024-03-11T18:36:11.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.944+0000 7f3754036000 20 client.4320 trim_cache size 0 max 16384 2024-03-11T18:36:11.953 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-03-11T18:36:11.953 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "76d56b10-6c29-476a-804b-8482ea90f9e2", 2024-03-11T18:36:11.953 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 0, 2024-03-11T18:36:11.953 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 4255, 2024-03-11T18:36:11.953 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-03-11T18:36:11.953 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-03-11T18:36:11.953 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-03-11T18:36:11.953 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 12.717568069, 2024-03-11T18:36:11.953 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 7, 2024-03-11T18:36:11.954 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 28, 2024-03-11T18:36:11.954 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 28, 2024-03-11T18:36:11.954 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 30.601609817 2024-03-11T18:36:11.954 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-03-11T18:36:11.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.944+0000 7f3750ecd640 1 -- 172.21.15.90:0/381523617 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f372c066e90 msgr2=0x7f372c069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:11.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.944+0000 7f3750ecd640 1 --2- 172.21.15.90:0/381523617 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f372c066e90 0x7f372c069350 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f373c0aa000 tx=0x7f373c005040 comp rx=0 tx=0).stop 2024-03-11T18:36:11.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.944+0000 7f3750ecd640 1 -- 172.21.15.90:0/381523617 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f374c1f1aa0 msgr2=0x7f374c06c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:11.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.944+0000 7f3750ecd640 1 --2- 172.21.15.90:0/381523617 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f374c1f1aa0 0x7f374c06c3c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f373400c680 tx=0x7f373400cb40 comp rx=0 tx=0).stop 2024-03-11T18:36:11.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.944+0000 7f3750ecd640 1 -- 172.21.15.90:0/381523617 shutdown_connections 2024-03-11T18:36:11.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.944+0000 7f3750ecd640 1 --2- 172.21.15.90:0/381523617 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f372c066e90 0x7f372c069350 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:11.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.944+0000 7f3750ecd640 1 --2- 172.21.15.90:0/381523617 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f374c147510 0x7f374c13eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:11.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.944+0000 7f3750ecd640 1 --2- 172.21.15.90:0/381523617 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f374c1f1aa0 0x7f374c06c3c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:11.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.944+0000 7f3750ecd640 1 --2- 172.21.15.90:0/381523617 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3738005b90 0x7f374c14afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:11.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.944+0000 7f3750ecd640 1 -- 172.21.15.90:0/381523617 >> 172.21.15.90:0/381523617 conn(0x7f374c071aa0 msgr2=0x7f374c0c6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:11.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.948+0000 7f3750ecd640 1 -- 172.21.15.90:0/381523617 shutdown_connections 2024-03-11T18:36:11.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:11.948+0000 7f3750ecd640 1 -- 172.21.15.90:0/381523617 wait complete. 2024-03-11T18:36:11.977 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-11T18:36:11.977 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-11T18:36:11.977 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-11T18:36:12.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.092+0000 7f8495daa640 1 Processor -- start 2024-03-11T18:36:12.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.092+0000 7f8495daa640 1 -- start start 2024-03-11T18:36:12.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.096+0000 7f8495daa640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f8490151d50 0x7f84900c3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:12.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.096+0000 7f8495daa640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f849006b860 0x7f84900c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:12.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.096+0000 7f8495daa640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8490157970 0x7f84900c5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:12.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.096+0000 7f8495daa640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84900c7430 con 0x7f8490151d50 2024-03-11T18:36:12.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.096+0000 7f8495daa640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8490152130 con 0x7f849006b860 2024-03-11T18:36:12.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.096+0000 7f8495daa640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84900c7a40 con 0x7f8490157970 2024-03-11T18:36:12.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.096+0000 7f848effd640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f849006b860 0x7f84900c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:12.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.096+0000 7f848ffff640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8490157970 0x7f84900c5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:12.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.096+0000 7f848effd640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f849006b860 0x7f84900c95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.90:50476/0 (socket says 172.21.15.90:50476) 2024-03-11T18:36:12.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.096+0000 7f848effd640 1 -- 172.21.15.90:0/1164072924 learned_addr learned my addr 172.21.15.90:0/1164072924 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:12.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.100+0000 7f848f7fe640 1 --2- 172.21.15.90:0/1164072924 >> v2:172.21.15.90:3300/0 conn(0x7f8490151d50 0x7f84900c3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:12.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.104+0000 7f848effd640 1 -- 172.21.15.90:0/1164072924 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8490157970 msgr2=0x7f84900c5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:12.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.104+0000 7f848effd640 1 --2- 172.21.15.90:0/1164072924 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8490157970 0x7f84900c5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:12.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.104+0000 7f848effd640 1 -- 172.21.15.90:0/1164072924 >> v2:172.21.15.90:3300/0 conn(0x7f8490151d50 msgr2=0x7f84900c3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:12.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.104+0000 7f848effd640 1 --2- 172.21.15.90:0/1164072924 >> v2:172.21.15.90:3300/0 conn(0x7f8490151d50 0x7f84900c3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:12.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.104+0000 7f848effd640 1 -- 172.21.15.90:0/1164072924 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84900c6e70 con 0x7f849006b860 2024-03-11T18:36:12.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.108+0000 7f848effd640 1 --2- 172.21.15.90:0/1164072924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f849006b860 0x7f84900c95f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8490074ac0 tx=0x7f847801ed10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=801991121ca27425 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:12.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.120+0000 7f848e7fc640 1 -- 172.21.15.90:0/1164072924 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84780a89d0 con 0x7f849006b860 2024-03-11T18:36:12.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.120+0000 7f848e7fc640 1 -- 172.21.15.90:0/1164072924 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84780a8b30 con 0x7f849006b860 2024-03-11T18:36:12.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.120+0000 7f848e7fc640 1 -- 172.21.15.90:0/1164072924 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84780b0440 con 0x7f849006b860 2024-03-11T18:36:12.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.120+0000 7f8495daa640 1 -- 172.21.15.90:0/1164072924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f849006b860 msgr2=0x7f84900c95f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:12.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.120+0000 7f8495daa640 1 --2- 172.21.15.90:0/1164072924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f849006b860 0x7f84900c95f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8490074ac0 tx=0x7f847801ed10 comp rx=0 tx=0).stop 2024-03-11T18:36:12.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.120+0000 7f8495daa640 1 -- 172.21.15.90:0/1164072924 shutdown_connections 2024-03-11T18:36:12.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.120+0000 7f8495daa640 1 --2- 172.21.15.90:0/1164072924 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8490157970 0x7f84900c5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:12.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.120+0000 7f8495daa640 1 --2- 172.21.15.90:0/1164072924 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f849006b860 0x7f84900c95f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:12.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.120+0000 7f8495daa640 1 --2- 172.21.15.90:0/1164072924 >> v2:172.21.15.90:3300/0 conn(0x7f8490151d50 0x7f84900c3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:12.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.120+0000 7f8495daa640 1 -- 172.21.15.90:0/1164072924 >> 172.21.15.90:0/1164072924 conn(0x7f8490071aa0 msgr2=0x7f8490071eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:12.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.120+0000 7f8495daa640 1 -- 172.21.15.90:0/1164072924 shutdown_connections 2024-03-11T18:36:12.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.120+0000 7f8495daa640 1 -- 172.21.15.90:0/1164072924 wait complete. 2024-03-11T18:36:12.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.120+0000 7f8495daa640 1 Processor -- start 2024-03-11T18:36:12.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.120+0000 7f8495daa640 1 -- start start 2024-03-11T18:36:12.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.120+0000 7f8495daa640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f849006b860 0x7f84901ed7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:12.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.120+0000 7f8495daa640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8490151d50 0x7f8490145de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:12.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.120+0000 7f8495daa640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8490157970 0x7f8490158020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:12.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.120+0000 7f8495daa640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84900ca530 con 0x7f8490151d50 2024-03-11T18:36:12.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.120+0000 7f8495daa640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84900ca810 con 0x7f8490157970 2024-03-11T18:36:12.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.120+0000 7f8495daa640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84900caaf0 con 0x7f849006b860 2024-03-11T18:36:12.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.124+0000 7f848effd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8490151d50 0x7f8490145de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:12.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.124+0000 7f848effd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8490151d50 0x7f8490145de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55550/0 (socket says 172.21.15.90:55550) 2024-03-11T18:36:12.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.124+0000 7f848effd640 1 -- 172.21.15.90:0/3142744816 learned_addr learned my addr 172.21.15.90:0/3142744816 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:12.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.124+0000 7f848f7fe640 1 --2- 172.21.15.90:0/3142744816 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f849006b860 0x7f84901ed7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:12.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.124+0000 7f848ffff640 1 --2- 172.21.15.90:0/3142744816 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8490157970 0x7f8490158020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:12.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.128+0000 7f848effd640 1 -- 172.21.15.90:0/3142744816 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f849006b860 msgr2=0x7f84901ed7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:12.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.128+0000 7f848effd640 1 --2- 172.21.15.90:0/3142744816 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f849006b860 0x7f84901ed7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:12.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.128+0000 7f848effd640 1 -- 172.21.15.90:0/3142744816 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8490157970 msgr2=0x7f8490158020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:12.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.128+0000 7f848effd640 1 --2- 172.21.15.90:0/3142744816 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8490157970 0x7f8490158020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:12.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.128+0000 7f848effd640 1 -- 172.21.15.90:0/3142744816 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84780094d0 con 0x7f8490151d50 2024-03-11T18:36:12.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.128+0000 7f848effd640 1 --2- 172.21.15.90:0/3142744816 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8490151d50 0x7f8490145de0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f847801d3c0 tx=0x7f847801bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:12.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.132+0000 7f8487fff640 1 -- 172.21.15.90:0/3142744816 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84780ba600 con 0x7f8490151d50 2024-03-11T18:36:12.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.132+0000 7f8487fff640 1 -- 172.21.15.90:0/3142744816 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84780ba760 con 0x7f8490151d50 2024-03-11T18:36:12.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.132+0000 7f8487fff640 1 -- 172.21.15.90:0/3142744816 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84780bab10 con 0x7f8490151d50 2024-03-11T18:36:12.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.132+0000 7f8495daa640 1 -- 172.21.15.90:0/3142744816 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84900cadd0 con 0x7f8490151d50 2024-03-11T18:36:12.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.132+0000 7f8495daa640 1 -- 172.21.15.90:0/3142744816 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84900cb390 con 0x7f8490151d50 2024-03-11T18:36:12.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.136+0000 7f8495daa640 1 -- 172.21.15.90:0/3142744816 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f849004eef0 con 0x7f8490151d50 2024-03-11T18:36:12.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.156+0000 7f8487fff640 1 -- 172.21.15.90:0/3142744816 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f84780bac70 con 0x7f8490151d50 2024-03-11T18:36:12.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.156+0000 7f8487fff640 1 --2- 172.21.15.90:0/3142744816 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f8470066e90 0x7f8470069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:12.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.156+0000 7f848f7fe640 1 --2- 172.21.15.90:0/3142744816 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f8470066e90 0x7f8470069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:12.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.156+0000 7f848f7fe640 1 --2- 172.21.15.90:0/3142744816 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f8470066e90 0x7f8470069350 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f84800045f0 tx=0x7f8480008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:12.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.156+0000 7f8487fff640 1 -- 172.21.15.90:0/3142744816 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f84780f4c60 con 0x7f8490151d50 2024-03-11T18:36:12.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.408+0000 7f8487fff640 1 -- 172.21.15.90:0/3142744816 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f84780f0800 con 0x7f8490151d50 2024-03-11T18:36:12.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.608+0000 7f8495daa640 1 -- 172.21.15.90:0/3142744816 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f84900c77e0 con 0x7f8490151d50 2024-03-11T18:36:12.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.636+0000 7f8487fff640 1 -- 172.21.15.90:0/3142744816 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7f84780b5030 con 0x7f8490151d50 2024-03-11T18:36:12.642 INFO:teuthology.orchestra.run.smithi090.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-11T18:36:12.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.640+0000 7f8495daa640 1 -- 172.21.15.90:0/3142744816 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f8470066e90 msgr2=0x7f8470069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:12.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.640+0000 7f8495daa640 1 --2- 172.21.15.90:0/3142744816 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f8470066e90 0x7f8470069350 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f84800045f0 tx=0x7f8480008040 comp rx=0 tx=0).stop 2024-03-11T18:36:12.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.640+0000 7f8495daa640 1 -- 172.21.15.90:0/3142744816 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8490151d50 msgr2=0x7f8490145de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:12.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.640+0000 7f8495daa640 1 --2- 172.21.15.90:0/3142744816 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8490151d50 0x7f8490145de0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f847801d3c0 tx=0x7f847801bab0 comp rx=0 tx=0).stop 2024-03-11T18:36:12.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.640+0000 7f8495daa640 1 -- 172.21.15.90:0/3142744816 shutdown_connections 2024-03-11T18:36:12.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.640+0000 7f8495daa640 1 --2- 172.21.15.90:0/3142744816 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f8470066e90 0x7f8470069350 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:12.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.640+0000 7f8495daa640 1 --2- 172.21.15.90:0/3142744816 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8490157970 0x7f8490158020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:12.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.640+0000 7f8495daa640 1 --2- 172.21.15.90:0/3142744816 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8490151d50 0x7f8490145de0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:12.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.640+0000 7f8495daa640 1 --2- 172.21.15.90:0/3142744816 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f849006b860 0x7f84901ed7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:12.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.640+0000 7f8495daa640 1 -- 172.21.15.90:0/3142744816 >> 172.21.15.90:0/3142744816 conn(0x7f8490071aa0 msgr2=0x7f84900c5ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:12.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.640+0000 7f8495daa640 1 -- 172.21.15.90:0/3142744816 shutdown_connections 2024-03-11T18:36:12.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:12.640+0000 7f8495daa640 1 -- 172.21.15.90:0/3142744816 wait complete. 2024-03-11T18:36:12.662 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-11T18:36:12.662 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-11T18:36:12.662 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi090.front.sepia.ceph.com 2024-03-11T18:36:12.662 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-03-11T18:36:12.662 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-11T18:36:12.662 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-11T18:36:12.663 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-11T18:36:12.663 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-11T18:36:12.663 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-03-11T18:36:12.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T18:36:12.663 DEBUG:teuthology.orchestra.run.smithi090:> ip addr 2024-03-11T18:36:12.668 INFO:teuthology.orchestra.run.smithi090.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-11T18:36:12.668 INFO:teuthology.orchestra.run.smithi090.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-11T18:36:12.668 INFO:teuthology.orchestra.run.smithi090.stdout: inet 127.0.0.1/8 scope host lo 2024-03-11T18:36:12.668 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-03-11T18:36:12.668 INFO:teuthology.orchestra.run.smithi090.stdout: inet6 ::1/128 scope host 2024-03-11T18:36:12.669 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-03-11T18:36:12.669 INFO:teuthology.orchestra.run.smithi090.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-11T18:36:12.669 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:d8:a1:a8 brd ff:ff:ff:ff:ff:ff 2024-03-11T18:36:12.669 INFO:teuthology.orchestra.run.smithi090.stdout: altname enp7s0f0 2024-03-11T18:36:12.669 INFO:teuthology.orchestra.run.smithi090.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-11T18:36:12.669 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:d8:a1:a9 brd ff:ff:ff:ff:ff:ff 2024-03-11T18:36:12.669 INFO:teuthology.orchestra.run.smithi090.stdout: altname enp7s0f1 2024-03-11T18:36:12.669 INFO:teuthology.orchestra.run.smithi090.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-11T18:36:12.669 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:8f:d7:66 brd ff:ff:ff:ff:ff:ff 2024-03-11T18:36:12.669 INFO:teuthology.orchestra.run.smithi090.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-11T18:36:12.669 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:8f:d7:67 brd ff:ff:ff:ff:ff:ff 2024-03-11T18:36:12.669 INFO:teuthology.orchestra.run.smithi090.stdout: inet 172.21.15.90/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-11T18:36:12.670 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft 42121sec preferred_lft 42121sec 2024-03-11T18:36:12.670 INFO:teuthology.orchestra.run.smithi090.stdout: inet6 fe80::ec4:7aff:fe8f:d767/64 scope link 2024-03-11T18:36:12.670 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-03-11T18:36:12.670 INFO:teuthology.orchestra.run.smithi090.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-11T18:36:12.670 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 02:42:26:17:e3:13 brd ff:ff:ff:ff:ff:ff 2024-03-11T18:36:12.670 INFO:teuthology.orchestra.run.smithi090.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-11T18:36:12.670 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-03-11T18:36:12.671 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-11T18:36:12.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T18:36:12.671 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec bash -c ' 2024-03-11T18:36:12.671 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-03-11T18:36:12.671 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link add name ceph-brx type bridge 2024-03-11T18:36:12.671 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip addr flush dev ceph-brx 2024-03-11T18:36:12.671 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link set ceph-brx up 2024-03-11T18:36:12.671 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-11T18:36:12.671 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-03-11T18:36:12.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T18:36:12.749 DEBUG:teuthology.orchestra.run.smithi090:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-11T18:36:12.801 INFO:teuthology.orchestra.run.smithi090.stdout:1 2024-03-11T18:36:12.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T18:36:12.802 DEBUG:teuthology.orchestra.run.smithi090:> route 2024-03-11T18:36:12.886 INFO:teuthology.orchestra.run.smithi090.stdout:Kernel IP routing table 2024-03-11T18:36:12.886 INFO:teuthology.orchestra.run.smithi090.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-11T18:36:12.886 INFO:teuthology.orchestra.run.smithi090.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-11T18:36:12.886 INFO:teuthology.orchestra.run.smithi090.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-11T18:36:12.886 INFO:teuthology.orchestra.run.smithi090.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-11T18:36:12.886 INFO:teuthology.orchestra.run.smithi090.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-11T18:36:12.886 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-11T18:36:12.887 INFO:teuthology.orchestra.run.smithi090.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-11T18:36:12.887 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-11T18:36:12.887 INFO:teuthology.orchestra.run.smithi090.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-11T18:36:12.887 INFO:teuthology.orchestra.run.smithi090.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-11T18:36:12.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T18:36:12.887 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec bash -c ' 2024-03-11T18:36:12.887 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-03-11T18:36:12.887 DEBUG:teuthology.orchestra.run.smithi090:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-11T18:36:12.887 DEBUG:teuthology.orchestra.run.smithi090:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-11T18:36:12.888 DEBUG:teuthology.orchestra.run.smithi090:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-11T18:36:12.888 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-03-11T18:36:12.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T18:36:12.960 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list 2024-03-11T18:36:13.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T18:36:13.006 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list-id 2024-03-11T18:36:13.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T18:36:13.053 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec bash -c ' 2024-03-11T18:36:13.053 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-03-11T18:36:13.053 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-11T18:36:13.054 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-03-11T18:36:13.054 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-03-11T18:36:13.127 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-03-11T18:36:13.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T18:36:13.127 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec bash -c ' 2024-03-11T18:36:13.127 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-03-11T18:36:13.127 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-03-11T18:36:13.128 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-11T18:36:13.128 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-03-11T18:36:13.128 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-03-11T18:36:13.128 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-03-11T18:36:13.128 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-03-11T18:36:13.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T18:36:13.238 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec bash -c ' 2024-03-11T18:36:13.238 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-03-11T18:36:13.238 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link set brx.0 up 2024-03-11T18:36:13.238 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link set dev brx.0 master ceph-brx 2024-03-11T18:36:13.238 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-03-11T18:36:13.257 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-03-11T18:36:13.257 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-11T18:36:13.257 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-03-11T18:36:13.304 INFO:teuthology.orchestra.run.smithi090.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-03-11T18:36:13.305 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-11T18:36:13.305 DEBUG:teuthology.orchestra.run.smithi090:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-03-11T18:36:13.356 DEBUG:teuthology.orchestra.run.smithi090:> sudo modprobe fuse 2024-03-11T18:36:13.411 DEBUG:teuthology.orchestra.run.smithi090:> cat /proc/self/mounts | awk '{print $2}' 2024-03-11T18:36:13.458 INFO:teuthology.orchestra.run.smithi090.stdout:/sys 2024-03-11T18:36:13.458 INFO:teuthology.orchestra.run.smithi090.stdout:/proc 2024-03-11T18:36:13.458 INFO:teuthology.orchestra.run.smithi090.stdout:/dev 2024-03-11T18:36:13.458 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/pts 2024-03-11T18:36:13.458 INFO:teuthology.orchestra.run.smithi090.stdout:/run 2024-03-11T18:36:13.459 INFO:teuthology.orchestra.run.smithi090.stdout:/ 2024-03-11T18:36:13.459 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/security 2024-03-11T18:36:13.459 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/shm 2024-03-11T18:36:13.459 INFO:teuthology.orchestra.run.smithi090.stdout:/run/lock 2024-03-11T18:36:13.459 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/cgroup 2024-03-11T18:36:13.459 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/pstore 2024-03-11T18:36:13.459 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/bpf 2024-03-11T18:36:13.459 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/sys/fs/binfmt_misc 2024-03-11T18:36:13.459 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/hugepages 2024-03-11T18:36:13.459 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/mqueue 2024-03-11T18:36:13.459 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/debug 2024-03-11T18:36:13.459 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/tracing 2024-03-11T18:36:13.459 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/fs/nfsd 2024-03-11T18:36:13.459 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/fuse/connections 2024-03-11T18:36:13.460 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/config 2024-03-11T18:36:13.460 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-sysusers.service 2024-03-11T18:36:13.460 INFO:teuthology.orchestra.run.smithi090.stdout:/run/rpc_pipefs 2024-03-11T18:36:13.460 INFO:teuthology.orchestra.run.smithi090.stdout:/run/user/1000 2024-03-11T18:36:13.460 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph 2024-03-11T18:36:13.460 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph/osd/ceph-0 2024-03-11T18:36:13.460 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph/osd/ceph-1 2024-03-11T18:36:13.460 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph/osd/ceph-2 2024-03-11T18:36:13.460 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph/osd/ceph-3 2024-03-11T18:36:13.460 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns 2024-03-11T18:36:13.460 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-11T18:36:13.461 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-11T18:36:13.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T18:36:13.462 DEBUG:teuthology.orchestra.run.smithi090:> ls /sys/fs/fuse/connections 2024-03-11T18:36:13.509 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-11T18:36:13.509 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-03-11T18:36:13.554 DEBUG:teuthology.orchestra.run.smithi090:> sudo modprobe fuse 2024-03-11T18:36:13.566 DEBUG:teuthology.orchestra.run.smithi090:> cat /proc/self/mounts | awk '{print $2}' 2024-03-11T18:36:13.613 INFO:teuthology.orchestra.run.smithi090.stdout:/sys 2024-03-11T18:36:13.613 INFO:teuthology.orchestra.run.smithi090.stdout:/proc 2024-03-11T18:36:13.613 INFO:teuthology.orchestra.run.smithi090.stdout:/dev 2024-03-11T18:36:13.613 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/pts 2024-03-11T18:36:13.613 INFO:teuthology.orchestra.run.smithi090.stdout:/run 2024-03-11T18:36:13.613 INFO:teuthology.orchestra.run.smithi090.stdout:/ 2024-03-11T18:36:13.613 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/security 2024-03-11T18:36:13.613 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/shm 2024-03-11T18:36:13.614 INFO:teuthology.orchestra.run.smithi090.stdout:/run/lock 2024-03-11T18:36:13.614 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/cgroup 2024-03-11T18:36:13.614 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/pstore 2024-03-11T18:36:13.614 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/bpf 2024-03-11T18:36:13.614 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/sys/fs/binfmt_misc 2024-03-11T18:36:13.614 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/hugepages 2024-03-11T18:36:13.614 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/mqueue 2024-03-11T18:36:13.614 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/debug 2024-03-11T18:36:13.614 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/tracing 2024-03-11T18:36:13.614 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/fs/nfsd 2024-03-11T18:36:13.614 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/fuse/connections 2024-03-11T18:36:13.614 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/config 2024-03-11T18:36:13.614 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-sysusers.service 2024-03-11T18:36:13.614 INFO:teuthology.orchestra.run.smithi090.stdout:/run/rpc_pipefs 2024-03-11T18:36:13.615 INFO:teuthology.orchestra.run.smithi090.stdout:/run/user/1000 2024-03-11T18:36:13.615 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph 2024-03-11T18:36:13.615 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph/osd/ceph-0 2024-03-11T18:36:13.615 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph/osd/ceph-1 2024-03-11T18:36:13.615 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph/osd/ceph-2 2024-03-11T18:36:13.615 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph/osd/ceph-3 2024-03-11T18:36:13.615 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns 2024-03-11T18:36:13.615 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-11T18:36:13.615 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-11T18:36:13.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T18:36:13.616 DEBUG:teuthology.orchestra.run.smithi090:> ls /sys/fs/fuse/connections 2024-03-11T18:36:13.661 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi090.stderr:2024-03-11T18:36:13.656+0000 7f5811eee580 -1 init, newargv = 0x558e35fb3990 newargc=15 2024-03-11T18:36:13.661 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi090.stderr:ceph-fuse[34127]: starting ceph client 2024-03-11T18:36:14.067 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi090.stderr:ceph-fuse[34127]: starting fuse 2024-03-11T18:36:14.766 DEBUG:teuthology.orchestra.run.smithi090:> sudo modprobe fuse 2024-03-11T18:36:14.783 DEBUG:teuthology.orchestra.run.smithi090:> cat /proc/self/mounts | awk '{print $2}' 2024-03-11T18:36:14.830 INFO:teuthology.orchestra.run.smithi090.stdout:/sys 2024-03-11T18:36:14.830 INFO:teuthology.orchestra.run.smithi090.stdout:/proc 2024-03-11T18:36:14.830 INFO:teuthology.orchestra.run.smithi090.stdout:/dev 2024-03-11T18:36:14.830 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/pts 2024-03-11T18:36:14.831 INFO:teuthology.orchestra.run.smithi090.stdout:/run 2024-03-11T18:36:14.831 INFO:teuthology.orchestra.run.smithi090.stdout:/ 2024-03-11T18:36:14.831 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/security 2024-03-11T18:36:14.831 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/shm 2024-03-11T18:36:14.831 INFO:teuthology.orchestra.run.smithi090.stdout:/run/lock 2024-03-11T18:36:14.831 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/cgroup 2024-03-11T18:36:14.831 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/pstore 2024-03-11T18:36:14.831 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/bpf 2024-03-11T18:36:14.831 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/sys/fs/binfmt_misc 2024-03-11T18:36:14.831 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/hugepages 2024-03-11T18:36:14.831 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/mqueue 2024-03-11T18:36:14.831 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/debug 2024-03-11T18:36:14.831 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/tracing 2024-03-11T18:36:14.832 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/fs/nfsd 2024-03-11T18:36:14.832 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/fuse/connections 2024-03-11T18:36:14.832 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/config 2024-03-11T18:36:14.832 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-sysusers.service 2024-03-11T18:36:14.832 INFO:teuthology.orchestra.run.smithi090.stdout:/run/rpc_pipefs 2024-03-11T18:36:14.832 INFO:teuthology.orchestra.run.smithi090.stdout:/run/user/1000 2024-03-11T18:36:14.832 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph 2024-03-11T18:36:14.832 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph/osd/ceph-0 2024-03-11T18:36:14.832 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph/osd/ceph-1 2024-03-11T18:36:14.832 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph/osd/ceph-2 2024-03-11T18:36:14.832 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph/osd/ceph-3 2024-03-11T18:36:14.832 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns 2024-03-11T18:36:14.832 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-11T18:36:14.832 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-11T18:36:14.832 INFO:teuthology.orchestra.run.smithi090.stdout:/home/ubuntu/cephtest/mnt.admin 2024-03-11T18:36:14.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T18:36:14.833 DEBUG:teuthology.orchestra.run.smithi090:> ls /sys/fs/fuse/connections 2024-03-11T18:36:14.886 INFO:teuthology.orchestra.run.smithi090.stdout:45 2024-03-11T18:36:14.886 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-03-11T18:36:14.886 DEBUG:teuthology.orchestra.run.smithi090:> sudo stdin-killer -- python3 -c ' 2024-03-11T18:36:14.887 DEBUG:teuthology.orchestra.run.smithi090:> import glob 2024-03-11T18:36:14.887 DEBUG:teuthology.orchestra.run.smithi090:> import re 2024-03-11T18:36:14.887 DEBUG:teuthology.orchestra.run.smithi090:> import os 2024-03-11T18:36:14.887 DEBUG:teuthology.orchestra.run.smithi090:> import subprocess 2024-03-11T18:36:14.887 DEBUG:teuthology.orchestra.run.smithi090:> 2024-03-11T18:36:14.887 DEBUG:teuthology.orchestra.run.smithi090:> def _find_admin_socket(client_name): 2024-03-11T18:36:14.887 DEBUG:teuthology.orchestra.run.smithi090:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-03-11T18:36:14.887 DEBUG:teuthology.orchestra.run.smithi090:> files = glob.glob(asok_path) 2024-03-11T18:36:14.887 DEBUG:teuthology.orchestra.run.smithi090:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-03-11T18:36:14.887 DEBUG:teuthology.orchestra.run.smithi090:> 2024-03-11T18:36:14.887 DEBUG:teuthology.orchestra.run.smithi090:> # Given a non-glob path, it better be there 2024-03-11T18:36:14.888 DEBUG:teuthology.orchestra.run.smithi090:> if "*" not in asok_path: 2024-03-11T18:36:14.888 DEBUG:teuthology.orchestra.run.smithi090:> assert(len(files) == 1) 2024-03-11T18:36:14.888 DEBUG:teuthology.orchestra.run.smithi090:> return files[0] 2024-03-11T18:36:14.888 DEBUG:teuthology.orchestra.run.smithi090:> 2024-03-11T18:36:14.888 DEBUG:teuthology.orchestra.run.smithi090:> for f in files: 2024-03-11T18:36:14.888 DEBUG:teuthology.orchestra.run.smithi090:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-11T18:36:14.888 DEBUG:teuthology.orchestra.run.smithi090:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-11T18:36:14.889 DEBUG:teuthology.orchestra.run.smithi090:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-11T18:36:14.889 DEBUG:teuthology.orchestra.run.smithi090:> contents = proc_f.read() 2024-03-11T18:36:14.889 DEBUG:teuthology.orchestra.run.smithi090:> if mountpoint in contents: 2024-03-11T18:36:14.889 DEBUG:teuthology.orchestra.run.smithi090:> return f 2024-03-11T18:36:14.889 DEBUG:teuthology.orchestra.run.smithi090:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-11T18:36:14.889 DEBUG:teuthology.orchestra.run.smithi090:> 2024-03-11T18:36:14.889 DEBUG:teuthology.orchestra.run.smithi090:> print(_find_admin_socket("client.admin")) 2024-03-11T18:36:14.889 DEBUG:teuthology.orchestra.run.smithi090:> ' 2024-03-11T18:36:15.009 INFO:teuthology.orchestra.run.smithi090.stdout:/var/run/ceph/ceph-client.admin.34127.asok 2024-03-11T18:36:15.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-11T18:36:15.021 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.34127.asok 2024-03-11T18:36:15.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T18:36:15.022 DEBUG:teuthology.orchestra.run.smithi090:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.34127.asok status 2024-03-11T18:36:15.170 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-03-11T18:36:15.170 INFO:teuthology.orchestra.run.smithi090.stdout: "metadata": { 2024-03-11T18:36:15.170 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_sha1": "94a539707733325322964c6a187c6a1b41be1daf", 2024-03-11T18:36:15.170 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_version": "ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev)", 2024-03-11T18:36:15.170 INFO:teuthology.orchestra.run.smithi090.stdout: "entity_id": "admin", 2024-03-11T18:36:15.170 INFO:teuthology.orchestra.run.smithi090.stdout: "hostname": "smithi090", 2024-03-11T18:36:15.170 INFO:teuthology.orchestra.run.smithi090.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-03-11T18:36:15.170 INFO:teuthology.orchestra.run.smithi090.stdout: "pid": "34127", 2024-03-11T18:36:15.170 INFO:teuthology.orchestra.run.smithi090.stdout: "root": "/" 2024-03-11T18:36:15.171 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-03-11T18:36:15.171 INFO:teuthology.orchestra.run.smithi090.stdout: "dentry_count": 0, 2024-03-11T18:36:15.171 INFO:teuthology.orchestra.run.smithi090.stdout: "dentry_pinned_count": 0, 2024-03-11T18:36:15.171 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 4332, 2024-03-11T18:36:15.171 INFO:teuthology.orchestra.run.smithi090.stdout: "inst": { 2024-03-11T18:36:15.171 INFO:teuthology.orchestra.run.smithi090.stdout: "name": { 2024-03-11T18:36:15.171 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "client", 2024-03-11T18:36:15.171 INFO:teuthology.orchestra.run.smithi090.stdout: "num": 4332 2024-03-11T18:36:15.171 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-03-11T18:36:15.171 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": { 2024-03-11T18:36:15.171 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v1", 2024-03-11T18:36:15.172 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "192.168.0.1:0", 2024-03-11T18:36:15.172 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 3520059632 2024-03-11T18:36:15.172 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-03-11T18:36:15.172 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-03-11T18:36:15.172 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": { 2024-03-11T18:36:15.172 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v1", 2024-03-11T18:36:15.172 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "192.168.0.1:0", 2024-03-11T18:36:15.172 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 3520059632 2024-03-11T18:36:15.172 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-03-11T18:36:15.172 INFO:teuthology.orchestra.run.smithi090.stdout: "inst_str": "client.4332 192.168.0.1:0/3520059632", 2024-03-11T18:36:15.172 INFO:teuthology.orchestra.run.smithi090.stdout: "addr_str": "192.168.0.1:0/3520059632", 2024-03-11T18:36:15.172 INFO:teuthology.orchestra.run.smithi090.stdout: "inode_count": 1, 2024-03-11T18:36:15.173 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_epoch": 7, 2024-03-11T18:36:15.173 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_epoch": 28, 2024-03-11T18:36:15.173 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_epoch_barrier": 0, 2024-03-11T18:36:15.173 INFO:teuthology.orchestra.run.smithi090.stdout: "blocklisted": false, 2024-03-11T18:36:15.173 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs" 2024-03-11T18:36:15.173 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-03-11T18:36:15.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T18:36:15.183 DEBUG:teuthology.orchestra.run.smithi090:> stat --file-system '--printf=%T 2024-03-11T18:36:15.183 DEBUG:teuthology.orchestra.run.smithi090:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-03-11T18:36:15.215 INFO:teuthology.orchestra.run.smithi090.stdout:fuseblk 2024-03-11T18:36:15.215 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-03-11T18:36:15.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T18:36:15.216 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-03-11T18:36:15.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T18:36:15.390 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-03-11T18:36:15.747 INFO:teuthology.orchestra.run.smithi090.stdout:# file: . 2024-03-11T18:36:15.747 INFO:teuthology.orchestra.run.smithi090.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-03-11T18:36:15.747 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:36:15.748 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-11T18:36:15.748 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-11T18:36:15.754 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T18:36:15.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:15.924+0000 7fce18b58640 1 Processor -- start 2024-03-11T18:36:15.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:15.924+0000 7fce18b58640 1 -- start start 2024-03-11T18:36:15.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:15.928+0000 7fce18b58640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fce141554a0 0x7fce140c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:15.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:15.928+0000 7fce18b58640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce140c2ae0 con 0x7fce1406fe60 2024-03-11T18:36:15.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:15.928+0000 7fce18b58640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce140c2560 con 0x7fce141550c0 2024-03-11T18:36:15.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:15.928+0000 7fce18b58640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce14157cc0 con 0x7fce141554a0 2024-03-11T18:36:15.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:15.928+0000 7fce11d9b640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fce141554a0 0x7fce140c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:15.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:15.928+0000 7fce1259c640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fce141550c0 0x7fce1415ba70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.90:54738/0 (socket says 172.21.15.90:54738) 2024-03-11T18:36:15.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:15.928+0000 7fce1259c640 1 -- 172.21.15.90:0/3060848438 learned_addr learned my addr 172.21.15.90:0/3060848438 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T18:36:15.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:15.936+0000 7fce11d9b640 1 -- 172.21.15.90:0/3060848438 >> v1:172.21.15.92:6789/0 conn(0x7fce141550c0 legacy=0x7fce1415ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:15.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:15.936+0000 7fce11d9b640 1 -- 172.21.15.90:0/3060848438 >> v1:172.21.15.90:6789/0 conn(0x7fce1406fe60 legacy=0x7fce140c2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:15.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:15.936+0000 7fce11d9b640 1 -- 172.21.15.90:0/3060848438 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce140c2800 con 0x7fce141554a0 2024-03-11T18:36:15.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:15.940+0000 7fce11d9b640 1 --2- 172.21.15.90:0/3060848438 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fce141554a0 0x7fce140c8fb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc0097f0 tx=0x7fcdfc01c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82feca32bf209c02 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:15.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:15.948+0000 7fce1159a640 1 -- 172.21.15.90:0/3060848438 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc0ab600 con 0x7fce141554a0 2024-03-11T18:36:15.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:15.948+0000 7fce1159a640 1 -- 172.21.15.90:0/3060848438 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdfc0a7050 con 0x7fce141554a0 2024-03-11T18:36:15.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:15.948+0000 7fce1159a640 1 -- 172.21.15.90:0/3060848438 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc0ab980 con 0x7fce141554a0 2024-03-11T18:36:15.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:15.952+0000 7fce18b58640 1 -- 172.21.15.90:0/3060848438 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fce141554a0 msgr2=0x7fce140c8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:15.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:15.952+0000 7fce18b58640 1 --2- 172.21.15.90:0/3060848438 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fce141554a0 0x7fce140c8fb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc0097f0 tx=0x7fcdfc01c410 comp rx=0 tx=0).stop 2024-03-11T18:36:15.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:15.952+0000 7fce18b58640 1 -- 172.21.15.90:0/3060848438 shutdown_connections 2024-03-11T18:36:15.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:15.952+0000 7fce18b58640 1 --2- 172.21.15.90:0/3060848438 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fce141554a0 0x7fce140c8fb0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:15.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:15.952+0000 7fce18b58640 1 -- 172.21.15.90:0/3060848438 >> 172.21.15.90:0/3060848438 conn(0x7fce14070e80 msgr2=0x7fce14066990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:15.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:15.952+0000 7fce18b58640 1 -- 172.21.15.90:0/3060848438 shutdown_connections 2024-03-11T18:36:15.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:15.952+0000 7fce18b58640 1 -- 172.21.15.90:0/3060848438 wait complete. 2024-03-11T18:36:15.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:15.952+0000 7fce18b58640 1 Processor -- start 2024-03-11T18:36:15.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:15.952+0000 7fce18b58640 1 -- start start 2024-03-11T18:36:15.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:15.952+0000 7fce18b58640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fce1406fe60 0x7fce1413fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:15.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:15.952+0000 7fce18b58640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce141550c0 0x7fce14155880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:15.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:15.952+0000 7fce18b58640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce141554a0 0x7fce140cb210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:15.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:15.952+0000 7fce18b58640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce140cd180 con 0x7fce141554a0 2024-03-11T18:36:15.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:15.952+0000 7fce18b58640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce14142890 con 0x7fce141550c0 2024-03-11T18:36:15.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:15.952+0000 7fce18b58640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce14162460 con 0x7fce1406fe60 2024-03-11T18:36:15.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:15.956+0000 7fce1259c640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fce1406fe60 0x7fce1413fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:15.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:15.956+0000 7fce1259c640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fce1406fe60 0x7fce1413fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.90:49744/0 (socket says 172.21.15.90:49744) 2024-03-11T18:36:15.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:15.956+0000 7fce1259c640 1 -- 172.21.15.90:0/2692965749 learned_addr learned my addr 172.21.15.90:0/2692965749 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:15.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:15.968+0000 7fce1259c640 1 -- 172.21.15.90:0/2692965749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce141550c0 msgr2=0x7fce14155880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:15.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:15.968+0000 7fce1259c640 1 --2- 172.21.15.90:0/2692965749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce141550c0 0x7fce14155880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:15.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:15.968+0000 7fce1259c640 1 -- 172.21.15.90:0/2692965749 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce141554a0 msgr2=0x7fce140cb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:15.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:15.968+0000 7fce1259c640 1 --2- 172.21.15.90:0/2692965749 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce141554a0 0x7fce140cb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:15.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:15.968+0000 7fce1259c640 1 -- 172.21.15.90:0/2692965749 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdfc009420 con 0x7fce1406fe60 2024-03-11T18:36:15.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:15.972+0000 7fce1259c640 1 --2- 172.21.15.90:0/2692965749 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fce1406fe60 0x7fce1413fcd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fce0400cbd0 tx=0x7fce0400a430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:15.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:15.980+0000 7fce0affd640 1 -- 172.21.15.90:0/2692965749 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0400ad40 con 0x7fce1406fe60 2024-03-11T18:36:15.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:15.980+0000 7fce0affd640 1 -- 172.21.15.90:0/2692965749 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce04012400 con 0x7fce1406fe60 2024-03-11T18:36:15.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:15.980+0000 7fce0affd640 1 -- 172.21.15.90:0/2692965749 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0400aea0 con 0x7fce1406fe60 2024-03-11T18:36:15.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:15.984+0000 7fce18b58640 1 -- 172.21.15.90:0/2692965749 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce140ccea0 con 0x7fce1406fe60 2024-03-11T18:36:15.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:15.984+0000 7fce18b58640 1 -- 172.21.15.90:0/2692965749 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce14144530 con 0x7fce1406fe60 2024-03-11T18:36:15.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:15.984+0000 7fce18b58640 1 -- 172.21.15.90:0/2692965749 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce1404eef0 con 0x7fce1406fe60 2024-03-11T18:36:16.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.320+0000 7fce0affd640 1 -- 172.21.15.90:0/2692965749 <== mon.2 v2:172.21.15.92:3301/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fce04007070 con 0x7fce1406fe60 2024-03-11T18:36:16.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.320+0000 7fce0affd640 1 --2- 172.21.15.90:0/2692965749 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fcdf4066ec0 0x7fcdf4069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:16.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.320+0000 7fce11d9b640 1 --2- 172.21.15.90:0/2692965749 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fcdf4066ec0 0x7fcdf4069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:16.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.320+0000 7fce11d9b640 1 --2- 172.21.15.90:0/2692965749 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fcdf4066ec0 0x7fcdf4069380 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc0ad000 tx=0x7fcdfc004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:16.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.324+0000 7fce0affd640 1 -- 172.21.15.90:0/2692965749 <== mon.2 v2:172.21.15.92:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fce04055cf0 con 0x7fce1406fe60 2024-03-11T18:36:16.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.460+0000 7fce0affd640 1 -- 172.21.15.90:0/2692965749 <== mon.2 v2:172.21.15.92:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fce04051f90 con 0x7fce1406fe60 2024-03-11T18:36:16.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.656+0000 7fce18b58640 1 -- 172.21.15.90:0/2692965749 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce1406fba0 con 0x7fce1406fe60 2024-03-11T18:36:16.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.744+0000 7fce0affd640 1 -- 172.21.15.90:0/2692965749 <== mon.2 v2:172.21.15.92:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18430 (secure 0 0 0) 0x7fce04021090 con 0x7fce1406fe60 2024-03-11T18:36:16.752 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:36:16.752 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":28,"fsid":"76d56b10-6c29-476a-804b-8482ea90f9e2","created":"2024-03-11T18:34:54.521741+0000","modified":"2024-03-11T18:36:05.526044+0000","last_up_change":"2024-03-11T18:35:16.207500+0000","last_in_change":"2024-03-11T18:35:05.213634+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T18:35:16.193842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T18:35:54.001469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T18:35:55.917321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T18:36:02.210943+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"34decff9-7280-4d84-9665-d6af8ea6de84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6819","nonce":1755667921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6821","nonce":1755667921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6825","nonce":1755667921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6823","nonce":1755667921}]},"public_addr":"172.21.15.90:6819/1755667921","cluster_addr":"172.21.15.90:6821/1755667921","heartbeat_back_addr":"172.21.15.90:6825/1755667921","heartbeat_front_addr":"172.21.15.90:6823/1755667921","state":["exists","up"]},{"osd":1,"uuid":"8ecdce9e-5486-415a-a609-dd16e945440e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6807","nonce":438682081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6809","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6811","nonce":438682081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6817","nonce":438682081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6815","nonce":438682081}]},"public_addr":"172.21.15.90:6807/438682081","cluster_addr":"172.21.15.90:6811/438682081","heartbeat_back_addr":"172.21.15.90:6817/438682081","heartbeat_front_addr":"172.21.15.90:6815/438682081","state":["exists","up"]},{"osd":2,"uuid":"737486d4-0e90-4c82-9e79-bf37c812e5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6827","nonce":2837704738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6829","nonce":2837704738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6833","nonce":2837704738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6831","nonce":2837704738}]},"public_addr":"172.21.15.90:6827/2837704738","cluster_addr":"172.21.15.90:6829/2837704738","heartbeat_back_addr":"172.21.15.90:6833/2837704738","heartbeat_front_addr":"172.21.15.90:6831/2837704738","state":["exists","up"]},{"osd":3,"uuid":"698bd661-bbef-4699-a619-f7bfe23ce12c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6803","nonce":240987971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6805","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6806","nonce":240987971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6813","nonce":240987971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6810","nonce":240987971}]},"public_addr":"172.21.15.90:6803/240987971","cluster_addr":"172.21.15.90:6806/240987971","heartbeat_back_addr":"172.21.15.90:6813/240987971","heartbeat_front_addr":"172.21.15.90:6810/240987971","state":["exists","up"]},{"osd":4,"uuid":"aba9e31f-73d9-43fe-a10b-2dc5832968b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6801","nonce":502685275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6803","nonce":502685275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6807","nonce":502685275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6805","nonce":502685275}]},"public_addr":"172.21.15.92:6801/502685275","cluster_addr":"172.21.15.92:6803/502685275","heartbeat_back_addr":"172.21.15.92:6807/502685275","heartbeat_front_addr":"172.21.15.92:6805/502685275","state":["exists","up"]},{"osd":5,"uuid":"bdd56766-4b28-4007-8de3-b5b08cf335d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6809","nonce":200142070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6811","nonce":200142070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6815","nonce":200142070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6813","nonce":200142070}]},"public_addr":"172.21.15.92:6809/200142070","cluster_addr":"172.21.15.92:6811/200142070","heartbeat_back_addr":"172.21.15.92:6815/200142070","heartbeat_front_addr":"172.21.15.92:6813/200142070","state":["exists","up"]},{"osd":6,"uuid":"6c580a59-610f-455e-a96b-033299dcc118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6817","nonce":1919010204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6819","nonce":1919010204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6823","nonce":1919010204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6821","nonce":1919010204}]},"public_addr":"172.21.15.92:6817/1919010204","cluster_addr":"172.21.15.92:6819/1919010204","heartbeat_back_addr":"172.21.15.92:6823/1919010204","heartbeat_front_addr":"172.21.15.92:6821/1919010204","state":["exists","up"]},{"osd":7,"uuid":"958184cc-aebf-4c5b-91a0-2a3e38a6b42b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6825","nonce":3475925719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6827","nonce":3475925719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6831","nonce":3475925719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6829","nonce":3475925719}]},"public_addr":"172.21.15.92:6825/3475925719","cluster_addr":"172.21.15.92:6827/3475925719","heartbeat_back_addr":"172.21.15.92:6831/3475925719","heartbeat_front_addr":"172.21.15.92:6829/3475925719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.816294+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.048084+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.137975+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.968246+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.016829+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.931676+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.041673+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.079404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T18:36:16.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.748+0000 7fce18b58640 1 -- 172.21.15.90:0/2692965749 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fcdf4066ec0 msgr2=0x7fcdf4069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:16.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.748+0000 7fce18b58640 1 --2- 172.21.15.90:0/2692965749 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fcdf4066ec0 0x7fcdf4069380 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc0ad000 tx=0x7fcdfc004740 comp rx=0 tx=0).stop 2024-03-11T18:36:16.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.748+0000 7fce18b58640 1 -- 172.21.15.90:0/2692965749 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fce1406fe60 msgr2=0x7fce1413fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:16.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.748+0000 7fce18b58640 1 --2- 172.21.15.90:0/2692965749 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fce1406fe60 0x7fce1413fcd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fce0400cbd0 tx=0x7fce0400a430 comp rx=0 tx=0).stop 2024-03-11T18:36:16.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.748+0000 7fce18b58640 1 -- 172.21.15.90:0/2692965749 shutdown_connections 2024-03-11T18:36:16.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.748+0000 7fce18b58640 1 --2- 172.21.15.90:0/2692965749 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fcdf4066ec0 0x7fcdf4069380 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:16.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.748+0000 7fce18b58640 1 --2- 172.21.15.90:0/2692965749 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fce141554a0 0x7fce140cb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:16.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.748+0000 7fce18b58640 1 --2- 172.21.15.90:0/2692965749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fce141550c0 0x7fce14155880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:16.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.748+0000 7fce18b58640 1 --2- 172.21.15.90:0/2692965749 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fce1406fe60 0x7fce1413fcd0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:16.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.748+0000 7fce18b58640 1 -- 172.21.15.90:0/2692965749 >> 172.21.15.90:0/2692965749 conn(0x7fce14070e80 msgr2=0x7fce14152800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:16.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.748+0000 7fce18b58640 1 -- 172.21.15.90:0/2692965749 shutdown_connections 2024-03-11T18:36:16.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.748+0000 7fce18b58640 1 -- 172.21.15.90:0/2692965749 wait complete. 2024-03-11T18:36:16.778 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-11T18:35:16.193842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-11T18:35:54.001469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-11T18:35:55.917321+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-11T18:36:02.210943+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-11T18:36:16.778 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-11T18:36:16.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.896+0000 7f36ee9b3640 1 Processor -- start 2024-03-11T18:36:16.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.896+0000 7f36ee9b3640 1 -- start start 2024-03-11T18:36:16.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.900+0000 7f36ee9b3640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f36e8151b30 0x7f36e806e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:16.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.900+0000 7f36ee9b3640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f36e80c4af0 0x7f36e80c4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:16.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.900+0000 7f36ee9b3640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36e806f430 con 0x7f36e8070b00 2024-03-11T18:36:16.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.900+0000 7f36ee9b3640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36e80c65b0 con 0x7f36e80c4af0 2024-03-11T18:36:16.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.900+0000 7f36ee9b3640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36e806e820 con 0x7f36e8151b30 2024-03-11T18:36:16.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.900+0000 7f36e77fe640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f36e8070b00 0x7f36e80c5de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:44364/0 (socket says 172.21.15.90:44364) 2024-03-11T18:36:16.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.900+0000 7f36e77fe640 1 -- 172.21.15.90:0/1040218847 learned_addr learned my addr 172.21.15.90:0/1040218847 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T18:36:16.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.900+0000 7f36ecf50640 1 --2- 172.21.15.90:0/1040218847 >> v2:172.21.15.92:3300/0 conn(0x7f36e80c4af0 0x7f36e80c4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:16.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.904+0000 7f36e6ffd640 1 -- 172.21.15.90:0/1040218847 <== mon.0 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3598784027 0 0) 0x7f36e806f430 con 0x7f36e8070b00 2024-03-11T18:36:16.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.904+0000 7f36e6ffd640 1 -- 172.21.15.90:0/1040218847 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36d4003530 con 0x7f36e8070b00 2024-03-11T18:36:16.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.904+0000 7f36e7fff640 1 --2- 172.21.15.90:0/1040218847 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f36e8151b30 0x7f36e806e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:16.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.904+0000 7f36e6ffd640 1 -- 172.21.15.90:0/1040218847 <== mon.0 v1:172.21.15.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1898201774 0 0) 0x7f36d4003530 con 0x7f36e8070b00 2024-03-11T18:36:16.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.904+0000 7f36e6ffd640 1 -- 172.21.15.90:0/1040218847 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f36e8151b30 msgr2=0x7f36e806e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:16.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.904+0000 7f36e6ffd640 1 --2- 172.21.15.90:0/1040218847 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f36e8151b30 0x7f36e806e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:16.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.908+0000 7f36e6ffd640 1 -- 172.21.15.90:0/1040218847 >> v2:172.21.15.92:3300/0 conn(0x7f36e80c4af0 msgr2=0x7f36e80c4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:16.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.908+0000 7f36e6ffd640 1 --2- 172.21.15.90:0/1040218847 >> v2:172.21.15.92:3300/0 conn(0x7f36e80c4af0 0x7f36e80c4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:16.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.908+0000 7f36e6ffd640 1 -- 172.21.15.90:0/1040218847 --> v1:172.21.15.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e806eb00 con 0x7f36e8070b00 2024-03-11T18:36:16.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.908+0000 7f36e6ffd640 1 -- 172.21.15.90:0/1040218847 <== mon.0 v1:172.21.15.90:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 691735435 0 0) 0x7f36d0001680 con 0x7f36e8070b00 2024-03-11T18:36:16.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.908+0000 7f36e6ffd640 1 -- 172.21.15.90:0/1040218847 >> v1:172.21.15.90:6789/0 conn(0x7f36e8070b00 legacy=0x7f36e80c5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:16.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.908+0000 7f36e6ffd640 1 --2- 172.21.15.90:0/1040218847 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36d4003980 0x7f36d4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:16.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.908+0000 7f36e6ffd640 1 --2- 172.21.15.90:0/1040218847 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36d40047d0 0x7f36d4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:16.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.908+0000 7f36e6ffd640 1 --2- 172.21.15.90:0/1040218847 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f36e8151b30 0x7f36d4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:16.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.908+0000 7f36e6ffd640 1 -- 172.21.15.90:0/1040218847 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e806e820 con 0x7f36d40047d0 2024-03-11T18:36:16.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.908+0000 7f36e6ffd640 1 -- 172.21.15.90:0/1040218847 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d400ba40 con 0x7f36d4003980 2024-03-11T18:36:16.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.908+0000 7f36e6ffd640 1 -- 172.21.15.90:0/1040218847 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36d400bc00 con 0x7f36e8151b30 2024-03-11T18:36:16.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.908+0000 7f36e77fe640 1 --2- 172.21.15.90:0/1040218847 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36d40047d0 0x7f36d4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:16.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.912+0000 7f36ecf50640 1 --2- 172.21.15.90:0/1040218847 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f36e8151b30 0x7f36d4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:16.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.912+0000 7f36e77fe640 1 -- 172.21.15.90:0/1040218847 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f36e8151b30 msgr2=0x7f36d4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:16.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.912+0000 7f36e77fe640 1 --2- 172.21.15.90:0/1040218847 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f36e8151b30 0x7f36d4005190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:16.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.912+0000 7f36e77fe640 1 -- 172.21.15.90:0/1040218847 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36d4003980 msgr2=0x7f36d4003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:16.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.912+0000 7f36e77fe640 1 --2- 172.21.15.90:0/1040218847 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36d4003980 0x7f36d4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:16.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.912+0000 7f36e77fe640 1 -- 172.21.15.90:0/1040218847 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d4003530 con 0x7f36d40047d0 2024-03-11T18:36:16.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.916+0000 7f36e77fe640 1 --2- 172.21.15.90:0/1040218847 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36d40047d0 0x7f36d4004c50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f36d0001060 tx=0x7f36d001f2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f31f5a15523694e server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:16.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.920+0000 7f36e6ffd640 1 -- 172.21.15.90:0/1040218847 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d00a8ce0 con 0x7f36d40047d0 2024-03-11T18:36:16.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.920+0000 7f36e6ffd640 1 -- 172.21.15.90:0/1040218847 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36d00a8e40 con 0x7f36d40047d0 2024-03-11T18:36:16.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.920+0000 7f36e6ffd640 1 -- 172.21.15.90:0/1040218847 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d0005b00 con 0x7f36d40047d0 2024-03-11T18:36:16.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.920+0000 7f36ee9b3640 1 -- 172.21.15.90:0/1040218847 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36d40047d0 msgr2=0x7f36d4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:16.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.920+0000 7f36ee9b3640 1 --2- 172.21.15.90:0/1040218847 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36d40047d0 0x7f36d4004c50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f36d0001060 tx=0x7f36d001f2b0 comp rx=0 tx=0).stop 2024-03-11T18:36:16.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.920+0000 7f36ecf50640 1 -- 172.21.15.90:0/1040218847 reap_dead start 2024-03-11T18:36:16.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.920+0000 7f36ee9b3640 1 -- 172.21.15.90:0/1040218847 shutdown_connections 2024-03-11T18:36:16.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.920+0000 7f36ee9b3640 1 -- 172.21.15.90:0/1040218847 >> 172.21.15.90:0/1040218847 conn(0x7f36e806ac00 msgr2=0x7f36e806b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:16.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.920+0000 7f36ee9b3640 1 -- 172.21.15.90:0/1040218847 shutdown_connections 2024-03-11T18:36:16.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.920+0000 7f36ee9b3640 1 -- 172.21.15.90:0/1040218847 wait complete. 2024-03-11T18:36:16.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.920+0000 7f36ee9b3640 1 Processor -- start 2024-03-11T18:36:16.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.920+0000 7f36ee9b3640 1 -- start start 2024-03-11T18:36:16.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.924+0000 7f36ee9b3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36d40047d0 0x7f36e81ee300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:16.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.924+0000 7f36ee9b3640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f36e815aa10 0x7f36e80c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:16.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.924+0000 7f36ee9b3640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36e8156970 0x7f36e8156e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:16.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.924+0000 7f36ee9b3640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e8153430 con 0x7f36e8156970 2024-03-11T18:36:16.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.924+0000 7f36ee9b3640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e81eddc0 con 0x7f36d40047d0 2024-03-11T18:36:16.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.924+0000 7f36ee9b3640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36e81ed890 con 0x7f36e815aa10 2024-03-11T18:36:16.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.924+0000 7f36ecf50640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36e8156970 0x7f36e8156e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:16.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.924+0000 7f36ecf50640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36e8156970 0x7f36e8156e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43000/0 (socket says 172.21.15.90:43000) 2024-03-11T18:36:16.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.924+0000 7f36ecf50640 1 -- 172.21.15.90:0/2711744345 learned_addr learned my addr 172.21.15.90:0/2711744345 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:16.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.924+0000 7f36e77fe640 1 --2- 172.21.15.90:0/2711744345 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f36e815aa10 0x7f36e80c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:16.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.924+0000 7f36e7fff640 1 --2- 172.21.15.90:0/2711744345 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36d40047d0 0x7f36e81ee300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:16.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.928+0000 7f36ecf50640 1 -- 172.21.15.90:0/2711744345 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f36e815aa10 msgr2=0x7f36e80c7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:16.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.928+0000 7f36ecf50640 1 --2- 172.21.15.90:0/2711744345 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f36e815aa10 0x7f36e80c7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:16.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.928+0000 7f36ecf50640 1 -- 172.21.15.90:0/2711744345 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36d40047d0 msgr2=0x7f36e81ee300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:16.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.928+0000 7f36ecf50640 1 --2- 172.21.15.90:0/2711744345 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36d40047d0 0x7f36e81ee300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:16.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.928+0000 7f36ecf50640 1 -- 172.21.15.90:0/2711744345 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e81ee840 con 0x7f36e8156970 2024-03-11T18:36:16.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.932+0000 7f36ecf50640 1 --2- 172.21.15.90:0/2711744345 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36e8156970 0x7f36e8156e20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f36e0010790 tx=0x7f36e0010c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:16.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.936+0000 7f36e4ff9640 1 -- 172.21.15.90:0/2711744345 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e000e8d0 con 0x7f36e8156970 2024-03-11T18:36:16.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.936+0000 7f36e4ff9640 1 -- 172.21.15.90:0/2711744345 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36e000ea30 con 0x7f36e8156970 2024-03-11T18:36:16.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.936+0000 7f36ee9b3640 1 -- 172.21.15.90:0/2711744345 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36e81edae0 con 0x7f36e8156970 2024-03-11T18:36:16.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.936+0000 7f36e4ff9640 1 -- 172.21.15.90:0/2711744345 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e000ede0 con 0x7f36e8156970 2024-03-11T18:36:16.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.936+0000 7f36ee9b3640 1 -- 172.21.15.90:0/2711744345 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36e815a210 con 0x7f36e8156970 2024-03-11T18:36:16.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.940+0000 7f36ee9b3640 1 -- 172.21.15.90:0/2711744345 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36e8159d90 con 0x7f36e8156970 2024-03-11T18:36:16.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.956+0000 7f36e4ff9640 1 -- 172.21.15.90:0/2711744345 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f36e0020410 con 0x7f36e8156970 2024-03-11T18:36:16.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.956+0000 7f36e4ff9640 1 --2- 172.21.15.90:0/2711744345 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f36c8066ec0 0x7f36c8069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:16.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.960+0000 7f36e7fff640 1 --2- 172.21.15.90:0/2711744345 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f36c8066ec0 0x7f36c8069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:16.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:16.960+0000 7f36e7fff640 1 --2- 172.21.15.90:0/2711744345 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f36c8066ec0 0x7f36c8069380 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f36d8004b40 tx=0x7f36d80076c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:17.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.240+0000 7f36e4ff9640 1 -- 172.21.15.90:0/2711744345 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f36e00556c0 con 0x7f36e8156970 2024-03-11T18:36:17.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.240+0000 7f36e4ff9640 1 -- 172.21.15.90:0/2711744345 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f36e0051960 con 0x7f36e8156970 2024-03-11T18:36:17.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.408+0000 7f36ee9b3640 1 -- 172.21.15.90:0/2711744345 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f36e8159d90 con 0x7f36e8156970 2024-03-11T18:36:17.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.420+0000 7f36e4ff9640 1 -- 172.21.15.90:0/2711744345 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f36e8159d90 con 0x7f36e8156970 2024-03-11T18:36:17.428 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 1 2024-03-11T18:36:17.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.424+0000 7f36ee9b3640 1 -- 172.21.15.90:0/2711744345 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f36c8066ec0 msgr2=0x7f36c8069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:17.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.424+0000 7f36ee9b3640 1 --2- 172.21.15.90:0/2711744345 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f36c8066ec0 0x7f36c8069380 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f36d8004b40 tx=0x7f36d80076c0 comp rx=0 tx=0).stop 2024-03-11T18:36:17.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.424+0000 7f36ee9b3640 1 -- 172.21.15.90:0/2711744345 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36e8156970 msgr2=0x7f36e8156e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:17.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.424+0000 7f36ee9b3640 1 --2- 172.21.15.90:0/2711744345 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36e8156970 0x7f36e8156e20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f36e0010790 tx=0x7f36e0010c50 comp rx=0 tx=0).stop 2024-03-11T18:36:17.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.424+0000 7f36ee9b3640 1 -- 172.21.15.90:0/2711744345 shutdown_connections 2024-03-11T18:36:17.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.424+0000 7f36ee9b3640 1 --2- 172.21.15.90:0/2711744345 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f36c8066ec0 0x7f36c8069380 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:17.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.424+0000 7f36ee9b3640 1 --2- 172.21.15.90:0/2711744345 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36e8156970 0x7f36e8156e20 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:17.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.424+0000 7f36ee9b3640 1 --2- 172.21.15.90:0/2711744345 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f36e815aa10 0x7f36e80c7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:17.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.424+0000 7f36ee9b3640 1 --2- 172.21.15.90:0/2711744345 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f36d40047d0 0x7f36e81ee300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:17.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.424+0000 7f36ee9b3640 1 -- 172.21.15.90:0/2711744345 >> 172.21.15.90:0/2711744345 conn(0x7f36e806ac00 msgr2=0x7f36e814a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:17.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.428+0000 7f36ee9b3640 1 -- 172.21.15.90:0/2711744345 shutdown_connections 2024-03-11T18:36:17.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.428+0000 7f36ee9b3640 1 -- 172.21.15.90:0/2711744345 wait complete. 2024-03-11T18:36:17.449 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-11T18:36:17.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.564+0000 7fc31455e640 1 Processor -- start 2024-03-11T18:36:17.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.564+0000 7fc31455e640 1 -- start start 2024-03-11T18:36:17.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.564+0000 7fc31455e640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fc30c150a90 0x7fc30c157220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:17.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.564+0000 7fc31455e640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc30c0c65b0 0x7fc30c0c37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:17.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.564+0000 7fc31455e640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc30c0c3d30 0x7fc30c0cc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:17.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.564+0000 7fc31455e640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc30c0c5e60 con 0x7fc30c150a90 2024-03-11T18:36:17.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.564+0000 7fc31455e640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc30c156a60 con 0x7fc30c0c3d30 2024-03-11T18:36:17.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.564+0000 7fc31455e640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc30c0c55c0 con 0x7fc30c0c65b0 2024-03-11T18:36:17.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.564+0000 7fc3122fa640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fc30c150a90 0x7fc30c157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:17.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.564+0000 7fc3122fa640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fc30c150a90 0x7fc30c157220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43016/0 (socket says 172.21.15.90:43016) 2024-03-11T18:36:17.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.564+0000 7fc3122fa640 1 -- 172.21.15.90:0/560363583 learned_addr learned my addr 172.21.15.90:0/560363583 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:17.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.564+0000 7fc311af9640 1 --2- 172.21.15.90:0/560363583 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc30c0c65b0 0x7fc30c0c37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:17.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.564+0000 7fc312afb640 1 --2- 172.21.15.90:0/560363583 >> v2:172.21.15.92:3300/0 conn(0x7fc30c0c3d30 0x7fc30c0cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:17.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.568+0000 7fc3122fa640 1 -- 172.21.15.90:0/560363583 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc30c0c65b0 msgr2=0x7fc30c0c37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:17.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.568+0000 7fc3122fa640 1 --2- 172.21.15.90:0/560363583 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc30c0c65b0 0x7fc30c0c37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:17.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.568+0000 7fc3122fa640 1 -- 172.21.15.90:0/560363583 >> v2:172.21.15.92:3300/0 conn(0x7fc30c0c3d30 msgr2=0x7fc30c0cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:17.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.568+0000 7fc3122fa640 1 --2- 172.21.15.90:0/560363583 >> v2:172.21.15.92:3300/0 conn(0x7fc30c0c3d30 0x7fc30c0cc280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:17.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.568+0000 7fc3122fa640 1 -- 172.21.15.90:0/560363583 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc30c0c58a0 con 0x7fc30c150a90 2024-03-11T18:36:17.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.568+0000 7fc3122fa640 1 --2- 172.21.15.90:0/560363583 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc30c150a90 0x7fc30c157220 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc30c156c30 tx=0x7fc30801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eac3979d73232ef9 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:17.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.572+0000 7fc3112f8640 1 -- 172.21.15.90:0/560363583 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3080ab600 con 0x7fc30c150a90 2024-03-11T18:36:17.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.572+0000 7fc3112f8640 1 -- 172.21.15.90:0/560363583 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3080a7050 con 0x7fc30c150a90 2024-03-11T18:36:17.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.572+0000 7fc3112f8640 1 -- 172.21.15.90:0/560363583 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3080ab980 con 0x7fc30c150a90 2024-03-11T18:36:17.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.576+0000 7fc31455e640 1 -- 172.21.15.90:0/560363583 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc30c150a90 msgr2=0x7fc30c157220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:17.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.576+0000 7fc31455e640 1 --2- 172.21.15.90:0/560363583 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc30c150a90 0x7fc30c157220 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc30c156c30 tx=0x7fc30801c500 comp rx=0 tx=0).stop 2024-03-11T18:36:17.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.576+0000 7fc31455e640 1 -- 172.21.15.90:0/560363583 shutdown_connections 2024-03-11T18:36:17.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.576+0000 7fc31455e640 1 --2- 172.21.15.90:0/560363583 >> v2:172.21.15.92:3300/0 conn(0x7fc30c0c3d30 0x7fc30c0cc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:17.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.576+0000 7fc31455e640 1 --2- 172.21.15.90:0/560363583 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc30c0c65b0 0x7fc30c0c37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:17.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.576+0000 7fc31455e640 1 --2- 172.21.15.90:0/560363583 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc30c150a90 0x7fc30c157220 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:17.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.576+0000 7fc31455e640 1 -- 172.21.15.90:0/560363583 >> 172.21.15.90:0/560363583 conn(0x7fc30c071aa0 msgr2=0x7fc30c071eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:17.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.576+0000 7fc31455e640 1 -- 172.21.15.90:0/560363583 shutdown_connections 2024-03-11T18:36:17.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.576+0000 7fc31455e640 1 -- 172.21.15.90:0/560363583 wait complete. 2024-03-11T18:36:17.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.576+0000 7fc31455e640 1 Processor -- start 2024-03-11T18:36:17.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.576+0000 7fc31455e640 1 -- start start 2024-03-11T18:36:17.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.576+0000 7fc31455e640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc30c0c3d30 0x7fc30c0cac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:17.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.576+0000 7fc31455e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc30c0c65b0 0x7fc30c14d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:17.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.576+0000 7fc31455e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc30c150a90 0x7fc30c145c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:17.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.576+0000 7fc31455e640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30c15f100 con 0x7fc30c150a90 2024-03-11T18:36:17.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.576+0000 7fc31455e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30c06c860 con 0x7fc30c0c65b0 2024-03-11T18:36:17.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.576+0000 7fc31455e640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc30c160a70 con 0x7fc30c0c3d30 2024-03-11T18:36:17.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.576+0000 7fc312afb640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc30c150a90 0x7fc30c145c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:17.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.576+0000 7fc312afb640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc30c150a90 0x7fc30c145c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43028/0 (socket says 172.21.15.90:43028) 2024-03-11T18:36:17.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.576+0000 7fc312afb640 1 -- 172.21.15.90:0/1293973734 learned_addr learned my addr 172.21.15.90:0/1293973734 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:17.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.576+0000 7fc311af9640 1 --2- 172.21.15.90:0/1293973734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc30c0c65b0 0x7fc30c14d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:17.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.580+0000 7fc3122fa640 1 --2- 172.21.15.90:0/1293973734 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc30c0c3d30 0x7fc30c0cac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:17.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.580+0000 7fc312afb640 1 -- 172.21.15.90:0/1293973734 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc30c0c3d30 msgr2=0x7fc30c0cac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:17.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.580+0000 7fc312afb640 1 --2- 172.21.15.90:0/1293973734 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc30c0c3d30 0x7fc30c0cac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:17.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.580+0000 7fc312afb640 1 -- 172.21.15.90:0/1293973734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc30c0c65b0 msgr2=0x7fc30c14d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:17.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.580+0000 7fc312afb640 1 --2- 172.21.15.90:0/1293973734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc30c0c65b0 0x7fc30c14d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:17.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.580+0000 7fc312afb640 1 -- 172.21.15.90:0/1293973734 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3080094d0 con 0x7fc30c150a90 2024-03-11T18:36:17.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.584+0000 7fc312afb640 1 --2- 172.21.15.90:0/1293973734 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc30c150a90 0x7fc30c145c80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc304007e30 tx=0x7fc30400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:17.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.588+0000 7fc302ffd640 1 -- 172.21.15.90:0/1293973734 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc304014420 con 0x7fc30c150a90 2024-03-11T18:36:17.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.588+0000 7fc302ffd640 1 -- 172.21.15.90:0/1293973734 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc304014580 con 0x7fc30c150a90 2024-03-11T18:36:17.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.588+0000 7fc302ffd640 1 -- 172.21.15.90:0/1293973734 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc304014930 con 0x7fc30c150a90 2024-03-11T18:36:17.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.588+0000 7fc31455e640 1 -- 172.21.15.90:0/1293973734 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc30c15ead0 con 0x7fc30c150a90 2024-03-11T18:36:17.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.588+0000 7fc31455e640 1 -- 172.21.15.90:0/1293973734 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc30c149c20 con 0x7fc30c150a90 2024-03-11T18:36:17.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.588+0000 7fc31455e640 1 -- 172.21.15.90:0/1293973734 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc30c15f240 con 0x7fc30c150a90 2024-03-11T18:36:17.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.860+0000 7fc302ffd640 1 -- 172.21.15.90:0/1293973734 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fc304014a90 con 0x7fc30c150a90 2024-03-11T18:36:17.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.864+0000 7fc302ffd640 1 --2- 172.21.15.90:0/1293973734 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fc2f0066e90 0x7fc2f0069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:17.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.864+0000 7fc3122fa640 1 --2- 172.21.15.90:0/1293973734 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fc2f0066e90 0x7fc2f0069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:17.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.864+0000 7fc3122fa640 1 --2- 172.21.15.90:0/1293973734 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fc2f0066e90 0x7fc2f0069350 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc3080ad000 tx=0x7fc3080a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:17.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.896+0000 7fc302ffd640 1 -- 172.21.15.90:0/1293973734 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc304056750 con 0x7fc30c150a90 2024-03-11T18:36:17.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:17.896+0000 7fc302ffd640 1 -- 172.21.15.90:0/1293973734 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc3040529f0 con 0x7fc30c150a90 2024-03-11T18:36:18.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.068+0000 7fc31455e640 1 -- 172.21.15.90:0/1293973734 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc30c15f240 con 0x7fc30c150a90 2024-03-11T18:36:18.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.096+0000 7fc302ffd640 1 -- 172.21.15.90:0/1293973734 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc30c15f240 con 0x7fc30c150a90 2024-03-11T18:36:18.102 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 32 2024-03-11T18:36:18.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.096+0000 7fc31455e640 1 -- 172.21.15.90:0/1293973734 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fc2f0066e90 msgr2=0x7fc2f0069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:18.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.096+0000 7fc31455e640 1 --2- 172.21.15.90:0/1293973734 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fc2f0066e90 0x7fc2f0069350 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc3080ad000 tx=0x7fc3080a6000 comp rx=0 tx=0).stop 2024-03-11T18:36:18.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.096+0000 7fc31455e640 1 -- 172.21.15.90:0/1293973734 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc30c150a90 msgr2=0x7fc30c145c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:18.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.096+0000 7fc31455e640 1 --2- 172.21.15.90:0/1293973734 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc30c150a90 0x7fc30c145c80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc304007e30 tx=0x7fc30400e4b0 comp rx=0 tx=0).stop 2024-03-11T18:36:18.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.100+0000 7fc31455e640 1 -- 172.21.15.90:0/1293973734 shutdown_connections 2024-03-11T18:36:18.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.100+0000 7fc31455e640 1 --2- 172.21.15.90:0/1293973734 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fc2f0066e90 0x7fc2f0069350 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:18.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.100+0000 7fc31455e640 1 --2- 172.21.15.90:0/1293973734 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc30c150a90 0x7fc30c145c80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:18.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.100+0000 7fc31455e640 1 --2- 172.21.15.90:0/1293973734 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc30c0c65b0 0x7fc30c14d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:18.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.100+0000 7fc31455e640 1 --2- 172.21.15.90:0/1293973734 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc30c0c3d30 0x7fc30c0cac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:18.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.100+0000 7fc31455e640 1 -- 172.21.15.90:0/1293973734 >> 172.21.15.90:0/1293973734 conn(0x7fc30c071aa0 msgr2=0x7fc30c14fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:18.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.100+0000 7fc31455e640 1 -- 172.21.15.90:0/1293973734 shutdown_connections 2024-03-11T18:36:18.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.100+0000 7fc31455e640 1 -- 172.21.15.90:0/1293973734 wait complete. 2024-03-11T18:36:18.119 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-11T18:36:18.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.224+0000 7f167911c640 1 Processor -- start 2024-03-11T18:36:18.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.224+0000 7f167911c640 1 -- start start 2024-03-11T18:36:18.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.224+0000 7f167911c640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1674151c70 0x7f167406bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:18.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.224+0000 7f167911c640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f16740c7f20 0x7f167406c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:18.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.224+0000 7f167911c640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16740c6720 con 0x7f1674066d60 2024-03-11T18:36:18.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.224+0000 7f167911c640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16740cc7b0 con 0x7f16740c7f20 2024-03-11T18:36:18.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.224+0000 7f167911c640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16740c5d20 con 0x7f1674151c70 2024-03-11T18:36:18.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.224+0000 7f167259c640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f16740c7f20 0x7f167406c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:18.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.224+0000 7f167259c640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f16740c7f20 0x7f167406c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43034/0 (socket says 172.21.15.90:43034) 2024-03-11T18:36:18.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.224+0000 7f167259c640 1 -- 172.21.15.90:0/1992874167 learned_addr learned my addr 172.21.15.90:0/1992874167 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:18.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.224+0000 7f1672d9d640 1 --2- 172.21.15.90:0/1992874167 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1674151c70 0x7f167406bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:18.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.228+0000 7f167259c640 1 -- 172.21.15.90:0/1992874167 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1674151c70 msgr2=0x7f167406bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:18.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.228+0000 7f167259c640 1 --2- 172.21.15.90:0/1992874167 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1674151c70 0x7f167406bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:18.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.228+0000 7f167259c640 1 -- 172.21.15.90:0/1992874167 >> v1:172.21.15.92:6789/0 conn(0x7f1674066d60 legacy=0x7f1674157ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:18.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.228+0000 7f167259c640 1 -- 172.21.15.90:0/1992874167 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16740c5a40 con 0x7f16740c7f20 2024-03-11T18:36:18.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.228+0000 7f167259c640 1 --2- 172.21.15.90:0/1992874167 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16740c7f20 0x7f167406c150 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f165c009870 tx=0x7f165c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d643a9380f29fe75 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:18.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.236+0000 7f1671d9b640 1 -- 172.21.15.90:0/1992874167 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165c0ab610 con 0x7f16740c7f20 2024-03-11T18:36:18.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.236+0000 7f1671d9b640 1 -- 172.21.15.90:0/1992874167 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f165c0ab770 con 0x7f16740c7f20 2024-03-11T18:36:18.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.236+0000 7f1671d9b640 1 -- 172.21.15.90:0/1992874167 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165c0abb00 con 0x7f16740c7f20 2024-03-11T18:36:18.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.236+0000 7f167911c640 1 -- 172.21.15.90:0/1992874167 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16740c7f20 msgr2=0x7f167406c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:18.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.236+0000 7f167911c640 1 --2- 172.21.15.90:0/1992874167 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16740c7f20 0x7f167406c150 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f165c009870 tx=0x7f165c01c410 comp rx=0 tx=0).stop 2024-03-11T18:36:18.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.236+0000 7f167911c640 1 -- 172.21.15.90:0/1992874167 shutdown_connections 2024-03-11T18:36:18.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.236+0000 7f167911c640 1 --2- 172.21.15.90:0/1992874167 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16740c7f20 0x7f167406c150 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:18.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.236+0000 7f167911c640 1 --2- 172.21.15.90:0/1992874167 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1674151c70 0x7f167406bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:18.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.236+0000 7f167911c640 1 -- 172.21.15.90:0/1992874167 >> 172.21.15.90:0/1992874167 conn(0x7f16740cb880 msgr2=0x7f16740cbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:18.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.236+0000 7f167911c640 1 -- 172.21.15.90:0/1992874167 shutdown_connections 2024-03-11T18:36:18.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.236+0000 7f167911c640 1 -- 172.21.15.90:0/1992874167 wait complete. 2024-03-11T18:36:18.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.240+0000 7f167911c640 1 Processor -- start 2024-03-11T18:36:18.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.240+0000 7f167911c640 1 -- start start 2024-03-11T18:36:18.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.240+0000 7f167911c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1674066d60 0x7f16741f1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:18.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.240+0000 7f167911c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16740c7f20 0x7f1674140c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:18.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.240+0000 7f167911c640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1674151c70 0x7f1674152350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:18.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.240+0000 7f167911c640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16741f1f80 con 0x7f16740c7f20 2024-03-11T18:36:18.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.240+0000 7f167911c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1674154610 con 0x7f1674066d60 2024-03-11T18:36:18.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.240+0000 7f167911c640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16741548b0 con 0x7f1674151c70 2024-03-11T18:36:18.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.240+0000 7f167259c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16740c7f20 0x7f1674140c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:18.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.240+0000 7f167259c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16740c7f20 0x7f1674140c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43044/0 (socket says 172.21.15.90:43044) 2024-03-11T18:36:18.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.240+0000 7f167259c640 1 -- 172.21.15.90:0/3733475300 learned_addr learned my addr 172.21.15.90:0/3733475300 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:18.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.240+0000 7f167359e640 1 --2- 172.21.15.90:0/3733475300 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1674151c70 0x7f1674152350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:18.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.240+0000 7f1672d9d640 1 --2- 172.21.15.90:0/3733475300 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1674066d60 0x7f16741f1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:18.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.244+0000 7f167259c640 1 -- 172.21.15.90:0/3733475300 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1674151c70 msgr2=0x7f1674152350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:18.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.244+0000 7f167259c640 1 --2- 172.21.15.90:0/3733475300 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1674151c70 0x7f1674152350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:18.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.244+0000 7f167259c640 1 -- 172.21.15.90:0/3733475300 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1674066d60 msgr2=0x7f16741f1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:18.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.244+0000 7f167259c640 1 --2- 172.21.15.90:0/3733475300 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1674066d60 0x7f16741f1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:18.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.244+0000 7f167259c640 1 -- 172.21.15.90:0/3733475300 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f165c0094d0 con 0x7f16740c7f20 2024-03-11T18:36:18.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.244+0000 7f167359e640 1 --2- 172.21.15.90:0/3733475300 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1674151c70 0x7f1674152350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T18:36:18.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.252+0000 7f167259c640 1 --2- 172.21.15.90:0/3733475300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16740c7f20 0x7f1674140c20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f165c01a040 tx=0x7f165c01ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:18.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.256+0000 7f166b7fe640 1 -- 172.21.15.90:0/3733475300 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165c0b4800 con 0x7f16740c7f20 2024-03-11T18:36:18.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.256+0000 7f166b7fe640 1 -- 172.21.15.90:0/3733475300 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f165c0b4980 con 0x7f16740c7f20 2024-03-11T18:36:18.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.256+0000 7f166b7fe640 1 -- 172.21.15.90:0/3733475300 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165c0b4d30 con 0x7f16740c7f20 2024-03-11T18:36:18.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.256+0000 7f167911c640 1 -- 172.21.15.90:0/3733475300 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1674154b90 con 0x7f16740c7f20 2024-03-11T18:36:18.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.256+0000 7f167911c640 1 -- 172.21.15.90:0/3733475300 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1674155150 con 0x7f16740c7f20 2024-03-11T18:36:18.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.256+0000 7f167911c640 1 -- 172.21.15.90:0/3733475300 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f167404eef0 con 0x7f16740c7f20 2024-03-11T18:36:18.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.276+0000 7f166b7fe640 1 -- 172.21.15.90:0/3733475300 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f165c0be3f0 con 0x7f16740c7f20 2024-03-11T18:36:18.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.276+0000 7f166b7fe640 1 --2- 172.21.15.90:0/3733475300 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1654066e90 0x7f1654069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:18.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.276+0000 7f1672d9d640 1 --2- 172.21.15.90:0/3733475300 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1654066e90 0x7f1654069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:18.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.276+0000 7f1672d9d640 1 --2- 172.21.15.90:0/3733475300 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1654066e90 0x7f1654069350 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f166c0079c0 tx=0x7f166c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:18.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.280+0000 7f166b7fe640 1 -- 172.21.15.90:0/3733475300 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f165c0f0510 con 0x7f16740c7f20 2024-03-11T18:36:18.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.556+0000 7f166b7fe640 1 -- 172.21.15.90:0/3733475300 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f165c0ec7b0 con 0x7f16740c7f20 2024-03-11T18:36:18.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.724+0000 7f167911c640 1 -- 172.21.15.90:0/3733475300 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f16740c79e0 con 0x7f16740c7f20 2024-03-11T18:36:18.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.824+0000 7f166b7fe640 1 -- 172.21.15.90:0/3733475300 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f165c0ab5d0 con 0x7f16740c7f20 2024-03-11T18:36:18.829 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 64 2024-03-11T18:36:18.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.828+0000 7f167911c640 1 -- 172.21.15.90:0/3733475300 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1654066e90 msgr2=0x7f1654069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:18.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.828+0000 7f167911c640 1 --2- 172.21.15.90:0/3733475300 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1654066e90 0x7f1654069350 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f166c0079c0 tx=0x7f166c008040 comp rx=0 tx=0).stop 2024-03-11T18:36:18.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.828+0000 7f167911c640 1 -- 172.21.15.90:0/3733475300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16740c7f20 msgr2=0x7f1674140c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:18.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.828+0000 7f167911c640 1 --2- 172.21.15.90:0/3733475300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16740c7f20 0x7f1674140c20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f165c01a040 tx=0x7f165c01ac50 comp rx=0 tx=0).stop 2024-03-11T18:36:18.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.828+0000 7f167911c640 1 -- 172.21.15.90:0/3733475300 shutdown_connections 2024-03-11T18:36:18.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.828+0000 7f167911c640 1 --2- 172.21.15.90:0/3733475300 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1654066e90 0x7f1654069350 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:18.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.828+0000 7f167911c640 1 --2- 172.21.15.90:0/3733475300 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1674151c70 0x7f1674152350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:18.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.828+0000 7f167911c640 1 --2- 172.21.15.90:0/3733475300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16740c7f20 0x7f1674140c20 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:18.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.828+0000 7f167911c640 1 --2- 172.21.15.90:0/3733475300 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1674066d60 0x7f16741f1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:18.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.828+0000 7f167911c640 1 -- 172.21.15.90:0/3733475300 >> 172.21.15.90:0/3733475300 conn(0x7f16740cb880 msgr2=0x7f1674150a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:18.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.828+0000 7f167911c640 1 -- 172.21.15.90:0/3733475300 shutdown_connections 2024-03-11T18:36:18.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.828+0000 7f167911c640 1 -- 172.21.15.90:0/3733475300 wait complete. 2024-03-11T18:36:18.856 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-11T18:36:18.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.976+0000 7fb5c10e8640 1 Processor -- start 2024-03-11T18:36:18.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.976+0000 7fb5c10e8640 1 -- start start 2024-03-11T18:36:18.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.980+0000 7fb5c10e8640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fb5bc1584c0 0x7fb5bc161fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:18.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.980+0000 7fb5c10e8640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5bc162520 con 0x7fb5bc06cc30 2024-03-11T18:36:18.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.980+0000 7fb5c10e8640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5bc06c0e0 con 0x7fb5bc0c7e10 2024-03-11T18:36:18.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.980+0000 7fb5c10e8640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5bc0b00a0 con 0x7fb5bc1584c0 2024-03-11T18:36:18.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.980+0000 7fb5bad9d640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fb5bc1584c0 0x7fb5bc161fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:18.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.980+0000 7fb5bb59e640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7fb5bc06cc30 0x7fb5bc0c8e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:44378/0 (socket says 172.21.15.90:44378) 2024-03-11T18:36:18.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.980+0000 7fb5bb59e640 1 -- 172.21.15.90:0/3837113016 learned_addr learned my addr 172.21.15.90:0/3837113016 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T18:36:18.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.984+0000 7fb5bad9d640 1 -- 172.21.15.90:0/3837113016 >> v1:172.21.15.92:6789/0 conn(0x7fb5bc0c7e10 legacy=0x7fb5bc06c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:18.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.984+0000 7fb5bad9d640 1 -- 172.21.15.90:0/3837113016 >> v1:172.21.15.90:6789/0 conn(0x7fb5bc06cc30 legacy=0x7fb5bc0c8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:18.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.984+0000 7fb5bad9d640 1 -- 172.21.15.90:0/3837113016 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5bc15e3b0 con 0x7fb5bc1584c0 2024-03-11T18:36:18.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.988+0000 7fb5bad9d640 1 --2- 172.21.15.90:0/3837113016 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5bc1584c0 0x7fb5bc161fe0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac01a7c0 tx=0x7fb5ac02d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=549dc1112f6720f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:18.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.992+0000 7fb5ba59c640 1 -- 172.21.15.90:0/3837113016 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac0bc600 con 0x7fb5bc1584c0 2024-03-11T18:36:18.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.992+0000 7fb5ba59c640 1 -- 172.21.15.90:0/3837113016 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5ac0b8050 con 0x7fb5bc1584c0 2024-03-11T18:36:19.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.992+0000 7fb5ba59c640 1 -- 172.21.15.90:0/3837113016 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac0bc980 con 0x7fb5bc1584c0 2024-03-11T18:36:19.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.992+0000 7fb5c10e8640 1 -- 172.21.15.90:0/3837113016 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5bc1584c0 msgr2=0x7fb5bc161fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:19.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.992+0000 7fb5c10e8640 1 --2- 172.21.15.90:0/3837113016 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5bc1584c0 0x7fb5bc161fe0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac01a7c0 tx=0x7fb5ac02d4a0 comp rx=0 tx=0).stop 2024-03-11T18:36:19.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.992+0000 7fb5c10e8640 1 -- 172.21.15.90:0/3837113016 shutdown_connections 2024-03-11T18:36:19.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.992+0000 7fb5c10e8640 1 --2- 172.21.15.90:0/3837113016 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5bc1584c0 0x7fb5bc161fe0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:19.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.992+0000 7fb5c10e8640 1 -- 172.21.15.90:0/3837113016 >> 172.21.15.90:0/3837113016 conn(0x7fb5bc070e80 msgr2=0x7fb5bc066990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:19.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.996+0000 7fb5c10e8640 1 -- 172.21.15.90:0/3837113016 shutdown_connections 2024-03-11T18:36:19.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.996+0000 7fb5c10e8640 1 -- 172.21.15.90:0/3837113016 wait complete. 2024-03-11T18:36:19.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.996+0000 7fb5c10e8640 1 Processor -- start 2024-03-11T18:36:19.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.996+0000 7fb5c10e8640 1 -- start start 2024-03-11T18:36:19.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.996+0000 7fb5c10e8640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5bc0c7e10 0x7fb5bc1545f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:19.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.996+0000 7fb5c10e8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5bc1584c0 0x7fb5bc0ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:19.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.996+0000 7fb5c10e8640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb5bc154b30 0x7fb5bc150020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:19.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.996+0000 7fb5c10e8640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5bc0cb860 con 0x7fb5bc0c7e10 2024-03-11T18:36:19.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.996+0000 7fb5c10e8640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5bc0cc360 con 0x7fb5bc1584c0 2024-03-11T18:36:19.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.996+0000 7fb5c10e8640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5bc153e80 con 0x7fb5bc154b30 2024-03-11T18:36:19.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.996+0000 7fb5bad9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5bc0c7e10 0x7fb5bc1545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:19.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.996+0000 7fb5bad9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5bc0c7e10 0x7fb5bc1545f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43060/0 (socket says 172.21.15.90:43060) 2024-03-11T18:36:19.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:18.996+0000 7fb5bad9d640 1 -- 172.21.15.90:0/3621026426 learned_addr learned my addr 172.21.15.90:0/3621026426 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:19.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.000+0000 7fb5bb59e640 1 --2- 172.21.15.90:0/3621026426 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb5bc154b30 0x7fb5bc150020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:19.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.000+0000 7fb5bad9d640 1 -- 172.21.15.90:0/3621026426 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb5bc154b30 msgr2=0x7fb5bc150020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:19.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.000+0000 7fb5bad9d640 1 --2- 172.21.15.90:0/3621026426 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb5bc154b30 0x7fb5bc150020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:19.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.000+0000 7fb5bad9d640 1 -- 172.21.15.90:0/3621026426 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5bc1584c0 msgr2=0x7fb5bc0ccc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:19.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.000+0000 7fb5bad9d640 1 --2- 172.21.15.90:0/3621026426 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5bc1584c0 0x7fb5bc0ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:19.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.000+0000 7fb5bad9d640 1 -- 172.21.15.90:0/3621026426 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5ac01a420 con 0x7fb5bc0c7e10 2024-03-11T18:36:19.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.004+0000 7fb5bad9d640 1 --2- 172.21.15.90:0/3621026426 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5bc0c7e10 0x7fb5bc1545f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac02a0b0 tx=0x7fb5ac02bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:19.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.008+0000 7fb5b37fe640 1 -- 172.21.15.90:0/3621026426 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac0c8650 con 0x7fb5bc0c7e10 2024-03-11T18:36:19.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.008+0000 7fb5b37fe640 1 -- 172.21.15.90:0/3621026426 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5ac0bce90 con 0x7fb5bc0c7e10 2024-03-11T18:36:19.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.008+0000 7fb5c10e8640 1 -- 172.21.15.90:0/3621026426 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5bc154160 con 0x7fb5bc0c7e10 2024-03-11T18:36:19.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.008+0000 7fb5b37fe640 1 -- 172.21.15.90:0/3621026426 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ac0c8a10 con 0x7fb5bc0c7e10 2024-03-11T18:36:19.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.008+0000 7fb5c10e8640 1 -- 172.21.15.90:0/3621026426 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5bc158d30 con 0x7fb5bc0c7e10 2024-03-11T18:36:19.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.008+0000 7fb5c10e8640 1 -- 172.21.15.90:0/3621026426 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5bc04eef0 con 0x7fb5bc0c7e10 2024-03-11T18:36:19.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.260+0000 7fb5b37fe640 1 -- 172.21.15.90:0/3621026426 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fb5ac0c8b70 con 0x7fb5bc0c7e10 2024-03-11T18:36:19.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.260+0000 7fb5b37fe640 1 --2- 172.21.15.90:0/3621026426 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fb59c066e90 0x7fb59c069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:19.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.260+0000 7fb5b3fff640 1 --2- 172.21.15.90:0/3621026426 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fb59c066e90 0x7fb59c069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:19.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.260+0000 7fb5b3fff640 1 --2- 172.21.15.90:0/3621026426 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fb59c066e90 0x7fb59c069350 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb5a4000f70 tx=0x7fb5a40079c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:19.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.272+0000 7fb5b37fe640 1 -- 172.21.15.90:0/3621026426 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb5ac0c8e20 con 0x7fb5bc0c7e10 2024-03-11T18:36:19.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.272+0000 7fb5b37fe640 1 -- 172.21.15.90:0/3621026426 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb5ac0fdfb0 con 0x7fb5bc0c7e10 2024-03-11T18:36:19.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.440+0000 7fb5c10e8640 1 -- 172.21.15.90:0/3621026426 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb5bc073f70 con 0x7fb5bc0c7e10 2024-03-11T18:36:19.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.444+0000 7fb5b37fe640 1 -- 172.21.15.90:0/3621026426 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb5ac0c1030 con 0x7fb5bc0c7e10 2024-03-11T18:36:19.452 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 32 2024-03-11T18:36:19.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.448+0000 7fb5c10e8640 1 -- 172.21.15.90:0/3621026426 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fb59c066e90 msgr2=0x7fb59c069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:19.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.448+0000 7fb5c10e8640 1 --2- 172.21.15.90:0/3621026426 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fb59c066e90 0x7fb59c069350 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb5a4000f70 tx=0x7fb5a40079c0 comp rx=0 tx=0).stop 2024-03-11T18:36:19.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.448+0000 7fb5c10e8640 1 -- 172.21.15.90:0/3621026426 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5bc0c7e10 msgr2=0x7fb5bc1545f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:19.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.448+0000 7fb5c10e8640 1 --2- 172.21.15.90:0/3621026426 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5bc0c7e10 0x7fb5bc1545f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac02a0b0 tx=0x7fb5ac02bce0 comp rx=0 tx=0).stop 2024-03-11T18:36:19.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.448+0000 7fb5c10e8640 1 -- 172.21.15.90:0/3621026426 shutdown_connections 2024-03-11T18:36:19.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.448+0000 7fb5c10e8640 1 --2- 172.21.15.90:0/3621026426 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fb59c066e90 0x7fb59c069350 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:19.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.448+0000 7fb5c10e8640 1 --2- 172.21.15.90:0/3621026426 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb5bc154b30 0x7fb5bc150020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:19.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.448+0000 7fb5c10e8640 1 --2- 172.21.15.90:0/3621026426 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb5bc1584c0 0x7fb5bc0ccc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:19.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.448+0000 7fb5c10e8640 1 --2- 172.21.15.90:0/3621026426 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5bc0c7e10 0x7fb5bc1545f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:19.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.448+0000 7fb5c10e8640 1 -- 172.21.15.90:0/3621026426 >> 172.21.15.90:0/3621026426 conn(0x7fb5bc070e80 msgr2=0x7fb5bc15fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:19.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.448+0000 7fb5c10e8640 1 -- 172.21.15.90:0/3621026426 shutdown_connections 2024-03-11T18:36:19.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.448+0000 7fb5c10e8640 1 -- 172.21.15.90:0/3621026426 wait complete. 2024-03-11T18:36:19.472 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-11T18:36:19.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.584+0000 7f1e4d1d5640 1 Processor -- start 2024-03-11T18:36:19.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.584+0000 7f1e4d1d5640 1 -- start start 2024-03-11T18:36:19.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.584+0000 7f1e4d1d5640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1e48153f10 0x7f1e4815a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:19.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.584+0000 7f1e4d1d5640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e4815abb0 con 0x7f1e480afb60 2024-03-11T18:36:19.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.584+0000 7f1e4d1d5640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e480c81e0 con 0x7f1e480c8e20 2024-03-11T18:36:19.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.584+0000 7f1e4d1d5640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4806c2d0 con 0x7f1e48153f10 2024-03-11T18:36:19.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.588+0000 7f1e4759e640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f1e480afb60 0x7f1e4806c520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:44380/0 (socket says 172.21.15.90:44380) 2024-03-11T18:36:19.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.588+0000 7f1e4759e640 1 -- 172.21.15.90:0/3206767538 learned_addr learned my addr 172.21.15.90:0/3206767538 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T18:36:19.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.588+0000 7f1e4659c640 1 -- 172.21.15.90:0/3206767538 <== mon.0 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4141645805 0 0) 0x7f1e400014a0 con 0x7f1e480afb60 2024-03-11T18:36:19.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.588+0000 7f1e4659c640 1 -- 172.21.15.90:0/3206767538 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e34003530 con 0x7f1e480afb60 2024-03-11T18:36:19.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.588+0000 7f1e46d9d640 1 --2- 172.21.15.90:0/3206767538 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1e48153f10 0x7f1e4815a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:19.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.596+0000 7f1e4659c640 1 -- 172.21.15.90:0/3206767538 <== mon.0 v1:172.21.15.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1706814850 0 0) 0x7f1e34003530 con 0x7f1e480afb60 2024-03-11T18:36:19.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.596+0000 7f1e4659c640 1 -- 172.21.15.90:0/3206767538 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1e48153f10 msgr2=0x7f1e4815a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:19.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.596+0000 7f1e4659c640 1 --2- 172.21.15.90:0/3206767538 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1e48153f10 0x7f1e4815a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:19.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.596+0000 7f1e4659c640 1 -- 172.21.15.90:0/3206767538 >> v1:172.21.15.92:6789/0 conn(0x7f1e480c8e20 legacy=0x7f1e4815db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:19.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.596+0000 7f1e4659c640 1 -- 172.21.15.90:0/3206767538 --> v1:172.21.15.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e4806bd10 con 0x7f1e480afb60 2024-03-11T18:36:19.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.596+0000 7f1e4659c640 1 -- 172.21.15.90:0/3206767538 <== mon.0 v1:172.21.15.90:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 691735435 0 0) 0x7f1e40001b90 con 0x7f1e480afb60 2024-03-11T18:36:19.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.596+0000 7f1e4659c640 1 -- 172.21.15.90:0/3206767538 >> v1:172.21.15.90:6789/0 conn(0x7f1e480afb60 legacy=0x7f1e4806c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:19.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.596+0000 7f1e4659c640 1 --2- 172.21.15.90:0/3206767538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e340039b0 0x7f1e34003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:19.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.596+0000 7f1e4659c640 1 --2- 172.21.15.90:0/3206767538 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e340047d0 0x7f1e34004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:19.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.596+0000 7f1e4659c640 1 --2- 172.21.15.90:0/3206767538 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1e48153f10 0x7f1e34005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:19.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.596+0000 7f1e4659c640 1 -- 172.21.15.90:0/3206767538 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3400ba40 con 0x7f1e340047d0 2024-03-11T18:36:19.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.596+0000 7f1e4659c640 1 -- 172.21.15.90:0/3206767538 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3400bbb0 con 0x7f1e340039b0 2024-03-11T18:36:19.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.596+0000 7f1e4659c640 1 -- 172.21.15.90:0/3206767538 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3400bd70 con 0x7f1e48153f10 2024-03-11T18:36:19.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.600+0000 7f1e46d9d640 1 --2- 172.21.15.90:0/3206767538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e340039b0 0x7f1e34003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:19.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.604+0000 7f1e4759e640 1 --2- 172.21.15.90:0/3206767538 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1e48153f10 0x7f1e34005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:19.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.628+0000 7f1e46d9d640 1 -- 172.21.15.90:0/3206767538 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1e48153f10 msgr2=0x7f1e34005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:19.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.628+0000 7f1e46d9d640 1 --2- 172.21.15.90:0/3206767538 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1e48153f10 0x7f1e34005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:19.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.628+0000 7f1e46d9d640 1 -- 172.21.15.90:0/3206767538 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e340047d0 msgr2=0x7f1e34004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:19.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.628+0000 7f1e46d9d640 1 --2- 172.21.15.90:0/3206767538 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e340047d0 0x7f1e34004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:19.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.628+0000 7f1e46d9d640 1 -- 172.21.15.90:0/3206767538 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e34003530 con 0x7f1e340039b0 2024-03-11T18:36:19.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.632+0000 7f1e46d9d640 1 --2- 172.21.15.90:0/3206767538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e340039b0 0x7f1e34003dd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1e38009ab0 tx=0x7f1e3801ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d106a220d101e01 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:19.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.732+0000 7f1e4659c640 1 -- 172.21.15.90:0/3206767538 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e380a5510 con 0x7f1e340039b0 2024-03-11T18:36:19.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.732+0000 7f1e4659c640 1 -- 172.21.15.90:0/3206767538 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e380a5670 con 0x7f1e340039b0 2024-03-11T18:36:19.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.732+0000 7f1e4659c640 1 -- 172.21.15.90:0/3206767538 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e380a5a60 con 0x7f1e340039b0 2024-03-11T18:36:19.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.732+0000 7f1e4d1d5640 1 -- 172.21.15.90:0/3206767538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e340039b0 msgr2=0x7f1e34003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:19.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.732+0000 7f1e4d1d5640 1 --2- 172.21.15.90:0/3206767538 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e340039b0 0x7f1e34003dd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1e38009ab0 tx=0x7f1e3801ca30 comp rx=0 tx=0).stop 2024-03-11T18:36:19.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.732+0000 7f1e4759e640 1 -- 172.21.15.90:0/3206767538 reap_dead start 2024-03-11T18:36:19.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.732+0000 7f1e4d1d5640 1 -- 172.21.15.90:0/3206767538 shutdown_connections 2024-03-11T18:36:19.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.732+0000 7f1e4d1d5640 1 -- 172.21.15.90:0/3206767538 >> 172.21.15.90:0/3206767538 conn(0x7f1e48070e80 msgr2=0x7f1e48066990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:19.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.732+0000 7f1e4d1d5640 1 -- 172.21.15.90:0/3206767538 shutdown_connections 2024-03-11T18:36:19.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.732+0000 7f1e4d1d5640 1 -- 172.21.15.90:0/3206767538 wait complete. 2024-03-11T18:36:19.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.736+0000 7f1e4d1d5640 1 Processor -- start 2024-03-11T18:36:19.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.736+0000 7f1e4d1d5640 1 -- start start 2024-03-11T18:36:19.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.736+0000 7f1e4d1d5640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1e340039b0 0x7f1e480b0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:19.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.736+0000 7f1e4d1d5640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e481493b0 0x7f1e480b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:19.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.736+0000 7f1e4d1d5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e481f3740 0x7f1e481f3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:19.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.736+0000 7f1e4d1d5640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4814ec00 con 0x7f1e481493b0 2024-03-11T18:36:19.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.736+0000 7f1e4d1d5640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4814e110 con 0x7f1e481f3740 2024-03-11T18:36:19.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.736+0000 7f1e4d1d5640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4814e360 con 0x7f1e340039b0 2024-03-11T18:36:19.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.736+0000 7f1e46d9d640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1e340039b0 0x7f1e480b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:19.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.736+0000 7f1e3e59c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e481493b0 0x7f1e480b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:19.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.736+0000 7f1e3e59c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e481493b0 0x7f1e480b26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43074/0 (socket says 172.21.15.90:43074) 2024-03-11T18:36:19.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.736+0000 7f1e3e59c640 1 -- 172.21.15.90:0/2671153583 learned_addr learned my addr 172.21.15.90:0/2671153583 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:19.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.740+0000 7f1e4759e640 1 --2- 172.21.15.90:0/2671153583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e481f3740 0x7f1e481f3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:19.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.740+0000 7f1e3e59c640 1 -- 172.21.15.90:0/2671153583 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1e340039b0 msgr2=0x7f1e480b0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:19.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.740+0000 7f1e3e59c640 1 --2- 172.21.15.90:0/2671153583 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1e340039b0 0x7f1e480b0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:19.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.740+0000 7f1e3e59c640 1 -- 172.21.15.90:0/2671153583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e481f3740 msgr2=0x7f1e481f3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:19.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.740+0000 7f1e3e59c640 1 --2- 172.21.15.90:0/2671153583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e481f3740 0x7f1e481f3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:19.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.740+0000 7f1e3e59c640 1 -- 172.21.15.90:0/2671153583 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e4814e640 con 0x7f1e481493b0 2024-03-11T18:36:19.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.744+0000 7f1e3e59c640 1 --2- 172.21.15.90:0/2671153583 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e481493b0 0x7f1e480b26d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1e3000c490 tx=0x7f1e3000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:19.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.748+0000 7f1e3ffff640 1 -- 172.21.15.90:0/2671153583 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e300052d0 con 0x7f1e481493b0 2024-03-11T18:36:19.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.748+0000 7f1e3ffff640 1 -- 172.21.15.90:0/2671153583 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e30005450 con 0x7f1e481493b0 2024-03-11T18:36:19.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.748+0000 7f1e4d1d5640 1 -- 172.21.15.90:0/2671153583 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e4814e920 con 0x7f1e481493b0 2024-03-11T18:36:19.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.748+0000 7f1e3ffff640 1 -- 172.21.15.90:0/2671153583 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e30003840 con 0x7f1e481493b0 2024-03-11T18:36:19.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.748+0000 7f1e4d1d5640 1 -- 172.21.15.90:0/2671153583 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e48149810 con 0x7f1e481493b0 2024-03-11T18:36:19.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.748+0000 7f1e4d1d5640 1 -- 172.21.15.90:0/2671153583 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e4804eef0 con 0x7f1e481493b0 2024-03-11T18:36:20.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.996+0000 7f1e3ffff640 1 -- 172.21.15.90:0/2671153583 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f1e300039a0 con 0x7f1e481493b0 2024-03-11T18:36:20.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.996+0000 7f1e3ffff640 1 --2- 172.21.15.90:0/2671153583 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1e28066ec0 0x7f1e28069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:20.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.996+0000 7f1e46d9d640 1 --2- 172.21.15.90:0/2671153583 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1e28066ec0 0x7f1e28069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:20.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:19.996+0000 7f1e46d9d640 1 --2- 172.21.15.90:0/2671153583 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1e28066ec0 0x7f1e28069380 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1e380af000 tx=0x7f1e380046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:20.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.024+0000 7f1e3ffff640 1 -- 172.21.15.90:0/2671153583 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f1e30055080 con 0x7f1e481493b0 2024-03-11T18:36:20.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.024+0000 7f1e3ffff640 1 -- 172.21.15.90:0/2671153583 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1e30051320 con 0x7f1e481493b0 2024-03-11T18:36:20.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.212+0000 7f1e4d1d5640 1 -- 172.21.15.90:0/2671153583 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1e48151750 con 0x7f1e481493b0 2024-03-11T18:36:20.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.224+0000 7f1e3ffff640 1 -- 172.21.15.90:0/2671153583 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1e30017020 con 0x7f1e481493b0 2024-03-11T18:36:20.229 INFO:teuthology.orchestra.run.smithi090.stderr:listed 0 entries 2024-03-11T18:36:20.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.224+0000 7f1e4d1d5640 1 -- 172.21.15.90:0/2671153583 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1e28066ec0 msgr2=0x7f1e28069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:20.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.224+0000 7f1e4d1d5640 1 --2- 172.21.15.90:0/2671153583 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1e28066ec0 0x7f1e28069380 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1e380af000 tx=0x7f1e380046d0 comp rx=0 tx=0).stop 2024-03-11T18:36:20.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.224+0000 7f1e4d1d5640 1 -- 172.21.15.90:0/2671153583 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e481493b0 msgr2=0x7f1e480b26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:20.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.224+0000 7f1e4d1d5640 1 --2- 172.21.15.90:0/2671153583 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e481493b0 0x7f1e480b26d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1e3000c490 tx=0x7f1e3000c950 comp rx=0 tx=0).stop 2024-03-11T18:36:20.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.224+0000 7f1e4d1d5640 1 -- 172.21.15.90:0/2671153583 shutdown_connections 2024-03-11T18:36:20.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.224+0000 7f1e4d1d5640 1 --2- 172.21.15.90:0/2671153583 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1e28066ec0 0x7f1e28069380 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:20.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.224+0000 7f1e4d1d5640 1 --2- 172.21.15.90:0/2671153583 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1e481f3740 0x7f1e481f3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:20.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.224+0000 7f1e4d1d5640 1 --2- 172.21.15.90:0/2671153583 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e481493b0 0x7f1e480b26d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:20.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.224+0000 7f1e4d1d5640 1 --2- 172.21.15.90:0/2671153583 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1e340039b0 0x7f1e480b0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:20.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.224+0000 7f1e4d1d5640 1 -- 172.21.15.90:0/2671153583 >> 172.21.15.90:0/2671153583 conn(0x7f1e48070e80 msgr2=0x7f1e480adc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:20.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.224+0000 7f1e4d1d5640 1 -- 172.21.15.90:0/2671153583 shutdown_connections 2024-03-11T18:36:20.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.224+0000 7f1e4d1d5640 1 -- 172.21.15.90:0/2671153583 wait complete. 2024-03-11T18:36:20.249 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-11T18:36:20.249 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-11T18:36:20.254 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T18:36:20.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.412+0000 7fa2cbe7e640 1 Processor -- start 2024-03-11T18:36:20.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.412+0000 7fa2cbe7e640 1 -- start start 2024-03-11T18:36:20.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.412+0000 7fa2cbe7e640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fa2c4153f10 0x7fa2c415a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:20.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.412+0000 7fa2cbe7e640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa2c40c8e20 0x7fa2c415db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:20.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.412+0000 7fa2cbe7e640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa2c40afb60 0x7fa2c406c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:20.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.412+0000 7fa2cbe7e640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2c406c2d0 con 0x7fa2c4153f10 2024-03-11T18:36:20.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.412+0000 7fa2cbe7e640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2c415abb0 con 0x7fa2c40c8e20 2024-03-11T18:36:20.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.412+0000 7fa2cbe7e640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c415e040 con 0x7fa2c40afb60 2024-03-11T18:36:20.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.412+0000 7fa2c9c1a640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fa2c4153f10 0x7fa2c415a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:20.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.412+0000 7fa2c9c1a640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fa2c4153f10 0x7fa2c415a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43080/0 (socket says 172.21.15.90:43080) 2024-03-11T18:36:20.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.412+0000 7fa2c9c1a640 1 -- 172.21.15.90:0/624097804 learned_addr learned my addr 172.21.15.90:0/624097804 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:20.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.412+0000 7fa2ca41b640 1 --2- 172.21.15.90:0/624097804 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa2c40afb60 0x7fa2c406c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:20.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.412+0000 7fa2c9419640 1 --2- 172.21.15.90:0/624097804 >> v2:172.21.15.92:3300/0 conn(0x7fa2c40c8e20 0x7fa2c415db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:20.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.416+0000 7fa2c9c1a640 1 -- 172.21.15.90:0/624097804 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa2c40afb60 msgr2=0x7fa2c406c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:20.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.416+0000 7fa2c9c1a640 1 --2- 172.21.15.90:0/624097804 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa2c40afb60 0x7fa2c406c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:20.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.416+0000 7fa2c9c1a640 1 -- 172.21.15.90:0/624097804 >> v2:172.21.15.92:3300/0 conn(0x7fa2c40c8e20 msgr2=0x7fa2c415db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:20.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.416+0000 7fa2c9c1a640 1 --2- 172.21.15.90:0/624097804 >> v2:172.21.15.92:3300/0 conn(0x7fa2c40c8e20 0x7fa2c415db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:20.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.416+0000 7fa2c9c1a640 1 -- 172.21.15.90:0/624097804 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c40c81e0 con 0x7fa2c4153f10 2024-03-11T18:36:20.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.420+0000 7fa2c9c1a640 1 --2- 172.21.15.90:0/624097804 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa2c4153f10 0x7fa2c415a670 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa2c406e2a0 tx=0x7fa2c001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=260445cfacf606c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:20.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.424+0000 7fa2c8c18640 1 -- 172.21.15.90:0/624097804 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c00a5870 con 0x7fa2c4153f10 2024-03-11T18:36:20.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.424+0000 7fa2c8c18640 1 -- 172.21.15.90:0/624097804 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2c00a59d0 con 0x7fa2c4153f10 2024-03-11T18:36:20.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.424+0000 7fa2c8c18640 1 -- 172.21.15.90:0/624097804 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c00a5d60 con 0x7fa2c4153f10 2024-03-11T18:36:20.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.424+0000 7fa2cbe7e640 1 -- 172.21.15.90:0/624097804 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa2c4153f10 msgr2=0x7fa2c415a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:20.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.424+0000 7fa2cbe7e640 1 --2- 172.21.15.90:0/624097804 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa2c4153f10 0x7fa2c415a670 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa2c406e2a0 tx=0x7fa2c001c9f0 comp rx=0 tx=0).stop 2024-03-11T18:36:20.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.424+0000 7fa2cbe7e640 1 -- 172.21.15.90:0/624097804 shutdown_connections 2024-03-11T18:36:20.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.424+0000 7fa2cbe7e640 1 --2- 172.21.15.90:0/624097804 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa2c40afb60 0x7fa2c406c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:20.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.424+0000 7fa2cbe7e640 1 --2- 172.21.15.90:0/624097804 >> v2:172.21.15.92:3300/0 conn(0x7fa2c40c8e20 0x7fa2c415db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:20.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.424+0000 7fa2cbe7e640 1 --2- 172.21.15.90:0/624097804 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa2c4153f10 0x7fa2c415a670 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:20.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.424+0000 7fa2cbe7e640 1 -- 172.21.15.90:0/624097804 >> 172.21.15.90:0/624097804 conn(0x7fa2c4070e80 msgr2=0x7fa2c4066990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:20.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.424+0000 7fa2cbe7e640 1 -- 172.21.15.90:0/624097804 shutdown_connections 2024-03-11T18:36:20.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.424+0000 7fa2cbe7e640 1 -- 172.21.15.90:0/624097804 wait complete. 2024-03-11T18:36:20.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.424+0000 7fa2cbe7e640 1 Processor -- start 2024-03-11T18:36:20.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.424+0000 7fa2cbe7e640 1 -- start start 2024-03-11T18:36:20.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.428+0000 7fa2cbe7e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2c40afb60 0x7fa2c4154e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:20.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.428+0000 7fa2cbe7e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa2c40c8e20 0x7fa2c4152d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:20.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.428+0000 7fa2cbe7e640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa2c4153f10 0x7fa2c4155f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:20.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.428+0000 7fa2cbe7e640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c414df90 con 0x7fa2c40c8e20 2024-03-11T18:36:20.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.428+0000 7fa2cbe7e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c414eb10 con 0x7fa2c40afb60 2024-03-11T18:36:20.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.428+0000 7fa2cbe7e640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c414f020 con 0x7fa2c4153f10 2024-03-11T18:36:20.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.428+0000 7fa2c9419640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa2c40c8e20 0x7fa2c4152d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:20.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.428+0000 7fa2c9419640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa2c40c8e20 0x7fa2c4152d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43090/0 (socket says 172.21.15.90:43090) 2024-03-11T18:36:20.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.428+0000 7fa2c9419640 1 -- 172.21.15.90:0/3734475717 learned_addr learned my addr 172.21.15.90:0/3734475717 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:20.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.428+0000 7fa2ca41b640 1 --2- 172.21.15.90:0/3734475717 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa2c4153f10 0x7fa2c4155f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:20.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.428+0000 7fa2c9c1a640 1 --2- 172.21.15.90:0/3734475717 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2c40afb60 0x7fa2c4154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:20.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.432+0000 7fa2c9419640 1 -- 172.21.15.90:0/3734475717 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa2c4153f10 msgr2=0x7fa2c4155f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:20.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.432+0000 7fa2c9419640 1 --2- 172.21.15.90:0/3734475717 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa2c4153f10 0x7fa2c4155f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:20.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.432+0000 7fa2c9419640 1 -- 172.21.15.90:0/3734475717 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2c40afb60 msgr2=0x7fa2c4154e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:20.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.432+0000 7fa2c9419640 1 --2- 172.21.15.90:0/3734475717 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2c40afb60 0x7fa2c4154e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:20.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.432+0000 7fa2c9419640 1 -- 172.21.15.90:0/3734475717 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c0009b30 con 0x7fa2c40c8e20 2024-03-11T18:36:20.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.432+0000 7fa2c9419640 1 --2- 172.21.15.90:0/3734475717 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa2c40c8e20 0x7fa2c4152d30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa2b400d620 tx=0x7fa2b400da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:20.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.436+0000 7fa2ba7fc640 1 -- 172.21.15.90:0/3734475717 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b40125f0 con 0x7fa2c40c8e20 2024-03-11T18:36:20.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.436+0000 7fa2ba7fc640 1 -- 172.21.15.90:0/3734475717 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b4012750 con 0x7fa2c40c8e20 2024-03-11T18:36:20.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.436+0000 7fa2ba7fc640 1 -- 172.21.15.90:0/3734475717 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4012ae0 con 0x7fa2c40c8e20 2024-03-11T18:36:20.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.436+0000 7fa2cbe7e640 1 -- 172.21.15.90:0/3734475717 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2c414ed40 con 0x7fa2c40c8e20 2024-03-11T18:36:20.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.436+0000 7fa2cbe7e640 1 -- 172.21.15.90:0/3734475717 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2c414dcb0 con 0x7fa2c40c8e20 2024-03-11T18:36:20.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.436+0000 7fa2cbe7e640 1 -- 172.21.15.90:0/3734475717 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2c404eef0 con 0x7fa2c40c8e20 2024-03-11T18:36:20.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.456+0000 7fa2ba7fc640 1 -- 172.21.15.90:0/3734475717 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fa2b4012c40 con 0x7fa2c40c8e20 2024-03-11T18:36:20.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.456+0000 7fa2ba7fc640 1 --2- 172.21.15.90:0/3734475717 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fa2a8066ec0 0x7fa2a8069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:20.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.456+0000 7fa2c9c1a640 1 --2- 172.21.15.90:0/3734475717 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fa2a8066ec0 0x7fa2a8069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:20.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.460+0000 7fa2c9c1a640 1 --2- 172.21.15.90:0/3734475717 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fa2a8066ec0 0x7fa2a8069380 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa2c00ae000 tx=0x7fa2c00a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:20.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.460+0000 7fa2ba7fc640 1 -- 172.21.15.90:0/3734475717 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa2b4056a40 con 0x7fa2c40c8e20 2024-03-11T18:36:20.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.720+0000 7fa2ba7fc640 1 -- 172.21.15.90:0/3734475717 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa2b40525e0 con 0x7fa2c40c8e20 2024-03-11T18:36:20.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.896+0000 7fa2cbe7e640 1 -- 172.21.15.90:0/3734475717 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa2c40c84f0 con 0x7fa2c40c8e20 2024-03-11T18:36:20.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.968+0000 7fa2ba7fc640 1 -- 172.21.15.90:0/3734475717 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18430 (secure 0 0 0) 0x7fa2b4007380 con 0x7fa2c40c8e20 2024-03-11T18:36:20.977 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:36:20.977 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":28,"fsid":"76d56b10-6c29-476a-804b-8482ea90f9e2","created":"2024-03-11T18:34:54.521741+0000","modified":"2024-03-11T18:36:05.526044+0000","last_up_change":"2024-03-11T18:35:16.207500+0000","last_in_change":"2024-03-11T18:35:05.213634+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T18:35:16.193842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T18:35:54.001469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T18:35:55.917321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T18:36:02.210943+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"34decff9-7280-4d84-9665-d6af8ea6de84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6819","nonce":1755667921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6821","nonce":1755667921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6825","nonce":1755667921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6823","nonce":1755667921}]},"public_addr":"172.21.15.90:6819/1755667921","cluster_addr":"172.21.15.90:6821/1755667921","heartbeat_back_addr":"172.21.15.90:6825/1755667921","heartbeat_front_addr":"172.21.15.90:6823/1755667921","state":["exists","up"]},{"osd":1,"uuid":"8ecdce9e-5486-415a-a609-dd16e945440e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6807","nonce":438682081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6809","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6811","nonce":438682081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6817","nonce":438682081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6815","nonce":438682081}]},"public_addr":"172.21.15.90:6807/438682081","cluster_addr":"172.21.15.90:6811/438682081","heartbeat_back_addr":"172.21.15.90:6817/438682081","heartbeat_front_addr":"172.21.15.90:6815/438682081","state":["exists","up"]},{"osd":2,"uuid":"737486d4-0e90-4c82-9e79-bf37c812e5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6827","nonce":2837704738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6829","nonce":2837704738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6833","nonce":2837704738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6831","nonce":2837704738}]},"public_addr":"172.21.15.90:6827/2837704738","cluster_addr":"172.21.15.90:6829/2837704738","heartbeat_back_addr":"172.21.15.90:6833/2837704738","heartbeat_front_addr":"172.21.15.90:6831/2837704738","state":["exists","up"]},{"osd":3,"uuid":"698bd661-bbef-4699-a619-f7bfe23ce12c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6803","nonce":240987971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6805","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6806","nonce":240987971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6813","nonce":240987971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6810","nonce":240987971}]},"public_addr":"172.21.15.90:6803/240987971","cluster_addr":"172.21.15.90:6806/240987971","heartbeat_back_addr":"172.21.15.90:6813/240987971","heartbeat_front_addr":"172.21.15.90:6810/240987971","state":["exists","up"]},{"osd":4,"uuid":"aba9e31f-73d9-43fe-a10b-2dc5832968b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6801","nonce":502685275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6803","nonce":502685275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6807","nonce":502685275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6805","nonce":502685275}]},"public_addr":"172.21.15.92:6801/502685275","cluster_addr":"172.21.15.92:6803/502685275","heartbeat_back_addr":"172.21.15.92:6807/502685275","heartbeat_front_addr":"172.21.15.92:6805/502685275","state":["exists","up"]},{"osd":5,"uuid":"bdd56766-4b28-4007-8de3-b5b08cf335d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6809","nonce":200142070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6811","nonce":200142070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6815","nonce":200142070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6813","nonce":200142070}]},"public_addr":"172.21.15.92:6809/200142070","cluster_addr":"172.21.15.92:6811/200142070","heartbeat_back_addr":"172.21.15.92:6815/200142070","heartbeat_front_addr":"172.21.15.92:6813/200142070","state":["exists","up"]},{"osd":6,"uuid":"6c580a59-610f-455e-a96b-033299dcc118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6817","nonce":1919010204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6819","nonce":1919010204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6823","nonce":1919010204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6821","nonce":1919010204}]},"public_addr":"172.21.15.92:6817/1919010204","cluster_addr":"172.21.15.92:6819/1919010204","heartbeat_back_addr":"172.21.15.92:6823/1919010204","heartbeat_front_addr":"172.21.15.92:6821/1919010204","state":["exists","up"]},{"osd":7,"uuid":"958184cc-aebf-4c5b-91a0-2a3e38a6b42b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6825","nonce":3475925719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6827","nonce":3475925719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6831","nonce":3475925719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6829","nonce":3475925719}]},"public_addr":"172.21.15.92:6825/3475925719","cluster_addr":"172.21.15.92:6827/3475925719","heartbeat_back_addr":"172.21.15.92:6831/3475925719","heartbeat_front_addr":"172.21.15.92:6829/3475925719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.816294+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.048084+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.137975+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.968246+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.016829+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.931676+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.041673+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.079404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T18:36:20.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.972+0000 7fa2cbe7e640 1 -- 172.21.15.90:0/3734475717 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fa2a8066ec0 msgr2=0x7fa2a8069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:20.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.972+0000 7fa2cbe7e640 1 --2- 172.21.15.90:0/3734475717 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fa2a8066ec0 0x7fa2a8069380 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa2c00ae000 tx=0x7fa2c00a7070 comp rx=0 tx=0).stop 2024-03-11T18:36:20.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.972+0000 7fa2cbe7e640 1 -- 172.21.15.90:0/3734475717 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa2c40c8e20 msgr2=0x7fa2c4152d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:20.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.972+0000 7fa2cbe7e640 1 --2- 172.21.15.90:0/3734475717 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa2c40c8e20 0x7fa2c4152d30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa2b400d620 tx=0x7fa2b400da20 comp rx=0 tx=0).stop 2024-03-11T18:36:20.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.972+0000 7fa2cbe7e640 1 -- 172.21.15.90:0/3734475717 shutdown_connections 2024-03-11T18:36:20.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.972+0000 7fa2cbe7e640 1 --2- 172.21.15.90:0/3734475717 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fa2a8066ec0 0x7fa2a8069380 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:20.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.972+0000 7fa2cbe7e640 1 --2- 172.21.15.90:0/3734475717 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa2c4153f10 0x7fa2c4155f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:20.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.972+0000 7fa2cbe7e640 1 --2- 172.21.15.90:0/3734475717 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa2c40c8e20 0x7fa2c4152d30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:20.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.972+0000 7fa2cbe7e640 1 --2- 172.21.15.90:0/3734475717 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa2c40afb60 0x7fa2c4154e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:20.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.972+0000 7fa2cbe7e640 1 -- 172.21.15.90:0/3734475717 >> 172.21.15.90:0/3734475717 conn(0x7fa2c4070e80 msgr2=0x7fa2c40c39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:20.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.972+0000 7fa2cbe7e640 1 -- 172.21.15.90:0/3734475717 shutdown_connections 2024-03-11T18:36:20.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:20.972+0000 7fa2cbe7e640 1 -- 172.21.15.90:0/3734475717 wait complete. 2024-03-11T18:36:21.001 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-11T18:35:16.193842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-11T18:35:54.001469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-11T18:35:55.917321+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-11T18:36:02.210943+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-11T18:36:21.002 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-11T18:36:21.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.120+0000 7f930f383640 1 Processor -- start 2024-03-11T18:36:21.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.120+0000 7f930f383640 1 -- start start 2024-03-11T18:36:21.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.120+0000 7f930f383640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f9308151b30 0x7f930806e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:21.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.120+0000 7f930f383640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f93080c4af0 0x7f93080c4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:21.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.120+0000 7f930f383640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f930806f430 con 0x7f9308070b00 2024-03-11T18:36:21.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.120+0000 7f930f383640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93080c65b0 con 0x7f9308151b30 2024-03-11T18:36:21.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.120+0000 7f930f383640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f930806e820 con 0x7f93080c4af0 2024-03-11T18:36:21.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.120+0000 7f930d11f640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f9308151b30 0x7f930806e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:21.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.120+0000 7f930d11f640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f9308151b30 0x7f930806e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43106/0 (socket says 172.21.15.90:43106) 2024-03-11T18:36:21.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.120+0000 7f930d11f640 1 -- 172.21.15.90:0/2400311676 learned_addr learned my addr 172.21.15.90:0/2400311676 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:21.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.120+0000 7f930d920640 1 --2- 172.21.15.90:0/2400311676 >> v2:172.21.15.92:3300/0 conn(0x7f93080c4af0 0x7f93080c4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:21.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.124+0000 7f930d11f640 1 -- 172.21.15.90:0/2400311676 >> v2:172.21.15.92:3300/0 conn(0x7f93080c4af0 msgr2=0x7f93080c4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:21.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.124+0000 7f930d11f640 1 --2- 172.21.15.90:0/2400311676 >> v2:172.21.15.92:3300/0 conn(0x7f93080c4af0 0x7f93080c4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:21.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.124+0000 7f930d11f640 1 -- 172.21.15.90:0/2400311676 >> v1:172.21.15.90:6789/0 conn(0x7f9308070b00 legacy=0x7f93080c5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:21.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.124+0000 7f930d11f640 1 -- 172.21.15.90:0/2400311676 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f930806eb00 con 0x7f9308151b30 2024-03-11T18:36:21.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.128+0000 7f930d11f640 1 --2- 172.21.15.90:0/2400311676 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9308151b30 0x7f930806e210 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f930806c220 tx=0x7f930401c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34afa93b9e61170a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:21.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.132+0000 7f92fffff640 1 -- 172.21.15.90:0/2400311676 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93040a5840 con 0x7f9308151b30 2024-03-11T18:36:21.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.132+0000 7f92fffff640 1 -- 172.21.15.90:0/2400311676 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93040a59a0 con 0x7f9308151b30 2024-03-11T18:36:21.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.132+0000 7f92fffff640 1 -- 172.21.15.90:0/2400311676 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93040a5d30 con 0x7f9308151b30 2024-03-11T18:36:21.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.132+0000 7f930f383640 1 -- 172.21.15.90:0/2400311676 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9308151b30 msgr2=0x7f930806e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:21.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.132+0000 7f930f383640 1 --2- 172.21.15.90:0/2400311676 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9308151b30 0x7f930806e210 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f930806c220 tx=0x7f930401c980 comp rx=0 tx=0).stop 2024-03-11T18:36:21.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.132+0000 7f930f383640 1 -- 172.21.15.90:0/2400311676 shutdown_connections 2024-03-11T18:36:21.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.132+0000 7f930f383640 1 --2- 172.21.15.90:0/2400311676 >> v2:172.21.15.92:3300/0 conn(0x7f93080c4af0 0x7f93080c4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:21.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.132+0000 7f930f383640 1 --2- 172.21.15.90:0/2400311676 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9308151b30 0x7f930806e210 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:21.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.132+0000 7f930f383640 1 -- 172.21.15.90:0/2400311676 >> 172.21.15.90:0/2400311676 conn(0x7f930806ac00 msgr2=0x7f930806b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:21.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.132+0000 7f930f383640 1 -- 172.21.15.90:0/2400311676 shutdown_connections 2024-03-11T18:36:21.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.132+0000 7f930f383640 1 -- 172.21.15.90:0/2400311676 wait complete. 2024-03-11T18:36:21.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.132+0000 7f930f383640 1 Processor -- start 2024-03-11T18:36:21.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.132+0000 7f930f383640 1 -- start start 2024-03-11T18:36:21.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.136+0000 7f930f383640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9308070b00 0x7f93081599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:21.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.136+0000 7f930f383640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f93080c4af0 0x7f9308147220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:21.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.136+0000 7f930f383640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9308151b30 0x7f9308143dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:21.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.136+0000 7f930f383640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930815be10 con 0x7f9308070b00 2024-03-11T18:36:21.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.136+0000 7f930f383640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93080cad30 con 0x7f9308151b30 2024-03-11T18:36:21.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.136+0000 7f930f383640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93080cb010 con 0x7f93080c4af0 2024-03-11T18:36:21.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.136+0000 7f930d11f640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9308070b00 0x7f93081599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:21.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.136+0000 7f930d11f640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9308070b00 0x7f93081599e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43112/0 (socket says 172.21.15.90:43112) 2024-03-11T18:36:21.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.136+0000 7f930d11f640 1 -- 172.21.15.90:0/1200841172 learned_addr learned my addr 172.21.15.90:0/1200841172 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:21.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.136+0000 7f930d920640 1 --2- 172.21.15.90:0/1200841172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9308151b30 0x7f9308143dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:21.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.136+0000 7f930c91e640 1 --2- 172.21.15.90:0/1200841172 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f93080c4af0 0x7f9308147220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:21.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.140+0000 7f930d11f640 1 -- 172.21.15.90:0/1200841172 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f93080c4af0 msgr2=0x7f9308147220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:21.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.140+0000 7f930d11f640 1 --2- 172.21.15.90:0/1200841172 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f93080c4af0 0x7f9308147220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:21.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.140+0000 7f930d11f640 1 -- 172.21.15.90:0/1200841172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9308151b30 msgr2=0x7f9308143dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:21.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.140+0000 7f930d11f640 1 --2- 172.21.15.90:0/1200841172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9308151b30 0x7f9308143dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:21.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.140+0000 7f930d11f640 1 -- 172.21.15.90:0/1200841172 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9304009b30 con 0x7f9308070b00 2024-03-11T18:36:21.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.144+0000 7f930d11f640 1 --2- 172.21.15.90:0/1200841172 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9308070b00 0x7f93081599e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f93040af040 tx=0x7f9304019720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:21.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.148+0000 7f92fdffb640 1 -- 172.21.15.90:0/1200841172 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93040b8670 con 0x7f9308070b00 2024-03-11T18:36:21.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.148+0000 7f92fdffb640 1 -- 172.21.15.90:0/1200841172 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93040b8810 con 0x7f9308070b00 2024-03-11T18:36:21.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.148+0000 7f92fdffb640 1 -- 172.21.15.90:0/1200841172 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93040b8bc0 con 0x7f9308070b00 2024-03-11T18:36:21.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.148+0000 7f930f383640 1 -- 172.21.15.90:0/1200841172 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9308146740 con 0x7f9308070b00 2024-03-11T18:36:21.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.148+0000 7f930f383640 1 -- 172.21.15.90:0/1200841172 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9308146d00 con 0x7f9308070b00 2024-03-11T18:36:21.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.148+0000 7f930f383640 1 -- 172.21.15.90:0/1200841172 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f930804eef0 con 0x7f9308070b00 2024-03-11T18:36:21.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.428+0000 7f92fdffb640 1 -- 172.21.15.90:0/1200841172 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f93040b8d20 con 0x7f9308070b00 2024-03-11T18:36:21.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.428+0000 7f92fdffb640 1 --2- 172.21.15.90:0/1200841172 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f92ec066ec0 0x7f92ec069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:21.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.432+0000 7f930c91e640 1 --2- 172.21.15.90:0/1200841172 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f92ec066ec0 0x7f92ec069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:21.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.432+0000 7f930c91e640 1 --2- 172.21.15.90:0/1200841172 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f92ec066ec0 0x7f92ec069380 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9308071930 tx=0x7f92f8009060 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:21.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.464+0000 7f92fdffb640 1 -- 172.21.15.90:0/1200841172 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f93040f1670 con 0x7f9308070b00 2024-03-11T18:36:21.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.464+0000 7f92fdffb640 1 -- 172.21.15.90:0/1200841172 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f93040ed210 con 0x7f9308070b00 2024-03-11T18:36:21.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.636+0000 7f930f383640 1 -- 172.21.15.90:0/1200841172 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f93081620d0 con 0x7f9308070b00 2024-03-11T18:36:21.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.644+0000 7f92fdffb640 1 -- 172.21.15.90:0/1200841172 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f93040f08d0 con 0x7f9308070b00 2024-03-11T18:36:21.649 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 1 2024-03-11T18:36:21.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.644+0000 7f930f383640 1 -- 172.21.15.90:0/1200841172 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f92ec066ec0 msgr2=0x7f92ec069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:21.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.644+0000 7f930f383640 1 --2- 172.21.15.90:0/1200841172 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f92ec066ec0 0x7f92ec069380 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9308071930 tx=0x7f92f8009060 comp rx=0 tx=0).stop 2024-03-11T18:36:21.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.644+0000 7f930f383640 1 -- 172.21.15.90:0/1200841172 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9308070b00 msgr2=0x7f93081599e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:21.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.644+0000 7f930f383640 1 --2- 172.21.15.90:0/1200841172 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9308070b00 0x7f93081599e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f93040af040 tx=0x7f9304019720 comp rx=0 tx=0).stop 2024-03-11T18:36:21.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.644+0000 7f930f383640 1 -- 172.21.15.90:0/1200841172 shutdown_connections 2024-03-11T18:36:21.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.644+0000 7f930f383640 1 --2- 172.21.15.90:0/1200841172 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f92ec066ec0 0x7f92ec069380 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:21.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.644+0000 7f930f383640 1 --2- 172.21.15.90:0/1200841172 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9308151b30 0x7f9308143dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:21.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.644+0000 7f930f383640 1 --2- 172.21.15.90:0/1200841172 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f93080c4af0 0x7f9308147220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:21.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.644+0000 7f930f383640 1 --2- 172.21.15.90:0/1200841172 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9308070b00 0x7f93081599e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:21.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.644+0000 7f930f383640 1 -- 172.21.15.90:0/1200841172 >> 172.21.15.90:0/1200841172 conn(0x7f930806ac00 msgr2=0x7f930814f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:21.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.644+0000 7f930f383640 1 -- 172.21.15.90:0/1200841172 shutdown_connections 2024-03-11T18:36:21.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.644+0000 7f930f383640 1 -- 172.21.15.90:0/1200841172 wait complete. 2024-03-11T18:36:21.667 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-11T18:36:21.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.788+0000 7f3c032fc640 1 Processor -- start 2024-03-11T18:36:21.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.788+0000 7f3c032fc640 1 -- start start 2024-03-11T18:36:21.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.788+0000 7f3c032fc640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f3bfc1509f0 0x7f3bfc157390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:21.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.788+0000 7f3c032fc640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3bfc150dd0 0x7f3bfc0c28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:21.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.788+0000 7f3c032fc640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bfc0c5de0 con 0x7f3bfc070b00 2024-03-11T18:36:21.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.788+0000 7f3c032fc640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bfc156910 con 0x7f3bfc1509f0 2024-03-11T18:36:21.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.788+0000 7f3c032fc640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bfc1578d0 con 0x7f3bfc150dd0 2024-03-11T18:36:21.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.788+0000 7f3c01098640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f3bfc1509f0 0x7f3bfc157390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:21.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.788+0000 7f3c01899640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f3bfc070b00 0x7f3bfc0c4580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:44412/0 (socket says 172.21.15.90:44412) 2024-03-11T18:36:21.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.788+0000 7f3c01899640 1 -- 172.21.15.90:0/3848354069 learned_addr learned my addr 172.21.15.90:0/3848354069 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T18:36:21.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.788+0000 7f3c00897640 1 --2- 172.21.15.90:0/3848354069 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3bfc150dd0 0x7f3bfc0c28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:21.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.792+0000 7f3c01098640 1 -- 172.21.15.90:0/3848354069 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3bfc150dd0 msgr2=0x7f3bfc0c28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:21.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.792+0000 7f3c01098640 1 --2- 172.21.15.90:0/3848354069 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3bfc150dd0 0x7f3bfc0c28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:21.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.792+0000 7f3c01098640 1 -- 172.21.15.90:0/3848354069 >> v1:172.21.15.90:6789/0 conn(0x7f3bfc070b00 legacy=0x7f3bfc0c4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:21.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.792+0000 7f3c01098640 1 -- 172.21.15.90:0/3848354069 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bfc0c6030 con 0x7f3bfc1509f0 2024-03-11T18:36:21.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.800+0000 7f3c01098640 1 --2- 172.21.15.90:0/3848354069 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3bfc1509f0 0x7f3bfc157390 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3bf80097c0 tx=0x7f3bf8020ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7fdf4e263bc1f04 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:21.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.824+0000 7f3bf3fff640 1 -- 172.21.15.90:0/3848354069 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf80aaba0 con 0x7f3bfc1509f0 2024-03-11T18:36:21.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.824+0000 7f3bf3fff640 1 -- 172.21.15.90:0/3848354069 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bf80aad00 con 0x7f3bfc1509f0 2024-03-11T18:36:21.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.824+0000 7f3bf3fff640 1 -- 172.21.15.90:0/3848354069 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf80b2660 con 0x7f3bfc1509f0 2024-03-11T18:36:21.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.824+0000 7f3c032fc640 1 -- 172.21.15.90:0/3848354069 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3bfc1509f0 msgr2=0x7f3bfc157390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:21.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.824+0000 7f3c032fc640 1 --2- 172.21.15.90:0/3848354069 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3bfc1509f0 0x7f3bfc157390 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3bf80097c0 tx=0x7f3bf8020ea0 comp rx=0 tx=0).stop 2024-03-11T18:36:21.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.824+0000 7f3c032fc640 1 -- 172.21.15.90:0/3848354069 shutdown_connections 2024-03-11T18:36:21.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.824+0000 7f3c032fc640 1 --2- 172.21.15.90:0/3848354069 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3bfc150dd0 0x7f3bfc0c28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:21.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.824+0000 7f3c032fc640 1 --2- 172.21.15.90:0/3848354069 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3bfc1509f0 0x7f3bfc157390 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:21.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.824+0000 7f3c032fc640 1 -- 172.21.15.90:0/3848354069 >> 172.21.15.90:0/3848354069 conn(0x7f3bfc06ac00 msgr2=0x7f3bfc06b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:21.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.824+0000 7f3c032fc640 1 -- 172.21.15.90:0/3848354069 shutdown_connections 2024-03-11T18:36:21.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.824+0000 7f3c032fc640 1 -- 172.21.15.90:0/3848354069 wait complete. 2024-03-11T18:36:21.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.824+0000 7f3c032fc640 1 Processor -- start 2024-03-11T18:36:21.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.824+0000 7f3c032fc640 1 -- start start 2024-03-11T18:36:21.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.828+0000 7f3c032fc640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3bfc070b00 0x7f3bfc1e0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:21.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.828+0000 7f3c032fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bfc1509f0 0x7f3bfc1e1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:21.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.828+0000 7f3c032fc640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3bfc150dd0 0x7f3bfc158200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:21.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.828+0000 7f3c032fc640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bfc157ad0 con 0x7f3bfc070b00 2024-03-11T18:36:21.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.828+0000 7f3c032fc640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bfc1e0e70 con 0x7f3bfc1509f0 2024-03-11T18:36:21.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.828+0000 7f3c032fc640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bfc140a60 con 0x7f3bfc150dd0 2024-03-11T18:36:21.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.828+0000 7f3c01098640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3bfc070b00 0x7f3bfc1e0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:21.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.828+0000 7f3c01098640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3bfc070b00 0x7f3bfc1e0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43140/0 (socket says 172.21.15.90:43140) 2024-03-11T18:36:21.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.828+0000 7f3c01098640 1 -- 172.21.15.90:0/3289523199 learned_addr learned my addr 172.21.15.90:0/3289523199 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:21.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.828+0000 7f3c00897640 1 --2- 172.21.15.90:0/3289523199 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bfc1509f0 0x7f3bfc1e1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:21.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.828+0000 7f3c01899640 1 --2- 172.21.15.90:0/3289523199 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3bfc150dd0 0x7f3bfc158200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:21.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.832+0000 7f3c01098640 1 -- 172.21.15.90:0/3289523199 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3bfc150dd0 msgr2=0x7f3bfc158200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:21.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.832+0000 7f3c01098640 1 --2- 172.21.15.90:0/3289523199 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3bfc150dd0 0x7f3bfc158200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:21.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.832+0000 7f3c01098640 1 -- 172.21.15.90:0/3289523199 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bfc1509f0 msgr2=0x7f3bfc1e1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:21.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.832+0000 7f3c01098640 1 --2- 172.21.15.90:0/3289523199 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bfc1509f0 0x7f3bfc1e1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:21.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.832+0000 7f3c01098640 1 -- 172.21.15.90:0/3289523199 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf8009420 con 0x7f3bfc070b00 2024-03-11T18:36:21.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.832+0000 7f3c01098640 1 --2- 172.21.15.90:0/3289523199 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3bfc070b00 0x7f3bfc1e0930 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3bf801ead0 tx=0x7f3bf801dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:21.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.836+0000 7f3bf1ffb640 1 -- 172.21.15.90:0/3289523199 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf80b3dc0 con 0x7f3bfc070b00 2024-03-11T18:36:21.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.836+0000 7f3bf1ffb640 1 -- 172.21.15.90:0/3289523199 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bf80bb410 con 0x7f3bfc070b00 2024-03-11T18:36:21.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.836+0000 7f3c032fc640 1 -- 172.21.15.90:0/3289523199 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bfc1407e0 con 0x7f3bfc070b00 2024-03-11T18:36:21.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.836+0000 7f3bf1ffb640 1 -- 172.21.15.90:0/3289523199 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf80bb7c0 con 0x7f3bfc070b00 2024-03-11T18:36:21.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.836+0000 7f3c032fc640 1 -- 172.21.15.90:0/3289523199 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bfc15a930 con 0x7f3bfc070b00 2024-03-11T18:36:21.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.840+0000 7f3c032fc640 1 -- 172.21.15.90:0/3289523199 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bfc04eef0 con 0x7f3bfc070b00 2024-03-11T18:36:21.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.856+0000 7f3bf1ffb640 1 -- 172.21.15.90:0/3289523199 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f3bf80b9040 con 0x7f3bfc070b00 2024-03-11T18:36:21.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.856+0000 7f3bf1ffb640 1 --2- 172.21.15.90:0/3289523199 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f3be0066e90 0x7f3be0069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:21.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.856+0000 7f3c00897640 1 --2- 172.21.15.90:0/3289523199 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f3be0066e90 0x7f3be0069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:21.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.860+0000 7f3c00897640 1 --2- 172.21.15.90:0/3289523199 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f3be0066e90 0x7f3be0069350 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc0c5b80 tx=0x7f3bec009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:21.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:21.860+0000 7f3bf1ffb640 1 -- 172.21.15.90:0/3289523199 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3bf80c8e50 con 0x7f3bfc070b00 2024-03-11T18:36:22.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.176+0000 7f3bf1ffb640 1 -- 172.21.15.90:0/3289523199 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3bf80f2080 con 0x7f3bfc070b00 2024-03-11T18:36:22.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.344+0000 7f3c032fc640 1 -- 172.21.15.90:0/3289523199 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3bfc142370 con 0x7f3bfc070b00 2024-03-11T18:36:22.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.364+0000 7f3bf1ffb640 1 -- 172.21.15.90:0/3289523199 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3bf80f5740 con 0x7f3bfc070b00 2024-03-11T18:36:22.371 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 32 2024-03-11T18:36:22.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.368+0000 7f3c032fc640 1 -- 172.21.15.90:0/3289523199 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f3be0066e90 msgr2=0x7f3be0069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:22.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.368+0000 7f3c032fc640 1 --2- 172.21.15.90:0/3289523199 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f3be0066e90 0x7f3be0069350 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc0c5b80 tx=0x7f3bec009190 comp rx=0 tx=0).stop 2024-03-11T18:36:22.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.368+0000 7f3c032fc640 1 -- 172.21.15.90:0/3289523199 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3bfc070b00 msgr2=0x7f3bfc1e0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:22.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.368+0000 7f3c032fc640 1 --2- 172.21.15.90:0/3289523199 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3bfc070b00 0x7f3bfc1e0930 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3bf801ead0 tx=0x7f3bf801dc40 comp rx=0 tx=0).stop 2024-03-11T18:36:22.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.368+0000 7f3c032fc640 1 -- 172.21.15.90:0/3289523199 shutdown_connections 2024-03-11T18:36:22.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.368+0000 7f3c032fc640 1 --2- 172.21.15.90:0/3289523199 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f3be0066e90 0x7f3be0069350 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:22.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.368+0000 7f3c032fc640 1 --2- 172.21.15.90:0/3289523199 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3bfc150dd0 0x7f3bfc158200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:22.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.368+0000 7f3c032fc640 1 --2- 172.21.15.90:0/3289523199 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3bfc1509f0 0x7f3bfc1e1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:22.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.368+0000 7f3c032fc640 1 --2- 172.21.15.90:0/3289523199 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3bfc070b00 0x7f3bfc1e0930 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:22.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.368+0000 7f3c032fc640 1 -- 172.21.15.90:0/3289523199 >> 172.21.15.90:0/3289523199 conn(0x7f3bfc06ac00 msgr2=0x7f3bfc1449e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:22.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.368+0000 7f3c032fc640 1 -- 172.21.15.90:0/3289523199 shutdown_connections 2024-03-11T18:36:22.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.368+0000 7f3c032fc640 1 -- 172.21.15.90:0/3289523199 wait complete. 2024-03-11T18:36:22.390 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-11T18:36:22.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.524+0000 7f9bbc68f640 1 Processor -- start 2024-03-11T18:36:22.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.524+0000 7f9bbc68f640 1 -- start start 2024-03-11T18:36:22.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.524+0000 7f9bbc68f640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f9bb4150c90 0x7f9bb4159f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:22.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.524+0000 7f9bbc68f640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9bb40c6070 0x7f9bb40c3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:22.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.524+0000 7f9bbc68f640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bb40c5be0 con 0x7f9bb40c3ea0 2024-03-11T18:36:22.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.524+0000 7f9bbc68f640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bb406cad0 con 0x7f9bb4150c90 2024-03-11T18:36:22.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.524+0000 7f9bbc68f640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb406c230 con 0x7f9bb40c6070 2024-03-11T18:36:22.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.540+0000 7f9bba42b640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f9bb4150c90 0x7f9bb4159f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:22.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.540+0000 7f9bba42b640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f9bb4150c90 0x7f9bb4159f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43150/0 (socket says 172.21.15.90:43150) 2024-03-11T18:36:22.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.540+0000 7f9bba42b640 1 -- 172.21.15.90:0/2367724668 learned_addr learned my addr 172.21.15.90:0/2367724668 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:22.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.544+0000 7f9bba42b640 1 -- 172.21.15.90:0/2367724668 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9bb40c6070 msgr2=0x7f9bb40c3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:22.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.544+0000 7f9bba42b640 1 --2- 172.21.15.90:0/2367724668 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9bb40c6070 0x7f9bb40c3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:22.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.544+0000 7f9bba42b640 1 -- 172.21.15.90:0/2367724668 >> v1:172.21.15.92:6789/0 conn(0x7f9bb40c3ea0 legacy=0x7f9bb40c8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:22.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.544+0000 7f9bba42b640 1 -- 172.21.15.90:0/2367724668 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb406c510 con 0x7f9bb4150c90 2024-03-11T18:36:22.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.548+0000 7f9bba42b640 1 --2- 172.21.15.90:0/2367724668 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9bb4150c90 0x7f9bb4159f60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0009870 tx=0x7f9bb001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=762e3fac53f56ed5 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:22.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.552+0000 7f9bb9429640 1 -- 172.21.15.90:0/2367724668 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb00aa640 con 0x7f9bb4150c90 2024-03-11T18:36:22.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.552+0000 7f9bb9429640 1 -- 172.21.15.90:0/2367724668 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bb00aa7a0 con 0x7f9bb4150c90 2024-03-11T18:36:22.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.552+0000 7f9bb9429640 1 -- 172.21.15.90:0/2367724668 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb00aab30 con 0x7f9bb4150c90 2024-03-11T18:36:22.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.552+0000 7f9bbc68f640 1 -- 172.21.15.90:0/2367724668 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9bb4150c90 msgr2=0x7f9bb4159f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:22.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.552+0000 7f9bbc68f640 1 --2- 172.21.15.90:0/2367724668 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9bb4150c90 0x7f9bb4159f60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0009870 tx=0x7f9bb001c520 comp rx=0 tx=0).stop 2024-03-11T18:36:22.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.552+0000 7f9bbc68f640 1 -- 172.21.15.90:0/2367724668 shutdown_connections 2024-03-11T18:36:22.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.552+0000 7f9bbc68f640 1 --2- 172.21.15.90:0/2367724668 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9bb40c6070 0x7f9bb40c3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:22.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.552+0000 7f9bbc68f640 1 --2- 172.21.15.90:0/2367724668 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9bb4150c90 0x7f9bb4159f60 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:22.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.552+0000 7f9bbc68f640 1 -- 172.21.15.90:0/2367724668 >> 172.21.15.90:0/2367724668 conn(0x7f9bb40cc630 msgr2=0x7f9bb40cca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:22.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.552+0000 7f9bbc68f640 1 -- 172.21.15.90:0/2367724668 shutdown_connections 2024-03-11T18:36:22.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.552+0000 7f9bbc68f640 1 -- 172.21.15.90:0/2367724668 wait complete. 2024-03-11T18:36:22.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.556+0000 7f9bbc68f640 1 Processor -- start 2024-03-11T18:36:22.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.556+0000 7f9bbc68f640 1 -- start start 2024-03-11T18:36:22.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.556+0000 7f9bbc68f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bb40c3ea0 0x7f9bb41f6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:22.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.556+0000 7f9bbc68f640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9bb40c6070 0x7f9bb41584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:22.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.556+0000 7f9bbc68f640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9bb4150c90 0x7f9bb4142080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:22.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.556+0000 7f9bbc68f640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb4152300 con 0x7f9bb4150c90 2024-03-11T18:36:22.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.556+0000 7f9bbc68f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb4153ad0 con 0x7f9bb40c3ea0 2024-03-11T18:36:22.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.556+0000 7f9bbc68f640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb41f68a0 con 0x7f9bb40c6070 2024-03-11T18:36:22.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.556+0000 7f9bbac2c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9bb4150c90 0x7f9bb4142080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:22.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.556+0000 7f9bbac2c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9bb4150c90 0x7f9bb4142080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43156/0 (socket says 172.21.15.90:43156) 2024-03-11T18:36:22.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.556+0000 7f9bbac2c640 1 -- 172.21.15.90:0/2886284227 learned_addr learned my addr 172.21.15.90:0/2886284227 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:22.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.560+0000 7f9bbac2c640 1 -- 172.21.15.90:0/2886284227 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9bb40c6070 msgr2=0x7f9bb41584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:22.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.560+0000 7f9bbac2c640 1 --2- 172.21.15.90:0/2886284227 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9bb40c6070 0x7f9bb41584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:22.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.560+0000 7f9bbac2c640 1 -- 172.21.15.90:0/2886284227 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bb40c3ea0 msgr2=0x7f9bb41f6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:22.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.560+0000 7f9bbac2c640 1 --2- 172.21.15.90:0/2886284227 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bb40c3ea0 0x7f9bb41f6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:22.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.560+0000 7f9bbac2c640 1 -- 172.21.15.90:0/2886284227 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb00094d0 con 0x7f9bb4150c90 2024-03-11T18:36:22.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.564+0000 7f9bbac2c640 1 --2- 172.21.15.90:0/2886284227 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9bb4150c90 0x7f9bb4142080 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9bac00c780 tx=0x7f9bac00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:22.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.568+0000 7f9baaffd640 1 -- 172.21.15.90:0/2886284227 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bac003d90 con 0x7f9bb4150c90 2024-03-11T18:36:22.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.568+0000 7f9baaffd640 1 -- 172.21.15.90:0/2886284227 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bac00ce90 con 0x7f9bb4150c90 2024-03-11T18:36:22.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.568+0000 7f9baaffd640 1 -- 172.21.15.90:0/2886284227 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bac0052b0 con 0x7f9bb4150c90 2024-03-11T18:36:22.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.568+0000 7f9bbc68f640 1 -- 172.21.15.90:0/2886284227 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bb4151760 con 0x7f9bb4150c90 2024-03-11T18:36:22.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.568+0000 7f9bbc68f640 1 -- 172.21.15.90:0/2886284227 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bb4153480 con 0x7f9bb4150c90 2024-03-11T18:36:22.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.568+0000 7f9bbc68f640 1 -- 172.21.15.90:0/2886284227 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bb4067f70 con 0x7f9bb4150c90 2024-03-11T18:36:22.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.836+0000 7f9baaffd640 1 -- 172.21.15.90:0/2886284227 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f9bac005410 con 0x7f9bb4150c90 2024-03-11T18:36:22.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.836+0000 7f9baaffd640 1 --2- 172.21.15.90:0/2886284227 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f9b98066e90 0x7f9b98069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:22.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.836+0000 7f9bba42b640 1 --2- 172.21.15.90:0/2886284227 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f9b98066e90 0x7f9b98069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:22.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.836+0000 7f9bba42b640 1 --2- 172.21.15.90:0/2886284227 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f9b98066e90 0x7f9b98069350 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9bb00ae000 tx=0x7f9bb0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:22.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.868+0000 7f9baaffd640 1 -- 172.21.15.90:0/2886284227 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f9bac0559a0 con 0x7f9bb4150c90 2024-03-11T18:36:22.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:22.868+0000 7f9baaffd640 1 -- 172.21.15.90:0/2886284227 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9bac051540 con 0x7f9bb4150c90 2024-03-11T18:36:23.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.036+0000 7f9bbc68f640 1 -- 172.21.15.90:0/2886284227 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9bb404eef0 con 0x7f9bb4150c90 2024-03-11T18:36:23.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.048+0000 7f9baaffd640 1 -- 172.21.15.90:0/2886284227 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9bac017070 con 0x7f9bb4150c90 2024-03-11T18:36:23.055 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 64 2024-03-11T18:36:23.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.052+0000 7f9bbc68f640 1 -- 172.21.15.90:0/2886284227 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f9b98066e90 msgr2=0x7f9b98069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:23.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.052+0000 7f9bbc68f640 1 --2- 172.21.15.90:0/2886284227 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f9b98066e90 0x7f9b98069350 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9bb00ae000 tx=0x7f9bb0008040 comp rx=0 tx=0).stop 2024-03-11T18:36:23.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.052+0000 7f9bbc68f640 1 -- 172.21.15.90:0/2886284227 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9bb4150c90 msgr2=0x7f9bb4142080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:23.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.052+0000 7f9bbc68f640 1 --2- 172.21.15.90:0/2886284227 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9bb4150c90 0x7f9bb4142080 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9bac00c780 tx=0x7f9bac00cc40 comp rx=0 tx=0).stop 2024-03-11T18:36:23.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.052+0000 7f9bbc68f640 1 -- 172.21.15.90:0/2886284227 shutdown_connections 2024-03-11T18:36:23.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.052+0000 7f9bbc68f640 1 --2- 172.21.15.90:0/2886284227 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f9b98066e90 0x7f9b98069350 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:23.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.052+0000 7f9bbc68f640 1 --2- 172.21.15.90:0/2886284227 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9bb4150c90 0x7f9bb4142080 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:23.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.052+0000 7f9bbc68f640 1 --2- 172.21.15.90:0/2886284227 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9bb40c6070 0x7f9bb41584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:23.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.052+0000 7f9bbc68f640 1 --2- 172.21.15.90:0/2886284227 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9bb40c3ea0 0x7f9bb41f6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:23.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.052+0000 7f9bbc68f640 1 -- 172.21.15.90:0/2886284227 >> 172.21.15.90:0/2886284227 conn(0x7f9bb40cc630 msgr2=0x7f9bb40cca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:23.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.052+0000 7f9bbc68f640 1 -- 172.21.15.90:0/2886284227 shutdown_connections 2024-03-11T18:36:23.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.052+0000 7f9bbc68f640 1 -- 172.21.15.90:0/2886284227 wait complete. 2024-03-11T18:36:23.075 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-11T18:36:23.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.192+0000 7f52aab19640 1 Processor -- start 2024-03-11T18:36:23.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.192+0000 7f52aab19640 1 -- start start 2024-03-11T18:36:23.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.196+0000 7f52aab19640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f52a40c4b60 0x7f52a415d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:23.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.196+0000 7f52aab19640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f52a40c42f0 0x7f52a40ca370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:23.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.196+0000 7f52aab19640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52a415a0b0 con 0x7f52a4153cf0 2024-03-11T18:36:23.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.196+0000 7f52aab19640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52a415e120 con 0x7f52a40c4b60 2024-03-11T18:36:23.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.196+0000 7f52aab19640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52a4159b30 con 0x7f52a40c42f0 2024-03-11T18:36:23.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.196+0000 7f52a3fff640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f52a40c4b60 0x7f52a415d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:23.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.196+0000 7f52a3fff640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f52a40c4b60 0x7f52a415d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43166/0 (socket says 172.21.15.90:43166) 2024-03-11T18:36:23.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.196+0000 7f52a3fff640 1 -- 172.21.15.90:0/1931499113 learned_addr learned my addr 172.21.15.90:0/1931499113 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:23.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.196+0000 7f52a90b6640 1 --2- 172.21.15.90:0/1931499113 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f52a40c42f0 0x7f52a40ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:23.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.200+0000 7f52a3fff640 1 -- 172.21.15.90:0/1931499113 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f52a40c42f0 msgr2=0x7f52a40ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:23.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.200+0000 7f52a3fff640 1 --2- 172.21.15.90:0/1931499113 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f52a40c42f0 0x7f52a40ca370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:23.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.200+0000 7f52a3fff640 1 -- 172.21.15.90:0/1931499113 >> v1:172.21.15.92:6789/0 conn(0x7f52a4153cf0 legacy=0x7f52a415a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:23.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.200+0000 7f52a3fff640 1 -- 172.21.15.90:0/1931499113 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52a415d880 con 0x7f52a40c4b60 2024-03-11T18:36:23.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.204+0000 7f52a3fff640 1 --2- 172.21.15.90:0/1931499113 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52a40c4b60 0x7f52a415d120 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5294009870 tx=0x7f529401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3efd7f0f6d4c6ce4 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:23.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.204+0000 7f52a37fe640 1 -- 172.21.15.90:0/1931499113 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52940ab650 con 0x7f52a40c4b60 2024-03-11T18:36:23.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.204+0000 7f52a37fe640 1 -- 172.21.15.90:0/1931499113 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52940ab7b0 con 0x7f52a40c4b60 2024-03-11T18:36:23.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.204+0000 7f52a37fe640 1 -- 172.21.15.90:0/1931499113 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52940abb40 con 0x7f52a40c4b60 2024-03-11T18:36:23.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.208+0000 7f52aab19640 1 -- 172.21.15.90:0/1931499113 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52a40c4b60 msgr2=0x7f52a415d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:23.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.208+0000 7f52aab19640 1 --2- 172.21.15.90:0/1931499113 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52a40c4b60 0x7f52a415d120 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5294009870 tx=0x7f529401c560 comp rx=0 tx=0).stop 2024-03-11T18:36:23.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.208+0000 7f52aab19640 1 -- 172.21.15.90:0/1931499113 shutdown_connections 2024-03-11T18:36:23.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.208+0000 7f52aab19640 1 --2- 172.21.15.90:0/1931499113 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f52a40c42f0 0x7f52a40ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:23.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.208+0000 7f52aab19640 1 --2- 172.21.15.90:0/1931499113 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52a40c4b60 0x7f52a415d120 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:23.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.208+0000 7f52aab19640 1 -- 172.21.15.90:0/1931499113 >> 172.21.15.90:0/1931499113 conn(0x7f52a406b640 msgr2=0x7f52a406ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:23.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.208+0000 7f52aab19640 1 -- 172.21.15.90:0/1931499113 shutdown_connections 2024-03-11T18:36:23.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.208+0000 7f52aab19640 1 -- 172.21.15.90:0/1931499113 wait complete. 2024-03-11T18:36:23.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.208+0000 7f52aab19640 1 Processor -- start 2024-03-11T18:36:23.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.208+0000 7f52aab19640 1 -- start start 2024-03-11T18:36:23.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.208+0000 7f52aab19640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52a40c42f0 0x7f52a40c8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:23.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.208+0000 7f52aab19640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52a40c4b60 0x7f52a415b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:23.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.208+0000 7f52aab19640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f52a4153cf0 0x7f52a415b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:23.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.208+0000 7f52aab19640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a40c7560 con 0x7f52a40c42f0 2024-03-11T18:36:23.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.208+0000 7f52aab19640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a415acf0 con 0x7f52a40c4b60 2024-03-11T18:36:23.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.208+0000 7f52aab19640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52a40c8d60 con 0x7f52a4153cf0 2024-03-11T18:36:23.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.208+0000 7f52a88b5640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52a40c42f0 0x7f52a40c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:23.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.208+0000 7f52a88b5640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52a40c42f0 0x7f52a40c8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43172/0 (socket says 172.21.15.90:43172) 2024-03-11T18:36:23.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.208+0000 7f52a88b5640 1 -- 172.21.15.90:0/41470883 learned_addr learned my addr 172.21.15.90:0/41470883 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:23.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.212+0000 7f52a3fff640 1 --2- 172.21.15.90:0/41470883 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52a40c4b60 0x7f52a415b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:23.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.212+0000 7f52a90b6640 1 --2- 172.21.15.90:0/41470883 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f52a4153cf0 0x7f52a415b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:23.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.212+0000 7f52a88b5640 1 -- 172.21.15.90:0/41470883 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f52a4153cf0 msgr2=0x7f52a415b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:23.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.212+0000 7f52a88b5640 1 --2- 172.21.15.90:0/41470883 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f52a4153cf0 0x7f52a415b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:23.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.212+0000 7f52a88b5640 1 -- 172.21.15.90:0/41470883 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52a40c4b60 msgr2=0x7f52a415b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:23.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.212+0000 7f52a88b5640 1 --2- 172.21.15.90:0/41470883 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52a40c4b60 0x7f52a415b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:23.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.212+0000 7f52a88b5640 1 -- 172.21.15.90:0/41470883 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52940094d0 con 0x7f52a40c42f0 2024-03-11T18:36:23.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.216+0000 7f52a88b5640 1 --2- 172.21.15.90:0/41470883 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52a40c42f0 0x7f52a40c8820 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f528c00c370 tx=0x7f528c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:23.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.220+0000 7f52a17fa640 1 -- 172.21.15.90:0/41470883 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c0052d0 con 0x7f52a40c42f0 2024-03-11T18:36:23.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.220+0000 7f52a17fa640 1 -- 172.21.15.90:0/41470883 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f528c005430 con 0x7f52a40c42f0 2024-03-11T18:36:23.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.220+0000 7f52a17fa640 1 -- 172.21.15.90:0/41470883 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c007260 con 0x7f52a40c42f0 2024-03-11T18:36:23.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.220+0000 7f52aab19640 1 -- 172.21.15.90:0/41470883 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52a40c7790 con 0x7f52a40c42f0 2024-03-11T18:36:23.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.220+0000 7f52aab19640 1 -- 172.21.15.90:0/41470883 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52a40c7d50 con 0x7f52a40c42f0 2024-03-11T18:36:23.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.220+0000 7f52aab19640 1 -- 172.21.15.90:0/41470883 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52a40c7f40 con 0x7f52a40c42f0 2024-03-11T18:36:23.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.512+0000 7f52a17fa640 1 -- 172.21.15.90:0/41470883 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f528c0073c0 con 0x7f52a40c42f0 2024-03-11T18:36:23.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.512+0000 7f52a17fa640 1 --2- 172.21.15.90:0/41470883 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f5284066e90 0x7f5284069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:23.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.512+0000 7f52a3fff640 1 --2- 172.21.15.90:0/41470883 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f5284066e90 0x7f5284069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:23.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.512+0000 7f52a3fff640 1 --2- 172.21.15.90:0/41470883 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f5284066e90 0x7f5284069350 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f52940af000 tx=0x7f52940b6040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:23.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.548+0000 7f52a17fa640 1 -- 172.21.15.90:0/41470883 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f528c007670 con 0x7f52a40c42f0 2024-03-11T18:36:23.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.552+0000 7f52a17fa640 1 -- 172.21.15.90:0/41470883 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f528c0510c0 con 0x7f52a40c42f0 2024-03-11T18:36:23.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.724+0000 7f52aab19640 1 -- 172.21.15.90:0/41470883 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f52a404eef0 con 0x7f52a40c42f0 2024-03-11T18:36:23.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.736+0000 7f52a17fa640 1 -- 172.21.15.90:0/41470883 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f528c0200a0 con 0x7f52a40c42f0 2024-03-11T18:36:23.743 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 32 2024-03-11T18:36:23.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.740+0000 7f52aab19640 1 -- 172.21.15.90:0/41470883 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f5284066e90 msgr2=0x7f5284069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:23.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.740+0000 7f52aab19640 1 --2- 172.21.15.90:0/41470883 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f5284066e90 0x7f5284069350 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f52940af000 tx=0x7f52940b6040 comp rx=0 tx=0).stop 2024-03-11T18:36:23.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.740+0000 7f52aab19640 1 -- 172.21.15.90:0/41470883 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52a40c42f0 msgr2=0x7f52a40c8820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:23.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.740+0000 7f52aab19640 1 --2- 172.21.15.90:0/41470883 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52a40c42f0 0x7f52a40c8820 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f528c00c370 tx=0x7f528c00c830 comp rx=0 tx=0).stop 2024-03-11T18:36:23.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.740+0000 7f52aab19640 1 -- 172.21.15.90:0/41470883 shutdown_connections 2024-03-11T18:36:23.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.740+0000 7f52aab19640 1 --2- 172.21.15.90:0/41470883 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f5284066e90 0x7f5284069350 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:23.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.740+0000 7f52aab19640 1 --2- 172.21.15.90:0/41470883 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f52a4153cf0 0x7f52a415b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:23.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.740+0000 7f52aab19640 1 --2- 172.21.15.90:0/41470883 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52a40c4b60 0x7f52a415b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:23.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.740+0000 7f52aab19640 1 --2- 172.21.15.90:0/41470883 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52a40c42f0 0x7f52a40c8820 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:23.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.740+0000 7f52aab19640 1 -- 172.21.15.90:0/41470883 >> 172.21.15.90:0/41470883 conn(0x7f52a406b640 msgr2=0x7f52a4158200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:23.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.740+0000 7f52aab19640 1 -- 172.21.15.90:0/41470883 shutdown_connections 2024-03-11T18:36:23.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.740+0000 7f52aab19640 1 -- 172.21.15.90:0/41470883 wait complete. 2024-03-11T18:36:23.763 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-11T18:36:23.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.880+0000 7fe4d756d640 1 Processor -- start 2024-03-11T18:36:23.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.880+0000 7fe4d756d640 1 -- start start 2024-03-11T18:36:23.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.880+0000 7fe4d756d640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe4d015da50 0x7fe4d00caf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:23.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.880+0000 7fe4d756d640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4d00cb5b0 con 0x7fe4d01541c0 2024-03-11T18:36:23.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.880+0000 7fe4d756d640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4d0159e10 con 0x7fe4d00cfda0 2024-03-11T18:36:23.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.880+0000 7fe4d756d640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d00b5410 con 0x7fe4d015da50 2024-03-11T18:36:23.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.884+0000 7fe4d4b08640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe4d00cfda0 0x7fe4d015d340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.90:40610/0 (socket says 172.21.15.90:40610) 2024-03-11T18:36:23.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.884+0000 7fe4d4b08640 1 -- 172.21.15.90:0/1767459089 learned_addr learned my addr 172.21.15.90:0/1767459089 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T18:36:23.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.884+0000 7fe4d5b0a640 1 --2- 172.21.15.90:0/1767459089 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe4d015da50 0x7fe4d00caf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:23.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.888+0000 7fe4c7fff640 1 -- 172.21.15.90:0/1767459089 <== mon.0 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2716114321 0 0) 0x7fe4d00cb5b0 con 0x7fe4d01541c0 2024-03-11T18:36:23.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.888+0000 7fe4c7fff640 1 -- 172.21.15.90:0/1767459089 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4bc003530 con 0x7fe4d01541c0 2024-03-11T18:36:23.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.888+0000 7fe4c7fff640 1 -- 172.21.15.90:0/1767459089 <== mon.0 v1:172.21.15.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2268554475 0 0) 0x7fe4bc003530 con 0x7fe4d01541c0 2024-03-11T18:36:23.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.888+0000 7fe4c7fff640 1 -- 172.21.15.90:0/1767459089 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe4d015da50 msgr2=0x7fe4d00caf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:23.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.888+0000 7fe4c7fff640 1 --2- 172.21.15.90:0/1767459089 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe4d015da50 0x7fe4d00caf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:23.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.888+0000 7fe4c7fff640 1 -- 172.21.15.90:0/1767459089 >> v1:172.21.15.92:6789/0 conn(0x7fe4d00cfda0 legacy=0x7fe4d015d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:23.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.888+0000 7fe4c7fff640 1 -- 172.21.15.90:0/1767459089 --> v1:172.21.15.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d015a0f0 con 0x7fe4d01541c0 2024-03-11T18:36:23.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.888+0000 7fe4c7fff640 1 -- 172.21.15.90:0/1767459089 <== mon.0 v1:172.21.15.90:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 691735435 0 0) 0x7fe4c00016a0 con 0x7fe4d01541c0 2024-03-11T18:36:23.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.888+0000 7fe4c7fff640 1 -- 172.21.15.90:0/1767459089 >> v1:172.21.15.90:6789/0 conn(0x7fe4d01541c0 legacy=0x7fe4d00ca180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:23.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.888+0000 7fe4c7fff640 1 --2- 172.21.15.90:0/1767459089 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4bc0039b0 0x7fe4bc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:23.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.888+0000 7fe4c7fff640 1 --2- 172.21.15.90:0/1767459089 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe4d015da50 0x7fe4bc0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:23.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.888+0000 7fe4c7fff640 1 --2- 172.21.15.90:0/1767459089 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4bc004de0 0x7fe4bc005260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:23.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.888+0000 7fe4c7fff640 1 -- 172.21.15.90:0/1767459089 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4bc00bc50 con 0x7fe4bc004de0 2024-03-11T18:36:23.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.888+0000 7fe4c7fff640 1 -- 172.21.15.90:0/1767459089 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4bc00bdc0 con 0x7fe4bc0039b0 2024-03-11T18:36:23.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.888+0000 7fe4c7fff640 1 -- 172.21.15.90:0/1767459089 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4bc00bf80 con 0x7fe4d015da50 2024-03-11T18:36:23.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.892+0000 7fe4d4b08640 1 --2- 172.21.15.90:0/1767459089 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe4d015da50 0x7fe4bc0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:23.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.892+0000 7fe4d5b0a640 1 --2- 172.21.15.90:0/1767459089 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4bc004de0 0x7fe4bc005260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:23.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.892+0000 7fe4d5309640 1 --2- 172.21.15.90:0/1767459089 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4bc0039b0 0x7fe4bc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:23.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.896+0000 7fe4d5b0a640 1 -- 172.21.15.90:0/1767459089 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe4d015da50 msgr2=0x7fe4bc0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:23.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.896+0000 7fe4d5b0a640 1 --2- 172.21.15.90:0/1767459089 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe4d015da50 0x7fe4bc0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:23.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.896+0000 7fe4d5b0a640 1 -- 172.21.15.90:0/1767459089 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4bc0039b0 msgr2=0x7fe4bc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:23.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.896+0000 7fe4d5b0a640 1 --2- 172.21.15.90:0/1767459089 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4bc0039b0 0x7fe4bc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:23.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.896+0000 7fe4d5b0a640 1 -- 172.21.15.90:0/1767459089 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4bc003530 con 0x7fe4bc004de0 2024-03-11T18:36:23.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.900+0000 7fe4d5b0a640 1 --2- 172.21.15.90:0/1767459089 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4bc004de0 0x7fe4bc005260 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8007f60 tx=0x7fe4c801ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63df25a60ec682ac server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:23.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.904+0000 7fe4c7fff640 1 -- 172.21.15.90:0/1767459089 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c80a5e60 con 0x7fe4bc004de0 2024-03-11T18:36:23.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.904+0000 7fe4c7fff640 1 -- 172.21.15.90:0/1767459089 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4c80ad440 con 0x7fe4bc004de0 2024-03-11T18:36:23.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.904+0000 7fe4c7fff640 1 -- 172.21.15.90:0/1767459089 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c80ad7b0 con 0x7fe4bc004de0 2024-03-11T18:36:23.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.904+0000 7fe4d756d640 1 -- 172.21.15.90:0/1767459089 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4bc004de0 msgr2=0x7fe4bc005260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:23.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.904+0000 7fe4d756d640 1 --2- 172.21.15.90:0/1767459089 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4bc004de0 0x7fe4bc005260 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8007f60 tx=0x7fe4c801ce50 comp rx=0 tx=0).stop 2024-03-11T18:36:23.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.904+0000 7fe4d5b0a640 1 -- 172.21.15.90:0/1767459089 reap_dead start 2024-03-11T18:36:23.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.904+0000 7fe4d756d640 1 -- 172.21.15.90:0/1767459089 shutdown_connections 2024-03-11T18:36:23.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.904+0000 7fe4d756d640 1 -- 172.21.15.90:0/1767459089 >> 172.21.15.90:0/1767459089 conn(0x7fe4d006b620 msgr2=0x7fe4d006ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:23.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.904+0000 7fe4d756d640 1 -- 172.21.15.90:0/1767459089 shutdown_connections 2024-03-11T18:36:23.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.904+0000 7fe4d756d640 1 -- 172.21.15.90:0/1767459089 wait complete. 2024-03-11T18:36:23.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.904+0000 7fe4d756d640 1 Processor -- start 2024-03-11T18:36:23.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.904+0000 7fe4d756d640 1 -- start start 2024-03-11T18:36:23.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.908+0000 7fe4d756d640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe4bc004de0 0x7fe4d0159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:23.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.908+0000 7fe4d756d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4d01effc0 0x7fe4d01f2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:23.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.908+0000 7fe4d756d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4d0150080 0x7fe4d0156de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:23.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.908+0000 7fe4d756d640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d01582b0 con 0x7fe4d01effc0 2024-03-11T18:36:23.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.908+0000 7fe4d756d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d015d0b0 con 0x7fe4d0150080 2024-03-11T18:36:23.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.908+0000 7fe4d756d640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d0159bd0 con 0x7fe4bc004de0 2024-03-11T18:36:23.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.908+0000 7fe4d4b08640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4d01effc0 0x7fe4d01f2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:23.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.908+0000 7fe4d4b08640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4d01effc0 0x7fe4d01f2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55610/0 (socket says 172.21.15.90:55610) 2024-03-11T18:36:23.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.908+0000 7fe4d4b08640 1 -- 172.21.15.90:0/2349456141 learned_addr learned my addr 172.21.15.90:0/2349456141 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:23.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.908+0000 7fe4d5b0a640 1 --2- 172.21.15.90:0/2349456141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4d0150080 0x7fe4d0156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:23.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.908+0000 7fe4d5309640 1 --2- 172.21.15.90:0/2349456141 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe4bc004de0 0x7fe4d0159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:23.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.912+0000 7fe4d4b08640 1 -- 172.21.15.90:0/2349456141 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe4bc004de0 msgr2=0x7fe4d0159690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:23.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.912+0000 7fe4d4b08640 1 --2- 172.21.15.90:0/2349456141 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe4bc004de0 0x7fe4d0159690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:23.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.912+0000 7fe4d4b08640 1 -- 172.21.15.90:0/2349456141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4d0150080 msgr2=0x7fe4d0156de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:23.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.912+0000 7fe4d4b08640 1 --2- 172.21.15.90:0/2349456141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4d0150080 0x7fe4d0156de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:23.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.912+0000 7fe4d4b08640 1 -- 172.21.15.90:0/2349456141 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d01ef290 con 0x7fe4d01effc0 2024-03-11T18:36:23.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.912+0000 7fe4d4b08640 1 --2- 172.21.15.90:0/2349456141 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4d01effc0 0x7fe4d01f2800 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc001350 tx=0x7fe4cc001810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:23.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.916+0000 7fe4c5ffb640 1 -- 172.21.15.90:0/2349456141 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc010510 con 0x7fe4d01effc0 2024-03-11T18:36:23.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.916+0000 7fe4c5ffb640 1 -- 172.21.15.90:0/2349456141 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4cc010670 con 0x7fe4d01effc0 2024-03-11T18:36:23.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.916+0000 7fe4d756d640 1 -- 172.21.15.90:0/2349456141 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4d0157cf0 con 0x7fe4d01effc0 2024-03-11T18:36:23.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.916+0000 7fe4c5ffb640 1 -- 172.21.15.90:0/2349456141 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc010a00 con 0x7fe4d01effc0 2024-03-11T18:36:23.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.916+0000 7fe4d756d640 1 -- 172.21.15.90:0/2349456141 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4d01ef4e0 con 0x7fe4d01effc0 2024-03-11T18:36:23.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.920+0000 7fe4d756d640 1 -- 172.21.15.90:0/2349456141 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4d004eef0 con 0x7fe4d01effc0 2024-03-11T18:36:23.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.944+0000 7fe4c5ffb640 1 -- 172.21.15.90:0/2349456141 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fe4cc010b60 con 0x7fe4d01effc0 2024-03-11T18:36:23.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.944+0000 7fe4c5ffb640 1 --2- 172.21.15.90:0/2349456141 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe4b4066ec0 0x7fe4b4069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:23.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.944+0000 7fe4d5309640 1 --2- 172.21.15.90:0/2349456141 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe4b4066ec0 0x7fe4b4069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:23.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:23.944+0000 7fe4d5309640 1 --2- 172.21.15.90:0/2349456141 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe4b4066ec0 0x7fe4b4069380 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0001610 tx=0x7fe4c00015a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:24.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:24.208+0000 7fe4c5ffb640 1 -- 172.21.15.90:0/2349456141 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe4cc055780 con 0x7fe4d01effc0 2024-03-11T18:36:24.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:24.208+0000 7fe4c5ffb640 1 -- 172.21.15.90:0/2349456141 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe4cc051a20 con 0x7fe4d01effc0 2024-03-11T18:36:24.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:24.392+0000 7fe4d756d640 1 -- 172.21.15.90:0/2349456141 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe4d01539f0 con 0x7fe4d01effc0 2024-03-11T18:36:24.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:24.404+0000 7fe4c5ffb640 1 -- 172.21.15.90:0/2349456141 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe4cc023070 con 0x7fe4d01effc0 2024-03-11T18:36:24.409 INFO:teuthology.orchestra.run.smithi090.stderr:listed 0 entries 2024-03-11T18:36:24.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:24.404+0000 7fe4d756d640 1 -- 172.21.15.90:0/2349456141 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe4b4066ec0 msgr2=0x7fe4b4069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:24.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:24.404+0000 7fe4d756d640 1 --2- 172.21.15.90:0/2349456141 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe4b4066ec0 0x7fe4b4069380 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0001610 tx=0x7fe4c00015a0 comp rx=0 tx=0).stop 2024-03-11T18:36:24.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:24.404+0000 7fe4d756d640 1 -- 172.21.15.90:0/2349456141 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4d01effc0 msgr2=0x7fe4d01f2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:24.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:24.404+0000 7fe4d756d640 1 --2- 172.21.15.90:0/2349456141 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4d01effc0 0x7fe4d01f2800 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc001350 tx=0x7fe4cc001810 comp rx=0 tx=0).stop 2024-03-11T18:36:24.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:24.404+0000 7fe4d756d640 1 -- 172.21.15.90:0/2349456141 shutdown_connections 2024-03-11T18:36:24.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:24.404+0000 7fe4d756d640 1 --2- 172.21.15.90:0/2349456141 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe4b4066ec0 0x7fe4b4069380 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:24.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:24.404+0000 7fe4d756d640 1 --2- 172.21.15.90:0/2349456141 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe4d0150080 0x7fe4d0156de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:24.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:24.404+0000 7fe4d756d640 1 --2- 172.21.15.90:0/2349456141 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4d01effc0 0x7fe4d01f2800 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:24.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:24.404+0000 7fe4d756d640 1 --2- 172.21.15.90:0/2349456141 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe4bc004de0 0x7fe4d0159690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:24.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:24.404+0000 7fe4d756d640 1 -- 172.21.15.90:0/2349456141 >> 172.21.15.90:0/2349456141 conn(0x7fe4d006b620 msgr2=0x7fe4d00cd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:24.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:24.404+0000 7fe4d756d640 1 -- 172.21.15.90:0/2349456141 shutdown_connections 2024-03-11T18:36:24.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:24.404+0000 7fe4d756d640 1 -- 172.21.15.90:0/2349456141 wait complete. 2024-03-11T18:36:24.429 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi090.front.sepia.ceph.com... 2024-03-11T18:36:24.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T18:36:24.429 DEBUG:teuthology.orchestra.run.smithi090:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-03-11T18:36:24.446 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi090.stderr:ceph-fuse[34127]: fuse finished with error 0 and tester_r 0 2024-03-11T18:36:24.447 INFO:teuthology.orchestra.run:waiting for 300 2024-03-11T18:36:30.548 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi090.front.sepia.ceph.com 2024-03-11T18:36:30.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T18:36:30.549 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-03-11T18:36:30.572 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T18:36:30.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.732+0000 7f0b6fdf2640 1 Processor -- start 2024-03-11T18:36:30.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.732+0000 7f0b6fdf2640 1 -- start start 2024-03-11T18:36:30.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.736+0000 7f0b6fdf2640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0b68158020 0x7f0b68158400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:30.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.736+0000 7f0b6fdf2640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f0b6806f1b0 0x7f0b6806dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:30.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.736+0000 7f0b6fdf2640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0b6815ed00 0x7f0b680c5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:30.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.736+0000 7f0b6fdf2640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b6806e4e0 con 0x7f0b6806f1b0 2024-03-11T18:36:30.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.736+0000 7f0b6fdf2640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b68071eb0 con 0x7f0b68158020 2024-03-11T18:36:30.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.736+0000 7f0b6fdf2640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b68072680 con 0x7f0b6815ed00 2024-03-11T18:36:30.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.736+0000 7f0b6d38d640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f0b6806f1b0 0x7f0b6806dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:30.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.736+0000 7f0b6d38d640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f0b6806f1b0 0x7f0b6806dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55612/0 (socket says 172.21.15.90:55612) 2024-03-11T18:36:30.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.736+0000 7f0b6d38d640 1 -- 172.21.15.90:0/1342845008 learned_addr learned my addr 172.21.15.90:0/1342845008 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:30.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.736+0000 7f0b6e38f640 1 --2- 172.21.15.90:0/1342845008 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0b6815ed00 0x7f0b680c5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:30.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.736+0000 7f0b6db8e640 1 --2- 172.21.15.90:0/1342845008 >> v2:172.21.15.92:3300/0 conn(0x7f0b68158020 0x7f0b68158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:30.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.740+0000 7f0b6d38d640 1 -- 172.21.15.90:0/1342845008 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0b6815ed00 msgr2=0x7f0b680c5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:30.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.740+0000 7f0b6d38d640 1 --2- 172.21.15.90:0/1342845008 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0b6815ed00 0x7f0b680c5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:30.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.740+0000 7f0b6d38d640 1 -- 172.21.15.90:0/1342845008 >> v2:172.21.15.92:3300/0 conn(0x7f0b68158020 msgr2=0x7f0b68158400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:30.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.740+0000 7f0b6d38d640 1 --2- 172.21.15.90:0/1342845008 >> v2:172.21.15.92:3300/0 conn(0x7f0b68158020 0x7f0b68158400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:30.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.740+0000 7f0b6d38d640 1 -- 172.21.15.90:0/1342845008 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b680720e0 con 0x7f0b6806f1b0 2024-03-11T18:36:30.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.740+0000 7f0b6db8e640 1 --2- 172.21.15.90:0/1342845008 >> v2:172.21.15.92:3300/0 conn(0x7f0b68158020 0x7f0b68158400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T18:36:30.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.744+0000 7f0b6d38d640 1 --2- 172.21.15.90:0/1342845008 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b6806f1b0 0x7f0b6806dfa0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0b6806ca40 tx=0x7f0b5801ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=545b92833e7449f server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:30.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.748+0000 7f0b6cb8c640 1 -- 172.21.15.90:0/1342845008 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b580a89d0 con 0x7f0b6806f1b0 2024-03-11T18:36:30.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.748+0000 7f0b6cb8c640 1 -- 172.21.15.90:0/1342845008 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b580a8b30 con 0x7f0b6806f1b0 2024-03-11T18:36:30.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.748+0000 7f0b6cb8c640 1 -- 172.21.15.90:0/1342845008 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b580b0440 con 0x7f0b6806f1b0 2024-03-11T18:36:30.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.748+0000 7f0b6fdf2640 1 -- 172.21.15.90:0/1342845008 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b6806f1b0 msgr2=0x7f0b6806dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:30.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.748+0000 7f0b6fdf2640 1 --2- 172.21.15.90:0/1342845008 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b6806f1b0 0x7f0b6806dfa0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0b6806ca40 tx=0x7f0b5801ed10 comp rx=0 tx=0).stop 2024-03-11T18:36:30.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.748+0000 7f0b6fdf2640 1 -- 172.21.15.90:0/1342845008 shutdown_connections 2024-03-11T18:36:30.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.748+0000 7f0b6fdf2640 1 --2- 172.21.15.90:0/1342845008 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0b6815ed00 0x7f0b680c5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:30.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.748+0000 7f0b6fdf2640 1 --2- 172.21.15.90:0/1342845008 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b6806f1b0 0x7f0b6806dfa0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:30.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.748+0000 7f0b6fdf2640 1 --2- 172.21.15.90:0/1342845008 >> v2:172.21.15.92:3300/0 conn(0x7f0b68158020 0x7f0b68158400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:30.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.748+0000 7f0b6fdf2640 1 -- 172.21.15.90:0/1342845008 >> 172.21.15.90:0/1342845008 conn(0x7f0b6806ac00 msgr2=0x7f0b6806b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:30.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.748+0000 7f0b6fdf2640 1 -- 172.21.15.90:0/1342845008 shutdown_connections 2024-03-11T18:36:30.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.748+0000 7f0b6fdf2640 1 -- 172.21.15.90:0/1342845008 wait complete. 2024-03-11T18:36:30.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.752+0000 7f0b6fdf2640 1 Processor -- start 2024-03-11T18:36:30.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.752+0000 7f0b6fdf2640 1 -- start start 2024-03-11T18:36:30.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.752+0000 7f0b6fdf2640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b6806f1b0 0x7f0b6814d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:30.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.752+0000 7f0b6fdf2640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0b68158020 0x7f0b681f4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:30.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.752+0000 7f0b6fdf2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b6815ed00 0x7f0b681f7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:30.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.752+0000 7f0b6fdf2640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6814ecc0 con 0x7f0b6806f1b0 2024-03-11T18:36:30.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.752+0000 7f0b6fdf2640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b68155480 con 0x7f0b6815ed00 2024-03-11T18:36:30.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.752+0000 7f0b6fdf2640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b681f5000 con 0x7f0b68158020 2024-03-11T18:36:30.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.752+0000 7f0b6db8e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b6806f1b0 0x7f0b6814d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:30.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.752+0000 7f0b6db8e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b6806f1b0 0x7f0b6814d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55628/0 (socket says 172.21.15.90:55628) 2024-03-11T18:36:30.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.752+0000 7f0b6db8e640 1 -- 172.21.15.90:0/1259672174 learned_addr learned my addr 172.21.15.90:0/1259672174 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:30.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.752+0000 7f0b6e38f640 1 --2- 172.21.15.90:0/1259672174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b6815ed00 0x7f0b681f7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:30.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.752+0000 7f0b6d38d640 1 --2- 172.21.15.90:0/1259672174 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0b68158020 0x7f0b681f4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:30.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.756+0000 7f0b6db8e640 1 -- 172.21.15.90:0/1259672174 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0b68158020 msgr2=0x7f0b681f4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:30.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.756+0000 7f0b6db8e640 1 --2- 172.21.15.90:0/1259672174 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0b68158020 0x7f0b681f4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:30.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.756+0000 7f0b6db8e640 1 -- 172.21.15.90:0/1259672174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b6815ed00 msgr2=0x7f0b681f7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:30.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.756+0000 7f0b6db8e640 1 --2- 172.21.15.90:0/1259672174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b6815ed00 0x7f0b681f7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:30.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.756+0000 7f0b6db8e640 1 -- 172.21.15.90:0/1259672174 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b580094d0 con 0x7f0b6806f1b0 2024-03-11T18:36:30.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.760+0000 7f0b6db8e640 1 --2- 172.21.15.90:0/1259672174 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b6806f1b0 0x7f0b6814d9e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0b6400d800 tx=0x7f0b6400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:30.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.764+0000 7f0b5e7fc640 1 -- 172.21.15.90:0/1259672174 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b64004030 con 0x7f0b6806f1b0 2024-03-11T18:36:30.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.764+0000 7f0b5e7fc640 1 -- 172.21.15.90:0/1259672174 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b64004190 con 0x7f0b6806f1b0 2024-03-11T18:36:30.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.764+0000 7f0b6fdf2640 1 -- 172.21.15.90:0/1259672174 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b681556b0 con 0x7f0b6806f1b0 2024-03-11T18:36:30.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.764+0000 7f0b5e7fc640 1 -- 172.21.15.90:0/1259672174 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b64004520 con 0x7f0b6806f1b0 2024-03-11T18:36:30.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.764+0000 7f0b6fdf2640 1 -- 172.21.15.90:0/1259672174 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b68155c70 con 0x7f0b6806f1b0 2024-03-11T18:36:30.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:30.764+0000 7f0b6fdf2640 1 -- 172.21.15.90:0/1259672174 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b68155e60 con 0x7f0b6806f1b0 2024-03-11T18:36:31.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.040+0000 7f0b5e7fc640 1 -- 172.21.15.90:0/1259672174 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f0b64009640 con 0x7f0b6806f1b0 2024-03-11T18:36:31.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.044+0000 7f0b5e7fc640 1 --2- 172.21.15.90:0/1259672174 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0b4c066e90 0x7f0b4c069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:31.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.044+0000 7f0b6d38d640 1 --2- 172.21.15.90:0/1259672174 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0b4c066e90 0x7f0b4c069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:31.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.044+0000 7f0b6d38d640 1 --2- 172.21.15.90:0/1259672174 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0b4c066e90 0x7f0b4c069350 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0b580b2000 tx=0x7f0b58009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:31.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.064+0000 7f0b5e7fc640 1 -- 172.21.15.90:0/1259672174 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0b64056280 con 0x7f0b6806f1b0 2024-03-11T18:36:31.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.068+0000 7f0b5e7fc640 1 -- 172.21.15.90:0/1259672174 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0b64052520 con 0x7f0b6806f1b0 2024-03-11T18:36:31.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.240+0000 7f0b6fdf2640 1 -- 172.21.15.90:0/1259672174 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0b6804eef0 con 0x7f0b6806f1b0 2024-03-11T18:36:31.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.324+0000 7f0b5e7fc640 1 -- 172.21.15.90:0/1259672174 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18430 (secure 0 0 0) 0x7f0b64018070 con 0x7f0b6806f1b0 2024-03-11T18:36:31.332 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:36:31.333 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":28,"fsid":"76d56b10-6c29-476a-804b-8482ea90f9e2","created":"2024-03-11T18:34:54.521741+0000","modified":"2024-03-11T18:36:05.526044+0000","last_up_change":"2024-03-11T18:35:16.207500+0000","last_in_change":"2024-03-11T18:35:05.213634+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T18:35:16.193842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T18:35:54.001469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T18:35:55.917321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T18:36:02.210943+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"34decff9-7280-4d84-9665-d6af8ea6de84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6819","nonce":1755667921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6821","nonce":1755667921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6825","nonce":1755667921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6823","nonce":1755667921}]},"public_addr":"172.21.15.90:6819/1755667921","cluster_addr":"172.21.15.90:6821/1755667921","heartbeat_back_addr":"172.21.15.90:6825/1755667921","heartbeat_front_addr":"172.21.15.90:6823/1755667921","state":["exists","up"]},{"osd":1,"uuid":"8ecdce9e-5486-415a-a609-dd16e945440e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6807","nonce":438682081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6809","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6811","nonce":438682081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6817","nonce":438682081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6815","nonce":438682081}]},"public_addr":"172.21.15.90:6807/438682081","cluster_addr":"172.21.15.90:6811/438682081","heartbeat_back_addr":"172.21.15.90:6817/438682081","heartbeat_front_addr":"172.21.15.90:6815/438682081","state":["exists","up"]},{"osd":2,"uuid":"737486d4-0e90-4c82-9e79-bf37c812e5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6827","nonce":2837704738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6829","nonce":2837704738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6833","nonce":2837704738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6831","nonce":2837704738}]},"public_addr":"172.21.15.90:6827/2837704738","cluster_addr":"172.21.15.90:6829/2837704738","heartbeat_back_addr":"172.21.15.90:6833/2837704738","heartbeat_front_addr":"172.21.15.90:6831/2837704738","state":["exists","up"]},{"osd":3,"uuid":"698bd661-bbef-4699-a619-f7bfe23ce12c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6803","nonce":240987971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6805","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6806","nonce":240987971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6813","nonce":240987971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6810","nonce":240987971}]},"public_addr":"172.21.15.90:6803/240987971","cluster_addr":"172.21.15.90:6806/240987971","heartbeat_back_addr":"172.21.15.90:6813/240987971","heartbeat_front_addr":"172.21.15.90:6810/240987971","state":["exists","up"]},{"osd":4,"uuid":"aba9e31f-73d9-43fe-a10b-2dc5832968b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6801","nonce":502685275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6803","nonce":502685275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6807","nonce":502685275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6805","nonce":502685275}]},"public_addr":"172.21.15.92:6801/502685275","cluster_addr":"172.21.15.92:6803/502685275","heartbeat_back_addr":"172.21.15.92:6807/502685275","heartbeat_front_addr":"172.21.15.92:6805/502685275","state":["exists","up"]},{"osd":5,"uuid":"bdd56766-4b28-4007-8de3-b5b08cf335d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6809","nonce":200142070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6811","nonce":200142070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6815","nonce":200142070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6813","nonce":200142070}]},"public_addr":"172.21.15.92:6809/200142070","cluster_addr":"172.21.15.92:6811/200142070","heartbeat_back_addr":"172.21.15.92:6815/200142070","heartbeat_front_addr":"172.21.15.92:6813/200142070","state":["exists","up"]},{"osd":6,"uuid":"6c580a59-610f-455e-a96b-033299dcc118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6817","nonce":1919010204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6819","nonce":1919010204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6823","nonce":1919010204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6821","nonce":1919010204}]},"public_addr":"172.21.15.92:6817/1919010204","cluster_addr":"172.21.15.92:6819/1919010204","heartbeat_back_addr":"172.21.15.92:6823/1919010204","heartbeat_front_addr":"172.21.15.92:6821/1919010204","state":["exists","up"]},{"osd":7,"uuid":"958184cc-aebf-4c5b-91a0-2a3e38a6b42b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6825","nonce":3475925719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6827","nonce":3475925719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6831","nonce":3475925719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6829","nonce":3475925719}]},"public_addr":"172.21.15.92:6825/3475925719","cluster_addr":"172.21.15.92:6827/3475925719","heartbeat_back_addr":"172.21.15.92:6831/3475925719","heartbeat_front_addr":"172.21.15.92:6829/3475925719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.816294+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.048084+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.137975+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.968246+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.016829+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.931676+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.041673+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.079404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T18:36:31.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.328+0000 7f0b6fdf2640 1 -- 172.21.15.90:0/1259672174 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0b4c066e90 msgr2=0x7f0b4c069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:31.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.328+0000 7f0b6fdf2640 1 --2- 172.21.15.90:0/1259672174 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0b4c066e90 0x7f0b4c069350 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0b580b2000 tx=0x7f0b58009040 comp rx=0 tx=0).stop 2024-03-11T18:36:31.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.328+0000 7f0b6fdf2640 1 -- 172.21.15.90:0/1259672174 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b6806f1b0 msgr2=0x7f0b6814d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:31.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.328+0000 7f0b6fdf2640 1 --2- 172.21.15.90:0/1259672174 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b6806f1b0 0x7f0b6814d9e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0b6400d800 tx=0x7f0b6400dcc0 comp rx=0 tx=0).stop 2024-03-11T18:36:31.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.328+0000 7f0b6fdf2640 1 -- 172.21.15.90:0/1259672174 shutdown_connections 2024-03-11T18:36:31.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.328+0000 7f0b6fdf2640 1 --2- 172.21.15.90:0/1259672174 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0b4c066e90 0x7f0b4c069350 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:31.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.328+0000 7f0b6fdf2640 1 --2- 172.21.15.90:0/1259672174 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0b6815ed00 0x7f0b681f7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:31.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.328+0000 7f0b6fdf2640 1 --2- 172.21.15.90:0/1259672174 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0b68158020 0x7f0b681f4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:31.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.328+0000 7f0b6fdf2640 1 --2- 172.21.15.90:0/1259672174 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b6806f1b0 0x7f0b6814d9e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:31.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.328+0000 7f0b6fdf2640 1 -- 172.21.15.90:0/1259672174 >> 172.21.15.90:0/1259672174 conn(0x7f0b6806ac00 msgr2=0x7f0b68071370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:31.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.328+0000 7f0b6fdf2640 1 -- 172.21.15.90:0/1259672174 shutdown_connections 2024-03-11T18:36:31.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.328+0000 7f0b6fdf2640 1 -- 172.21.15.90:0/1259672174 wait complete. 2024-03-11T18:36:31.357 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T18:36:31.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.472+0000 7ff238ca6640 1 Processor -- start 2024-03-11T18:36:31.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.472+0000 7ff238ca6640 1 -- start start 2024-03-11T18:36:31.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.472+0000 7ff238ca6640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7ff234151b30 0x7ff23406e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:31.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.472+0000 7ff238ca6640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff2340c4af0 0x7ff2340c4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:31.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.472+0000 7ff238ca6640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff23406f430 con 0x7ff234070b00 2024-03-11T18:36:31.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.472+0000 7ff238ca6640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2340c65b0 con 0x7ff234151b30 2024-03-11T18:36:31.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.472+0000 7ff238ca6640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff23406e820 con 0x7ff2340c4af0 2024-03-11T18:36:31.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.472+0000 7ff23259c640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7ff234151b30 0x7ff23406e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:31.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.472+0000 7ff23259c640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7ff234151b30 0x7ff23406e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55642/0 (socket says 172.21.15.90:55642) 2024-03-11T18:36:31.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.472+0000 7ff23259c640 1 -- 172.21.15.90:0/3839328471 learned_addr learned my addr 172.21.15.90:0/3839328471 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:31.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.476+0000 7ff232d9d640 1 --2- 172.21.15.90:0/3839328471 >> v2:172.21.15.92:3300/0 conn(0x7ff2340c4af0 0x7ff2340c4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:31.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.476+0000 7ff23259c640 1 -- 172.21.15.90:0/3839328471 >> v2:172.21.15.92:3300/0 conn(0x7ff2340c4af0 msgr2=0x7ff2340c4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:31.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.476+0000 7ff23259c640 1 --2- 172.21.15.90:0/3839328471 >> v2:172.21.15.92:3300/0 conn(0x7ff2340c4af0 0x7ff2340c4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:31.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.476+0000 7ff23259c640 1 -- 172.21.15.90:0/3839328471 >> v1:172.21.15.90:6789/0 conn(0x7ff234070b00 legacy=0x7ff2340c5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:31.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.476+0000 7ff23259c640 1 -- 172.21.15.90:0/3839328471 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff23406eb00 con 0x7ff234151b30 2024-03-11T18:36:31.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.484+0000 7ff23259c640 1 --2- 172.21.15.90:0/3839328471 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff234151b30 0x7ff23406e210 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff23406c220 tx=0x7ff22401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9662f7561d5a3d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:31.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.484+0000 7ff23159a640 1 -- 172.21.15.90:0/3839328471 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2240aa650 con 0x7ff234151b30 2024-03-11T18:36:31.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.488+0000 7ff23159a640 1 -- 172.21.15.90:0/3839328471 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2240aa7b0 con 0x7ff234151b30 2024-03-11T18:36:31.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.488+0000 7ff23159a640 1 -- 172.21.15.90:0/3839328471 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2240aab40 con 0x7ff234151b30 2024-03-11T18:36:31.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.488+0000 7ff238ca6640 1 -- 172.21.15.90:0/3839328471 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff234151b30 msgr2=0x7ff23406e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:31.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.488+0000 7ff238ca6640 1 --2- 172.21.15.90:0/3839328471 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff234151b30 0x7ff23406e210 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff23406c220 tx=0x7ff22401c4f0 comp rx=0 tx=0).stop 2024-03-11T18:36:31.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.488+0000 7ff238ca6640 1 -- 172.21.15.90:0/3839328471 shutdown_connections 2024-03-11T18:36:31.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.488+0000 7ff238ca6640 1 --2- 172.21.15.90:0/3839328471 >> v2:172.21.15.92:3300/0 conn(0x7ff2340c4af0 0x7ff2340c4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:31.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.488+0000 7ff238ca6640 1 --2- 172.21.15.90:0/3839328471 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff234151b30 0x7ff23406e210 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:31.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.488+0000 7ff238ca6640 1 -- 172.21.15.90:0/3839328471 >> 172.21.15.90:0/3839328471 conn(0x7ff23406ac00 msgr2=0x7ff23406b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:31.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.488+0000 7ff238ca6640 1 -- 172.21.15.90:0/3839328471 shutdown_connections 2024-03-11T18:36:31.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.488+0000 7ff238ca6640 1 -- 172.21.15.90:0/3839328471 wait complete. 2024-03-11T18:36:31.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.488+0000 7ff238ca6640 1 Processor -- start 2024-03-11T18:36:31.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.488+0000 7ff238ca6640 1 -- start start 2024-03-11T18:36:31.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.492+0000 7ff238ca6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff234070b00 0x7ff2341f41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:31.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.492+0000 7ff238ca6640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff2340c4af0 0x7ff2340ca600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:31.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.492+0000 7ff238ca6640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff234151b30 0x7ff23415ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:31.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.492+0000 7ff238ca6640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2341598b0 con 0x7ff2340c4af0 2024-03-11T18:36:31.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.492+0000 7ff238ca6640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff234141950 con 0x7ff234070b00 2024-03-11T18:36:31.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.492+0000 7ff238ca6640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff23415a890 con 0x7ff234151b30 2024-03-11T18:36:31.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.492+0000 7ff231d9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff2340c4af0 0x7ff2340ca600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:31.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.492+0000 7ff231d9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff2340c4af0 0x7ff2340ca600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55654/0 (socket says 172.21.15.90:55654) 2024-03-11T18:36:31.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.492+0000 7ff231d9b640 1 -- 172.21.15.90:0/542957160 learned_addr learned my addr 172.21.15.90:0/542957160 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:31.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.492+0000 7ff23259c640 1 --2- 172.21.15.90:0/542957160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff234070b00 0x7ff2341f41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:31.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.492+0000 7ff232d9d640 1 --2- 172.21.15.90:0/542957160 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff234151b30 0x7ff23415ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:31.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.496+0000 7ff231d9b640 1 -- 172.21.15.90:0/542957160 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff234151b30 msgr2=0x7ff23415ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:31.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.496+0000 7ff231d9b640 1 --2- 172.21.15.90:0/542957160 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff234151b30 0x7ff23415ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:31.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.496+0000 7ff231d9b640 1 -- 172.21.15.90:0/542957160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff234070b00 msgr2=0x7ff2341f41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:31.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.496+0000 7ff231d9b640 1 --2- 172.21.15.90:0/542957160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff234070b00 0x7ff2341f41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:31.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.496+0000 7ff231d9b640 1 -- 172.21.15.90:0/542957160 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2240094d0 con 0x7ff2340c4af0 2024-03-11T18:36:31.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.496+0000 7ff231d9b640 1 --2- 172.21.15.90:0/542957160 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff2340c4af0 0x7ff2340ca600 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff22800c370 tx=0x7ff22800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:31.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.500+0000 7ff222ffd640 1 -- 172.21.15.90:0/542957160 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2280052d0 con 0x7ff2340c4af0 2024-03-11T18:36:31.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.500+0000 7ff222ffd640 1 -- 172.21.15.90:0/542957160 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff228005430 con 0x7ff2340c4af0 2024-03-11T18:36:31.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.500+0000 7ff222ffd640 1 -- 172.21.15.90:0/542957160 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff228007260 con 0x7ff2340c4af0 2024-03-11T18:36:31.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.500+0000 7ff238ca6640 1 -- 172.21.15.90:0/542957160 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff234156bb0 con 0x7ff2340c4af0 2024-03-11T18:36:31.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.500+0000 7ff238ca6640 1 -- 172.21.15.90:0/542957160 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2341570f0 con 0x7ff2340c4af0 2024-03-11T18:36:31.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.500+0000 7ff238ca6640 1 -- 172.21.15.90:0/542957160 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff23404eef0 con 0x7ff2340c4af0 2024-03-11T18:36:31.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.520+0000 7ff222ffd640 1 -- 172.21.15.90:0/542957160 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7ff2280073c0 con 0x7ff2340c4af0 2024-03-11T18:36:31.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.520+0000 7ff222ffd640 1 --2- 172.21.15.90:0/542957160 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7ff214066e90 0x7ff214069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:31.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.520+0000 7ff23259c640 1 --2- 172.21.15.90:0/542957160 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7ff214066e90 0x7ff214069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:31.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.520+0000 7ff23259c640 1 --2- 172.21.15.90:0/542957160 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7ff214066e90 0x7ff214069350 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff2240ae000 tx=0x7ff2240b5040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:31.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.748+0000 7ff222ffd640 1 -- 172.21.15.90:0/542957160 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff228007670 con 0x7ff2340c4af0 2024-03-11T18:36:31.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.752+0000 7ff222ffd640 1 -- 172.21.15.90:0/542957160 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff228051400 con 0x7ff2340c4af0 2024-03-11T18:36:31.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:31.916+0000 7ff238ca6640 1 -- 172.21.15.90:0/542957160 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff2341454b0 con 0x7ff2340c4af0 2024-03-11T18:36:32.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.000+0000 7ff222ffd640 1 -- 172.21.15.90:0/542957160 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18430 (secure 0 0 0) 0x7ff228020020 con 0x7ff2340c4af0 2024-03-11T18:36:32.006 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:36:32.007 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":28,"fsid":"76d56b10-6c29-476a-804b-8482ea90f9e2","created":"2024-03-11T18:34:54.521741+0000","modified":"2024-03-11T18:36:05.526044+0000","last_up_change":"2024-03-11T18:35:16.207500+0000","last_in_change":"2024-03-11T18:35:05.213634+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T18:35:16.193842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T18:35:54.001469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T18:35:55.917321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T18:36:02.210943+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"34decff9-7280-4d84-9665-d6af8ea6de84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6819","nonce":1755667921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6821","nonce":1755667921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6825","nonce":1755667921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6823","nonce":1755667921}]},"public_addr":"172.21.15.90:6819/1755667921","cluster_addr":"172.21.15.90:6821/1755667921","heartbeat_back_addr":"172.21.15.90:6825/1755667921","heartbeat_front_addr":"172.21.15.90:6823/1755667921","state":["exists","up"]},{"osd":1,"uuid":"8ecdce9e-5486-415a-a609-dd16e945440e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6807","nonce":438682081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6809","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6811","nonce":438682081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6817","nonce":438682081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6815","nonce":438682081}]},"public_addr":"172.21.15.90:6807/438682081","cluster_addr":"172.21.15.90:6811/438682081","heartbeat_back_addr":"172.21.15.90:6817/438682081","heartbeat_front_addr":"172.21.15.90:6815/438682081","state":["exists","up"]},{"osd":2,"uuid":"737486d4-0e90-4c82-9e79-bf37c812e5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6827","nonce":2837704738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6829","nonce":2837704738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6833","nonce":2837704738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6831","nonce":2837704738}]},"public_addr":"172.21.15.90:6827/2837704738","cluster_addr":"172.21.15.90:6829/2837704738","heartbeat_back_addr":"172.21.15.90:6833/2837704738","heartbeat_front_addr":"172.21.15.90:6831/2837704738","state":["exists","up"]},{"osd":3,"uuid":"698bd661-bbef-4699-a619-f7bfe23ce12c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6803","nonce":240987971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6805","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6806","nonce":240987971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6813","nonce":240987971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6810","nonce":240987971}]},"public_addr":"172.21.15.90:6803/240987971","cluster_addr":"172.21.15.90:6806/240987971","heartbeat_back_addr":"172.21.15.90:6813/240987971","heartbeat_front_addr":"172.21.15.90:6810/240987971","state":["exists","up"]},{"osd":4,"uuid":"aba9e31f-73d9-43fe-a10b-2dc5832968b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6801","nonce":502685275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6803","nonce":502685275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6807","nonce":502685275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6805","nonce":502685275}]},"public_addr":"172.21.15.92:6801/502685275","cluster_addr":"172.21.15.92:6803/502685275","heartbeat_back_addr":"172.21.15.92:6807/502685275","heartbeat_front_addr":"172.21.15.92:6805/502685275","state":["exists","up"]},{"osd":5,"uuid":"bdd56766-4b28-4007-8de3-b5b08cf335d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6809","nonce":200142070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6811","nonce":200142070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6815","nonce":200142070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6813","nonce":200142070}]},"public_addr":"172.21.15.92:6809/200142070","cluster_addr":"172.21.15.92:6811/200142070","heartbeat_back_addr":"172.21.15.92:6815/200142070","heartbeat_front_addr":"172.21.15.92:6813/200142070","state":["exists","up"]},{"osd":6,"uuid":"6c580a59-610f-455e-a96b-033299dcc118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6817","nonce":1919010204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6819","nonce":1919010204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6823","nonce":1919010204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6821","nonce":1919010204}]},"public_addr":"172.21.15.92:6817/1919010204","cluster_addr":"172.21.15.92:6819/1919010204","heartbeat_back_addr":"172.21.15.92:6823/1919010204","heartbeat_front_addr":"172.21.15.92:6821/1919010204","state":["exists","up"]},{"osd":7,"uuid":"958184cc-aebf-4c5b-91a0-2a3e38a6b42b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6825","nonce":3475925719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6827","nonce":3475925719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6831","nonce":3475925719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6829","nonce":3475925719}]},"public_addr":"172.21.15.92:6825/3475925719","cluster_addr":"172.21.15.92:6827/3475925719","heartbeat_back_addr":"172.21.15.92:6831/3475925719","heartbeat_front_addr":"172.21.15.92:6829/3475925719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.816294+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.048084+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.137975+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.968246+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.016829+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.931676+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.041673+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.079404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T18:36:32.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.004+0000 7ff238ca6640 1 -- 172.21.15.90:0/542957160 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7ff214066e90 msgr2=0x7ff214069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:32.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.004+0000 7ff238ca6640 1 --2- 172.21.15.90:0/542957160 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7ff214066e90 0x7ff214069350 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff2240ae000 tx=0x7ff2240b5040 comp rx=0 tx=0).stop 2024-03-11T18:36:32.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.004+0000 7ff238ca6640 1 -- 172.21.15.90:0/542957160 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff2340c4af0 msgr2=0x7ff2340ca600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:32.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.004+0000 7ff238ca6640 1 --2- 172.21.15.90:0/542957160 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff2340c4af0 0x7ff2340ca600 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff22800c370 tx=0x7ff22800c830 comp rx=0 tx=0).stop 2024-03-11T18:36:32.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.004+0000 7ff238ca6640 1 -- 172.21.15.90:0/542957160 shutdown_connections 2024-03-11T18:36:32.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.004+0000 7ff238ca6640 1 --2- 172.21.15.90:0/542957160 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7ff214066e90 0x7ff214069350 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:32.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.004+0000 7ff238ca6640 1 --2- 172.21.15.90:0/542957160 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff234151b30 0x7ff23415ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:32.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.004+0000 7ff238ca6640 1 --2- 172.21.15.90:0/542957160 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff2340c4af0 0x7ff2340ca600 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:32.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.004+0000 7ff238ca6640 1 --2- 172.21.15.90:0/542957160 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff234070b00 0x7ff2341f41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:32.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.004+0000 7ff238ca6640 1 -- 172.21.15.90:0/542957160 >> 172.21.15.90:0/542957160 conn(0x7ff23406ac00 msgr2=0x7ff23414b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:32.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.004+0000 7ff238ca6640 1 -- 172.21.15.90:0/542957160 shutdown_connections 2024-03-11T18:36:32.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.004+0000 7ff238ca6640 1 -- 172.21.15.90:0/542957160 wait complete. 2024-03-11T18:36:32.029 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-11T18:36:32.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.144+0000 7fe102218640 1 Processor -- start 2024-03-11T18:36:32.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.144+0000 7fe102218640 1 -- start start 2024-03-11T18:36:32.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.148+0000 7fe102218640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fe0fc150a90 0x7fe0fc157220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:32.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.148+0000 7fe102218640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0fc0c5e60 con 0x7fe0fc0c65b0 2024-03-11T18:36:32.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.148+0000 7fe102218640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0fc156a60 con 0x7fe0fc0cc280 2024-03-11T18:36:32.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.148+0000 7fe102218640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0fc0c55c0 con 0x7fe0fc150a90 2024-03-11T18:36:32.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.148+0000 7fe0fbfff640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe0fc0cc280 0x7fe0fc161240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.90:40618/0 (socket says 172.21.15.90:40618) 2024-03-11T18:36:32.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.148+0000 7fe0fbfff640 1 -- 172.21.15.90:0/1813833783 learned_addr learned my addr 172.21.15.90:0/1813833783 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T18:36:32.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.264+0000 7fe0fb7fe640 1 --2- 172.21.15.90:0/1813833783 >> v2:172.21.15.90:3300/0 conn(0x7fe0fc150a90 0x7fe0fc157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:32.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.276+0000 7fe0fb7fe640 1 -- 172.21.15.90:0/1813833783 >> v1:172.21.15.92:6789/0 conn(0x7fe0fc0cc280 legacy=0x7fe0fc161240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:32.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.276+0000 7fe0fb7fe640 1 -- 172.21.15.90:0/1813833783 >> v1:172.21.15.90:6789/0 conn(0x7fe0fc0c65b0 legacy=0x7fe0fc0c37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:32.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.276+0000 7fe0fb7fe640 1 -- 172.21.15.90:0/1813833783 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0fc0c58a0 con 0x7fe0fc150a90 2024-03-11T18:36:32.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.292+0000 7fe0fb7fe640 1 --2- 172.21.15.90:0/1813833783 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0fc150a90 0x7fe0fc157220 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe0f4009d90 tx=0x7fe0f401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d98eccc77614b7d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:32.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.300+0000 7fe0fa7fc640 1 -- 172.21.15.90:0/1813833783 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f40a5810 con 0x7fe0fc150a90 2024-03-11T18:36:32.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.300+0000 7fe0fa7fc640 1 -- 172.21.15.90:0/1813833783 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f40a5970 con 0x7fe0fc150a90 2024-03-11T18:36:32.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.300+0000 7fe0fa7fc640 1 -- 172.21.15.90:0/1813833783 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f40a5d00 con 0x7fe0fc150a90 2024-03-11T18:36:32.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.300+0000 7fe102218640 1 -- 172.21.15.90:0/1813833783 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0fc150a90 msgr2=0x7fe0fc157220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:32.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.300+0000 7fe102218640 1 --2- 172.21.15.90:0/1813833783 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0fc150a90 0x7fe0fc157220 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe0f4009d90 tx=0x7fe0f401c970 comp rx=0 tx=0).stop 2024-03-11T18:36:32.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.300+0000 7fe102218640 1 -- 172.21.15.90:0/1813833783 shutdown_connections 2024-03-11T18:36:32.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.300+0000 7fe102218640 1 --2- 172.21.15.90:0/1813833783 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0fc150a90 0x7fe0fc157220 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:32.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.300+0000 7fe102218640 1 -- 172.21.15.90:0/1813833783 >> 172.21.15.90:0/1813833783 conn(0x7fe0fc071aa0 msgr2=0x7fe0fc071eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:32.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.304+0000 7fe102218640 1 -- 172.21.15.90:0/1813833783 shutdown_connections 2024-03-11T18:36:32.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.304+0000 7fe102218640 1 -- 172.21.15.90:0/1813833783 wait complete. 2024-03-11T18:36:32.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.304+0000 7fe102218640 1 Processor -- start 2024-03-11T18:36:32.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.304+0000 7fe102218640 1 -- start start 2024-03-11T18:36:32.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.304+0000 7fe102218640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe0fc0c65b0 0x7fe0fc152340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:32.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.304+0000 7fe102218640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0fc0cc280 0x7fe0fc13b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:32.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.304+0000 7fe102218640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0fc150a90 0x7fe0fc0ca640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:32.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.304+0000 7fe102218640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc153aa0 con 0x7fe0fc150a90 2024-03-11T18:36:32.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.304+0000 7fe102218640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc0ca190 con 0x7fe0fc0cc280 2024-03-11T18:36:32.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.304+0000 7fe102218640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc1f4630 con 0x7fe0fc0c65b0 2024-03-11T18:36:32.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.308+0000 7fe0fbfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0fc150a90 0x7fe0fc0ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:32.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.308+0000 7fe0fbfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0fc150a90 0x7fe0fc0ca640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55664/0 (socket says 172.21.15.90:55664) 2024-03-11T18:36:32.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.308+0000 7fe0fbfff640 1 -- 172.21.15.90:0/2601683069 learned_addr learned my addr 172.21.15.90:0/2601683069 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:32.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.312+0000 7fe0fbfff640 1 -- 172.21.15.90:0/2601683069 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe0fc0c65b0 msgr2=0x7fe0fc152340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:32.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.312+0000 7fe0fbfff640 1 --2- 172.21.15.90:0/2601683069 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe0fc0c65b0 0x7fe0fc152340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:32.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.312+0000 7fe0fbfff640 1 -- 172.21.15.90:0/2601683069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0fc0cc280 msgr2=0x7fe0fc13b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:32.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.312+0000 7fe0fbfff640 1 --2- 172.21.15.90:0/2601683069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0fc0cc280 0x7fe0fc13b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:32.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.312+0000 7fe0fbfff640 1 -- 172.21.15.90:0/2601683069 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f4009a70 con 0x7fe0fc150a90 2024-03-11T18:36:32.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.316+0000 7fe0fbfff640 1 --2- 172.21.15.90:0/2601683069 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0fc150a90 0x7fe0fc0ca640 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec00c400 tx=0x7fe0ec00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:32.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.320+0000 7fe0e3fff640 1 -- 172.21.15.90:0/2601683069 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec005cf0 con 0x7fe0fc150a90 2024-03-11T18:36:32.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.320+0000 7fe0e3fff640 1 -- 172.21.15.90:0/2601683069 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0ec005e50 con 0x7fe0fc150a90 2024-03-11T18:36:32.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.320+0000 7fe0e3fff640 1 -- 172.21.15.90:0/2601683069 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec0072d0 con 0x7fe0fc150a90 2024-03-11T18:36:32.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.320+0000 7fe102218640 1 -- 172.21.15.90:0/2601683069 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0fc1f48f0 con 0x7fe0fc150a90 2024-03-11T18:36:32.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.320+0000 7fe102218640 1 -- 172.21.15.90:0/2601683069 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0fc1f4eb0 con 0x7fe0fc150a90 2024-03-11T18:36:32.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.320+0000 7fe102218640 1 -- 172.21.15.90:0/2601683069 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0fc0ca330 con 0x7fe0fc150a90 2024-03-11T18:36:32.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.344+0000 7fe0e3fff640 1 -- 172.21.15.90:0/2601683069 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fe0ec007430 con 0x7fe0fc150a90 2024-03-11T18:36:32.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.344+0000 7fe0e3fff640 1 --2- 172.21.15.90:0/2601683069 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe0e8066e90 0x7fe0e8069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:32.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.344+0000 7fe0fb7fe640 1 --2- 172.21.15.90:0/2601683069 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe0e8066e90 0x7fe0e8069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:32.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.344+0000 7fe0fb7fe640 1 --2- 172.21.15.90:0/2601683069 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe0e8066e90 0x7fe0e8069350 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe0f40af000 tx=0x7fe0f4004ac0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:32.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.656+0000 7fe0e3fff640 1 -- 172.21.15.90:0/2601683069 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe0ec0553c0 con 0x7fe0fc150a90 2024-03-11T18:36:32.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.656+0000 7fe0e3fff640 1 -- 172.21.15.90:0/2601683069 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe0ec051660 con 0x7fe0fc150a90 2024-03-11T18:36:32.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:32.868+0000 7fe102218640 1 -- 172.21.15.90:0/2601683069 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe0fc04eef0 con 0x7fe0fc150a90 2024-03-11T18:36:33.972 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:36:33.975 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:36:34.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.060+0000 7fe0e3fff640 1 -- 172.21.15.90:0/2601683069 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe0ec054d20 con 0x7fe0fc150a90 2024-03-11T18:36:34.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.064+0000 7fe102218640 1 -- 172.21.15.90:0/2601683069 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe0e8066e90 msgr2=0x7fe0e8069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:34.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.064+0000 7fe102218640 1 --2- 172.21.15.90:0/2601683069 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe0e8066e90 0x7fe0e8069350 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe0f40af000 tx=0x7fe0f4004ac0 comp rx=0 tx=0).stop 2024-03-11T18:36:34.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.064+0000 7fe102218640 1 -- 172.21.15.90:0/2601683069 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0fc150a90 msgr2=0x7fe0fc0ca640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:34.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.064+0000 7fe102218640 1 --2- 172.21.15.90:0/2601683069 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0fc150a90 0x7fe0fc0ca640 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec00c400 tx=0x7fe0ec00c8c0 comp rx=0 tx=0).stop 2024-03-11T18:36:34.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.064+0000 7fe102218640 1 -- 172.21.15.90:0/2601683069 shutdown_connections 2024-03-11T18:36:34.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.064+0000 7fe102218640 1 --2- 172.21.15.90:0/2601683069 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe0e8066e90 0x7fe0e8069350 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:34.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.064+0000 7fe102218640 1 --2- 172.21.15.90:0/2601683069 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0fc150a90 0x7fe0fc0ca640 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:34.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.064+0000 7fe102218640 1 --2- 172.21.15.90:0/2601683069 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe0fc0cc280 0x7fe0fc13b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:34.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.064+0000 7fe102218640 1 --2- 172.21.15.90:0/2601683069 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe0fc0c65b0 0x7fe0fc152340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:34.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.064+0000 7fe102218640 1 -- 172.21.15.90:0/2601683069 >> 172.21.15.90:0/2601683069 conn(0x7fe0fc071aa0 msgr2=0x7fe0fc14fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:34.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.064+0000 7fe102218640 1 -- 172.21.15.90:0/2601683069 shutdown_connections 2024-03-11T18:36:34.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.064+0000 7fe102218640 1 -- 172.21.15.90:0/2601683069 wait complete. 2024-03-11T18:36:34.093 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300} 2024-03-11T18:36:34.093 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-03-11T18:36:34.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.216+0000 7fb0713d2640 1 Processor -- start 2024-03-11T18:36:34.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.216+0000 7fb0713d2640 1 -- start start 2024-03-11T18:36:34.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.220+0000 7fb0713d2640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb06c151c70 0x7fb06c06bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:34.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.220+0000 7fb0713d2640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb06c0c7f20 0x7fb06c06c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:34.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.220+0000 7fb0713d2640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb06c0c6720 con 0x7fb06c066d60 2024-03-11T18:36:34.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.220+0000 7fb0713d2640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb06c0cc7b0 con 0x7fb06c151c70 2024-03-11T18:36:34.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.220+0000 7fb0713d2640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb06c0c5d20 con 0x7fb06c0c7f20 2024-03-11T18:36:34.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.220+0000 7fb06affd640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb06c151c70 0x7fb06c06bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:34.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.220+0000 7fb0627fc640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb06c0c7f20 0x7fb06c06c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:34.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.220+0000 7fb06affd640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb06c151c70 0x7fb06c06bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.90:45208/0 (socket says 172.21.15.90:45208) 2024-03-11T18:36:34.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.220+0000 7fb06affd640 1 -- 172.21.15.90:0/167371828 learned_addr learned my addr 172.21.15.90:0/167371828 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:34.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.228+0000 7fb06affd640 1 -- 172.21.15.90:0/167371828 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb06c0c7f20 msgr2=0x7fb06c06c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:34.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.228+0000 7fb06affd640 1 --2- 172.21.15.90:0/167371828 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb06c0c7f20 0x7fb06c06c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:34.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.228+0000 7fb06affd640 1 -- 172.21.15.90:0/167371828 >> v1:172.21.15.90:6789/0 conn(0x7fb06c066d60 legacy=0x7fb06c157ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:34.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.228+0000 7fb06affd640 1 -- 172.21.15.90:0/167371828 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb06c0c5a40 con 0x7fb06c151c70 2024-03-11T18:36:34.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.232+0000 7fb06affd640 1 --2- 172.21.15.90:0/167371828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb06c151c70 0x7fb06c06bc10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb05c009e50 tx=0x7fb05c01c8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7eb0b7770d3477b server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:34.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.240+0000 7fb06a7fc640 1 -- 172.21.15.90:0/167371828 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05c0a5720 con 0x7fb06c151c70 2024-03-11T18:36:34.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.240+0000 7fb06a7fc640 1 -- 172.21.15.90:0/167371828 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb05c0a5880 con 0x7fb06c151c70 2024-03-11T18:36:34.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.240+0000 7fb06a7fc640 1 -- 172.21.15.90:0/167371828 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05c0a5c10 con 0x7fb06c151c70 2024-03-11T18:36:34.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.240+0000 7fb0713d2640 1 -- 172.21.15.90:0/167371828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb06c151c70 msgr2=0x7fb06c06bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:34.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.240+0000 7fb0713d2640 1 --2- 172.21.15.90:0/167371828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb06c151c70 0x7fb06c06bc10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb05c009e50 tx=0x7fb05c01c8a0 comp rx=0 tx=0).stop 2024-03-11T18:36:34.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.240+0000 7fb0713d2640 1 -- 172.21.15.90:0/167371828 shutdown_connections 2024-03-11T18:36:34.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.240+0000 7fb0713d2640 1 --2- 172.21.15.90:0/167371828 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb06c0c7f20 0x7fb06c06c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:34.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.240+0000 7fb0713d2640 1 --2- 172.21.15.90:0/167371828 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb06c151c70 0x7fb06c06bc10 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:34.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.240+0000 7fb0713d2640 1 -- 172.21.15.90:0/167371828 >> 172.21.15.90:0/167371828 conn(0x7fb06c0cb880 msgr2=0x7fb06c0cbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:34.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.240+0000 7fb0713d2640 1 -- 172.21.15.90:0/167371828 shutdown_connections 2024-03-11T18:36:34.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.240+0000 7fb0713d2640 1 -- 172.21.15.90:0/167371828 wait complete. 2024-03-11T18:36:34.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.240+0000 7fb0713d2640 1 Processor -- start 2024-03-11T18:36:34.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.240+0000 7fb0713d2640 1 -- start start 2024-03-11T18:36:34.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.240+0000 7fb0713d2640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb06c066d60 0x7fb06c0c4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:34.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.240+0000 7fb0713d2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb06c0c7f20 0x7fb06c1f6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:34.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.240+0000 7fb0713d2640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb06c151c70 0x7fb06c1f6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:34.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.240+0000 7fb0713d2640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb06c0c27f0 con 0x7fb06c066d60 2024-03-11T18:36:34.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.240+0000 7fb0713d2640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb06c14fe20 con 0x7fb06c0c7f20 2024-03-11T18:36:34.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.240+0000 7fb0713d2640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb06c144230 con 0x7fb06c151c70 2024-03-11T18:36:34.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.244+0000 7fb0627fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb06c0c7f20 0x7fb06c1f6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:34.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.244+0000 7fb0627fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb06c0c7f20 0x7fb06c1f6530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.90:45214/0 (socket says 172.21.15.90:45214) 2024-03-11T18:36:34.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.244+0000 7fb0627fc640 1 -- 172.21.15.90:0/572747561 learned_addr learned my addr 172.21.15.90:0/572747561 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:34.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.248+0000 7fb06affd640 1 --2- 172.21.15.90:0/572747561 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb06c066d60 0x7fb06c0c4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:34.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.252+0000 7fb0627fc640 1 -- 172.21.15.90:0/572747561 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb06c151c70 msgr2=0x7fb06c1f6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:34.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.252+0000 7fb0627fc640 1 --2- 172.21.15.90:0/572747561 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb06c151c70 0x7fb06c1f6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:34.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.252+0000 7fb0627fc640 1 -- 172.21.15.90:0/572747561 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb06c066d60 msgr2=0x7fb06c0c4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:34.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.252+0000 7fb0627fc640 1 --2- 172.21.15.90:0/572747561 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb06c066d60 0x7fb06c0c4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:34.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.252+0000 7fb0627fc640 1 -- 172.21.15.90:0/572747561 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb05c009b30 con 0x7fb06c0c7f20 2024-03-11T18:36:34.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.260+0000 7fb0627fc640 1 --2- 172.21.15.90:0/572747561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb06c0c7f20 0x7fb06c1f6530 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb05400e870 tx=0x7fb05400ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:34.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.272+0000 7fb063fff640 1 -- 172.21.15.90:0/572747561 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb054009510 con 0x7fb06c0c7f20 2024-03-11T18:36:34.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.272+0000 7fb063fff640 1 -- 172.21.15.90:0/572747561 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb05400eec0 con 0x7fb06c0c7f20 2024-03-11T18:36:34.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.272+0000 7fb063fff640 1 -- 172.21.15.90:0/572747561 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05400cb80 con 0x7fb06c0c7f20 2024-03-11T18:36:34.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.272+0000 7fb0713d2640 1 -- 172.21.15.90:0/572747561 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb06c14db80 con 0x7fb06c0c7f20 2024-03-11T18:36:34.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.272+0000 7fb0713d2640 1 -- 172.21.15.90:0/572747561 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb06c144440 con 0x7fb06c0c7f20 2024-03-11T18:36:34.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.272+0000 7fb0713d2640 1 -- 172.21.15.90:0/572747561 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb06c1f6330 con 0x7fb06c0c7f20 2024-03-11T18:36:34.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.616+0000 7fb063fff640 1 -- 172.21.15.90:0/572747561 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fb05400cce0 con 0x7fb06c0c7f20 2024-03-11T18:36:34.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.616+0000 7fb063fff640 1 --2- 172.21.15.90:0/572747561 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fb04c066e90 0x7fb04c069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:34.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.616+0000 7fb06affd640 1 --2- 172.21.15.90:0/572747561 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fb04c066e90 0x7fb04c069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:34.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.616+0000 7fb06affd640 1 --2- 172.21.15.90:0/572747561 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fb04c066e90 0x7fb04c069350 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb05c0ae000 tx=0x7fb05c0a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:34.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.620+0000 7fb063fff640 1 -- 172.21.15.90:0/572747561 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb054055df0 con 0x7fb06c0c7f20 2024-03-11T18:36:34.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.648+0000 7fb063fff640 1 -- 172.21.15.90:0/572747561 <== mon.1 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb054052090 con 0x7fb06c0c7f20 2024-03-11T18:36:34.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:34.868+0000 7fb0713d2640 1 -- 172.21.15.90:0/572747561 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7fb06c04eef0 con 0x7fb06c0c7f20 2024-03-11T18:36:36.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.200+0000 7fb063fff640 1 -- 172.21.15.90:0/572747561 <== mon.1 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v9) v1 ==== 105+0+0 (secure 0 0 0) 0x7fb0540170b0 con 0x7fb06c0c7f20 2024-03-11T18:36:36.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.204+0000 7fb0713d2640 1 -- 172.21.15.90:0/572747561 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fb04c066e90 msgr2=0x7fb04c069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:36.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.204+0000 7fb0713d2640 1 --2- 172.21.15.90:0/572747561 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fb04c066e90 0x7fb04c069350 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb05c0ae000 tx=0x7fb05c0a7070 comp rx=0 tx=0).stop 2024-03-11T18:36:36.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.204+0000 7fb0713d2640 1 -- 172.21.15.90:0/572747561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb06c0c7f20 msgr2=0x7fb06c1f6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:36.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.204+0000 7fb0713d2640 1 --2- 172.21.15.90:0/572747561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb06c0c7f20 0x7fb06c1f6530 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb05400e870 tx=0x7fb05400ec70 comp rx=0 tx=0).stop 2024-03-11T18:36:36.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.204+0000 7fb0713d2640 1 -- 172.21.15.90:0/572747561 shutdown_connections 2024-03-11T18:36:36.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.204+0000 7fb0713d2640 1 --2- 172.21.15.90:0/572747561 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fb04c066e90 0x7fb04c069350 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:36.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.204+0000 7fb0713d2640 1 --2- 172.21.15.90:0/572747561 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb06c151c70 0x7fb06c1f6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:36.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.204+0000 7fb0713d2640 1 --2- 172.21.15.90:0/572747561 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb06c0c7f20 0x7fb06c1f6530 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:36.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.204+0000 7fb0713d2640 1 --2- 172.21.15.90:0/572747561 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb06c066d60 0x7fb06c0c4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:36.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.204+0000 7fb0713d2640 1 -- 172.21.15.90:0/572747561 >> 172.21.15.90:0/572747561 conn(0x7fb06c0cb880 msgr2=0x7fb06c160490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:36.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.204+0000 7fb0713d2640 1 -- 172.21.15.90:0/572747561 shutdown_connections 2024-03-11T18:36:36.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.204+0000 7fb0713d2640 1 -- 172.21.15.90:0/572747561 wait complete. 2024-03-11T18:36:36.234 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-11T18:36:36.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.356+0000 7f99a294e640 1 Processor -- start 2024-03-11T18:36:36.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.356+0000 7f99a294e640 1 -- start start 2024-03-11T18:36:36.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.356+0000 7f99a294e640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f999c151000 0x7f999c0c2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:36.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.356+0000 7f99a294e640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f999c0c5b50 0x7f999c0c4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:36.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.356+0000 7f99a294e640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f999c0c7980 con 0x7f999c066d60 2024-03-11T18:36:36.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.356+0000 7f99a294e640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f999c0cc820 con 0x7f999c0c5b50 2024-03-11T18:36:36.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.356+0000 7f99a294e640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f999c0caca0 con 0x7f999c151000 2024-03-11T18:36:36.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.356+0000 7f99937fe640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f999c0c5b50 0x7f999c0c4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:36.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.356+0000 7f99937fe640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f999c0c5b50 0x7f999c0c4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50204/0 (socket says 172.21.15.90:50204) 2024-03-11T18:36:36.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.356+0000 7f99937fe640 1 -- 172.21.15.90:0/1480663824 learned_addr learned my addr 172.21.15.90:0/1480663824 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:36.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.360+0000 7f99937fe640 1 -- 172.21.15.90:0/1480663824 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f999c151000 msgr2=0x7f999c0c2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:36.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.360+0000 7f99937fe640 1 --2- 172.21.15.90:0/1480663824 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f999c151000 0x7f999c0c2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:36.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.360+0000 7f99937fe640 1 -- 172.21.15.90:0/1480663824 >> v1:172.21.15.92:6789/0 conn(0x7f999c066d60 legacy=0x7f999c0c6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:36.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.360+0000 7f99937fe640 1 -- 172.21.15.90:0/1480663824 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f999c0c2ad0 con 0x7f999c0c5b50 2024-03-11T18:36:36.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.364+0000 7f99937fe640 1 --2- 172.21.15.90:0/1480663824 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f999c0c5b50 0x7f999c0c4ad0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f998400bb10 tx=0x7f998401eae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6529009cb92bf2bf server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:36.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.368+0000 7f999b7fe640 1 -- 172.21.15.90:0/1480663824 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99840a7bb0 con 0x7f999c0c5b50 2024-03-11T18:36:36.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.368+0000 7f999b7fe640 1 -- 172.21.15.90:0/1480663824 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99840a7d10 con 0x7f999c0c5b50 2024-03-11T18:36:36.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.368+0000 7f999b7fe640 1 -- 172.21.15.90:0/1480663824 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99840af660 con 0x7f999c0c5b50 2024-03-11T18:36:36.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.368+0000 7f99a294e640 1 -- 172.21.15.90:0/1480663824 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f999c0c5b50 msgr2=0x7f999c0c4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:36.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.368+0000 7f99a294e640 1 --2- 172.21.15.90:0/1480663824 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f999c0c5b50 0x7f999c0c4ad0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f998400bb10 tx=0x7f998401eae0 comp rx=0 tx=0).stop 2024-03-11T18:36:36.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.368+0000 7f99a294e640 1 -- 172.21.15.90:0/1480663824 shutdown_connections 2024-03-11T18:36:36.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.368+0000 7f99a294e640 1 --2- 172.21.15.90:0/1480663824 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f999c0c5b50 0x7f999c0c4ad0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:36.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.368+0000 7f99a294e640 1 --2- 172.21.15.90:0/1480663824 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f999c151000 0x7f999c0c2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:36.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.368+0000 7f99a294e640 1 -- 172.21.15.90:0/1480663824 >> 172.21.15.90:0/1480663824 conn(0x7f999c0cb880 msgr2=0x7f999c0cbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:36.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.368+0000 7f99a294e640 1 -- 172.21.15.90:0/1480663824 shutdown_connections 2024-03-11T18:36:36.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.368+0000 7f99a294e640 1 -- 172.21.15.90:0/1480663824 wait complete. 2024-03-11T18:36:36.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.372+0000 7f99a294e640 1 Processor -- start 2024-03-11T18:36:36.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.372+0000 7f99a294e640 1 -- start start 2024-03-11T18:36:36.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.372+0000 7f99a294e640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f999c066d60 0x7f999c150290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:36.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.372+0000 7f99a294e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f999c0c5b50 0x7f999c141ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:36.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.372+0000 7f99a294e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f999c151000 0x7f999c14eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:36.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.372+0000 7f99a294e640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f999c0c25d0 con 0x7f999c0c5b50 2024-03-11T18:36:36.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.372+0000 7f99a294e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f999c13bc20 con 0x7f999c151000 2024-03-11T18:36:36.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.372+0000 7f99a294e640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f999c1507d0 con 0x7f999c066d60 2024-03-11T18:36:36.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.372+0000 7f99937fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f999c0c5b50 0x7f999c141ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:36.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.372+0000 7f99937fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f999c0c5b50 0x7f999c141ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50210/0 (socket says 172.21.15.90:50210) 2024-03-11T18:36:36.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.372+0000 7f99937fe640 1 -- 172.21.15.90:0/3284005026 learned_addr learned my addr 172.21.15.90:0/3284005026 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:36.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.376+0000 7f99a0eeb640 1 --2- 172.21.15.90:0/3284005026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f999c151000 0x7f999c14eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:36.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.376+0000 7f99937fe640 1 -- 172.21.15.90:0/3284005026 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f999c066d60 msgr2=0x7f999c150290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:36.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.376+0000 7f99937fe640 1 --2- 172.21.15.90:0/3284005026 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f999c066d60 0x7f999c150290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:36.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.376+0000 7f99937fe640 1 -- 172.21.15.90:0/3284005026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f999c151000 msgr2=0x7f999c14eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:36.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.376+0000 7f99937fe640 1 --2- 172.21.15.90:0/3284005026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f999c151000 0x7f999c14eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:36.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.376+0000 7f99937fe640 1 -- 172.21.15.90:0/3284005026 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f998400b770 con 0x7f999c0c5b50 2024-03-11T18:36:36.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.380+0000 7f99937fe640 1 --2- 172.21.15.90:0/3284005026 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f999c0c5b50 0x7f999c141ab0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f998400b2c0 tx=0x7f998401b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:36.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.384+0000 7f99997fa640 1 -- 172.21.15.90:0/3284005026 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99840afe60 con 0x7f999c0c5b50 2024-03-11T18:36:36.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.384+0000 7f99997fa640 1 -- 172.21.15.90:0/3284005026 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99840b9410 con 0x7f999c0c5b50 2024-03-11T18:36:36.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.384+0000 7f99997fa640 1 -- 172.21.15.90:0/3284005026 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99840b97c0 con 0x7f999c0c5b50 2024-03-11T18:36:36.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.384+0000 7f99a294e640 1 -- 172.21.15.90:0/3284005026 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f999c1f1de0 con 0x7f999c0c5b50 2024-03-11T18:36:36.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.384+0000 7f99a294e640 1 -- 172.21.15.90:0/3284005026 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f999c1f2340 con 0x7f999c0c5b50 2024-03-11T18:36:36.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.384+0000 7f99a294e640 1 -- 172.21.15.90:0/3284005026 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f999c04eef0 con 0x7f999c0c5b50 2024-03-11T18:36:36.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.664+0000 7f99997fa640 1 -- 172.21.15.90:0/3284005026 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f99840b6050 con 0x7f999c0c5b50 2024-03-11T18:36:36.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.668+0000 7f99997fa640 1 --2- 172.21.15.90:0/3284005026 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f997c066e90 0x7f997c069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:36.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.668+0000 7f999bfff640 1 --2- 172.21.15.90:0/3284005026 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f997c066e90 0x7f997c069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:36.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.668+0000 7f999bfff640 1 --2- 172.21.15.90:0/3284005026 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f997c066e90 0x7f997c069350 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f998c009730 tx=0x7f998c006ca0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:36.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.700+0000 7f99997fa640 1 -- 172.21.15.90:0/3284005026 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f99840f33e0 con 0x7f999c0c5b50 2024-03-11T18:36:36.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.700+0000 7f99997fa640 1 -- 172.21.15.90:0/3284005026 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f99840eef80 con 0x7f999c0c5b50 2024-03-11T18:36:36.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:36.916+0000 7f99a294e640 1 -- 172.21.15.90:0/3284005026 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f999c0c5f30 con 0x7f999c0c5b50 2024-03-11T18:36:38.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.140+0000 7f99997fa640 1 -- 172.21.15.90:0/3284005026 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v11) v1 ==== 122+0+0 (secure 0 0 0) 0x7f99840b4020 con 0x7f999c0c5b50 2024-03-11T18:36:38.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.144+0000 7f99a294e640 1 -- 172.21.15.90:0/3284005026 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f997c066e90 msgr2=0x7f997c069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:38.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.144+0000 7f99a294e640 1 --2- 172.21.15.90:0/3284005026 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f997c066e90 0x7f997c069350 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f998c009730 tx=0x7f998c006ca0 comp rx=0 tx=0).stop 2024-03-11T18:36:38.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.144+0000 7f99a294e640 1 -- 172.21.15.90:0/3284005026 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f999c0c5b50 msgr2=0x7f999c141ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:38.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.144+0000 7f99a294e640 1 --2- 172.21.15.90:0/3284005026 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f999c0c5b50 0x7f999c141ab0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f998400b2c0 tx=0x7f998401b880 comp rx=0 tx=0).stop 2024-03-11T18:36:38.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.144+0000 7f99a294e640 1 -- 172.21.15.90:0/3284005026 shutdown_connections 2024-03-11T18:36:38.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.144+0000 7f99a294e640 1 --2- 172.21.15.90:0/3284005026 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f997c066e90 0x7f997c069350 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:38.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.144+0000 7f99a294e640 1 --2- 172.21.15.90:0/3284005026 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f999c151000 0x7f999c14eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:38.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.144+0000 7f99a294e640 1 --2- 172.21.15.90:0/3284005026 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f999c0c5b50 0x7f999c141ab0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:38.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.144+0000 7f99a294e640 1 --2- 172.21.15.90:0/3284005026 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f999c066d60 0x7f999c150290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:38.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.144+0000 7f99a294e640 1 -- 172.21.15.90:0/3284005026 >> 172.21.15.90:0/3284005026 conn(0x7f999c0cb880 msgr2=0x7f999c06b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:38.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.144+0000 7f99a294e640 1 -- 172.21.15.90:0/3284005026 shutdown_connections 2024-03-11T18:36:38.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.144+0000 7f99a294e640 1 -- 172.21.15.90:0/3284005026 wait complete. 2024-03-11T18:36:38.171 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-03-11T18:36:38.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.296+0000 7ff3f36bb640 1 Processor -- start 2024-03-11T18:36:38.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.296+0000 7ff3f36bb640 1 -- start start 2024-03-11T18:36:38.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.300+0000 7ff3f36bb640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff3ec151c10 0x7ff3ec06bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:38.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.300+0000 7ff3f36bb640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff3ec151ff0 0x7ff3ec06c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:38.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.300+0000 7ff3f36bb640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3ec0c6e60 con 0x7ff3ec0c8b50 2024-03-11T18:36:38.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.300+0000 7ff3f36bb640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3ec0c75f0 con 0x7ff3ec151ff0 2024-03-11T18:36:38.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.300+0000 7ff3f36bb640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec0c7090 con 0x7ff3ec151c10 2024-03-11T18:36:38.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.300+0000 7ff3f0c56640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7ff3ec0c8b50 0x7ff3ec15b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:39158/0 (socket says 172.21.15.90:39158) 2024-03-11T18:36:38.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.300+0000 7ff3f0c56640 1 -- 172.21.15.90:0/1688052773 learned_addr learned my addr 172.21.15.90:0/1688052773 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T18:36:38.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.300+0000 7ff3f1c58640 1 --2- 172.21.15.90:0/1688052773 >> v2:172.21.15.92:3300/0 conn(0x7ff3ec151ff0 0x7ff3ec06c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:38.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.300+0000 7ff3e3fff640 1 -- 172.21.15.90:0/1688052773 <== mon.0 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 639187075 0 0) 0x7ff3ec0c6e60 con 0x7ff3ec0c8b50 2024-03-11T18:36:38.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.300+0000 7ff3e3fff640 1 -- 172.21.15.90:0/1688052773 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3d8003530 con 0x7ff3ec0c8b50 2024-03-11T18:36:38.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.304+0000 7ff3e3fff640 1 -- 172.21.15.90:0/1688052773 <== mon.0 v1:172.21.15.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 810841608 0 0) 0x7ff3d8003530 con 0x7ff3ec0c8b50 2024-03-11T18:36:38.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.304+0000 7ff3e3fff640 1 -- 172.21.15.90:0/1688052773 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff3ec151c10 msgr2=0x7ff3ec06bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:38.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.304+0000 7ff3e3fff640 1 --2- 172.21.15.90:0/1688052773 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff3ec151c10 0x7ff3ec06bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:38.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.304+0000 7ff3e3fff640 1 -- 172.21.15.90:0/1688052773 >> v2:172.21.15.92:3300/0 conn(0x7ff3ec151ff0 msgr2=0x7ff3ec06c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:38.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.304+0000 7ff3e3fff640 1 --2- 172.21.15.90:0/1688052773 >> v2:172.21.15.92:3300/0 conn(0x7ff3ec151ff0 0x7ff3ec06c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:38.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.304+0000 7ff3e3fff640 1 -- 172.21.15.90:0/1688052773 --> v1:172.21.15.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ec0c7330 con 0x7ff3ec0c8b50 2024-03-11T18:36:38.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.304+0000 7ff3e3fff640 1 -- 172.21.15.90:0/1688052773 <== mon.0 v1:172.21.15.90:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 691735435 0 0) 0x7ff3dc001680 con 0x7ff3ec0c8b50 2024-03-11T18:36:38.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.304+0000 7ff3e3fff640 1 -- 172.21.15.90:0/1688052773 >> v1:172.21.15.90:6789/0 conn(0x7ff3ec0c8b50 legacy=0x7ff3ec15b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:38.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.304+0000 7ff3e3fff640 1 --2- 172.21.15.90:0/1688052773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3d8003980 0x7ff3d8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:38.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.304+0000 7ff3e3fff640 1 --2- 172.21.15.90:0/1688052773 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff3ec151c10 0x7ff3d80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:38.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.304+0000 7ff3e3fff640 1 --2- 172.21.15.90:0/1688052773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3d8004d10 0x7ff3d8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:38.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.304+0000 7ff3e3fff640 1 -- 172.21.15.90:0/1688052773 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec0c75f0 con 0x7ff3d8003980 2024-03-11T18:36:38.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.304+0000 7ff3e3fff640 1 -- 172.21.15.90:0/1688052773 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec0c7090 con 0x7ff3d8004d10 2024-03-11T18:36:38.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.304+0000 7ff3e3fff640 1 -- 172.21.15.90:0/1688052773 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d800bba0 con 0x7ff3ec151c10 2024-03-11T18:36:38.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.308+0000 7ff3f1457640 1 --2- 172.21.15.90:0/1688052773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3d8003980 0x7ff3d8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:38.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.312+0000 7ff3f1c58640 1 --2- 172.21.15.90:0/1688052773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3d8004d10 0x7ff3d8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:38.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.312+0000 7ff3f1457640 1 -- 172.21.15.90:0/1688052773 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff3ec151c10 msgr2=0x7ff3d80047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:38.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.312+0000 7ff3f1457640 1 --2- 172.21.15.90:0/1688052773 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff3ec151c10 0x7ff3d80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:38.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.312+0000 7ff3f1457640 1 -- 172.21.15.90:0/1688052773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3d8004d10 msgr2=0x7ff3d8005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:38.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.312+0000 7ff3f1457640 1 --2- 172.21.15.90:0/1688052773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3d8004d10 0x7ff3d8005190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:38.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.312+0000 7ff3f1457640 1 -- 172.21.15.90:0/1688052773 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d8003530 con 0x7ff3d8003980 2024-03-11T18:36:38.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.316+0000 7ff3f1457640 1 --2- 172.21.15.90:0/1688052773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3d8003980 0x7ff3d8003dc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff3e8009890 tx=0x7ff3e801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=522dd3a8348f8174 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:38.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.320+0000 7ff3e3fff640 1 -- 172.21.15.90:0/1688052773 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e80ab600 con 0x7ff3d8003980 2024-03-11T18:36:38.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.320+0000 7ff3e3fff640 1 -- 172.21.15.90:0/1688052773 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3e80a7050 con 0x7ff3d8003980 2024-03-11T18:36:38.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.320+0000 7ff3e3fff640 1 -- 172.21.15.90:0/1688052773 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e80aba00 con 0x7ff3d8003980 2024-03-11T18:36:38.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.324+0000 7ff3f36bb640 1 -- 172.21.15.90:0/1688052773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3d8003980 msgr2=0x7ff3d8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:38.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.324+0000 7ff3f36bb640 1 --2- 172.21.15.90:0/1688052773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3d8003980 0x7ff3d8003dc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff3e8009890 tx=0x7ff3e801c3c0 comp rx=0 tx=0).stop 2024-03-11T18:36:38.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.324+0000 7ff3f1c58640 1 -- 172.21.15.90:0/1688052773 reap_dead start 2024-03-11T18:36:38.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.324+0000 7ff3f36bb640 1 -- 172.21.15.90:0/1688052773 shutdown_connections 2024-03-11T18:36:38.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.324+0000 7ff3f36bb640 1 -- 172.21.15.90:0/1688052773 >> 172.21.15.90:0/1688052773 conn(0x7ff3ec0cc630 msgr2=0x7ff3ec0cca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:38.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.324+0000 7ff3f36bb640 1 -- 172.21.15.90:0/1688052773 shutdown_connections 2024-03-11T18:36:38.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.324+0000 7ff3f36bb640 1 -- 172.21.15.90:0/1688052773 wait complete. 2024-03-11T18:36:38.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.324+0000 7ff3f36bb640 1 Processor -- start 2024-03-11T18:36:38.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.324+0000 7ff3f36bb640 1 -- start start 2024-03-11T18:36:38.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.324+0000 7ff3f36bb640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff3d8003980 0x7ff3ec157ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:38.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.324+0000 7ff3f36bb640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3ec1f6480 0x7ff3ec158420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:38.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.324+0000 7ff3f36bb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3ec14ac80 0x7ff3ec14b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:38.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.324+0000 7ff3f36bb640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec15fde0 con 0x7ff3ec1f6480 2024-03-11T18:36:38.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.324+0000 7ff3f36bb640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec14b640 con 0x7ff3ec14ac80 2024-03-11T18:36:38.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.324+0000 7ff3f36bb640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec160010 con 0x7ff3d8003980 2024-03-11T18:36:38.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.324+0000 7ff3f1c58640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3ec14ac80 0x7ff3ec14b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:38.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.328+0000 7ff3f1c58640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3ec14ac80 0x7ff3ec14b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.90:45246/0 (socket says 172.21.15.90:45246) 2024-03-11T18:36:38.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.328+0000 7ff3f1c58640 1 -- 172.21.15.90:0/4013284161 learned_addr learned my addr 172.21.15.90:0/4013284161 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:38.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.332+0000 7ff3f1457640 1 --2- 172.21.15.90:0/4013284161 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff3d8003980 0x7ff3ec157ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:38.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.336+0000 7ff3f1c58640 1 -- 172.21.15.90:0/4013284161 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff3d8003980 msgr2=0x7ff3ec157ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:38.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.336+0000 7ff3f1c58640 1 --2- 172.21.15.90:0/4013284161 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff3d8003980 0x7ff3ec157ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:38.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.336+0000 7ff3f1c58640 1 -- 172.21.15.90:0/4013284161 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3ec1f6480 msgr2=0x7ff3ec158420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:38.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.336+0000 7ff3f1c58640 1 --2- 172.21.15.90:0/4013284161 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3ec1f6480 0x7ff3ec158420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:38.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.336+0000 7ff3f1c58640 1 -- 172.21.15.90:0/4013284161 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ec160e50 con 0x7ff3ec14ac80 2024-03-11T18:36:38.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.344+0000 7ff3f1c58640 1 --2- 172.21.15.90:0/4013284161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3ec14ac80 0x7ff3ec14b100 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8004750 tx=0x7ff3e400f520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:38.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.352+0000 7ff3e1ffb640 1 -- 172.21.15.90:0/4013284161 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e4016620 con 0x7ff3ec14ac80 2024-03-11T18:36:38.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.352+0000 7ff3e1ffb640 1 -- 172.21.15.90:0/4013284161 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3e4016780 con 0x7ff3ec14ac80 2024-03-11T18:36:38.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.352+0000 7ff3e1ffb640 1 -- 172.21.15.90:0/4013284161 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e4016b30 con 0x7ff3ec14ac80 2024-03-11T18:36:38.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.352+0000 7ff3f36bb640 1 -- 172.21.15.90:0/4013284161 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3ec14c220 con 0x7ff3ec14ac80 2024-03-11T18:36:38.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.352+0000 7ff3f36bb640 1 -- 172.21.15.90:0/4013284161 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3ec1605b0 con 0x7ff3ec14ac80 2024-03-11T18:36:38.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.352+0000 7ff3f36bb640 1 -- 172.21.15.90:0/4013284161 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3ec04eef0 con 0x7ff3ec14ac80 2024-03-11T18:36:38.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.688+0000 7ff3e1ffb640 1 -- 172.21.15.90:0/4013284161 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7ff3e4016c90 con 0x7ff3ec14ac80 2024-03-11T18:36:38.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.688+0000 7ff3e1ffb640 1 --2- 172.21.15.90:0/4013284161 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7ff3d0066e90 0x7ff3d0069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:38.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.688+0000 7ff3f1457640 1 --2- 172.21.15.90:0/4013284161 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7ff3d0066e90 0x7ff3d0069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:38.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.688+0000 7ff3f1457640 1 --2- 172.21.15.90:0/4013284161 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7ff3d0066e90 0x7ff3d0069350 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff3e80ad000 tx=0x7ff3e8004850 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:38.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.692+0000 7ff3e1ffb640 1 -- 172.21.15.90:0/4013284161 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff3e400fe80 con 0x7ff3ec14ac80 2024-03-11T18:36:38.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.728+0000 7ff3e1ffb640 1 -- 172.21.15.90:0/4013284161 <== mon.1 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff3e4052fc0 con 0x7ff3ec14ac80 2024-03-11T18:36:38.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:38.928+0000 7ff3f36bb640 1 -- 172.21.15.90:0/4013284161 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7ff3ec151440 con 0x7ff3ec14ac80 2024-03-11T18:36:40.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.144+0000 7ff3e1ffb640 1 -- 172.21.15.90:0/4013284161 <== mon.1 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v12) v1 ==== 115+0+0 (secure 0 0 0) 0x7ff3e4056680 con 0x7ff3ec14ac80 2024-03-11T18:36:40.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.148+0000 7ff3f36bb640 1 -- 172.21.15.90:0/4013284161 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7ff3d0066e90 msgr2=0x7ff3d0069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:40.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.148+0000 7ff3f36bb640 1 --2- 172.21.15.90:0/4013284161 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7ff3d0066e90 0x7ff3d0069350 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff3e80ad000 tx=0x7ff3e8004850 comp rx=0 tx=0).stop 2024-03-11T18:36:40.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.148+0000 7ff3f36bb640 1 -- 172.21.15.90:0/4013284161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3ec14ac80 msgr2=0x7ff3ec14b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:40.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.148+0000 7ff3f36bb640 1 --2- 172.21.15.90:0/4013284161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3ec14ac80 0x7ff3ec14b100 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8004750 tx=0x7ff3e400f520 comp rx=0 tx=0).stop 2024-03-11T18:36:40.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.148+0000 7ff3f36bb640 1 -- 172.21.15.90:0/4013284161 shutdown_connections 2024-03-11T18:36:40.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.148+0000 7ff3f36bb640 1 --2- 172.21.15.90:0/4013284161 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7ff3d0066e90 0x7ff3d0069350 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:40.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.148+0000 7ff3f36bb640 1 --2- 172.21.15.90:0/4013284161 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff3ec14ac80 0x7ff3ec14b100 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:40.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.148+0000 7ff3f36bb640 1 --2- 172.21.15.90:0/4013284161 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3ec1f6480 0x7ff3ec158420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:40.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.148+0000 7ff3f36bb640 1 --2- 172.21.15.90:0/4013284161 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff3d8003980 0x7ff3ec157ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:40.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.148+0000 7ff3f36bb640 1 -- 172.21.15.90:0/4013284161 >> 172.21.15.90:0/4013284161 conn(0x7ff3ec0cc630 msgr2=0x7ff3ec157260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:40.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.148+0000 7ff3f36bb640 1 -- 172.21.15.90:0/4013284161 shutdown_connections 2024-03-11T18:36:40.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.148+0000 7ff3f36bb640 1 -- 172.21.15.90:0/4013284161 wait complete. 2024-03-11T18:36:40.176 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-11T18:36:40.176 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T18:36:40.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.296+0000 7f395508c640 1 Processor -- start 2024-03-11T18:36:40.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.296+0000 7f395508c640 1 -- start start 2024-03-11T18:36:40.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.296+0000 7f395508c640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f39501550c0 0x7f39500c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:40.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.296+0000 7f395508c640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f395006fe60 0x7f39500c2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:40.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.296+0000 7f395508c640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39500c2ae0 con 0x7f39501554a0 2024-03-11T18:36:40.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.296+0000 7f395508c640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39500c2560 con 0x7f395006fe60 2024-03-11T18:36:40.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.296+0000 7f395508c640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3950157cc0 con 0x7f39501550c0 2024-03-11T18:36:40.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.300+0000 7f394e59c640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f39501554a0 0x7f395015ba70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.90:33678/0 (socket says 172.21.15.90:33678) 2024-03-11T18:36:40.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.300+0000 7f394e59c640 1 -- 172.21.15.90:0/1288527384 learned_addr learned my addr 172.21.15.90:0/1288527384 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T18:36:40.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.320+0000 7f394ed9d640 1 --2- 172.21.15.90:0/1288527384 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f39501550c0 0x7f39500c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:40.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.328+0000 7f394ed9d640 1 -- 172.21.15.90:0/1288527384 >> v2:172.21.15.90:3300/0 conn(0x7f395006fe60 msgr2=0x7f39500c2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:40.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.328+0000 7f394ed9d640 1 --2- 172.21.15.90:0/1288527384 >> v2:172.21.15.90:3300/0 conn(0x7f395006fe60 0x7f39500c2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:40.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.328+0000 7f394ed9d640 1 -- 172.21.15.90:0/1288527384 >> v1:172.21.15.92:6789/0 conn(0x7f39501554a0 legacy=0x7f395015ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:40.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.328+0000 7f394ed9d640 1 -- 172.21.15.90:0/1288527384 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39500c2800 con 0x7f39501550c0 2024-03-11T18:36:40.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.332+0000 7f394ed9d640 1 --2- 172.21.15.90:0/1288527384 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f39501550c0 0x7f39500c8fb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3948009d90 tx=0x7f394801c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=650aea2ccafe2640 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:40.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.340+0000 7f394dd9b640 1 -- 172.21.15.90:0/1288527384 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39480a5890 con 0x7f39501550c0 2024-03-11T18:36:40.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.340+0000 7f394dd9b640 1 -- 172.21.15.90:0/1288527384 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39480a59f0 con 0x7f39501550c0 2024-03-11T18:36:40.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.340+0000 7f394dd9b640 1 -- 172.21.15.90:0/1288527384 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39480a5d80 con 0x7f39501550c0 2024-03-11T18:36:40.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.340+0000 7f395508c640 1 -- 172.21.15.90:0/1288527384 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f39501550c0 msgr2=0x7f39500c8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:40.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.340+0000 7f395508c640 1 --2- 172.21.15.90:0/1288527384 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f39501550c0 0x7f39500c8fb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3948009d90 tx=0x7f394801c8e0 comp rx=0 tx=0).stop 2024-03-11T18:36:40.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.340+0000 7f395508c640 1 -- 172.21.15.90:0/1288527384 shutdown_connections 2024-03-11T18:36:40.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.340+0000 7f395508c640 1 --2- 172.21.15.90:0/1288527384 >> v2:172.21.15.90:3300/0 conn(0x7f395006fe60 0x7f39500c2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:40.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.340+0000 7f395508c640 1 --2- 172.21.15.90:0/1288527384 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f39501550c0 0x7f39500c8fb0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:40.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.340+0000 7f395508c640 1 -- 172.21.15.90:0/1288527384 >> 172.21.15.90:0/1288527384 conn(0x7f3950070e80 msgr2=0x7f3950066990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:40.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.340+0000 7f395508c640 1 -- 172.21.15.90:0/1288527384 shutdown_connections 2024-03-11T18:36:40.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.340+0000 7f395508c640 1 -- 172.21.15.90:0/1288527384 wait complete. 2024-03-11T18:36:40.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.344+0000 7f395508c640 1 Processor -- start 2024-03-11T18:36:40.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.344+0000 7f395508c640 1 -- start start 2024-03-11T18:36:40.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.344+0000 7f395508c640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f395006fe60 0x7f3950141c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:40.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.344+0000 7f395508c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f39501550c0 0x7f3950146d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:40.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.344+0000 7f395508c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39501554a0 0x7f39501472d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:40.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.344+0000 7f395508c640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39500b3130 con 0x7f39501550c0 2024-03-11T18:36:40.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.344+0000 7f395508c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39500af750 con 0x7f39501554a0 2024-03-11T18:36:40.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.344+0000 7f395508c640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f395006d7c0 con 0x7f395006fe60 2024-03-11T18:36:40.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.344+0000 7f394ed9d640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f395006fe60 0x7f3950141c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:40.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.344+0000 7f394f59e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39501554a0 0x7f39501472d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:40.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.344+0000 7f394ed9d640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f395006fe60 0x7f3950141c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.90:43894/0 (socket says 172.21.15.90:43894) 2024-03-11T18:36:40.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.344+0000 7f394ed9d640 1 -- 172.21.15.90:0/3984024662 learned_addr learned my addr 172.21.15.90:0/3984024662 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:40.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.352+0000 7f394ed9d640 1 -- 172.21.15.90:0/3984024662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39501554a0 msgr2=0x7f39501472d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:40.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.352+0000 7f394ed9d640 1 --2- 172.21.15.90:0/3984024662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39501554a0 0x7f39501472d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:40.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.352+0000 7f394ed9d640 1 -- 172.21.15.90:0/3984024662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f39501550c0 msgr2=0x7f3950146d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:40.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.352+0000 7f394ed9d640 1 --2- 172.21.15.90:0/3984024662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f39501550c0 0x7f3950146d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:40.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.352+0000 7f394ed9d640 1 -- 172.21.15.90:0/3984024662 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3948009a70 con 0x7f395006fe60 2024-03-11T18:36:40.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.356+0000 7f394ed9d640 1 --2- 172.21.15.90:0/3984024662 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f395006fe60 0x7f3950141c60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f394801a510 tx=0x7f3948019680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:40.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.364+0000 7f39377fe640 1 -- 172.21.15.90:0/3984024662 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39480ad850 con 0x7f395006fe60 2024-03-11T18:36:40.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.364+0000 7f39377fe640 1 -- 172.21.15.90:0/3984024662 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39480ad9b0 con 0x7f395006fe60 2024-03-11T18:36:40.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.364+0000 7f39377fe640 1 -- 172.21.15.90:0/3984024662 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39480adda0 con 0x7f395006fe60 2024-03-11T18:36:40.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.364+0000 7f395508c640 1 -- 172.21.15.90:0/3984024662 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f395006dd10 con 0x7f395006fe60 2024-03-11T18:36:40.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.364+0000 7f395508c640 1 -- 172.21.15.90:0/3984024662 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f395006da30 con 0x7f395006fe60 2024-03-11T18:36:40.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.368+0000 7f395508c640 1 -- 172.21.15.90:0/3984024662 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39501f6480 con 0x7f395006fe60 2024-03-11T18:36:40.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.700+0000 7f39377fe640 1 -- 172.21.15.90:0/3984024662 <== mon.2 v2:172.21.15.92:3301/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f39480b4050 con 0x7f395006fe60 2024-03-11T18:36:40.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.700+0000 7f39377fe640 1 --2- 172.21.15.90:0/3984024662 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f393c066ec0 0x7f393c069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:40.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.704+0000 7f394e59c640 1 --2- 172.21.15.90:0/3984024662 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f393c066ec0 0x7f393c069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:40.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.704+0000 7f394e59c640 1 --2- 172.21.15.90:0/3984024662 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f393c066ec0 0x7f393c069380 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3944000f70 tx=0x7f3944007a90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:40.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.708+0000 7f39377fe640 1 -- 172.21.15.90:0/3984024662 <== mon.2 v2:172.21.15.92:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f39480f1170 con 0x7f395006fe60 2024-03-11T18:36:40.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:40.796+0000 7f39377fe640 1 -- 172.21.15.90:0/3984024662 <== mon.2 v2:172.21.15.92:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f39480ecd10 con 0x7f395006fe60 2024-03-11T18:36:41.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.000+0000 7f395508c640 1 -- 172.21.15.90:0/3984024662 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f39501f6480 con 0x7f395006fe60 2024-03-11T18:36:41.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.068+0000 7f39377fe640 1 -- 172.21.15.90:0/3984024662 <== mon.2 v2:172.21.15.92:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+6528 (secure 0 0 0) 0x7f39501f6480 con 0x7f395006fe60 2024-03-11T18:36:41.076 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:36:41.076 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/641527992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":641527992},{"type":"v1","addr":"172.21.15.90:6839","nonce":641527992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4229,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/460417177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":460417177},{"type":"v1","addr":"172.21.15.90:6835","nonce":460417177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":4238,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6835/3568438077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":3568438077},{"type":"v1","addr":"172.21.15.92:6835","nonce":3568438077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.92:6834/2306487548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":2306487548},{"type":"v1","addr":"172.21.15.92:6834","nonce":2306487548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-11T18:35:57.363570+0000","modified":"2024-03-11T18:36:39.014666+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1],"up":{"mds_0":4255,"mds_1":4253},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4253":{"gid":4253,"name":"f","rank":1,"incarnation":10,"state":"up:creating","state_seq":1,"addr":"172.21.15.92:6837/3209534444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":3209534444},{"type":"v1","addr":"172.21.15.92:6837","nonce":3209534444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4255":{"gid":4255,"name":"e","rank":0,"incarnation":5,"state":"up:active","state_seq":6,"addr":"172.21.15.90:6837/3485943375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3485943375},{"type":"v1","addr":"172.21.15.90:6837","nonce":3485943375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T18:36:41.077 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 12 2024-03-11T18:36:41.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.072+0000 7f395508c640 1 -- 172.21.15.90:0/3984024662 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f393c066ec0 msgr2=0x7f393c069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:41.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.072+0000 7f395508c640 1 --2- 172.21.15.90:0/3984024662 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f393c066ec0 0x7f393c069380 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3944000f70 tx=0x7f3944007a90 comp rx=0 tx=0).stop 2024-03-11T18:36:41.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.072+0000 7f395508c640 1 -- 172.21.15.90:0/3984024662 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f395006fe60 msgr2=0x7f3950141c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:41.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.072+0000 7f395508c640 1 --2- 172.21.15.90:0/3984024662 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f395006fe60 0x7f3950141c60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f394801a510 tx=0x7f3948019680 comp rx=0 tx=0).stop 2024-03-11T18:36:41.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.072+0000 7f395508c640 1 -- 172.21.15.90:0/3984024662 shutdown_connections 2024-03-11T18:36:41.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.072+0000 7f395508c640 1 --2- 172.21.15.90:0/3984024662 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f393c066ec0 0x7f393c069380 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:41.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.072+0000 7f395508c640 1 --2- 172.21.15.90:0/3984024662 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f39501554a0 0x7f39501472d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:41.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.072+0000 7f395508c640 1 --2- 172.21.15.90:0/3984024662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f39501550c0 0x7f3950146d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:41.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.072+0000 7f395508c640 1 --2- 172.21.15.90:0/3984024662 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f395006fe60 0x7f3950141c60 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:41.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.072+0000 7f395508c640 1 -- 172.21.15.90:0/3984024662 >> 172.21.15.90:0/3984024662 conn(0x7f3950070e80 msgr2=0x7f3950154030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:41.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.072+0000 7f395508c640 1 -- 172.21.15.90:0/3984024662 shutdown_connections 2024-03-11T18:36:41.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.072+0000 7f395508c640 1 -- 172.21.15.90:0/3984024662 wait complete. 2024-03-11T18:36:41.100 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T18:36:41.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.228+0000 7fd575d50640 1 Processor -- start 2024-03-11T18:36:41.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.228+0000 7fd575d50640 1 -- start start 2024-03-11T18:36:41.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.228+0000 7fd575d50640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd570153f10 0x7fd57015a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:41.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.228+0000 7fd575d50640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd5700afb60 0x7fd57006c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:41.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.228+0000 7fd575d50640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd57015abb0 con 0x7fd5700c8e20 2024-03-11T18:36:41.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.228+0000 7fd575d50640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd57006c2d0 con 0x7fd5700afb60 2024-03-11T18:36:41.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.228+0000 7fd575d50640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5700c81e0 con 0x7fd570153f10 2024-03-11T18:36:41.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.268+0000 7fd56effd640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7fd5700c8e20 0x7fd57015db00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:39160/0 (socket says 172.21.15.90:39160) 2024-03-11T18:36:41.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.268+0000 7fd56effd640 1 -- 172.21.15.90:0/3754647888 learned_addr learned my addr 172.21.15.90:0/3754647888 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T18:36:41.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.272+0000 7fd56e7fc640 1 -- 172.21.15.90:0/3754647888 <== mon.0 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2241399330 0 0) 0x7fd560001120 con 0x7fd5700c8e20 2024-03-11T18:36:41.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.272+0000 7fd56e7fc640 1 -- 172.21.15.90:0/3754647888 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd558003530 con 0x7fd5700c8e20 2024-03-11T18:36:41.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.284+0000 7fd56e7fc640 1 -- 172.21.15.90:0/3754647888 <== mon.0 v1:172.21.15.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1175653505 0 0) 0x7fd558003530 con 0x7fd5700c8e20 2024-03-11T18:36:41.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.284+0000 7fd56e7fc640 1 -- 172.21.15.90:0/3754647888 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd570153f10 msgr2=0x7fd57015a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:41.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.284+0000 7fd56e7fc640 1 --2- 172.21.15.90:0/3754647888 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd570153f10 0x7fd57015a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:41.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.284+0000 7fd56e7fc640 1 -- 172.21.15.90:0/3754647888 >> v2:172.21.15.92:3300/0 conn(0x7fd5700afb60 msgr2=0x7fd57006c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:41.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.284+0000 7fd56e7fc640 1 --2- 172.21.15.90:0/3754647888 >> v2:172.21.15.92:3300/0 conn(0x7fd5700afb60 0x7fd57006c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:41.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.284+0000 7fd56e7fc640 1 -- 172.21.15.90:0/3754647888 --> v1:172.21.15.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd57006bd10 con 0x7fd5700c8e20 2024-03-11T18:36:41.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.284+0000 7fd56e7fc640 1 -- 172.21.15.90:0/3754647888 <== mon.0 v1:172.21.15.90:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 691735435 0 0) 0x7fd560001810 con 0x7fd5700c8e20 2024-03-11T18:36:41.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.284+0000 7fd56e7fc640 1 -- 172.21.15.90:0/3754647888 >> v1:172.21.15.90:6789/0 conn(0x7fd5700c8e20 legacy=0x7fd57015db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:41.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.284+0000 7fd56e7fc640 1 --2- 172.21.15.90:0/3754647888 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd558003980 0x7fd558003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:41.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.284+0000 7fd56e7fc640 1 --2- 172.21.15.90:0/3754647888 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5580047d0 0x7fd558004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:41.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.284+0000 7fd56e7fc640 1 --2- 172.21.15.90:0/3754647888 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd5580052b0 0x7fd55800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:41.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.284+0000 7fd56e7fc640 1 -- 172.21.15.90:0/3754647888 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5700c81e0 con 0x7fd558003980 2024-03-11T18:36:41.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.284+0000 7fd56e7fc640 1 -- 172.21.15.90:0/3754647888 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd55800c110 con 0x7fd5580047d0 2024-03-11T18:36:41.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.284+0000 7fd56e7fc640 1 -- 172.21.15.90:0/3754647888 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd55800c2d0 con 0x7fd5580052b0 2024-03-11T18:36:41.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.288+0000 7fd56f7fe640 1 --2- 172.21.15.90:0/3754647888 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd558003980 0x7fd558003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:41.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.296+0000 7fd56f7fe640 1 -- 172.21.15.90:0/3754647888 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd5580052b0 msgr2=0x7fd55800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:41.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.296+0000 7fd56f7fe640 1 --2- 172.21.15.90:0/3754647888 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd5580052b0 0x7fd55800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:41.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.296+0000 7fd56f7fe640 1 -- 172.21.15.90:0/3754647888 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5580047d0 msgr2=0x7fd558004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:41.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.296+0000 7fd56f7fe640 1 --2- 172.21.15.90:0/3754647888 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5580047d0 0x7fd558004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:41.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.296+0000 7fd56f7fe640 1 -- 172.21.15.90:0/3754647888 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd558003530 con 0x7fd558003980 2024-03-11T18:36:41.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.304+0000 7fd56f7fe640 1 --2- 172.21.15.90:0/3754647888 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd558003980 0x7fd558003da0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd5700caa00 tx=0x7fd56401c280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69609fbee7ae9e8c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:41.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.312+0000 7fd56e7fc640 1 -- 172.21.15.90:0/3754647888 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5640ab600 con 0x7fd558003980 2024-03-11T18:36:41.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.312+0000 7fd56e7fc640 1 -- 172.21.15.90:0/3754647888 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5640a7050 con 0x7fd558003980 2024-03-11T18:36:41.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.312+0000 7fd56e7fc640 1 -- 172.21.15.90:0/3754647888 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5640aba20 con 0x7fd558003980 2024-03-11T18:36:41.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.316+0000 7fd575d50640 1 -- 172.21.15.90:0/3754647888 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd558003980 msgr2=0x7fd558003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:41.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.316+0000 7fd575d50640 1 --2- 172.21.15.90:0/3754647888 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd558003980 0x7fd558003da0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd5700caa00 tx=0x7fd56401c280 comp rx=0 tx=0).stop 2024-03-11T18:36:41.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.316+0000 7fd56ffff640 1 -- 172.21.15.90:0/3754647888 reap_dead start 2024-03-11T18:36:41.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.316+0000 7fd575d50640 1 -- 172.21.15.90:0/3754647888 shutdown_connections 2024-03-11T18:36:41.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.316+0000 7fd575d50640 1 -- 172.21.15.90:0/3754647888 >> 172.21.15.90:0/3754647888 conn(0x7fd570070e80 msgr2=0x7fd570066990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:41.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.316+0000 7fd575d50640 1 -- 172.21.15.90:0/3754647888 shutdown_connections 2024-03-11T18:36:41.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.316+0000 7fd575d50640 1 -- 172.21.15.90:0/3754647888 wait complete. 2024-03-11T18:36:41.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.316+0000 7fd575d50640 1 Processor -- start 2024-03-11T18:36:41.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.316+0000 7fd575d50640 1 -- start start 2024-03-11T18:36:41.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.316+0000 7fd575d50640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5700ae090 0x7fd570157140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:41.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.316+0000 7fd575d50640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd5700ae470 0x7fd570154560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:41.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.316+0000 7fd575d50640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd5701f2700 0x7fd570152ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:41.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.316+0000 7fd575d50640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd570160890 con 0x7fd5700ae470 2024-03-11T18:36:41.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.316+0000 7fd575d50640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd570154d80 con 0x7fd5700ae090 2024-03-11T18:36:41.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.316+0000 7fd575d50640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd570155060 con 0x7fd5701f2700 2024-03-11T18:36:41.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.320+0000 7fd56effd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd5700ae470 0x7fd570154560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:41.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.320+0000 7fd56effd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd5700ae470 0x7fd570154560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50280/0 (socket says 172.21.15.90:50280) 2024-03-11T18:36:41.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.320+0000 7fd56effd640 1 -- 172.21.15.90:0/2306133656 learned_addr learned my addr 172.21.15.90:0/2306133656 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:41.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.320+0000 7fd56f7fe640 1 --2- 172.21.15.90:0/2306133656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5700ae090 0x7fd570157140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:41.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.328+0000 7fd56effd640 1 -- 172.21.15.90:0/2306133656 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd5701f2700 msgr2=0x7fd570152ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:41.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.328+0000 7fd56effd640 1 --2- 172.21.15.90:0/2306133656 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd5701f2700 0x7fd570152ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:41.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.328+0000 7fd56effd640 1 -- 172.21.15.90:0/2306133656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5700ae090 msgr2=0x7fd570157140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:41.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.328+0000 7fd56effd640 1 --2- 172.21.15.90:0/2306133656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5700ae090 0x7fd570157140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:41.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.328+0000 7fd56effd640 1 -- 172.21.15.90:0/2306133656 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd570160b30 con 0x7fd5700ae470 2024-03-11T18:36:41.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.328+0000 7fd56effd640 1 --2- 172.21.15.90:0/2306133656 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd5700ae470 0x7fd570154560 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd56000ca70 tx=0x7fd56000cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:41.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.332+0000 7fd557fff640 1 -- 172.21.15.90:0/2306133656 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd560004cc0 con 0x7fd5700ae470 2024-03-11T18:36:41.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.332+0000 7fd557fff640 1 -- 172.21.15.90:0/2306133656 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd560004e20 con 0x7fd5700ae470 2024-03-11T18:36:41.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.332+0000 7fd557fff640 1 -- 172.21.15.90:0/2306133656 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd560007310 con 0x7fd5700ae470 2024-03-11T18:36:41.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.332+0000 7fd575d50640 1 -- 172.21.15.90:0/2306133656 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd570160e10 con 0x7fd5700ae470 2024-03-11T18:36:41.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.332+0000 7fd575d50640 1 -- 172.21.15.90:0/2306133656 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd570154aa0 con 0x7fd5700ae470 2024-03-11T18:36:41.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.336+0000 7fd575d50640 1 -- 172.21.15.90:0/2306133656 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd570161330 con 0x7fd5700ae470 2024-03-11T18:36:41.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.620+0000 7fd557fff640 1 -- 172.21.15.90:0/2306133656 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fd560007470 con 0x7fd5700ae470 2024-03-11T18:36:41.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.620+0000 7fd557fff640 1 --2- 172.21.15.90:0/2306133656 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fd55c066e90 0x7fd55c069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:41.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.620+0000 7fd56f7fe640 1 --2- 172.21.15.90:0/2306133656 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fd55c066e90 0x7fd55c069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:41.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.620+0000 7fd56f7fe640 1 --2- 172.21.15.90:0/2306133656 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fd55c066e90 0x7fd55c069350 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd5700c47e0 tx=0x7fd5640047e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:41.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.656+0000 7fd557fff640 1 -- 172.21.15.90:0/2306133656 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd560007720 con 0x7fd5700ae470 2024-03-11T18:36:41.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.656+0000 7fd557fff640 1 -- 172.21.15.90:0/2306133656 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd560051330 con 0x7fd5700ae470 2024-03-11T18:36:41.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.808+0000 7fd575d50640 1 -- 172.21.15.90:0/2306133656 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd57015abb0 con 0x7fd5700ae470 2024-03-11T18:36:41.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.896+0000 7fd557fff640 1 -- 172.21.15.90:0/2306133656 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18430 (secure 0 0 0) 0x7fd57015abb0 con 0x7fd5700ae470 2024-03-11T18:36:41.902 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:36:41.902 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":28,"fsid":"76d56b10-6c29-476a-804b-8482ea90f9e2","created":"2024-03-11T18:34:54.521741+0000","modified":"2024-03-11T18:36:05.526044+0000","last_up_change":"2024-03-11T18:35:16.207500+0000","last_in_change":"2024-03-11T18:35:05.213634+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T18:35:16.193842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T18:35:54.001469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T18:35:55.917321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T18:36:02.210943+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"34decff9-7280-4d84-9665-d6af8ea6de84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6819","nonce":1755667921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6821","nonce":1755667921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6825","nonce":1755667921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6823","nonce":1755667921}]},"public_addr":"172.21.15.90:6819/1755667921","cluster_addr":"172.21.15.90:6821/1755667921","heartbeat_back_addr":"172.21.15.90:6825/1755667921","heartbeat_front_addr":"172.21.15.90:6823/1755667921","state":["exists","up"]},{"osd":1,"uuid":"8ecdce9e-5486-415a-a609-dd16e945440e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6807","nonce":438682081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6809","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6811","nonce":438682081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6817","nonce":438682081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6815","nonce":438682081}]},"public_addr":"172.21.15.90:6807/438682081","cluster_addr":"172.21.15.90:6811/438682081","heartbeat_back_addr":"172.21.15.90:6817/438682081","heartbeat_front_addr":"172.21.15.90:6815/438682081","state":["exists","up"]},{"osd":2,"uuid":"737486d4-0e90-4c82-9e79-bf37c812e5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6827","nonce":2837704738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6829","nonce":2837704738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6833","nonce":2837704738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6831","nonce":2837704738}]},"public_addr":"172.21.15.90:6827/2837704738","cluster_addr":"172.21.15.90:6829/2837704738","heartbeat_back_addr":"172.21.15.90:6833/2837704738","heartbeat_front_addr":"172.21.15.90:6831/2837704738","state":["exists","up"]},{"osd":3,"uuid":"698bd661-bbef-4699-a619-f7bfe23ce12c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6803","nonce":240987971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6805","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6806","nonce":240987971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6813","nonce":240987971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6810","nonce":240987971}]},"public_addr":"172.21.15.90:6803/240987971","cluster_addr":"172.21.15.90:6806/240987971","heartbeat_back_addr":"172.21.15.90:6813/240987971","heartbeat_front_addr":"172.21.15.90:6810/240987971","state":["exists","up"]},{"osd":4,"uuid":"aba9e31f-73d9-43fe-a10b-2dc5832968b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6801","nonce":502685275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6803","nonce":502685275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6807","nonce":502685275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6805","nonce":502685275}]},"public_addr":"172.21.15.92:6801/502685275","cluster_addr":"172.21.15.92:6803/502685275","heartbeat_back_addr":"172.21.15.92:6807/502685275","heartbeat_front_addr":"172.21.15.92:6805/502685275","state":["exists","up"]},{"osd":5,"uuid":"bdd56766-4b28-4007-8de3-b5b08cf335d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6809","nonce":200142070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6811","nonce":200142070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6815","nonce":200142070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6813","nonce":200142070}]},"public_addr":"172.21.15.92:6809/200142070","cluster_addr":"172.21.15.92:6811/200142070","heartbeat_back_addr":"172.21.15.92:6815/200142070","heartbeat_front_addr":"172.21.15.92:6813/200142070","state":["exists","up"]},{"osd":6,"uuid":"6c580a59-610f-455e-a96b-033299dcc118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6817","nonce":1919010204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6819","nonce":1919010204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6823","nonce":1919010204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6821","nonce":1919010204}]},"public_addr":"172.21.15.92:6817/1919010204","cluster_addr":"172.21.15.92:6819/1919010204","heartbeat_back_addr":"172.21.15.92:6823/1919010204","heartbeat_front_addr":"172.21.15.92:6821/1919010204","state":["exists","up"]},{"osd":7,"uuid":"958184cc-aebf-4c5b-91a0-2a3e38a6b42b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6825","nonce":3475925719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6827","nonce":3475925719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6831","nonce":3475925719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6829","nonce":3475925719}]},"public_addr":"172.21.15.92:6825/3475925719","cluster_addr":"172.21.15.92:6827/3475925719","heartbeat_back_addr":"172.21.15.92:6831/3475925719","heartbeat_front_addr":"172.21.15.92:6829/3475925719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.816294+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.048084+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.137975+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.968246+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.016829+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.931676+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.041673+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.079404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T18:36:41.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.900+0000 7fd575d50640 1 -- 172.21.15.90:0/2306133656 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fd55c066e90 msgr2=0x7fd55c069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:41.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.900+0000 7fd575d50640 1 --2- 172.21.15.90:0/2306133656 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fd55c066e90 0x7fd55c069350 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd5700c47e0 tx=0x7fd5640047e0 comp rx=0 tx=0).stop 2024-03-11T18:36:41.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.900+0000 7fd575d50640 1 -- 172.21.15.90:0/2306133656 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd5700ae470 msgr2=0x7fd570154560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:41.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.900+0000 7fd575d50640 1 --2- 172.21.15.90:0/2306133656 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd5700ae470 0x7fd570154560 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd56000ca70 tx=0x7fd56000cf30 comp rx=0 tx=0).stop 2024-03-11T18:36:41.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.900+0000 7fd575d50640 1 -- 172.21.15.90:0/2306133656 shutdown_connections 2024-03-11T18:36:41.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.900+0000 7fd575d50640 1 --2- 172.21.15.90:0/2306133656 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fd55c066e90 0x7fd55c069350 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:41.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.900+0000 7fd575d50640 1 --2- 172.21.15.90:0/2306133656 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd5701f2700 0x7fd570152ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:41.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.900+0000 7fd575d50640 1 --2- 172.21.15.90:0/2306133656 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd5700ae470 0x7fd570154560 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:41.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.900+0000 7fd575d50640 1 --2- 172.21.15.90:0/2306133656 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd5700ae090 0x7fd570157140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:41.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.900+0000 7fd575d50640 1 -- 172.21.15.90:0/2306133656 >> 172.21.15.90:0/2306133656 conn(0x7fd570070e80 msgr2=0x7fd5700c2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:41.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.900+0000 7fd575d50640 1 -- 172.21.15.90:0/2306133656 shutdown_connections 2024-03-11T18:36:41.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:41.900+0000 7fd575d50640 1 -- 172.21.15.90:0/2306133656 wait complete. 2024-03-11T18:36:41.923 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-11T18:36:41.923 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-11T18:36:42.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.040+0000 7f018fd1c640 1 Processor -- start 2024-03-11T18:36:42.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.040+0000 7f018fd1c640 1 -- start start 2024-03-11T18:36:42.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.040+0000 7f018fd1c640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f0188070b00 0x7f01880c4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:42.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.040+0000 7f018fd1c640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01880c5de0 con 0x7f01881509f0 2024-03-11T18:36:42.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.040+0000 7f018fd1c640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0188156910 con 0x7f0188150dd0 2024-03-11T18:36:42.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.040+0000 7f018fd1c640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01880c6030 con 0x7f0188070b00 2024-03-11T18:36:42.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.040+0000 7f018e2b9640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f0188070b00 0x7f01880c4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:42.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.040+0000 7f018dab8640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f01881509f0 0x7f0188157390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:39172/0 (socket says 172.21.15.90:39172) 2024-03-11T18:36:42.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.040+0000 7f018dab8640 1 -- 172.21.15.90:0/740371206 learned_addr learned my addr 172.21.15.90:0/740371206 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T18:36:42.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.044+0000 7f018e2b9640 1 -- 172.21.15.90:0/740371206 >> v1:172.21.15.92:6789/0 conn(0x7f0188150dd0 legacy=0x7f01880c28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:42.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.044+0000 7f018e2b9640 1 -- 172.21.15.90:0/740371206 >> v1:172.21.15.90:6789/0 conn(0x7f01881509f0 legacy=0x7f0188157390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:42.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.044+0000 7f018e2b9640 1 -- 172.21.15.90:0/740371206 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01880c6310 con 0x7f0188070b00 2024-03-11T18:36:42.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.048+0000 7f018e2b9640 1 --2- 172.21.15.90:0/740371206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0188070b00 0x7f01880c4580 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f01800097c0 tx=0x7f018001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ad6680c5939cc7a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:42.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.056+0000 7f018cab6640 1 -- 172.21.15.90:0/740371206 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01800a6040 con 0x7f0188070b00 2024-03-11T18:36:42.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.056+0000 7f018cab6640 1 -- 172.21.15.90:0/740371206 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01800aa600 con 0x7f0188070b00 2024-03-11T18:36:42.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.056+0000 7f018cab6640 1 -- 172.21.15.90:0/740371206 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01800aa990 con 0x7f0188070b00 2024-03-11T18:36:42.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.056+0000 7f018fd1c640 1 -- 172.21.15.90:0/740371206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0188070b00 msgr2=0x7f01880c4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:42.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.056+0000 7f018fd1c640 1 --2- 172.21.15.90:0/740371206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0188070b00 0x7f01880c4580 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f01800097c0 tx=0x7f018001c4a0 comp rx=0 tx=0).stop 2024-03-11T18:36:42.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.056+0000 7f018fd1c640 1 -- 172.21.15.90:0/740371206 shutdown_connections 2024-03-11T18:36:42.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.056+0000 7f018fd1c640 1 --2- 172.21.15.90:0/740371206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0188070b00 0x7f01880c4580 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:42.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.056+0000 7f018fd1c640 1 -- 172.21.15.90:0/740371206 >> 172.21.15.90:0/740371206 conn(0x7f018806ac00 msgr2=0x7f018806b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:42.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.056+0000 7f018fd1c640 1 -- 172.21.15.90:0/740371206 shutdown_connections 2024-03-11T18:36:42.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.056+0000 7f018fd1c640 1 -- 172.21.15.90:0/740371206 wait complete. 2024-03-11T18:36:42.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.056+0000 7f018fd1c640 1 Processor -- start 2024-03-11T18:36:42.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.056+0000 7f018fd1c640 1 -- start start 2024-03-11T18:36:42.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.056+0000 7f018fd1c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0188070b00 0x7f0188140de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:42.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.056+0000 7f018fd1c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01881509f0 0x7f018814e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:42.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.056+0000 7f018fd1c640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0188150dd0 0x7f018814e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:42.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.056+0000 7f018fd1c640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0188139bd0 con 0x7f01881509f0 2024-03-11T18:36:42.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.056+0000 7f018fd1c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f018813a430 con 0x7f0188070b00 2024-03-11T18:36:42.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.056+0000 7f018fd1c640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0188138e30 con 0x7f0188150dd0 2024-03-11T18:36:42.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.060+0000 7f018d2b7640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01881509f0 0x7f018814e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:42.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.060+0000 7f018d2b7640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01881509f0 0x7f018814e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50298/0 (socket says 172.21.15.90:50298) 2024-03-11T18:36:42.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.060+0000 7f018d2b7640 1 -- 172.21.15.90:0/1783884884 learned_addr learned my addr 172.21.15.90:0/1783884884 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:42.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.060+0000 7f018dab8640 1 --2- 172.21.15.90:0/1783884884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0188070b00 0x7f0188140de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:42.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.060+0000 7f018e2b9640 1 --2- 172.21.15.90:0/1783884884 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0188150dd0 0x7f018814e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:42.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.060+0000 7f018d2b7640 1 -- 172.21.15.90:0/1783884884 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0188150dd0 msgr2=0x7f018814e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:42.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.060+0000 7f018d2b7640 1 --2- 172.21.15.90:0/1783884884 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0188150dd0 0x7f018814e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:42.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.060+0000 7f018d2b7640 1 -- 172.21.15.90:0/1783884884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0188070b00 msgr2=0x7f0188140de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:42.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.060+0000 7f018d2b7640 1 --2- 172.21.15.90:0/1783884884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0188070b00 0x7f0188140de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:42.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.060+0000 7f018d2b7640 1 -- 172.21.15.90:0/1783884884 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0180009420 con 0x7f01881509f0 2024-03-11T18:36:42.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.064+0000 7f018d2b7640 1 --2- 172.21.15.90:0/1783884884 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01881509f0 0x7f018814e080 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f017800c370 tx=0x7f017800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:42.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.068+0000 7f017e7fc640 1 -- 172.21.15.90:0/1783884884 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01780052d0 con 0x7f01881509f0 2024-03-11T18:36:42.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.068+0000 7f017e7fc640 1 -- 172.21.15.90:0/1783884884 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0178005430 con 0x7f01881509f0 2024-03-11T18:36:42.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.068+0000 7f017e7fc640 1 -- 172.21.15.90:0/1783884884 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0178007260 con 0x7f01881509f0 2024-03-11T18:36:42.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.068+0000 7f018fd1c640 1 -- 172.21.15.90:0/1783884884 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0188139de0 con 0x7f01881509f0 2024-03-11T18:36:42.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.068+0000 7f018fd1c640 1 -- 172.21.15.90:0/1783884884 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f018813aa00 con 0x7f01881509f0 2024-03-11T18:36:42.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.068+0000 7f018fd1c640 1 -- 172.21.15.90:0/1783884884 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f018813a290 con 0x7f01881509f0 2024-03-11T18:36:42.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.088+0000 7f017e7fc640 1 -- 172.21.15.90:0/1783884884 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f01780073c0 con 0x7f01881509f0 2024-03-11T18:36:42.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.092+0000 7f017e7fc640 1 --2- 172.21.15.90:0/1783884884 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f016c066e90 0x7f016c069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:42.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.092+0000 7f018dab8640 1 --2- 172.21.15.90:0/1783884884 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f016c066e90 0x7f016c069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:42.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.092+0000 7f018dab8640 1 --2- 172.21.15.90:0/1783884884 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f016c066e90 0x7f016c069350 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0184005c10 tx=0x7f0184009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:42.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.092+0000 7f017e7fc640 1 -- 172.21.15.90:0/1783884884 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0178007670 con 0x7f01881509f0 2024-03-11T18:36:42.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.364+0000 7f017e7fc640 1 -- 172.21.15.90:0/1783884884 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0178050ef0 con 0x7f01881509f0 2024-03-11T18:36:42.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.532+0000 7f018fd1c640 1 -- 172.21.15.90:0/1783884884 --> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f018813a290 con 0x7f016c066e90 2024-03-11T18:36:42.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.544+0000 7f017e7fc640 1 -- 172.21.15.90:0/1783884884 <== mgr.4101 v2:172.21.15.90:6800/1999720161 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332612 (secure 0 0 0) 0x7f018813a290 con 0x7f016c066e90 2024-03-11T18:36:42.556 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:36:42.558 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-03-11T18:36:42.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.552+0000 7f018fd1c640 1 -- 172.21.15.90:0/1783884884 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f016c066e90 msgr2=0x7f016c069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:42.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.552+0000 7f018fd1c640 1 --2- 172.21.15.90:0/1783884884 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f016c066e90 0x7f016c069350 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0184005c10 tx=0x7f0184009040 comp rx=0 tx=0).stop 2024-03-11T18:36:42.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.552+0000 7f018fd1c640 1 -- 172.21.15.90:0/1783884884 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01881509f0 msgr2=0x7f018814e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:42.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.552+0000 7f018fd1c640 1 --2- 172.21.15.90:0/1783884884 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01881509f0 0x7f018814e080 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f017800c370 tx=0x7f017800c830 comp rx=0 tx=0).stop 2024-03-11T18:36:42.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.552+0000 7f018fd1c640 1 -- 172.21.15.90:0/1783884884 shutdown_connections 2024-03-11T18:36:42.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.552+0000 7f018fd1c640 1 --2- 172.21.15.90:0/1783884884 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f016c066e90 0x7f016c069350 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:42.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.552+0000 7f018fd1c640 1 --2- 172.21.15.90:0/1783884884 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0188150dd0 0x7f018814e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:42.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.552+0000 7f018fd1c640 1 --2- 172.21.15.90:0/1783884884 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01881509f0 0x7f018814e080 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:42.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.552+0000 7f018fd1c640 1 --2- 172.21.15.90:0/1783884884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0188070b00 0x7f0188140de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:42.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.552+0000 7f018fd1c640 1 -- 172.21.15.90:0/1783884884 >> 172.21.15.90:0/1783884884 conn(0x7f018806ac00 msgr2=0x7f0188073bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:42.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.552+0000 7f018fd1c640 1 -- 172.21.15.90:0/1783884884 shutdown_connections 2024-03-11T18:36:42.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.552+0000 7f018fd1c640 1 -- 172.21.15.90:0/1783884884 wait complete. 2024-03-11T18:36:42.574 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2024-03-11T18:36:41.806780+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596302,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":221,"num_write_kb":2439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":173,"ondisk_log_size":173,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219664,"kb_used_data":3516,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755056,"statfs":{"total":773094113280,"available":772869177344,"internally_reserved":0,"allocated":3600384,"data_stored":2123708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":221236788},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1294,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005016"},"pg_stats":[{"pgid":"2.1f","version":"23'4","reported_seq":31,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.406537+0000","last_change":"2024-03-11T18:35:56.290140+0000","last_active":"2024-03-11T18:36:06.406537+0000","last_peered":"2024-03-11T18:36:06.406537+0000","last_clean":"2024-03-11T18:36:06.406537+0000","last_became_active":"2024-03-11T18:35:56.289178+0000","last_became_peered":"2024-03-11T18:35:56.289178+0000","last_unstale":"2024-03-11T18:36:06.406537+0000","last_undegraded":"2024-03-11T18:36:06.406537+0000","last_fullsized":"2024-03-11T18:36:06.406537+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:59:46.211033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.405302+0000","last_change":"2024-03-11T18:35:57.676047+0000","last_active":"2024-03-11T18:36:06.405302+0000","last_peered":"2024-03-11T18:36:06.405302+0000","last_clean":"2024-03-11T18:36:06.405302+0000","last_became_active":"2024-03-11T18:35:57.675105+0000","last_became_peered":"2024-03-11T18:35:57.675105+0000","last_unstale":"2024-03-11T18:36:06.405302+0000","last_undegraded":"2024-03-11T18:36:06.405302+0000","last_fullsized":"2024-03-11T18:36:06.405302+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:45:39.057074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.401561+0000","last_change":"2024-03-11T18:36:04.607351+0000","last_active":"2024-03-11T18:36:06.401561+0000","last_peered":"2024-03-11T18:36:06.401561+0000","last_clean":"2024-03-11T18:36:06.401561+0000","last_became_active":"2024-03-11T18:36:04.599562+0000","last_became_peered":"2024-03-11T18:36:04.599562+0000","last_unstale":"2024-03-11T18:36:06.401561+0000","last_undegraded":"2024-03-11T18:36:06.401561+0000","last_fullsized":"2024-03-11T18:36:06.401561+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T18:57:51.134410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"23'1","reported_seq":28,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.405632+0000","last_change":"2024-03-11T18:35:56.291253+0000","last_active":"2024-03-11T18:36:06.405632+0000","last_peered":"2024-03-11T18:36:06.405632+0000","last_clean":"2024-03-11T18:36:06.405632+0000","last_became_active":"2024-03-11T18:35:56.290862+0000","last_became_peered":"2024-03-11T18:35:56.290862+0000","last_unstale":"2024-03-11T18:36:06.405632+0000","last_undegraded":"2024-03-11T18:36:06.405632+0000","last_fullsized":"2024-03-11T18:36:06.405632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:36:17.809744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.405636+0000","last_change":"2024-03-11T18:35:57.636861+0000","last_active":"2024-03-11T18:36:06.405636+0000","last_peered":"2024-03-11T18:36:06.405636+0000","last_clean":"2024-03-11T18:36:06.405636+0000","last_became_active":"2024-03-11T18:35:57.636111+0000","last_became_peered":"2024-03-11T18:35:57.636111+0000","last_unstale":"2024-03-11T18:36:06.405636+0000","last_undegraded":"2024-03-11T18:36:06.405636+0000","last_fullsized":"2024-03-11T18:36:06.405636+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:23:11.238170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.408420+0000","last_change":"2024-03-11T18:36:04.643426+0000","last_active":"2024-03-11T18:36:06.408420+0000","last_peered":"2024-03-11T18:36:06.408420+0000","last_clean":"2024-03-11T18:36:06.408420+0000","last_became_active":"2024-03-11T18:36:04.642597+0000","last_became_peered":"2024-03-11T18:36:04.642597+0000","last_unstale":"2024-03-11T18:36:06.408420+0000","last_undegraded":"2024-03-11T18:36:06.408420+0000","last_fullsized":"2024-03-11T18:36:06.408420+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:31:27.049273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"23'2","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.355107+0000","last_change":"2024-03-11T18:35:56.355147+0000","last_active":"2024-03-11T18:36:06.355107+0000","last_peered":"2024-03-11T18:36:06.355107+0000","last_clean":"2024-03-11T18:36:06.355107+0000","last_became_active":"2024-03-11T18:35:56.354388+0000","last_became_peered":"2024-03-11T18:35:56.354388+0000","last_unstale":"2024-03-11T18:36:06.355107+0000","last_undegraded":"2024-03-11T18:36:06.355107+0000","last_fullsized":"2024-03-11T18:36:06.355107+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:03:55.046045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.410658+0000","last_change":"2024-03-11T18:35:57.641486+0000","last_active":"2024-03-11T18:36:06.410658+0000","last_peered":"2024-03-11T18:36:06.410658+0000","last_clean":"2024-03-11T18:36:06.410658+0000","last_became_active":"2024-03-11T18:35:57.641052+0000","last_became_peered":"2024-03-11T18:35:57.641052+0000","last_unstale":"2024-03-11T18:36:06.410658+0000","last_undegraded":"2024-03-11T18:36:06.410658+0000","last_fullsized":"2024-03-11T18:36:06.410658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:29:45.662029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.411193+0000","last_change":"2024-03-11T18:36:04.629291+0000","last_active":"2024-03-11T18:36:06.411193+0000","last_peered":"2024-03-11T18:36:06.411193+0000","last_clean":"2024-03-11T18:36:06.411193+0000","last_became_active":"2024-03-11T18:36:04.624110+0000","last_became_peered":"2024-03-11T18:36:04.624110+0000","last_unstale":"2024-03-11T18:36:06.411193+0000","last_undegraded":"2024-03-11T18:36:06.411193+0000","last_fullsized":"2024-03-11T18:36:06.411193+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:04:03.368322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"23'2","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.353273+0000","last_change":"2024-03-11T18:35:56.298903+0000","last_active":"2024-03-11T18:36:06.353273+0000","last_peered":"2024-03-11T18:36:06.353273+0000","last_clean":"2024-03-11T18:36:06.353273+0000","last_became_active":"2024-03-11T18:35:56.297582+0000","last_became_peered":"2024-03-11T18:35:56.297582+0000","last_unstale":"2024-03-11T18:36:06.353273+0000","last_undegraded":"2024-03-11T18:36:06.353273+0000","last_fullsized":"2024-03-11T18:36:06.353273+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:53:25.278916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.413016+0000","last_change":"2024-03-11T18:35:57.640122+0000","last_active":"2024-03-11T18:36:06.413016+0000","last_peered":"2024-03-11T18:36:06.413016+0000","last_clean":"2024-03-11T18:36:06.413016+0000","last_became_active":"2024-03-11T18:35:57.639770+0000","last_became_peered":"2024-03-11T18:35:57.639770+0000","last_unstale":"2024-03-11T18:36:06.413016+0000","last_undegraded":"2024-03-11T18:36:06.413016+0000","last_fullsized":"2024-03-11T18:36:06.413016+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:39:25.767184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.410463+0000","last_change":"2024-03-11T18:36:04.627149+0000","last_active":"2024-03-11T18:36:06.410463+0000","last_peered":"2024-03-11T18:36:06.410463+0000","last_clean":"2024-03-11T18:36:06.410463+0000","last_became_active":"2024-03-11T18:36:04.626108+0000","last_became_peered":"2024-03-11T18:36:04.626108+0000","last_unstale":"2024-03-11T18:36:06.410463+0000","last_undegraded":"2024-03-11T18:36:06.410463+0000","last_fullsized":"2024-03-11T18:36:06.410463+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:03:40.619618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.410013+0000","last_change":"2024-03-11T18:35:57.616739+0000","last_active":"2024-03-11T18:36:06.410013+0000","last_peered":"2024-03-11T18:36:06.410013+0000","last_clean":"2024-03-11T18:36:06.410013+0000","last_became_active":"2024-03-11T18:35:57.616398+0000","last_became_peered":"2024-03-11T18:35:57.616398+0000","last_unstale":"2024-03-11T18:36:06.410013+0000","last_undegraded":"2024-03-11T18:36:06.410013+0000","last_fullsized":"2024-03-11T18:36:06.410013+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:53:30.324898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"23'2","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.353981+0000","last_change":"2024-03-11T18:35:56.304762+0000","last_active":"2024-03-11T18:36:06.353981+0000","last_peered":"2024-03-11T18:36:06.353981+0000","last_clean":"2024-03-11T18:36:06.353981+0000","last_became_active":"2024-03-11T18:35:56.302803+0000","last_became_peered":"2024-03-11T18:35:56.302803+0000","last_unstale":"2024-03-11T18:36:06.353981+0000","last_undegraded":"2024-03-11T18:36:06.353981+0000","last_fullsized":"2024-03-11T18:36:06.353981+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:31:18.726996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.412227+0000","last_change":"2024-03-11T18:36:04.626968+0000","last_active":"2024-03-11T18:36:06.412227+0000","last_peered":"2024-03-11T18:36:06.412227+0000","last_clean":"2024-03-11T18:36:06.412227+0000","last_became_active":"2024-03-11T18:36:04.626337+0000","last_became_peered":"2024-03-11T18:36:04.626337+0000","last_unstale":"2024-03-11T18:36:06.412227+0000","last_undegraded":"2024-03-11T18:36:06.412227+0000","last_fullsized":"2024-03-11T18:36:06.412227+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:53:37.432826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.402625+0000","last_change":"2024-03-11T18:35:57.689835+0000","last_active":"2024-03-11T18:36:06.402625+0000","last_peered":"2024-03-11T18:36:06.402625+0000","last_clean":"2024-03-11T18:36:06.402625+0000","last_became_active":"2024-03-11T18:35:57.689039+0000","last_became_peered":"2024-03-11T18:35:57.689039+0000","last_unstale":"2024-03-11T18:36:06.402625+0000","last_undegraded":"2024-03-11T18:36:06.402625+0000","last_fullsized":"2024-03-11T18:36:06.402625+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:38:27.733669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.358819+0000","last_change":"2024-03-11T18:35:56.291070+0000","last_active":"2024-03-11T18:36:06.358819+0000","last_peered":"2024-03-11T18:36:06.358819+0000","last_clean":"2024-03-11T18:36:06.358819+0000","last_became_active":"2024-03-11T18:35:56.290504+0000","last_became_peered":"2024-03-11T18:35:56.290504+0000","last_unstale":"2024-03-11T18:36:06.358819+0000","last_undegraded":"2024-03-11T18:36:06.358819+0000","last_fullsized":"2024-03-11T18:36:06.358819+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:29:31.660243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.411988+0000","last_change":"2024-03-11T18:36:04.626150+0000","last_active":"2024-03-11T18:36:06.411988+0000","last_peered":"2024-03-11T18:36:06.411988+0000","last_clean":"2024-03-11T18:36:06.411988+0000","last_became_active":"2024-03-11T18:36:04.625084+0000","last_became_peered":"2024-03-11T18:36:04.625084+0000","last_unstale":"2024-03-11T18:36:06.411988+0000","last_undegraded":"2024-03-11T18:36:06.411988+0000","last_fullsized":"2024-03-11T18:36:06.411988+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:28:58.373219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.400936+0000","last_change":"2024-03-11T18:35:57.684535+0000","last_active":"2024-03-11T18:36:06.400936+0000","last_peered":"2024-03-11T18:36:06.400936+0000","last_clean":"2024-03-11T18:36:06.400936+0000","last_became_active":"2024-03-11T18:35:57.681098+0000","last_became_peered":"2024-03-11T18:35:57.681098+0000","last_unstale":"2024-03-11T18:36:06.400936+0000","last_undegraded":"2024-03-11T18:36:06.400936+0000","last_fullsized":"2024-03-11T18:36:06.400936+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:25:49.707568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"28'3","reported_seq":30,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:37.719417+0000","last_change":"2024-03-11T18:35:56.290745+0000","last_active":"2024-03-11T18:36:37.719417+0000","last_peered":"2024-03-11T18:36:37.719417+0000","last_clean":"2024-03-11T18:36:37.719417+0000","last_became_active":"2024-03-11T18:35:56.290006+0000","last_became_peered":"2024-03-11T18:35:56.290006+0000","last_unstale":"2024-03-11T18:36:37.719417+0000","last_undegraded":"2024-03-11T18:36:37.719417+0000","last_fullsized":"2024-03-11T18:36:37.719417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:06:02.844765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.359859+0000","last_change":"2024-03-11T18:36:04.620689+0000","last_active":"2024-03-11T18:36:06.359859+0000","last_peered":"2024-03-11T18:36:06.359859+0000","last_clean":"2024-03-11T18:36:06.359859+0000","last_became_active":"2024-03-11T18:36:04.620282+0000","last_became_peered":"2024-03-11T18:36:04.620282+0000","last_unstale":"2024-03-11T18:36:06.359859+0000","last_undegraded":"2024-03-11T18:36:06.359859+0000","last_fullsized":"2024-03-11T18:36:06.359859+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:54:16.807299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.413230+0000","last_change":"2024-03-11T18:35:57.615234+0000","last_active":"2024-03-11T18:36:06.413230+0000","last_peered":"2024-03-11T18:36:06.413230+0000","last_clean":"2024-03-11T18:36:06.413230+0000","last_became_active":"2024-03-11T18:35:57.614885+0000","last_became_peered":"2024-03-11T18:35:57.614885+0000","last_unstale":"2024-03-11T18:36:06.413230+0000","last_undegraded":"2024-03-11T18:36:06.413230+0000","last_fullsized":"2024-03-11T18:36:06.413230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:22:13.730630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.352619+0000","last_change":"2024-03-11T18:35:56.304617+0000","last_active":"2024-03-11T18:36:06.352619+0000","last_peered":"2024-03-11T18:36:06.352619+0000","last_clean":"2024-03-11T18:36:06.352619+0000","last_became_active":"2024-03-11T18:35:56.301698+0000","last_became_peered":"2024-03-11T18:35:56.301698+0000","last_unstale":"2024-03-11T18:36:06.352619+0000","last_undegraded":"2024-03-11T18:36:06.352619+0000","last_fullsized":"2024-03-11T18:36:06.352619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:29:31.660243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.412949+0000","last_change":"2024-03-11T18:36:04.643279+0000","last_active":"2024-03-11T18:36:06.412949+0000","last_peered":"2024-03-11T18:36:06.412949+0000","last_clean":"2024-03-11T18:36:06.412949+0000","last_became_active":"2024-03-11T18:36:04.641983+0000","last_became_peered":"2024-03-11T18:36:04.641983+0000","last_unstale":"2024-03-11T18:36:06.412949+0000","last_undegraded":"2024-03-11T18:36:06.412949+0000","last_fullsized":"2024-03-11T18:36:06.412949+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:13:47.499965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.406548+0000","last_change":"2024-03-11T18:35:57.506963+0000","last_active":"2024-03-11T18:36:06.406548+0000","last_peered":"2024-03-11T18:36:06.406548+0000","last_clean":"2024-03-11T18:36:06.406548+0000","last_became_active":"2024-03-11T18:35:57.506038+0000","last_became_peered":"2024-03-11T18:35:57.506038+0000","last_unstale":"2024-03-11T18:36:06.406548+0000","last_undegraded":"2024-03-11T18:36:06.406548+0000","last_fullsized":"2024-03-11T18:36:06.406548+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:52:06.024245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"23'2","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.411969+0000","last_change":"2024-03-11T18:35:56.364512+0000","last_active":"2024-03-11T18:36:06.411969+0000","last_peered":"2024-03-11T18:36:06.411969+0000","last_clean":"2024-03-11T18:36:06.411969+0000","last_became_active":"2024-03-11T18:35:56.363731+0000","last_became_peered":"2024-03-11T18:35:56.363731+0000","last_unstale":"2024-03-11T18:36:06.411969+0000","last_undegraded":"2024-03-11T18:36:06.411969+0000","last_fullsized":"2024-03-11T18:36:06.411969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:36:17.809744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.408707+0000","last_change":"2024-03-11T18:36:04.643782+0000","last_active":"2024-03-11T18:36:06.408707+0000","last_peered":"2024-03-11T18:36:06.408707+0000","last_clean":"2024-03-11T18:36:06.408707+0000","last_became_active":"2024-03-11T18:36:04.642133+0000","last_became_peered":"2024-03-11T18:36:04.642133+0000","last_unstale":"2024-03-11T18:36:06.408707+0000","last_undegraded":"2024-03-11T18:36:06.408707+0000","last_fullsized":"2024-03-11T18:36:06.408707+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:53:33.601193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.405392+0000","last_change":"2024-03-11T18:35:57.675207+0000","last_active":"2024-03-11T18:36:06.405392+0000","last_peered":"2024-03-11T18:36:06.405392+0000","last_clean":"2024-03-11T18:36:06.405392+0000","last_became_active":"2024-03-11T18:35:57.674472+0000","last_became_peered":"2024-03-11T18:35:57.674472+0000","last_unstale":"2024-03-11T18:36:06.405392+0000","last_undegraded":"2024-03-11T18:36:06.405392+0000","last_fullsized":"2024-03-11T18:36:06.405392+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:29:45.662029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"24'1","reported_seq":28,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.359192+0000","last_change":"2024-03-11T18:35:56.348479+0000","last_active":"2024-03-11T18:36:06.359192+0000","last_peered":"2024-03-11T18:36:06.359192+0000","last_clean":"2024-03-11T18:36:06.359192+0000","last_became_active":"2024-03-11T18:35:56.348040+0000","last_became_peered":"2024-03-11T18:35:56.348040+0000","last_unstale":"2024-03-11T18:36:06.359192+0000","last_undegraded":"2024-03-11T18:36:06.359192+0000","last_fullsized":"2024-03-11T18:36:06.359192+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:27:26.471550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.401155+0000","last_change":"2024-03-11T18:36:04.599805+0000","last_active":"2024-03-11T18:36:06.401155+0000","last_peered":"2024-03-11T18:36:06.401155+0000","last_clean":"2024-03-11T18:36:06.401155+0000","last_became_active":"2024-03-11T18:36:04.599065+0000","last_became_peered":"2024-03-11T18:36:04.599065+0000","last_unstale":"2024-03-11T18:36:06.401155+0000","last_undegraded":"2024-03-11T18:36:06.401155+0000","last_fullsized":"2024-03-11T18:36:06.401155+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:36:07.187003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.359096+0000","last_change":"2024-03-11T18:35:57.538551+0000","last_active":"2024-03-11T18:36:06.359096+0000","last_peered":"2024-03-11T18:36:06.359096+0000","last_clean":"2024-03-11T18:36:06.359096+0000","last_became_active":"2024-03-11T18:35:57.537706+0000","last_became_peered":"2024-03-11T18:35:57.537706+0000","last_unstale":"2024-03-11T18:36:06.359096+0000","last_undegraded":"2024-03-11T18:36:06.359096+0000","last_fullsized":"2024-03-11T18:36:06.359096+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:03:56.260394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.355377+0000","last_change":"2024-03-11T18:35:56.302575+0000","last_active":"2024-03-11T18:36:06.355377+0000","last_peered":"2024-03-11T18:36:06.355377+0000","last_clean":"2024-03-11T18:36:06.355377+0000","last_became_active":"2024-03-11T18:35:56.301819+0000","last_became_peered":"2024-03-11T18:35:56.301819+0000","last_unstale":"2024-03-11T18:36:06.355377+0000","last_undegraded":"2024-03-11T18:36:06.355377+0000","last_fullsized":"2024-03-11T18:36:06.355377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:38:26.519320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.404840+0000","last_change":"2024-03-11T18:36:04.609282+0000","last_active":"2024-03-11T18:36:06.404840+0000","last_peered":"2024-03-11T18:36:06.404840+0000","last_clean":"2024-03-11T18:36:06.404840+0000","last_became_active":"2024-03-11T18:36:04.608260+0000","last_became_peered":"2024-03-11T18:36:04.608260+0000","last_unstale":"2024-03-11T18:36:06.404840+0000","last_undegraded":"2024-03-11T18:36:06.404840+0000","last_fullsized":"2024-03-11T18:36:06.404840+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:49:28.143755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.413195+0000","last_change":"2024-03-11T18:35:57.688727+0000","last_active":"2024-03-11T18:36:06.413195+0000","last_peered":"2024-03-11T18:36:06.413195+0000","last_clean":"2024-03-11T18:36:06.413195+0000","last_became_active":"2024-03-11T18:35:57.688209+0000","last_became_peered":"2024-03-11T18:35:57.688209+0000","last_unstale":"2024-03-11T18:36:06.413195+0000","last_undegraded":"2024-03-11T18:36:06.413195+0000","last_fullsized":"2024-03-11T18:36:06.413195+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:58:13.389498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.359326+0000","last_change":"2024-03-11T18:35:56.293842+0000","last_active":"2024-03-11T18:36:06.359326+0000","last_peered":"2024-03-11T18:36:06.359326+0000","last_clean":"2024-03-11T18:36:06.359326+0000","last_became_active":"2024-03-11T18:35:56.293276+0000","last_became_peered":"2024-03-11T18:35:56.293276+0000","last_unstale":"2024-03-11T18:36:06.359326+0000","last_undegraded":"2024-03-11T18:36:06.359326+0000","last_fullsized":"2024-03-11T18:36:06.359326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:04:13.101160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.358354+0000","last_change":"2024-03-11T18:36:04.626320+0000","last_active":"2024-03-11T18:36:06.358354+0000","last_peered":"2024-03-11T18:36:06.358354+0000","last_clean":"2024-03-11T18:36:06.358354+0000","last_became_active":"2024-03-11T18:36:04.625663+0000","last_became_peered":"2024-03-11T18:36:04.625663+0000","last_unstale":"2024-03-11T18:36:06.358354+0000","last_undegraded":"2024-03-11T18:36:06.358354+0000","last_fullsized":"2024-03-11T18:36:06.358354+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:39:32.875112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.411275+0000","last_change":"2024-03-11T18:35:57.612242+0000","last_active":"2024-03-11T18:36:06.411275+0000","last_peered":"2024-03-11T18:36:06.411275+0000","last_clean":"2024-03-11T18:36:06.411275+0000","last_became_active":"2024-03-11T18:35:57.611509+0000","last_became_peered":"2024-03-11T18:35:57.611509+0000","last_unstale":"2024-03-11T18:36:06.411275+0000","last_undegraded":"2024-03-11T18:36:06.411275+0000","last_fullsized":"2024-03-11T18:36:06.411275+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:45:28.543623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.410118+0000","last_change":"2024-03-11T18:35:56.298766+0000","last_active":"2024-03-11T18:36:06.410118+0000","last_peered":"2024-03-11T18:36:06.410118+0000","last_clean":"2024-03-11T18:36:06.410118+0000","last_became_active":"2024-03-11T18:35:56.297747+0000","last_became_peered":"2024-03-11T18:35:56.297747+0000","last_unstale":"2024-03-11T18:36:06.410118+0000","last_undegraded":"2024-03-11T18:36:06.410118+0000","last_fullsized":"2024-03-11T18:36:06.410118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:13:39.177689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.358743+0000","last_change":"2024-03-11T18:36:04.635773+0000","last_active":"2024-03-11T18:36:06.358743+0000","last_peered":"2024-03-11T18:36:06.358743+0000","last_clean":"2024-03-11T18:36:06.358743+0000","last_became_active":"2024-03-11T18:36:04.635261+0000","last_became_peered":"2024-03-11T18:36:04.635261+0000","last_unstale":"2024-03-11T18:36:06.358743+0000","last_undegraded":"2024-03-11T18:36:06.358743+0000","last_fullsized":"2024-03-11T18:36:06.358743+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:44:18.985469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.358192+0000","last_change":"2024-03-11T18:35:57.538552+0000","last_active":"2024-03-11T18:36:06.358192+0000","last_peered":"2024-03-11T18:36:06.358192+0000","last_clean":"2024-03-11T18:36:06.358192+0000","last_became_active":"2024-03-11T18:35:57.537717+0000","last_became_peered":"2024-03-11T18:35:57.537717+0000","last_unstale":"2024-03-11T18:36:06.358192+0000","last_undegraded":"2024-03-11T18:36:06.358192+0000","last_fullsized":"2024-03-11T18:36:06.358192+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:53:26.493265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"28'1","reported_seq":28,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.401191+0000","last_change":"2024-03-11T18:35:56.294572+0000","last_active":"2024-03-11T18:36:06.401191+0000","last_peered":"2024-03-11T18:36:06.401191+0000","last_clean":"2024-03-11T18:36:06.401191+0000","last_became_active":"2024-03-11T18:35:56.294096+0000","last_became_peered":"2024-03-11T18:35:56.294096+0000","last_unstale":"2024-03-11T18:36:06.401191+0000","last_undegraded":"2024-03-11T18:36:06.401191+0000","last_fullsized":"2024-03-11T18:36:06.401191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:22:12.516281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.359857+0000","last_change":"2024-03-11T18:36:04.620762+0000","last_active":"2024-03-11T18:36:06.359857+0000","last_peered":"2024-03-11T18:36:06.359857+0000","last_clean":"2024-03-11T18:36:06.359857+0000","last_became_active":"2024-03-11T18:36:04.620356+0000","last_became_peered":"2024-03-11T18:36:04.620356+0000","last_unstale":"2024-03-11T18:36:06.359857+0000","last_undegraded":"2024-03-11T18:36:06.359857+0000","last_fullsized":"2024-03-11T18:36:06.359857+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:58:20.497426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.404612+0000","last_change":"2024-03-11T18:35:57.602773+0000","last_active":"2024-03-11T18:36:06.404612+0000","last_peered":"2024-03-11T18:36:06.404612+0000","last_clean":"2024-03-11T18:36:06.404612+0000","last_became_active":"2024-03-11T18:35:57.602060+0000","last_became_peered":"2024-03-11T18:35:57.602060+0000","last_unstale":"2024-03-11T18:36:06.404612+0000","last_undegraded":"2024-03-11T18:36:06.404612+0000","last_fullsized":"2024-03-11T18:36:06.404612+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:36:22.201707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.352640+0000","last_change":"2024-03-11T18:35:56.305283+0000","last_active":"2024-03-11T18:36:06.352640+0000","last_peered":"2024-03-11T18:36:06.352640+0000","last_clean":"2024-03-11T18:36:06.352640+0000","last_became_active":"2024-03-11T18:35:56.303063+0000","last_became_peered":"2024-03-11T18:35:56.303063+0000","last_unstale":"2024-03-11T18:36:06.352640+0000","last_undegraded":"2024-03-11T18:36:06.352640+0000","last_fullsized":"2024-03-11T18:36:06.352640+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T18:51:00.795706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.406282+0000","last_change":"2024-03-11T18:36:04.617247+0000","last_active":"2024-03-11T18:36:06.406282+0000","last_peered":"2024-03-11T18:36:06.406282+0000","last_clean":"2024-03-11T18:36:06.406282+0000","last_became_active":"2024-03-11T18:36:04.610765+0000","last_became_peered":"2024-03-11T18:36:04.610765+0000","last_unstale":"2024-03-11T18:36:06.406282+0000","last_undegraded":"2024-03-11T18:36:06.406282+0000","last_fullsized":"2024-03-11T18:36:06.406282+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:22:19.187685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.401101+0000","last_change":"2024-03-11T18:35:57.684207+0000","last_active":"2024-03-11T18:36:06.401101+0000","last_peered":"2024-03-11T18:36:06.401101+0000","last_clean":"2024-03-11T18:36:06.401101+0000","last_became_active":"2024-03-11T18:35:57.682653+0000","last_became_peered":"2024-03-11T18:35:57.682653+0000","last_unstale":"2024-03-11T18:36:06.401101+0000","last_undegraded":"2024-03-11T18:36:06.401101+0000","last_fullsized":"2024-03-11T18:36:06.401101+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:28:51.265291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.409922+0000","last_change":"2024-03-11T18:35:56.297078+0000","last_active":"2024-03-11T18:36:06.409922+0000","last_peered":"2024-03-11T18:36:06.409922+0000","last_clean":"2024-03-11T18:36:06.409922+0000","last_became_active":"2024-03-11T18:35:56.296705+0000","last_became_peered":"2024-03-11T18:35:56.296705+0000","last_unstale":"2024-03-11T18:36:06.409922+0000","last_undegraded":"2024-03-11T18:36:06.409922+0000","last_fullsized":"2024-03-11T18:36:06.409922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:53:25.278916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":182,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.352930+0000","last_change":"2024-03-11T18:35:18.199528+0000","last_active":"2024-03-11T18:36:06.352930+0000","last_peered":"2024-03-11T18:36:06.352930+0000","last_clean":"2024-03-11T18:36:06.352930+0000","last_became_active":"2024-03-11T18:35:18.197797+0000","last_became_peered":"2024-03-11T18:35:18.197797+0000","last_unstale":"2024-03-11T18:36:06.352930+0000","last_undegraded":"2024-03-11T18:36:06.352930+0000","last_fullsized":"2024-03-11T18:36:06.352930+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:16.207500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:16.207500+0000","last_clean_scrub_stamp":"2024-03-11T18:35:16.207500+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:29:00.357496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.411807+0000","last_change":"2024-03-11T18:36:04.599815+0000","last_active":"2024-03-11T18:36:06.411807+0000","last_peered":"2024-03-11T18:36:06.411807+0000","last_clean":"2024-03-11T18:36:06.411807+0000","last_became_active":"2024-03-11T18:36:04.598777+0000","last_became_peered":"2024-03-11T18:36:04.598777+0000","last_unstale":"2024-03-11T18:36:06.411807+0000","last_undegraded":"2024-03-11T18:36:06.411807+0000","last_fullsized":"2024-03-11T18:36:06.411807+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:39:32.875112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.354816+0000","last_change":"2024-03-11T18:35:57.687766+0000","last_active":"2024-03-11T18:36:06.354816+0000","last_peered":"2024-03-11T18:36:06.354816+0000","last_clean":"2024-03-11T18:36:06.354816+0000","last_became_active":"2024-03-11T18:35:57.687059+0000","last_became_peered":"2024-03-11T18:35:57.687059+0000","last_unstale":"2024-03-11T18:36:06.354816+0000","last_undegraded":"2024-03-11T18:36:06.354816+0000","last_fullsized":"2024-03-11T18:36:06.354816+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:39:25.767184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.402184+0000","last_change":"2024-03-11T18:35:56.294626+0000","last_active":"2024-03-11T18:36:06.402184+0000","last_peered":"2024-03-11T18:36:06.402184+0000","last_clean":"2024-03-11T18:36:06.402184+0000","last_became_active":"2024-03-11T18:35:56.293961+0000","last_became_peered":"2024-03-11T18:35:56.293961+0000","last_unstale":"2024-03-11T18:36:06.402184+0000","last_undegraded":"2024-03-11T18:36:06.402184+0000","last_fullsized":"2024-03-11T18:36:06.402184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:50:38.709532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.409872+0000","last_change":"2024-03-11T18:36:04.619862+0000","last_active":"2024-03-11T18:36:06.409872+0000","last_peered":"2024-03-11T18:36:06.409872+0000","last_clean":"2024-03-11T18:36:06.409872+0000","last_became_active":"2024-03-11T18:36:04.618908+0000","last_became_peered":"2024-03-11T18:36:04.618908+0000","last_unstale":"2024-03-11T18:36:06.409872+0000","last_undegraded":"2024-03-11T18:36:06.409872+0000","last_fullsized":"2024-03-11T18:36:06.409872+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:34:46.433500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.410912+0000","last_change":"2024-03-11T18:35:57.686416+0000","last_active":"2024-03-11T18:36:06.410912+0000","last_peered":"2024-03-11T18:36:06.410912+0000","last_clean":"2024-03-11T18:36:06.410912+0000","last_became_active":"2024-03-11T18:35:57.685828+0000","last_became_peered":"2024-03-11T18:35:57.685828+0000","last_unstale":"2024-03-11T18:36:06.410912+0000","last_undegraded":"2024-03-11T18:36:06.410912+0000","last_fullsized":"2024-03-11T18:36:06.410912+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:41:42.263652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"23'1","reported_seq":28,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.359093+0000","last_change":"2024-03-11T18:35:56.295539+0000","last_active":"2024-03-11T18:36:06.359093+0000","last_peered":"2024-03-11T18:36:06.359093+0000","last_clean":"2024-03-11T18:36:06.359093+0000","last_became_active":"2024-03-11T18:35:56.295123+0000","last_became_peered":"2024-03-11T18:35:56.295123+0000","last_unstale":"2024-03-11T18:36:06.359093+0000","last_undegraded":"2024-03-11T18:36:06.359093+0000","last_fullsized":"2024-03-11T18:36:06.359093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:03:06.585728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.406058+0000","last_change":"2024-03-11T18:36:04.616750+0000","last_active":"2024-03-11T18:36:06.406058+0000","last_peered":"2024-03-11T18:36:06.406058+0000","last_clean":"2024-03-11T18:36:06.406058+0000","last_became_active":"2024-03-11T18:36:04.616338+0000","last_became_peered":"2024-03-11T18:36:04.616338+0000","last_unstale":"2024-03-11T18:36:06.406058+0000","last_undegraded":"2024-03-11T18:36:06.406058+0000","last_fullsized":"2024-03-11T18:36:06.406058+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:14:42.558491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.354272+0000","last_change":"2024-03-11T18:35:57.689888+0000","last_active":"2024-03-11T18:36:06.354272+0000","last_peered":"2024-03-11T18:36:06.354272+0000","last_clean":"2024-03-11T18:36:06.354272+0000","last_became_active":"2024-03-11T18:35:57.689424+0000","last_became_peered":"2024-03-11T18:35:57.689424+0000","last_unstale":"2024-03-11T18:36:06.354272+0000","last_undegraded":"2024-03-11T18:36:06.354272+0000","last_fullsized":"2024-03-11T18:36:06.354272+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:25:16.564261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"23'2","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.402166+0000","last_change":"2024-03-11T18:35:56.293862+0000","last_active":"2024-03-11T18:36:06.402166+0000","last_peered":"2024-03-11T18:36:06.402166+0000","last_clean":"2024-03-11T18:36:06.402166+0000","last_became_active":"2024-03-11T18:35:56.293585+0000","last_became_peered":"2024-03-11T18:35:56.293585+0000","last_unstale":"2024-03-11T18:36:06.402166+0000","last_undegraded":"2024-03-11T18:36:06.402166+0000","last_fullsized":"2024-03-11T18:36:06.402166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:45:27.329274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.408514+0000","last_change":"2024-03-11T18:36:04.627729+0000","last_active":"2024-03-11T18:36:06.408514+0000","last_peered":"2024-03-11T18:36:06.408514+0000","last_clean":"2024-03-11T18:36:06.408514+0000","last_became_active":"2024-03-11T18:36:04.626596+0000","last_became_peered":"2024-03-11T18:36:04.626596+0000","last_unstale":"2024-03-11T18:36:06.408514+0000","last_undegraded":"2024-03-11T18:36:06.408514+0000","last_fullsized":"2024-03-11T18:36:06.408514+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:51:28.017349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.353860+0000","last_change":"2024-03-11T18:35:57.690506+0000","last_active":"2024-03-11T18:36:06.353860+0000","last_peered":"2024-03-11T18:36:06.353860+0000","last_clean":"2024-03-11T18:36:06.353860+0000","last_became_active":"2024-03-11T18:35:57.690223+0000","last_became_peered":"2024-03-11T18:35:57.690223+0000","last_unstale":"2024-03-11T18:36:06.353860+0000","last_undegraded":"2024-03-11T18:36:06.353860+0000","last_fullsized":"2024-03-11T18:36:06.353860+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:58:13.389498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"23'1","reported_seq":28,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.414711+0000","last_change":"2024-03-11T18:35:56.257209+0000","last_active":"2024-03-11T18:36:06.414711+0000","last_peered":"2024-03-11T18:36:06.414711+0000","last_clean":"2024-03-11T18:36:06.414711+0000","last_became_active":"2024-03-11T18:35:56.256454+0000","last_became_peered":"2024-03-11T18:35:56.256454+0000","last_unstale":"2024-03-11T18:36:06.414711+0000","last_undegraded":"2024-03-11T18:36:06.414711+0000","last_fullsized":"2024-03-11T18:36:06.414711+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:32:06.369248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.408104+0000","last_change":"2024-03-11T18:36:04.644184+0000","last_active":"2024-03-11T18:36:06.408104+0000","last_peered":"2024-03-11T18:36:06.408104+0000","last_clean":"2024-03-11T18:36:06.408104+0000","last_became_active":"2024-03-11T18:36:04.642079+0000","last_became_peered":"2024-03-11T18:36:04.642079+0000","last_unstale":"2024-03-11T18:36:06.408104+0000","last_undegraded":"2024-03-11T18:36:06.408104+0000","last_fullsized":"2024-03-11T18:36:06.408104+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:36:26.132020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.353155+0000","last_change":"2024-03-11T18:35:57.687311+0000","last_active":"2024-03-11T18:36:06.353155+0000","last_peered":"2024-03-11T18:36:06.353155+0000","last_clean":"2024-03-11T18:36:06.353155+0000","last_became_active":"2024-03-11T18:35:57.686850+0000","last_became_peered":"2024-03-11T18:35:57.686850+0000","last_unstale":"2024-03-11T18:36:06.353155+0000","last_undegraded":"2024-03-11T18:36:06.353155+0000","last_fullsized":"2024-03-11T18:36:06.353155+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:29:45.662029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"28'10","reported_seq":37,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:37.713866+0000","last_change":"2024-03-11T18:35:56.291232+0000","last_active":"2024-03-11T18:36:37.713866+0000","last_peered":"2024-03-11T18:36:37.713866+0000","last_clean":"2024-03-11T18:36:37.713866+0000","last_became_active":"2024-03-11T18:35:56.290944+0000","last_became_peered":"2024-03-11T18:35:56.290944+0000","last_unstale":"2024-03-11T18:36:37.713866+0000","last_undegraded":"2024-03-11T18:36:37.713866+0000","last_fullsized":"2024-03-11T18:36:37.713866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:49:27.827765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.405992+0000","last_change":"2024-03-11T18:36:04.602142+0000","last_active":"2024-03-11T18:36:06.405992+0000","last_peered":"2024-03-11T18:36:06.405992+0000","last_clean":"2024-03-11T18:36:06.405992+0000","last_became_active":"2024-03-11T18:36:04.601224+0000","last_became_peered":"2024-03-11T18:36:04.601224+0000","last_unstale":"2024-03-11T18:36:06.405992+0000","last_undegraded":"2024-03-11T18:36:06.405992+0000","last_fullsized":"2024-03-11T18:36:06.405992+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:17:42.075355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.409133+0000","last_change":"2024-03-11T18:35:57.611305+0000","last_active":"2024-03-11T18:36:06.409133+0000","last_peered":"2024-03-11T18:36:06.409133+0000","last_clean":"2024-03-11T18:36:06.409133+0000","last_became_active":"2024-03-11T18:35:57.610955+0000","last_became_peered":"2024-03-11T18:35:57.610955+0000","last_unstale":"2024-03-11T18:36:06.409133+0000","last_undegraded":"2024-03-11T18:36:06.409133+0000","last_fullsized":"2024-03-11T18:36:06.409133+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:31:19.941345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.410833+0000","last_change":"2024-03-11T18:35:56.361952+0000","last_active":"2024-03-11T18:36:06.410833+0000","last_peered":"2024-03-11T18:36:06.410833+0000","last_clean":"2024-03-11T18:36:06.410833+0000","last_became_active":"2024-03-11T18:35:56.361141+0000","last_became_peered":"2024-03-11T18:35:56.361141+0000","last_unstale":"2024-03-11T18:36:06.410833+0000","last_undegraded":"2024-03-11T18:36:06.410833+0000","last_fullsized":"2024-03-11T18:36:06.410833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:04:13.101160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.405270+0000","last_change":"2024-03-11T18:36:04.593604+0000","last_active":"2024-03-11T18:36:06.405270+0000","last_peered":"2024-03-11T18:36:06.405270+0000","last_clean":"2024-03-11T18:36:06.405270+0000","last_became_active":"2024-03-11T18:36:04.592304+0000","last_became_peered":"2024-03-11T18:36:04.592304+0000","last_unstale":"2024-03-11T18:36:06.405270+0000","last_undegraded":"2024-03-11T18:36:06.405270+0000","last_fullsized":"2024-03-11T18:36:06.405270+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:31:04.738940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.353525+0000","last_change":"2024-03-11T18:35:57.689404+0000","last_active":"2024-03-11T18:36:06.353525+0000","last_peered":"2024-03-11T18:36:06.353525+0000","last_clean":"2024-03-11T18:36:06.353525+0000","last_became_active":"2024-03-11T18:35:57.689139+0000","last_became_peered":"2024-03-11T18:35:57.689139+0000","last_unstale":"2024-03-11T18:36:06.353525+0000","last_undegraded":"2024-03-11T18:36:06.353525+0000","last_fullsized":"2024-03-11T18:36:06.353525+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:54:09.699371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.409872+0000","last_change":"2024-03-11T18:35:56.297682+0000","last_active":"2024-03-11T18:36:06.409872+0000","last_peered":"2024-03-11T18:36:06.409872+0000","last_clean":"2024-03-11T18:36:06.409872+0000","last_became_active":"2024-03-11T18:35:56.296426+0000","last_became_peered":"2024-03-11T18:35:56.296426+0000","last_unstale":"2024-03-11T18:36:06.409872+0000","last_undegraded":"2024-03-11T18:36:06.409872+0000","last_fullsized":"2024-03-11T18:36:06.409872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:28:50.050942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.403363+0000","last_change":"2024-03-11T18:36:04.590780+0000","last_active":"2024-03-11T18:36:06.403363+0000","last_peered":"2024-03-11T18:36:06.403363+0000","last_clean":"2024-03-11T18:36:06.403363+0000","last_became_active":"2024-03-11T18:36:04.590014+0000","last_became_peered":"2024-03-11T18:36:04.590014+0000","last_unstale":"2024-03-11T18:36:06.403363+0000","last_undegraded":"2024-03-11T18:36:06.403363+0000","last_fullsized":"2024-03-11T18:36:06.403363+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:51:13.205384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.407150+0000","last_change":"2024-03-11T18:35:57.671638+0000","last_active":"2024-03-11T18:36:06.407150+0000","last_peered":"2024-03-11T18:36:06.407150+0000","last_clean":"2024-03-11T18:36:06.407150+0000","last_became_active":"2024-03-11T18:35:57.670893+0000","last_became_peered":"2024-03-11T18:35:57.670893+0000","last_unstale":"2024-03-11T18:36:06.407150+0000","last_undegraded":"2024-03-11T18:36:06.407150+0000","last_fullsized":"2024-03-11T18:36:06.407150+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:31:21.395133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"28'4","reported_seq":31,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:37.719433+0000","last_change":"2024-03-11T18:35:56.251580+0000","last_active":"2024-03-11T18:36:37.719433+0000","last_peered":"2024-03-11T18:36:37.719433+0000","last_clean":"2024-03-11T18:36:37.719433+0000","last_became_active":"2024-03-11T18:35:56.251287+0000","last_became_peered":"2024-03-11T18:35:56.251287+0000","last_unstale":"2024-03-11T18:36:37.719433+0000","last_undegraded":"2024-03-11T18:36:37.719433+0000","last_fullsized":"2024-03-11T18:36:37.719433+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:30:39.907360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.412899+0000","last_change":"2024-03-11T18:36:04.625563+0000","last_active":"2024-03-11T18:36:06.412899+0000","last_peered":"2024-03-11T18:36:06.412899+0000","last_clean":"2024-03-11T18:36:06.412899+0000","last_became_active":"2024-03-11T18:36:04.618911+0000","last_became_peered":"2024-03-11T18:36:04.618911+0000","last_unstale":"2024-03-11T18:36:06.412899+0000","last_undegraded":"2024-03-11T18:36:06.412899+0000","last_fullsized":"2024-03-11T18:36:06.412899+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:31:04.738940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.354510+0000","last_change":"2024-03-11T18:35:57.697641+0000","last_active":"2024-03-11T18:36:06.354510+0000","last_peered":"2024-03-11T18:36:06.354510+0000","last_clean":"2024-03-11T18:36:06.354510+0000","last_became_active":"2024-03-11T18:35:57.697263+0000","last_became_peered":"2024-03-11T18:35:57.697263+0000","last_unstale":"2024-03-11T18:36:06.354510+0000","last_undegraded":"2024-03-11T18:36:06.354510+0000","last_fullsized":"2024-03-11T18:36:06.354510+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:34:39.325572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"28'10","reported_seq":37,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:15.644151+0000","last_change":"2024-03-11T18:35:56.355612+0000","last_active":"2024-03-11T18:36:15.644151+0000","last_peered":"2024-03-11T18:36:15.644151+0000","last_clean":"2024-03-11T18:36:15.644151+0000","last_became_active":"2024-03-11T18:35:56.354827+0000","last_became_peered":"2024-03-11T18:35:56.354827+0000","last_unstale":"2024-03-11T18:36:15.644151+0000","last_undegraded":"2024-03-11T18:36:15.644151+0000","last_fullsized":"2024-03-11T18:36:15.644151+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:06:02.844765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.358459+0000","last_change":"2024-03-11T18:36:04.617561+0000","last_active":"2024-03-11T18:36:06.358459+0000","last_peered":"2024-03-11T18:36:06.358459+0000","last_clean":"2024-03-11T18:36:06.358459+0000","last_became_active":"2024-03-11T18:36:04.616938+0000","last_became_peered":"2024-03-11T18:36:04.616938+0000","last_unstale":"2024-03-11T18:36:06.358459+0000","last_undegraded":"2024-03-11T18:36:06.358459+0000","last_fullsized":"2024-03-11T18:36:06.358459+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:16:10.508913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.412500+0000","last_change":"2024-03-11T18:35:57.679035+0000","last_active":"2024-03-11T18:36:06.412500+0000","last_peered":"2024-03-11T18:36:06.412500+0000","last_clean":"2024-03-11T18:36:06.412500+0000","last_became_active":"2024-03-11T18:35:57.678565+0000","last_became_peered":"2024-03-11T18:35:57.678565+0000","last_unstale":"2024-03-11T18:36:06.412500+0000","last_undegraded":"2024-03-11T18:36:06.412500+0000","last_fullsized":"2024-03-11T18:36:06.412500+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:18:05.980207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"28'4","reported_seq":31,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.358900+0000","last_change":"2024-03-11T18:35:56.288132+0000","last_active":"2024-03-11T18:36:06.358900+0000","last_peered":"2024-03-11T18:36:06.358900+0000","last_clean":"2024-03-11T18:36:06.358900+0000","last_became_active":"2024-03-11T18:35:56.287846+0000","last_became_peered":"2024-03-11T18:35:56.287846+0000","last_unstale":"2024-03-11T18:36:06.358900+0000","last_undegraded":"2024-03-11T18:36:06.358900+0000","last_fullsized":"2024-03-11T18:36:06.358900+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T18:51:00.795706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.411589+0000","last_change":"2024-03-11T18:36:04.643164+0000","last_active":"2024-03-11T18:36:06.411589+0000","last_peered":"2024-03-11T18:36:06.411589+0000","last_clean":"2024-03-11T18:36:06.411589+0000","last_became_active":"2024-03-11T18:36:04.642374+0000","last_became_peered":"2024-03-11T18:36:04.642374+0000","last_unstale":"2024-03-11T18:36:06.411589+0000","last_undegraded":"2024-03-11T18:36:06.411589+0000","last_fullsized":"2024-03-11T18:36:06.411589+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:25:56.815496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.405796+0000","last_change":"2024-03-11T18:35:57.636550+0000","last_active":"2024-03-11T18:36:06.405796+0000","last_peered":"2024-03-11T18:36:06.405796+0000","last_clean":"2024-03-11T18:36:06.405796+0000","last_became_active":"2024-03-11T18:35:57.636093+0000","last_became_peered":"2024-03-11T18:35:57.636093+0000","last_unstale":"2024-03-11T18:36:06.405796+0000","last_undegraded":"2024-03-11T18:36:06.405796+0000","last_fullsized":"2024-03-11T18:36:06.405796+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:33:31.344465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.413301+0000","last_change":"2024-03-11T18:35:56.355480+0000","last_active":"2024-03-11T18:36:06.413301+0000","last_peered":"2024-03-11T18:36:06.413301+0000","last_clean":"2024-03-11T18:36:06.413301+0000","last_became_active":"2024-03-11T18:35:56.354759+0000","last_became_peered":"2024-03-11T18:35:56.354759+0000","last_unstale":"2024-03-11T18:36:06.413301+0000","last_undegraded":"2024-03-11T18:36:06.413301+0000","last_fullsized":"2024-03-11T18:36:06.413301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:45:37.842725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.400682+0000","last_change":"2024-03-11T18:36:04.617364+0000","last_active":"2024-03-11T18:36:06.400682+0000","last_peered":"2024-03-11T18:36:06.400682+0000","last_clean":"2024-03-11T18:36:06.400682+0000","last_became_active":"2024-03-11T18:36:04.613159+0000","last_became_peered":"2024-03-11T18:36:04.613159+0000","last_unstale":"2024-03-11T18:36:06.400682+0000","last_undegraded":"2024-03-11T18:36:06.400682+0000","last_fullsized":"2024-03-11T18:36:06.400682+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:25:23.672189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.412882+0000","last_change":"2024-03-11T18:35:57.683426+0000","last_active":"2024-03-11T18:36:06.412882+0000","last_peered":"2024-03-11T18:36:06.412882+0000","last_clean":"2024-03-11T18:36:06.412882+0000","last_became_active":"2024-03-11T18:35:57.683150+0000","last_became_peered":"2024-03-11T18:35:57.683150+0000","last_unstale":"2024-03-11T18:36:06.412882+0000","last_undegraded":"2024-03-11T18:36:06.412882+0000","last_fullsized":"2024-03-11T18:36:06.412882+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:36:00.079075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.411507+0000","last_change":"2024-03-11T18:35:56.361613+0000","last_active":"2024-03-11T18:36:06.411507+0000","last_peered":"2024-03-11T18:36:06.411507+0000","last_clean":"2024-03-11T18:36:06.411507+0000","last_became_active":"2024-03-11T18:35:56.359629+0000","last_became_peered":"2024-03-11T18:35:56.359629+0000","last_unstale":"2024-03-11T18:36:06.411507+0000","last_undegraded":"2024-03-11T18:36:06.411507+0000","last_fullsized":"2024-03-11T18:36:06.411507+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:03:06.585728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.401756+0000","last_change":"2024-03-11T18:36:04.608168+0000","last_active":"2024-03-11T18:36:06.401756+0000","last_peered":"2024-03-11T18:36:06.401756+0000","last_clean":"2024-03-11T18:36:06.401756+0000","last_became_active":"2024-03-11T18:36:04.607063+0000","last_became_peered":"2024-03-11T18:36:04.607063+0000","last_unstale":"2024-03-11T18:36:06.401756+0000","last_undegraded":"2024-03-11T18:36:06.401756+0000","last_fullsized":"2024-03-11T18:36:06.401756+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:53:41.996024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.405114+0000","last_change":"2024-03-11T18:36:04.607992+0000","last_active":"2024-03-11T18:36:06.405114+0000","last_peered":"2024-03-11T18:36:06.405114+0000","last_clean":"2024-03-11T18:36:06.405114+0000","last_became_active":"2024-03-11T18:36:04.607109+0000","last_became_peered":"2024-03-11T18:36:04.607109+0000","last_unstale":"2024-03-11T18:36:06.405114+0000","last_undegraded":"2024-03-11T18:36:06.405114+0000","last_fullsized":"2024-03-11T18:36:06.405114+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:14:37.678854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.357424+0000","last_change":"2024-03-11T18:35:57.680534+0000","last_active":"2024-03-11T18:36:06.357424+0000","last_peered":"2024-03-11T18:36:06.357424+0000","last_clean":"2024-03-11T18:36:06.357424+0000","last_became_active":"2024-03-11T18:35:57.679642+0000","last_became_peered":"2024-03-11T18:35:57.679642+0000","last_unstale":"2024-03-11T18:36:06.357424+0000","last_undegraded":"2024-03-11T18:36:06.357424+0000","last_fullsized":"2024-03-11T18:36:06.357424+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:25:49.707568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.409300+0000","last_change":"2024-03-11T18:35:56.303510+0000","last_active":"2024-03-11T18:36:06.409300+0000","last_peered":"2024-03-11T18:36:06.409300+0000","last_clean":"2024-03-11T18:36:06.409300+0000","last_became_active":"2024-03-11T18:35:56.302758+0000","last_became_peered":"2024-03-11T18:35:56.302758+0000","last_unstale":"2024-03-11T18:36:06.409300+0000","last_undegraded":"2024-03-11T18:36:06.409300+0000","last_fullsized":"2024-03-11T18:36:06.409300+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:03:32.297342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.412847+0000","last_change":"2024-03-11T18:36:04.618818+0000","last_active":"2024-03-11T18:36:06.412847+0000","last_peered":"2024-03-11T18:36:06.412847+0000","last_clean":"2024-03-11T18:36:06.412847+0000","last_became_active":"2024-03-11T18:36:04.617868+0000","last_became_peered":"2024-03-11T18:36:04.617868+0000","last_unstale":"2024-03-11T18:36:06.412847+0000","last_undegraded":"2024-03-11T18:36:06.412847+0000","last_fullsized":"2024-03-11T18:36:06.412847+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:16:10.508913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.400811+0000","last_change":"2024-03-11T18:35:57.690347+0000","last_active":"2024-03-11T18:36:06.400811+0000","last_peered":"2024-03-11T18:36:06.400811+0000","last_clean":"2024-03-11T18:36:06.400811+0000","last_became_active":"2024-03-11T18:35:57.689146+0000","last_became_peered":"2024-03-11T18:35:57.689146+0000","last_unstale":"2024-03-11T18:36:06.400811+0000","last_undegraded":"2024-03-11T18:36:06.400811+0000","last_fullsized":"2024-03-11T18:36:06.400811+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:03:56.260394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.402147+0000","last_change":"2024-03-11T18:35:56.348355+0000","last_active":"2024-03-11T18:36:06.402147+0000","last_peered":"2024-03-11T18:36:06.402147+0000","last_clean":"2024-03-11T18:36:06.402147+0000","last_became_active":"2024-03-11T18:35:56.347631+0000","last_became_peered":"2024-03-11T18:35:56.347631+0000","last_unstale":"2024-03-11T18:36:06.402147+0000","last_undegraded":"2024-03-11T18:36:06.402147+0000","last_fullsized":"2024-03-11T18:36:06.402147+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:06:02.844765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"23'1","reported_seq":28,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.410764+0000","last_change":"2024-03-11T18:35:56.361254+0000","last_active":"2024-03-11T18:36:06.410764+0000","last_peered":"2024-03-11T18:36:06.410764+0000","last_clean":"2024-03-11T18:36:06.410764+0000","last_became_active":"2024-03-11T18:35:56.360580+0000","last_became_peered":"2024-03-11T18:35:56.360580+0000","last_unstale":"2024-03-11T18:36:06.410764+0000","last_undegraded":"2024-03-11T18:36:06.410764+0000","last_fullsized":"2024-03-11T18:36:06.410764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:27:26.471550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.406163+0000","last_change":"2024-03-11T18:35:57.602778+0000","last_active":"2024-03-11T18:36:06.406163+0000","last_peered":"2024-03-11T18:36:06.406163+0000","last_clean":"2024-03-11T18:36:06.406163+0000","last_became_active":"2024-03-11T18:35:57.601990+0000","last_became_peered":"2024-03-11T18:35:57.601990+0000","last_unstale":"2024-03-11T18:36:06.406163+0000","last_undegraded":"2024-03-11T18:36:06.406163+0000","last_fullsized":"2024-03-11T18:36:06.406163+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:56:59.014871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.401528+0000","last_change":"2024-03-11T18:36:04.606098+0000","last_active":"2024-03-11T18:36:06.401528+0000","last_peered":"2024-03-11T18:36:06.401528+0000","last_clean":"2024-03-11T18:36:06.401528+0000","last_became_active":"2024-03-11T18:36:04.604823+0000","last_became_peered":"2024-03-11T18:36:04.604823+0000","last_unstale":"2024-03-11T18:36:06.401528+0000","last_undegraded":"2024-03-11T18:36:06.401528+0000","last_fullsized":"2024-03-11T18:36:06.401528+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:52:04.862452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"28'5","reported_seq":32,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:37.941879+0000","last_change":"2024-03-11T18:35:56.296312+0000","last_active":"2024-03-11T18:36:37.941879+0000","last_peered":"2024-03-11T18:36:37.941879+0000","last_clean":"2024-03-11T18:36:37.941879+0000","last_became_active":"2024-03-11T18:35:56.296090+0000","last_became_peered":"2024-03-11T18:35:56.296090+0000","last_unstale":"2024-03-11T18:36:37.941879+0000","last_undegraded":"2024-03-11T18:36:37.941879+0000","last_fullsized":"2024-03-11T18:36:37.941879+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:03:06.585728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.410741+0000","last_change":"2024-03-11T18:35:57.611809+0000","last_active":"2024-03-11T18:36:06.410741+0000","last_peered":"2024-03-11T18:36:06.410741+0000","last_clean":"2024-03-11T18:36:06.410741+0000","last_became_active":"2024-03-11T18:35:57.611097+0000","last_became_peered":"2024-03-11T18:35:57.611097+0000","last_unstale":"2024-03-11T18:36:06.410741+0000","last_undegraded":"2024-03-11T18:36:06.410741+0000","last_fullsized":"2024-03-11T18:36:06.410741+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:49:29.042114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.403540+0000","last_change":"2024-03-11T18:36:04.590064+0000","last_active":"2024-03-11T18:36:06.403540+0000","last_peered":"2024-03-11T18:36:06.403540+0000","last_clean":"2024-03-11T18:36:06.403540+0000","last_became_active":"2024-03-11T18:36:04.589037+0000","last_became_peered":"2024-03-11T18:36:04.589037+0000","last_unstale":"2024-03-11T18:36:06.403540+0000","last_undegraded":"2024-03-11T18:36:06.403540+0000","last_fullsized":"2024-03-11T18:36:06.403540+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:51:28.017349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.407866+0000","last_change":"2024-03-11T18:35:57.614944+0000","last_active":"2024-03-11T18:36:06.407866+0000","last_peered":"2024-03-11T18:36:06.407866+0000","last_clean":"2024-03-11T18:36:06.407866+0000","last_became_active":"2024-03-11T18:35:57.614144+0000","last_became_peered":"2024-03-11T18:35:57.614144+0000","last_unstale":"2024-03-11T18:36:06.407866+0000","last_undegraded":"2024-03-11T18:36:06.407866+0000","last_fullsized":"2024-03-11T18:36:06.407866+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:53:26.493265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.400811+0000","last_change":"2024-03-11T18:35:57.630211+0000","last_active":"2024-03-11T18:36:06.400811+0000","last_peered":"2024-03-11T18:36:06.400811+0000","last_clean":"2024-03-11T18:36:06.400811+0000","last_became_active":"2024-03-11T18:35:57.629599+0000","last_became_peered":"2024-03-11T18:35:57.629599+0000","last_unstale":"2024-03-11T18:36:06.400811+0000","last_undegraded":"2024-03-11T18:36:06.400811+0000","last_fullsized":"2024-03-11T18:36:06.400811+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:18:05.980207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.409629+0000","last_change":"2024-03-11T18:35:57.688700+0000","last_active":"2024-03-11T18:36:06.409629+0000","last_peered":"2024-03-11T18:36:06.409629+0000","last_clean":"2024-03-11T18:36:06.409629+0000","last_became_active":"2024-03-11T18:35:57.687968+0000","last_became_peered":"2024-03-11T18:35:57.687968+0000","last_unstale":"2024-03-11T18:36:06.409629+0000","last_undegraded":"2024-03-11T18:36:06.409629+0000","last_fullsized":"2024-03-11T18:36:06.409629+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:25:16.564261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.405998+0000","last_change":"2024-03-11T18:35:57.680730+0000","last_active":"2024-03-11T18:36:06.405998+0000","last_peered":"2024-03-11T18:36:06.405998+0000","last_clean":"2024-03-11T18:36:06.405998+0000","last_became_active":"2024-03-11T18:35:57.679935+0000","last_became_peered":"2024-03-11T18:35:57.679935+0000","last_unstale":"2024-03-11T18:36:06.405998+0000","last_undegraded":"2024-03-11T18:36:06.405998+0000","last_fullsized":"2024-03-11T18:36:06.405998+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:21:21.894805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.403523+0000","last_change":"2024-03-11T18:35:57.690799+0000","last_active":"2024-03-11T18:36:06.403523+0000","last_peered":"2024-03-11T18:36:06.403523+0000","last_clean":"2024-03-11T18:36:06.403523+0000","last_became_active":"2024-03-11T18:35:57.686623+0000","last_became_peered":"2024-03-11T18:35:57.686623+0000","last_unstale":"2024-03-11T18:36:06.403523+0000","last_undegraded":"2024-03-11T18:36:06.403523+0000","last_fullsized":"2024-03-11T18:36:06.403523+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:53:30.324898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.358254+0000","last_change":"2024-03-11T18:35:57.691955+0000","last_active":"2024-03-11T18:36:06.358254+0000","last_peered":"2024-03-11T18:36:06.358254+0000","last_clean":"2024-03-11T18:36:06.358254+0000","last_became_active":"2024-03-11T18:35:57.691413+0000","last_became_peered":"2024-03-11T18:35:57.691413+0000","last_unstale":"2024-03-11T18:36:06.358254+0000","last_undegraded":"2024-03-11T18:36:06.358254+0000","last_fullsized":"2024-03-11T18:36:06.358254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:31:19.941345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.410285+0000","last_change":"2024-03-11T18:35:57.684695+0000","last_active":"2024-03-11T18:36:06.410285+0000","last_peered":"2024-03-11T18:36:06.410285+0000","last_clean":"2024-03-11T18:36:06.410285+0000","last_became_active":"2024-03-11T18:35:57.684227+0000","last_became_peered":"2024-03-11T18:35:57.684227+0000","last_unstale":"2024-03-11T18:36:06.410285+0000","last_undegraded":"2024-03-11T18:36:06.410285+0000","last_fullsized":"2024-03-11T18:36:06.410285+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:28:51.265291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.405217+0000","last_change":"2024-03-11T18:35:57.683549+0000","last_active":"2024-03-11T18:36:06.405217+0000","last_peered":"2024-03-11T18:36:06.405217+0000","last_clean":"2024-03-11T18:36:06.405217+0000","last_became_active":"2024-03-11T18:35:57.680610+0000","last_became_peered":"2024-03-11T18:35:57.680610+0000","last_unstale":"2024-03-11T18:36:06.405217+0000","last_undegraded":"2024-03-11T18:36:06.405217+0000","last_fullsized":"2024-03-11T18:36:06.405217+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:25:52.716744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.409054+0000","last_change":"2024-03-11T18:35:57.686833+0000","last_active":"2024-03-11T18:36:06.409054+0000","last_peered":"2024-03-11T18:36:06.409054+0000","last_clean":"2024-03-11T18:36:06.409054+0000","last_became_active":"2024-03-11T18:35:57.686144+0000","last_became_peered":"2024-03-11T18:35:57.686144+0000","last_unstale":"2024-03-11T18:36:06.409054+0000","last_undegraded":"2024-03-11T18:36:06.409054+0000","last_fullsized":"2024-03-11T18:36:06.409054+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:16:03.400985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.401952+0000","last_change":"2024-03-11T18:35:57.681630+0000","last_active":"2024-03-11T18:36:06.401952+0000","last_peered":"2024-03-11T18:36:06.401952+0000","last_clean":"2024-03-11T18:36:06.401952+0000","last_became_active":"2024-03-11T18:35:57.678876+0000","last_became_peered":"2024-03-11T18:35:57.678876+0000","last_unstale":"2024-03-11T18:36:06.401952+0000","last_undegraded":"2024-03-11T18:36:06.401952+0000","last_fullsized":"2024-03-11T18:36:06.401952+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:03:33.511691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.409370+0000","last_change":"2024-03-11T18:35:57.679589+0000","last_active":"2024-03-11T18:36:06.409370+0000","last_peered":"2024-03-11T18:36:06.409370+0000","last_clean":"2024-03-11T18:36:06.409370+0000","last_became_active":"2024-03-11T18:35:57.678838+0000","last_became_peered":"2024-03-11T18:35:57.678838+0000","last_unstale":"2024-03-11T18:36:06.409370+0000","last_undegraded":"2024-03-11T18:36:06.409370+0000","last_fullsized":"2024-03-11T18:36:06.409370+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:54:09.699371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.404736+0000","last_change":"2024-03-11T18:35:57.674571+0000","last_active":"2024-03-11T18:36:06.404736+0000","last_peered":"2024-03-11T18:36:06.404736+0000","last_clean":"2024-03-11T18:36:06.404736+0000","last_became_active":"2024-03-11T18:35:57.671573+0000","last_became_peered":"2024-03-11T18:35:57.671573+0000","last_unstale":"2024-03-11T18:36:06.404736+0000","last_undegraded":"2024-03-11T18:36:06.404736+0000","last_fullsized":"2024-03-11T18:36:06.404736+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:35:10.403666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.410712+0000","last_change":"2024-03-11T18:35:57.615939+0000","last_active":"2024-03-11T18:36:06.410712+0000","last_peered":"2024-03-11T18:36:06.410712+0000","last_clean":"2024-03-11T18:36:06.410712+0000","last_became_active":"2024-03-11T18:35:57.615579+0000","last_became_peered":"2024-03-11T18:35:57.615579+0000","last_unstale":"2024-03-11T18:36:06.410712+0000","last_undegraded":"2024-03-11T18:36:06.410712+0000","last_fullsized":"2024-03-11T18:36:06.410712+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:03:33.511691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.355121+0000","last_change":"2024-03-11T18:35:57.688955+0000","last_active":"2024-03-11T18:36:06.355121+0000","last_peered":"2024-03-11T18:36:06.355121+0000","last_clean":"2024-03-11T18:36:06.355121+0000","last_became_active":"2024-03-11T18:35:57.688365+0000","last_became_peered":"2024-03-11T18:35:57.688365+0000","last_unstale":"2024-03-11T18:36:06.355121+0000","last_undegraded":"2024-03-11T18:36:06.355121+0000","last_fullsized":"2024-03-11T18:36:06.355121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:44:11.877541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.401903+0000","last_change":"2024-03-11T18:35:57.602044+0000","last_active":"2024-03-11T18:36:06.401903+0000","last_peered":"2024-03-11T18:36:06.401903+0000","last_clean":"2024-03-11T18:36:06.401903+0000","last_became_active":"2024-03-11T18:35:57.601334+0000","last_became_peered":"2024-03-11T18:35:57.601334+0000","last_unstale":"2024-03-11T18:36:06.401903+0000","last_undegraded":"2024-03-11T18:36:06.401903+0000","last_fullsized":"2024-03-11T18:36:06.401903+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:39:12.693619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.413873+0000","last_change":"2024-03-11T18:35:57.677114+0000","last_active":"2024-03-11T18:36:06.413873+0000","last_peered":"2024-03-11T18:36:06.413873+0000","last_clean":"2024-03-11T18:36:06.413873+0000","last_became_active":"2024-03-11T18:35:57.676075+0000","last_became_peered":"2024-03-11T18:35:57.676075+0000","last_unstale":"2024-03-11T18:36:06.413873+0000","last_undegraded":"2024-03-11T18:36:06.413873+0000","last_fullsized":"2024-03-11T18:36:06.413873+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:44:11.877541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.408799+0000","last_change":"2024-03-11T18:35:57.641392+0000","last_active":"2024-03-11T18:36:06.408799+0000","last_peered":"2024-03-11T18:36:06.408799+0000","last_clean":"2024-03-11T18:36:06.408799+0000","last_became_active":"2024-03-11T18:35:57.640934+0000","last_became_peered":"2024-03-11T18:35:57.640934+0000","last_unstale":"2024-03-11T18:36:06.408799+0000","last_undegraded":"2024-03-11T18:36:06.408799+0000","last_fullsized":"2024-03-11T18:36:06.408799+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:25:52.716744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.357659+0000","last_change":"2024-03-11T18:35:57.605291+0000","last_active":"2024-03-11T18:36:06.357659+0000","last_peered":"2024-03-11T18:36:06.357659+0000","last_clean":"2024-03-11T18:36:06.357659+0000","last_became_active":"2024-03-11T18:35:57.604645+0000","last_became_peered":"2024-03-11T18:35:57.604645+0000","last_unstale":"2024-03-11T18:36:06.357659+0000","last_undegraded":"2024-03-11T18:36:06.357659+0000","last_fullsized":"2024-03-11T18:36:06.357659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:03:33.511691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.404894+0000","last_change":"2024-03-11T18:35:57.503994+0000","last_active":"2024-03-11T18:36:06.404894+0000","last_peered":"2024-03-11T18:36:06.404894+0000","last_clean":"2024-03-11T18:36:06.404894+0000","last_became_active":"2024-03-11T18:35:57.503362+0000","last_became_peered":"2024-03-11T18:35:57.503362+0000","last_unstale":"2024-03-11T18:36:06.404894+0000","last_undegraded":"2024-03-11T18:36:06.404894+0000","last_fullsized":"2024-03-11T18:36:06.404894+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:03:58.875164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.402309+0000","last_change":"2024-03-11T18:35:57.679571+0000","last_active":"2024-03-11T18:36:06.402309+0000","last_peered":"2024-03-11T18:36:06.402309+0000","last_clean":"2024-03-11T18:36:06.402309+0000","last_became_active":"2024-03-11T18:35:57.678787+0000","last_became_peered":"2024-03-11T18:35:57.678787+0000","last_unstale":"2024-03-11T18:36:06.402309+0000","last_undegraded":"2024-03-11T18:36:06.402309+0000","last_fullsized":"2024-03-11T18:36:06.402309+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:13:40.392038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.412355+0000","last_change":"2024-03-11T18:35:57.613374+0000","last_active":"2024-03-11T18:36:06.412355+0000","last_peered":"2024-03-11T18:36:06.412355+0000","last_clean":"2024-03-11T18:36:06.412355+0000","last_became_active":"2024-03-11T18:35:57.613050+0000","last_became_peered":"2024-03-11T18:35:57.613050+0000","last_unstale":"2024-03-11T18:36:06.412355+0000","last_undegraded":"2024-03-11T18:36:06.412355+0000","last_fullsized":"2024-03-11T18:36:06.412355+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:50:39.923881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.411942+0000","last_change":"2024-03-11T18:35:57.636565+0000","last_active":"2024-03-11T18:36:06.411942+0000","last_peered":"2024-03-11T18:36:06.411942+0000","last_clean":"2024-03-11T18:36:06.411942+0000","last_became_active":"2024-03-11T18:35:57.635978+0000","last_became_peered":"2024-03-11T18:35:57.635978+0000","last_unstale":"2024-03-11T18:36:06.411942+0000","last_undegraded":"2024-03-11T18:36:06.411942+0000","last_fullsized":"2024-03-11T18:36:06.411942+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:03:56.260394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.410011+0000","last_change":"2024-03-11T18:35:57.641714+0000","last_active":"2024-03-11T18:36:06.410011+0000","last_peered":"2024-03-11T18:36:06.410011+0000","last_clean":"2024-03-11T18:36:06.410011+0000","last_became_active":"2024-03-11T18:35:57.641417+0000","last_became_peered":"2024-03-11T18:35:57.641417+0000","last_unstale":"2024-03-11T18:36:06.410011+0000","last_undegraded":"2024-03-11T18:36:06.410011+0000","last_fullsized":"2024-03-11T18:36:06.410011+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:25:49.707568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.354140+0000","last_change":"2024-03-11T18:35:57.689703+0000","last_active":"2024-03-11T18:36:06.354140+0000","last_peered":"2024-03-11T18:36:06.354140+0000","last_clean":"2024-03-11T18:36:06.354140+0000","last_became_active":"2024-03-11T18:35:57.689056+0000","last_became_peered":"2024-03-11T18:35:57.689056+0000","last_unstale":"2024-03-11T18:36:06.354140+0000","last_undegraded":"2024-03-11T18:36:06.354140+0000","last_fullsized":"2024-03-11T18:36:06.354140+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:41:42.263652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.400467+0000","last_change":"2024-03-11T18:35:57.684305+0000","last_active":"2024-03-11T18:36:06.400467+0000","last_peered":"2024-03-11T18:36:06.400467+0000","last_clean":"2024-03-11T18:36:06.400467+0000","last_became_active":"2024-03-11T18:35:57.682947+0000","last_became_peered":"2024-03-11T18:35:57.682947+0000","last_unstale":"2024-03-11T18:36:06.400467+0000","last_undegraded":"2024-03-11T18:36:06.400467+0000","last_fullsized":"2024-03-11T18:36:06.400467+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:53:26.493265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.352832+0000","last_change":"2024-03-11T18:35:57.692717+0000","last_active":"2024-03-11T18:36:06.352832+0000","last_peered":"2024-03-11T18:36:06.352832+0000","last_clean":"2024-03-11T18:36:06.352832+0000","last_became_active":"2024-03-11T18:35:57.692504+0000","last_became_peered":"2024-03-11T18:35:57.692504+0000","last_unstale":"2024-03-11T18:36:06.352832+0000","last_undegraded":"2024-03-11T18:36:06.352832+0000","last_fullsized":"2024-03-11T18:36:06.352832+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:25:52.716744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.409110+0000","last_change":"2024-03-11T18:35:57.640483+0000","last_active":"2024-03-11T18:36:06.409110+0000","last_peered":"2024-03-11T18:36:06.409110+0000","last_clean":"2024-03-11T18:36:06.409110+0000","last_became_active":"2024-03-11T18:35:57.639886+0000","last_became_peered":"2024-03-11T18:35:57.639886+0000","last_unstale":"2024-03-11T18:36:06.409110+0000","last_undegraded":"2024-03-11T18:36:06.409110+0000","last_fullsized":"2024-03-11T18:36:06.409110+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:34:39.325572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.352837+0000","last_change":"2024-03-11T18:35:57.687096+0000","last_active":"2024-03-11T18:36:06.352837+0000","last_peered":"2024-03-11T18:36:06.352837+0000","last_clean":"2024-03-11T18:36:06.352837+0000","last_became_active":"2024-03-11T18:35:57.686771+0000","last_became_peered":"2024-03-11T18:35:57.686771+0000","last_unstale":"2024-03-11T18:36:06.352837+0000","last_undegraded":"2024-03-11T18:36:06.352837+0000","last_fullsized":"2024-03-11T18:36:06.352837+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:16:03.400985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.409284+0000","last_change":"2024-03-11T18:35:57.694707+0000","last_active":"2024-03-11T18:36:06.409284+0000","last_peered":"2024-03-11T18:36:06.409284+0000","last_clean":"2024-03-11T18:36:06.409284+0000","last_became_active":"2024-03-11T18:35:57.694104+0000","last_became_peered":"2024-03-11T18:35:57.694104+0000","last_unstale":"2024-03-11T18:36:06.409284+0000","last_undegraded":"2024-03-11T18:36:06.409284+0000","last_fullsized":"2024-03-11T18:36:06.409284+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:47:28.579467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.405456+0000","last_change":"2024-03-11T18:35:57.538548+0000","last_active":"2024-03-11T18:36:06.405456+0000","last_peered":"2024-03-11T18:36:06.405456+0000","last_clean":"2024-03-11T18:36:06.405456+0000","last_became_active":"2024-03-11T18:35:57.537704+0000","last_became_peered":"2024-03-11T18:35:57.537704+0000","last_unstale":"2024-03-11T18:36:06.405456+0000","last_undegraded":"2024-03-11T18:36:06.405456+0000","last_fullsized":"2024-03-11T18:36:06.405456+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:40:52.251981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.359086+0000","last_change":"2024-03-11T18:35:57.507126+0000","last_active":"2024-03-11T18:36:06.359086+0000","last_peered":"2024-03-11T18:36:06.359086+0000","last_clean":"2024-03-11T18:36:06.359086+0000","last_became_active":"2024-03-11T18:35:57.506429+0000","last_became_peered":"2024-03-11T18:35:57.506429+0000","last_unstale":"2024-03-11T18:36:06.359086+0000","last_undegraded":"2024-03-11T18:36:06.359086+0000","last_fullsized":"2024-03-11T18:36:06.359086+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:28:51.265291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.353858+0000","last_change":"2024-03-11T18:35:57.687817+0000","last_active":"2024-03-11T18:36:06.353858+0000","last_peered":"2024-03-11T18:36:06.353858+0000","last_clean":"2024-03-11T18:36:06.353858+0000","last_became_active":"2024-03-11T18:35:57.687532+0000","last_became_peered":"2024-03-11T18:35:57.687532+0000","last_unstale":"2024-03-11T18:36:06.353858+0000","last_undegraded":"2024-03-11T18:36:06.353858+0000","last_fullsized":"2024-03-11T18:36:06.353858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:47:28.579467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5934,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":62,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":10596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":56,"ondisk_log_size":56,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476758,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":888832,"data_stored":707497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476758,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":303104,"data_stored":120389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476756,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":294912,"data_stored":116637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476757,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":303104,"data_stored":116673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509461,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":311296,"data_stored":121011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509461,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":116547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509462,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":307200,"data_stored":117379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542163,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343944,"statfs":{"total":96636764160,"available":96608198656,"internally_reserved":0,"allocated":901120,"data_stored":707575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-11T18:36:42.580 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-11T18:36:42.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.728+0000 7efd358bc640 1 Processor -- start 2024-03-11T18:36:42.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.732+0000 7efd358bc640 1 -- start start 2024-03-11T18:36:42.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.732+0000 7efd358bc640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7efd3006f220 0x7efd300c5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:42.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.732+0000 7efd358bc640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7efd300c8720 0x7efd30070b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:42.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.732+0000 7efd358bc640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd30071f50 con 0x7efd30071540 2024-03-11T18:36:42.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.732+0000 7efd358bc640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd300c8300 con 0x7efd3006f220 2024-03-11T18:36:42.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.732+0000 7efd358bc640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd300c7c70 con 0x7efd300c8720 2024-03-11T18:36:42.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.732+0000 7efd2e7fc640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7efd3006f220 0x7efd300c5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:42.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.732+0000 7efd2e7fc640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7efd3006f220 0x7efd300c5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50314/0 (socket says 172.21.15.90:50314) 2024-03-11T18:36:42.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.732+0000 7efd2e7fc640 1 -- 172.21.15.90:0/2470806553 learned_addr learned my addr 172.21.15.90:0/2470806553 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:42.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.736+0000 7efd2e7fc640 1 -- 172.21.15.90:0/2470806553 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7efd300c8720 msgr2=0x7efd30070b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:42.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.736+0000 7efd2e7fc640 1 --2- 172.21.15.90:0/2470806553 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7efd300c8720 0x7efd30070b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:42.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.736+0000 7efd2e7fc640 1 -- 172.21.15.90:0/2470806553 >> v1:172.21.15.90:6789/0 conn(0x7efd30071540 legacy=0x7efd30159c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:42.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.736+0000 7efd2e7fc640 1 -- 172.21.15.90:0/2470806553 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd300721f0 con 0x7efd3006f220 2024-03-11T18:36:42.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.740+0000 7efd2e7fc640 1 --2- 172.21.15.90:0/2470806553 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd3006f220 0x7efd300c5de0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7efd24009870 tx=0x7efd2401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6843f7ddeb7df35e server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:42.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.744+0000 7efd2dffb640 1 -- 172.21.15.90:0/2470806553 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd240a95f0 con 0x7efd3006f220 2024-03-11T18:36:42.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.744+0000 7efd2dffb640 1 -- 172.21.15.90:0/2470806553 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd240a9750 con 0x7efd3006f220 2024-03-11T18:36:42.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.744+0000 7efd2dffb640 1 -- 172.21.15.90:0/2470806553 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd240a9ae0 con 0x7efd3006f220 2024-03-11T18:36:42.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.744+0000 7efd358bc640 1 -- 172.21.15.90:0/2470806553 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd3006f220 msgr2=0x7efd300c5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:42.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.744+0000 7efd358bc640 1 --2- 172.21.15.90:0/2470806553 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd3006f220 0x7efd300c5de0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7efd24009870 tx=0x7efd2401c530 comp rx=0 tx=0).stop 2024-03-11T18:36:42.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.744+0000 7efd358bc640 1 -- 172.21.15.90:0/2470806553 shutdown_connections 2024-03-11T18:36:42.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.744+0000 7efd358bc640 1 --2- 172.21.15.90:0/2470806553 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7efd300c8720 0x7efd30070b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:42.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.744+0000 7efd358bc640 1 --2- 172.21.15.90:0/2470806553 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd3006f220 0x7efd300c5de0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:42.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.744+0000 7efd358bc640 1 -- 172.21.15.90:0/2470806553 >> 172.21.15.90:0/2470806553 conn(0x7efd3006ac00 msgr2=0x7efd3006b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:42.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.744+0000 7efd358bc640 1 -- 172.21.15.90:0/2470806553 shutdown_connections 2024-03-11T18:36:42.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.744+0000 7efd358bc640 1 -- 172.21.15.90:0/2470806553 wait complete. 2024-03-11T18:36:42.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.748+0000 7efd358bc640 1 Processor -- start 2024-03-11T18:36:42.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.748+0000 7efd358bc640 1 -- start start 2024-03-11T18:36:42.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.748+0000 7efd358bc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd3006f220 0x7efd30154f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:42.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.748+0000 7efd358bc640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7efd30071540 0x7efd30155640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:42.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.748+0000 7efd358bc640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd300c8720 0x7efd30156170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:42.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.748+0000 7efd358bc640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd30150720 con 0x7efd300c8720 2024-03-11T18:36:42.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.748+0000 7efd358bc640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd301ec4f0 con 0x7efd3006f220 2024-03-11T18:36:42.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.748+0000 7efd358bc640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd301501a0 con 0x7efd30071540 2024-03-11T18:36:42.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.748+0000 7efd2f7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd300c8720 0x7efd30156170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:42.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.748+0000 7efd2f7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd300c8720 0x7efd30156170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50316/0 (socket says 172.21.15.90:50316) 2024-03-11T18:36:42.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.748+0000 7efd2f7fe640 1 -- 172.21.15.90:0/4257794212 learned_addr learned my addr 172.21.15.90:0/4257794212 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:42.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.752+0000 7efd2f7fe640 1 -- 172.21.15.90:0/4257794212 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7efd30071540 msgr2=0x7efd30155640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:42.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.752+0000 7efd2f7fe640 1 --2- 172.21.15.90:0/4257794212 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7efd30071540 0x7efd30155640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:42.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.752+0000 7efd2f7fe640 1 -- 172.21.15.90:0/4257794212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd3006f220 msgr2=0x7efd30154f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:42.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.752+0000 7efd2f7fe640 1 --2- 172.21.15.90:0/4257794212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd3006f220 0x7efd30154f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:42.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.752+0000 7efd2f7fe640 1 -- 172.21.15.90:0/4257794212 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd240094d0 con 0x7efd300c8720 2024-03-11T18:36:42.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.756+0000 7efd2f7fe640 1 --2- 172.21.15.90:0/4257794212 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd300c8720 0x7efd30156170 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7efd2800b390 tx=0x7efd2800b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:42.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.760+0000 7efd17fff640 1 -- 172.21.15.90:0/4257794212 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd28011830 con 0x7efd300c8720 2024-03-11T18:36:42.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.760+0000 7efd17fff640 1 -- 172.21.15.90:0/4257794212 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd28011990 con 0x7efd300c8720 2024-03-11T18:36:42.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.760+0000 7efd358bc640 1 -- 172.21.15.90:0/4257794212 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd30154ac0 con 0x7efd300c8720 2024-03-11T18:36:42.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.760+0000 7efd17fff640 1 -- 172.21.15.90:0/4257794212 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd28011d40 con 0x7efd300c8720 2024-03-11T18:36:42.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.760+0000 7efd358bc640 1 -- 172.21.15.90:0/4257794212 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd301ebb50 con 0x7efd300c8720 2024-03-11T18:36:42.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:42.760+0000 7efd358bc640 1 -- 172.21.15.90:0/4257794212 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd3004eef0 con 0x7efd300c8720 2024-03-11T18:36:43.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.048+0000 7efd17fff640 1 -- 172.21.15.90:0/4257794212 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7efd2801a400 con 0x7efd300c8720 2024-03-11T18:36:43.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.048+0000 7efd17fff640 1 --2- 172.21.15.90:0/4257794212 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7efd1c066e90 0x7efd1c069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:43.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.048+0000 7efd2effd640 1 --2- 172.21.15.90:0/4257794212 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7efd1c066e90 0x7efd1c069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:43.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.048+0000 7efd2effd640 1 --2- 172.21.15.90:0/4257794212 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7efd1c066e90 0x7efd1c069350 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efd20007fd0 tx=0x7efd20007d60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:43.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.084+0000 7efd17fff640 1 -- 172.21.15.90:0/4257794212 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7efd280564c0 con 0x7efd300c8720 2024-03-11T18:36:43.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.084+0000 7efd17fff640 1 -- 172.21.15.90:0/4257794212 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efd28052060 con 0x7efd300c8720 2024-03-11T18:36:43.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.248+0000 7efd358bc640 1 -- 172.21.15.90:0/4257794212 --> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efd30071150 con 0x7efd1c066e90 2024-03-11T18:36:43.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.272+0000 7efd17fff640 1 -- 172.21.15.90:0/4257794212 <== mgr.4101 v2:172.21.15.90:6800/1999720161 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332612 (secure 0 0 0) 0x7efd30071150 con 0x7efd1c066e90 2024-03-11T18:36:43.281 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:36:43.283 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-03-11T18:36:43.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.276+0000 7efd358bc640 1 -- 172.21.15.90:0/4257794212 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7efd1c066e90 msgr2=0x7efd1c069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:43.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.276+0000 7efd358bc640 1 --2- 172.21.15.90:0/4257794212 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7efd1c066e90 0x7efd1c069350 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efd20007fd0 tx=0x7efd20007d60 comp rx=0 tx=0).stop 2024-03-11T18:36:43.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.276+0000 7efd358bc640 1 -- 172.21.15.90:0/4257794212 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd300c8720 msgr2=0x7efd30156170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:43.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.276+0000 7efd358bc640 1 --2- 172.21.15.90:0/4257794212 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd300c8720 0x7efd30156170 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7efd2800b390 tx=0x7efd2800b850 comp rx=0 tx=0).stop 2024-03-11T18:36:43.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.276+0000 7efd358bc640 1 -- 172.21.15.90:0/4257794212 shutdown_connections 2024-03-11T18:36:43.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.276+0000 7efd358bc640 1 --2- 172.21.15.90:0/4257794212 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7efd1c066e90 0x7efd1c069350 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:43.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.276+0000 7efd358bc640 1 --2- 172.21.15.90:0/4257794212 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd300c8720 0x7efd30156170 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:43.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.276+0000 7efd358bc640 1 --2- 172.21.15.90:0/4257794212 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7efd30071540 0x7efd30155640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:43.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.276+0000 7efd358bc640 1 --2- 172.21.15.90:0/4257794212 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7efd3006f220 0x7efd30154f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:43.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.276+0000 7efd358bc640 1 -- 172.21.15.90:0/4257794212 >> 172.21.15.90:0/4257794212 conn(0x7efd3006ac00 msgr2=0x7efd301531b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:43.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.276+0000 7efd358bc640 1 -- 172.21.15.90:0/4257794212 shutdown_connections 2024-03-11T18:36:43.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.276+0000 7efd358bc640 1 -- 172.21.15.90:0/4257794212 wait complete. 2024-03-11T18:36:43.301 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2024-03-11T18:36:41.806780+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596302,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":221,"num_write_kb":2439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":173,"ondisk_log_size":173,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219664,"kb_used_data":3516,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755056,"statfs":{"total":773094113280,"available":772869177344,"internally_reserved":0,"allocated":3600384,"data_stored":2123708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":221236788},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1294,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005016"},"pg_stats":[{"pgid":"2.1f","version":"23'4","reported_seq":31,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.406537+0000","last_change":"2024-03-11T18:35:56.290140+0000","last_active":"2024-03-11T18:36:06.406537+0000","last_peered":"2024-03-11T18:36:06.406537+0000","last_clean":"2024-03-11T18:36:06.406537+0000","last_became_active":"2024-03-11T18:35:56.289178+0000","last_became_peered":"2024-03-11T18:35:56.289178+0000","last_unstale":"2024-03-11T18:36:06.406537+0000","last_undegraded":"2024-03-11T18:36:06.406537+0000","last_fullsized":"2024-03-11T18:36:06.406537+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:59:46.211033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.405302+0000","last_change":"2024-03-11T18:35:57.676047+0000","last_active":"2024-03-11T18:36:06.405302+0000","last_peered":"2024-03-11T18:36:06.405302+0000","last_clean":"2024-03-11T18:36:06.405302+0000","last_became_active":"2024-03-11T18:35:57.675105+0000","last_became_peered":"2024-03-11T18:35:57.675105+0000","last_unstale":"2024-03-11T18:36:06.405302+0000","last_undegraded":"2024-03-11T18:36:06.405302+0000","last_fullsized":"2024-03-11T18:36:06.405302+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:45:39.057074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.401561+0000","last_change":"2024-03-11T18:36:04.607351+0000","last_active":"2024-03-11T18:36:06.401561+0000","last_peered":"2024-03-11T18:36:06.401561+0000","last_clean":"2024-03-11T18:36:06.401561+0000","last_became_active":"2024-03-11T18:36:04.599562+0000","last_became_peered":"2024-03-11T18:36:04.599562+0000","last_unstale":"2024-03-11T18:36:06.401561+0000","last_undegraded":"2024-03-11T18:36:06.401561+0000","last_fullsized":"2024-03-11T18:36:06.401561+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T18:57:51.134410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"23'1","reported_seq":28,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.405632+0000","last_change":"2024-03-11T18:35:56.291253+0000","last_active":"2024-03-11T18:36:06.405632+0000","last_peered":"2024-03-11T18:36:06.405632+0000","last_clean":"2024-03-11T18:36:06.405632+0000","last_became_active":"2024-03-11T18:35:56.290862+0000","last_became_peered":"2024-03-11T18:35:56.290862+0000","last_unstale":"2024-03-11T18:36:06.405632+0000","last_undegraded":"2024-03-11T18:36:06.405632+0000","last_fullsized":"2024-03-11T18:36:06.405632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:36:17.809744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.405636+0000","last_change":"2024-03-11T18:35:57.636861+0000","last_active":"2024-03-11T18:36:06.405636+0000","last_peered":"2024-03-11T18:36:06.405636+0000","last_clean":"2024-03-11T18:36:06.405636+0000","last_became_active":"2024-03-11T18:35:57.636111+0000","last_became_peered":"2024-03-11T18:35:57.636111+0000","last_unstale":"2024-03-11T18:36:06.405636+0000","last_undegraded":"2024-03-11T18:36:06.405636+0000","last_fullsized":"2024-03-11T18:36:06.405636+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:23:11.238170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.408420+0000","last_change":"2024-03-11T18:36:04.643426+0000","last_active":"2024-03-11T18:36:06.408420+0000","last_peered":"2024-03-11T18:36:06.408420+0000","last_clean":"2024-03-11T18:36:06.408420+0000","last_became_active":"2024-03-11T18:36:04.642597+0000","last_became_peered":"2024-03-11T18:36:04.642597+0000","last_unstale":"2024-03-11T18:36:06.408420+0000","last_undegraded":"2024-03-11T18:36:06.408420+0000","last_fullsized":"2024-03-11T18:36:06.408420+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:31:27.049273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"23'2","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.355107+0000","last_change":"2024-03-11T18:35:56.355147+0000","last_active":"2024-03-11T18:36:06.355107+0000","last_peered":"2024-03-11T18:36:06.355107+0000","last_clean":"2024-03-11T18:36:06.355107+0000","last_became_active":"2024-03-11T18:35:56.354388+0000","last_became_peered":"2024-03-11T18:35:56.354388+0000","last_unstale":"2024-03-11T18:36:06.355107+0000","last_undegraded":"2024-03-11T18:36:06.355107+0000","last_fullsized":"2024-03-11T18:36:06.355107+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:03:55.046045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.410658+0000","last_change":"2024-03-11T18:35:57.641486+0000","last_active":"2024-03-11T18:36:06.410658+0000","last_peered":"2024-03-11T18:36:06.410658+0000","last_clean":"2024-03-11T18:36:06.410658+0000","last_became_active":"2024-03-11T18:35:57.641052+0000","last_became_peered":"2024-03-11T18:35:57.641052+0000","last_unstale":"2024-03-11T18:36:06.410658+0000","last_undegraded":"2024-03-11T18:36:06.410658+0000","last_fullsized":"2024-03-11T18:36:06.410658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:29:45.662029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.411193+0000","last_change":"2024-03-11T18:36:04.629291+0000","last_active":"2024-03-11T18:36:06.411193+0000","last_peered":"2024-03-11T18:36:06.411193+0000","last_clean":"2024-03-11T18:36:06.411193+0000","last_became_active":"2024-03-11T18:36:04.624110+0000","last_became_peered":"2024-03-11T18:36:04.624110+0000","last_unstale":"2024-03-11T18:36:06.411193+0000","last_undegraded":"2024-03-11T18:36:06.411193+0000","last_fullsized":"2024-03-11T18:36:06.411193+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:04:03.368322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"23'2","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.353273+0000","last_change":"2024-03-11T18:35:56.298903+0000","last_active":"2024-03-11T18:36:06.353273+0000","last_peered":"2024-03-11T18:36:06.353273+0000","last_clean":"2024-03-11T18:36:06.353273+0000","last_became_active":"2024-03-11T18:35:56.297582+0000","last_became_peered":"2024-03-11T18:35:56.297582+0000","last_unstale":"2024-03-11T18:36:06.353273+0000","last_undegraded":"2024-03-11T18:36:06.353273+0000","last_fullsized":"2024-03-11T18:36:06.353273+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:53:25.278916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.413016+0000","last_change":"2024-03-11T18:35:57.640122+0000","last_active":"2024-03-11T18:36:06.413016+0000","last_peered":"2024-03-11T18:36:06.413016+0000","last_clean":"2024-03-11T18:36:06.413016+0000","last_became_active":"2024-03-11T18:35:57.639770+0000","last_became_peered":"2024-03-11T18:35:57.639770+0000","last_unstale":"2024-03-11T18:36:06.413016+0000","last_undegraded":"2024-03-11T18:36:06.413016+0000","last_fullsized":"2024-03-11T18:36:06.413016+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:39:25.767184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.410463+0000","last_change":"2024-03-11T18:36:04.627149+0000","last_active":"2024-03-11T18:36:06.410463+0000","last_peered":"2024-03-11T18:36:06.410463+0000","last_clean":"2024-03-11T18:36:06.410463+0000","last_became_active":"2024-03-11T18:36:04.626108+0000","last_became_peered":"2024-03-11T18:36:04.626108+0000","last_unstale":"2024-03-11T18:36:06.410463+0000","last_undegraded":"2024-03-11T18:36:06.410463+0000","last_fullsized":"2024-03-11T18:36:06.410463+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:03:40.619618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.410013+0000","last_change":"2024-03-11T18:35:57.616739+0000","last_active":"2024-03-11T18:36:06.410013+0000","last_peered":"2024-03-11T18:36:06.410013+0000","last_clean":"2024-03-11T18:36:06.410013+0000","last_became_active":"2024-03-11T18:35:57.616398+0000","last_became_peered":"2024-03-11T18:35:57.616398+0000","last_unstale":"2024-03-11T18:36:06.410013+0000","last_undegraded":"2024-03-11T18:36:06.410013+0000","last_fullsized":"2024-03-11T18:36:06.410013+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:53:30.324898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"23'2","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.353981+0000","last_change":"2024-03-11T18:35:56.304762+0000","last_active":"2024-03-11T18:36:06.353981+0000","last_peered":"2024-03-11T18:36:06.353981+0000","last_clean":"2024-03-11T18:36:06.353981+0000","last_became_active":"2024-03-11T18:35:56.302803+0000","last_became_peered":"2024-03-11T18:35:56.302803+0000","last_unstale":"2024-03-11T18:36:06.353981+0000","last_undegraded":"2024-03-11T18:36:06.353981+0000","last_fullsized":"2024-03-11T18:36:06.353981+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:31:18.726996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.412227+0000","last_change":"2024-03-11T18:36:04.626968+0000","last_active":"2024-03-11T18:36:06.412227+0000","last_peered":"2024-03-11T18:36:06.412227+0000","last_clean":"2024-03-11T18:36:06.412227+0000","last_became_active":"2024-03-11T18:36:04.626337+0000","last_became_peered":"2024-03-11T18:36:04.626337+0000","last_unstale":"2024-03-11T18:36:06.412227+0000","last_undegraded":"2024-03-11T18:36:06.412227+0000","last_fullsized":"2024-03-11T18:36:06.412227+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:53:37.432826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.402625+0000","last_change":"2024-03-11T18:35:57.689835+0000","last_active":"2024-03-11T18:36:06.402625+0000","last_peered":"2024-03-11T18:36:06.402625+0000","last_clean":"2024-03-11T18:36:06.402625+0000","last_became_active":"2024-03-11T18:35:57.689039+0000","last_became_peered":"2024-03-11T18:35:57.689039+0000","last_unstale":"2024-03-11T18:36:06.402625+0000","last_undegraded":"2024-03-11T18:36:06.402625+0000","last_fullsized":"2024-03-11T18:36:06.402625+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:38:27.733669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.358819+0000","last_change":"2024-03-11T18:35:56.291070+0000","last_active":"2024-03-11T18:36:06.358819+0000","last_peered":"2024-03-11T18:36:06.358819+0000","last_clean":"2024-03-11T18:36:06.358819+0000","last_became_active":"2024-03-11T18:35:56.290504+0000","last_became_peered":"2024-03-11T18:35:56.290504+0000","last_unstale":"2024-03-11T18:36:06.358819+0000","last_undegraded":"2024-03-11T18:36:06.358819+0000","last_fullsized":"2024-03-11T18:36:06.358819+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:29:31.660243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.411988+0000","last_change":"2024-03-11T18:36:04.626150+0000","last_active":"2024-03-11T18:36:06.411988+0000","last_peered":"2024-03-11T18:36:06.411988+0000","last_clean":"2024-03-11T18:36:06.411988+0000","last_became_active":"2024-03-11T18:36:04.625084+0000","last_became_peered":"2024-03-11T18:36:04.625084+0000","last_unstale":"2024-03-11T18:36:06.411988+0000","last_undegraded":"2024-03-11T18:36:06.411988+0000","last_fullsized":"2024-03-11T18:36:06.411988+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:28:58.373219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.400936+0000","last_change":"2024-03-11T18:35:57.684535+0000","last_active":"2024-03-11T18:36:06.400936+0000","last_peered":"2024-03-11T18:36:06.400936+0000","last_clean":"2024-03-11T18:36:06.400936+0000","last_became_active":"2024-03-11T18:35:57.681098+0000","last_became_peered":"2024-03-11T18:35:57.681098+0000","last_unstale":"2024-03-11T18:36:06.400936+0000","last_undegraded":"2024-03-11T18:36:06.400936+0000","last_fullsized":"2024-03-11T18:36:06.400936+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:25:49.707568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"28'3","reported_seq":30,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:37.719417+0000","last_change":"2024-03-11T18:35:56.290745+0000","last_active":"2024-03-11T18:36:37.719417+0000","last_peered":"2024-03-11T18:36:37.719417+0000","last_clean":"2024-03-11T18:36:37.719417+0000","last_became_active":"2024-03-11T18:35:56.290006+0000","last_became_peered":"2024-03-11T18:35:56.290006+0000","last_unstale":"2024-03-11T18:36:37.719417+0000","last_undegraded":"2024-03-11T18:36:37.719417+0000","last_fullsized":"2024-03-11T18:36:37.719417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:06:02.844765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.359859+0000","last_change":"2024-03-11T18:36:04.620689+0000","last_active":"2024-03-11T18:36:06.359859+0000","last_peered":"2024-03-11T18:36:06.359859+0000","last_clean":"2024-03-11T18:36:06.359859+0000","last_became_active":"2024-03-11T18:36:04.620282+0000","last_became_peered":"2024-03-11T18:36:04.620282+0000","last_unstale":"2024-03-11T18:36:06.359859+0000","last_undegraded":"2024-03-11T18:36:06.359859+0000","last_fullsized":"2024-03-11T18:36:06.359859+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:54:16.807299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.413230+0000","last_change":"2024-03-11T18:35:57.615234+0000","last_active":"2024-03-11T18:36:06.413230+0000","last_peered":"2024-03-11T18:36:06.413230+0000","last_clean":"2024-03-11T18:36:06.413230+0000","last_became_active":"2024-03-11T18:35:57.614885+0000","last_became_peered":"2024-03-11T18:35:57.614885+0000","last_unstale":"2024-03-11T18:36:06.413230+0000","last_undegraded":"2024-03-11T18:36:06.413230+0000","last_fullsized":"2024-03-11T18:36:06.413230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:22:13.730630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.352619+0000","last_change":"2024-03-11T18:35:56.304617+0000","last_active":"2024-03-11T18:36:06.352619+0000","last_peered":"2024-03-11T18:36:06.352619+0000","last_clean":"2024-03-11T18:36:06.352619+0000","last_became_active":"2024-03-11T18:35:56.301698+0000","last_became_peered":"2024-03-11T18:35:56.301698+0000","last_unstale":"2024-03-11T18:36:06.352619+0000","last_undegraded":"2024-03-11T18:36:06.352619+0000","last_fullsized":"2024-03-11T18:36:06.352619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:29:31.660243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.412949+0000","last_change":"2024-03-11T18:36:04.643279+0000","last_active":"2024-03-11T18:36:06.412949+0000","last_peered":"2024-03-11T18:36:06.412949+0000","last_clean":"2024-03-11T18:36:06.412949+0000","last_became_active":"2024-03-11T18:36:04.641983+0000","last_became_peered":"2024-03-11T18:36:04.641983+0000","last_unstale":"2024-03-11T18:36:06.412949+0000","last_undegraded":"2024-03-11T18:36:06.412949+0000","last_fullsized":"2024-03-11T18:36:06.412949+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:13:47.499965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.406548+0000","last_change":"2024-03-11T18:35:57.506963+0000","last_active":"2024-03-11T18:36:06.406548+0000","last_peered":"2024-03-11T18:36:06.406548+0000","last_clean":"2024-03-11T18:36:06.406548+0000","last_became_active":"2024-03-11T18:35:57.506038+0000","last_became_peered":"2024-03-11T18:35:57.506038+0000","last_unstale":"2024-03-11T18:36:06.406548+0000","last_undegraded":"2024-03-11T18:36:06.406548+0000","last_fullsized":"2024-03-11T18:36:06.406548+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:52:06.024245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"23'2","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.411969+0000","last_change":"2024-03-11T18:35:56.364512+0000","last_active":"2024-03-11T18:36:06.411969+0000","last_peered":"2024-03-11T18:36:06.411969+0000","last_clean":"2024-03-11T18:36:06.411969+0000","last_became_active":"2024-03-11T18:35:56.363731+0000","last_became_peered":"2024-03-11T18:35:56.363731+0000","last_unstale":"2024-03-11T18:36:06.411969+0000","last_undegraded":"2024-03-11T18:36:06.411969+0000","last_fullsized":"2024-03-11T18:36:06.411969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:36:17.809744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.408707+0000","last_change":"2024-03-11T18:36:04.643782+0000","last_active":"2024-03-11T18:36:06.408707+0000","last_peered":"2024-03-11T18:36:06.408707+0000","last_clean":"2024-03-11T18:36:06.408707+0000","last_became_active":"2024-03-11T18:36:04.642133+0000","last_became_peered":"2024-03-11T18:36:04.642133+0000","last_unstale":"2024-03-11T18:36:06.408707+0000","last_undegraded":"2024-03-11T18:36:06.408707+0000","last_fullsized":"2024-03-11T18:36:06.408707+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:53:33.601193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.405392+0000","last_change":"2024-03-11T18:35:57.675207+0000","last_active":"2024-03-11T18:36:06.405392+0000","last_peered":"2024-03-11T18:36:06.405392+0000","last_clean":"2024-03-11T18:36:06.405392+0000","last_became_active":"2024-03-11T18:35:57.674472+0000","last_became_peered":"2024-03-11T18:35:57.674472+0000","last_unstale":"2024-03-11T18:36:06.405392+0000","last_undegraded":"2024-03-11T18:36:06.405392+0000","last_fullsized":"2024-03-11T18:36:06.405392+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:29:45.662029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"24'1","reported_seq":28,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.359192+0000","last_change":"2024-03-11T18:35:56.348479+0000","last_active":"2024-03-11T18:36:06.359192+0000","last_peered":"2024-03-11T18:36:06.359192+0000","last_clean":"2024-03-11T18:36:06.359192+0000","last_became_active":"2024-03-11T18:35:56.348040+0000","last_became_peered":"2024-03-11T18:35:56.348040+0000","last_unstale":"2024-03-11T18:36:06.359192+0000","last_undegraded":"2024-03-11T18:36:06.359192+0000","last_fullsized":"2024-03-11T18:36:06.359192+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:27:26.471550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.401155+0000","last_change":"2024-03-11T18:36:04.599805+0000","last_active":"2024-03-11T18:36:06.401155+0000","last_peered":"2024-03-11T18:36:06.401155+0000","last_clean":"2024-03-11T18:36:06.401155+0000","last_became_active":"2024-03-11T18:36:04.599065+0000","last_became_peered":"2024-03-11T18:36:04.599065+0000","last_unstale":"2024-03-11T18:36:06.401155+0000","last_undegraded":"2024-03-11T18:36:06.401155+0000","last_fullsized":"2024-03-11T18:36:06.401155+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:36:07.187003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.359096+0000","last_change":"2024-03-11T18:35:57.538551+0000","last_active":"2024-03-11T18:36:06.359096+0000","last_peered":"2024-03-11T18:36:06.359096+0000","last_clean":"2024-03-11T18:36:06.359096+0000","last_became_active":"2024-03-11T18:35:57.537706+0000","last_became_peered":"2024-03-11T18:35:57.537706+0000","last_unstale":"2024-03-11T18:36:06.359096+0000","last_undegraded":"2024-03-11T18:36:06.359096+0000","last_fullsized":"2024-03-11T18:36:06.359096+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:03:56.260394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.355377+0000","last_change":"2024-03-11T18:35:56.302575+0000","last_active":"2024-03-11T18:36:06.355377+0000","last_peered":"2024-03-11T18:36:06.355377+0000","last_clean":"2024-03-11T18:36:06.355377+0000","last_became_active":"2024-03-11T18:35:56.301819+0000","last_became_peered":"2024-03-11T18:35:56.301819+0000","last_unstale":"2024-03-11T18:36:06.355377+0000","last_undegraded":"2024-03-11T18:36:06.355377+0000","last_fullsized":"2024-03-11T18:36:06.355377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:38:26.519320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.404840+0000","last_change":"2024-03-11T18:36:04.609282+0000","last_active":"2024-03-11T18:36:06.404840+0000","last_peered":"2024-03-11T18:36:06.404840+0000","last_clean":"2024-03-11T18:36:06.404840+0000","last_became_active":"2024-03-11T18:36:04.608260+0000","last_became_peered":"2024-03-11T18:36:04.608260+0000","last_unstale":"2024-03-11T18:36:06.404840+0000","last_undegraded":"2024-03-11T18:36:06.404840+0000","last_fullsized":"2024-03-11T18:36:06.404840+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:49:28.143755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.413195+0000","last_change":"2024-03-11T18:35:57.688727+0000","last_active":"2024-03-11T18:36:06.413195+0000","last_peered":"2024-03-11T18:36:06.413195+0000","last_clean":"2024-03-11T18:36:06.413195+0000","last_became_active":"2024-03-11T18:35:57.688209+0000","last_became_peered":"2024-03-11T18:35:57.688209+0000","last_unstale":"2024-03-11T18:36:06.413195+0000","last_undegraded":"2024-03-11T18:36:06.413195+0000","last_fullsized":"2024-03-11T18:36:06.413195+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:58:13.389498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.359326+0000","last_change":"2024-03-11T18:35:56.293842+0000","last_active":"2024-03-11T18:36:06.359326+0000","last_peered":"2024-03-11T18:36:06.359326+0000","last_clean":"2024-03-11T18:36:06.359326+0000","last_became_active":"2024-03-11T18:35:56.293276+0000","last_became_peered":"2024-03-11T18:35:56.293276+0000","last_unstale":"2024-03-11T18:36:06.359326+0000","last_undegraded":"2024-03-11T18:36:06.359326+0000","last_fullsized":"2024-03-11T18:36:06.359326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:04:13.101160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.358354+0000","last_change":"2024-03-11T18:36:04.626320+0000","last_active":"2024-03-11T18:36:06.358354+0000","last_peered":"2024-03-11T18:36:06.358354+0000","last_clean":"2024-03-11T18:36:06.358354+0000","last_became_active":"2024-03-11T18:36:04.625663+0000","last_became_peered":"2024-03-11T18:36:04.625663+0000","last_unstale":"2024-03-11T18:36:06.358354+0000","last_undegraded":"2024-03-11T18:36:06.358354+0000","last_fullsized":"2024-03-11T18:36:06.358354+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:39:32.875112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.411275+0000","last_change":"2024-03-11T18:35:57.612242+0000","last_active":"2024-03-11T18:36:06.411275+0000","last_peered":"2024-03-11T18:36:06.411275+0000","last_clean":"2024-03-11T18:36:06.411275+0000","last_became_active":"2024-03-11T18:35:57.611509+0000","last_became_peered":"2024-03-11T18:35:57.611509+0000","last_unstale":"2024-03-11T18:36:06.411275+0000","last_undegraded":"2024-03-11T18:36:06.411275+0000","last_fullsized":"2024-03-11T18:36:06.411275+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:45:28.543623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.410118+0000","last_change":"2024-03-11T18:35:56.298766+0000","last_active":"2024-03-11T18:36:06.410118+0000","last_peered":"2024-03-11T18:36:06.410118+0000","last_clean":"2024-03-11T18:36:06.410118+0000","last_became_active":"2024-03-11T18:35:56.297747+0000","last_became_peered":"2024-03-11T18:35:56.297747+0000","last_unstale":"2024-03-11T18:36:06.410118+0000","last_undegraded":"2024-03-11T18:36:06.410118+0000","last_fullsized":"2024-03-11T18:36:06.410118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:13:39.177689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.358743+0000","last_change":"2024-03-11T18:36:04.635773+0000","last_active":"2024-03-11T18:36:06.358743+0000","last_peered":"2024-03-11T18:36:06.358743+0000","last_clean":"2024-03-11T18:36:06.358743+0000","last_became_active":"2024-03-11T18:36:04.635261+0000","last_became_peered":"2024-03-11T18:36:04.635261+0000","last_unstale":"2024-03-11T18:36:06.358743+0000","last_undegraded":"2024-03-11T18:36:06.358743+0000","last_fullsized":"2024-03-11T18:36:06.358743+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:44:18.985469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.358192+0000","last_change":"2024-03-11T18:35:57.538552+0000","last_active":"2024-03-11T18:36:06.358192+0000","last_peered":"2024-03-11T18:36:06.358192+0000","last_clean":"2024-03-11T18:36:06.358192+0000","last_became_active":"2024-03-11T18:35:57.537717+0000","last_became_peered":"2024-03-11T18:35:57.537717+0000","last_unstale":"2024-03-11T18:36:06.358192+0000","last_undegraded":"2024-03-11T18:36:06.358192+0000","last_fullsized":"2024-03-11T18:36:06.358192+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:53:26.493265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"28'1","reported_seq":28,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.401191+0000","last_change":"2024-03-11T18:35:56.294572+0000","last_active":"2024-03-11T18:36:06.401191+0000","last_peered":"2024-03-11T18:36:06.401191+0000","last_clean":"2024-03-11T18:36:06.401191+0000","last_became_active":"2024-03-11T18:35:56.294096+0000","last_became_peered":"2024-03-11T18:35:56.294096+0000","last_unstale":"2024-03-11T18:36:06.401191+0000","last_undegraded":"2024-03-11T18:36:06.401191+0000","last_fullsized":"2024-03-11T18:36:06.401191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:22:12.516281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.359857+0000","last_change":"2024-03-11T18:36:04.620762+0000","last_active":"2024-03-11T18:36:06.359857+0000","last_peered":"2024-03-11T18:36:06.359857+0000","last_clean":"2024-03-11T18:36:06.359857+0000","last_became_active":"2024-03-11T18:36:04.620356+0000","last_became_peered":"2024-03-11T18:36:04.620356+0000","last_unstale":"2024-03-11T18:36:06.359857+0000","last_undegraded":"2024-03-11T18:36:06.359857+0000","last_fullsized":"2024-03-11T18:36:06.359857+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:58:20.497426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.404612+0000","last_change":"2024-03-11T18:35:57.602773+0000","last_active":"2024-03-11T18:36:06.404612+0000","last_peered":"2024-03-11T18:36:06.404612+0000","last_clean":"2024-03-11T18:36:06.404612+0000","last_became_active":"2024-03-11T18:35:57.602060+0000","last_became_peered":"2024-03-11T18:35:57.602060+0000","last_unstale":"2024-03-11T18:36:06.404612+0000","last_undegraded":"2024-03-11T18:36:06.404612+0000","last_fullsized":"2024-03-11T18:36:06.404612+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:36:22.201707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.352640+0000","last_change":"2024-03-11T18:35:56.305283+0000","last_active":"2024-03-11T18:36:06.352640+0000","last_peered":"2024-03-11T18:36:06.352640+0000","last_clean":"2024-03-11T18:36:06.352640+0000","last_became_active":"2024-03-11T18:35:56.303063+0000","last_became_peered":"2024-03-11T18:35:56.303063+0000","last_unstale":"2024-03-11T18:36:06.352640+0000","last_undegraded":"2024-03-11T18:36:06.352640+0000","last_fullsized":"2024-03-11T18:36:06.352640+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T18:51:00.795706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.406282+0000","last_change":"2024-03-11T18:36:04.617247+0000","last_active":"2024-03-11T18:36:06.406282+0000","last_peered":"2024-03-11T18:36:06.406282+0000","last_clean":"2024-03-11T18:36:06.406282+0000","last_became_active":"2024-03-11T18:36:04.610765+0000","last_became_peered":"2024-03-11T18:36:04.610765+0000","last_unstale":"2024-03-11T18:36:06.406282+0000","last_undegraded":"2024-03-11T18:36:06.406282+0000","last_fullsized":"2024-03-11T18:36:06.406282+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:22:19.187685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.401101+0000","last_change":"2024-03-11T18:35:57.684207+0000","last_active":"2024-03-11T18:36:06.401101+0000","last_peered":"2024-03-11T18:36:06.401101+0000","last_clean":"2024-03-11T18:36:06.401101+0000","last_became_active":"2024-03-11T18:35:57.682653+0000","last_became_peered":"2024-03-11T18:35:57.682653+0000","last_unstale":"2024-03-11T18:36:06.401101+0000","last_undegraded":"2024-03-11T18:36:06.401101+0000","last_fullsized":"2024-03-11T18:36:06.401101+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:28:51.265291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.409922+0000","last_change":"2024-03-11T18:35:56.297078+0000","last_active":"2024-03-11T18:36:06.409922+0000","last_peered":"2024-03-11T18:36:06.409922+0000","last_clean":"2024-03-11T18:36:06.409922+0000","last_became_active":"2024-03-11T18:35:56.296705+0000","last_became_peered":"2024-03-11T18:35:56.296705+0000","last_unstale":"2024-03-11T18:36:06.409922+0000","last_undegraded":"2024-03-11T18:36:06.409922+0000","last_fullsized":"2024-03-11T18:36:06.409922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:53:25.278916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":182,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.352930+0000","last_change":"2024-03-11T18:35:18.199528+0000","last_active":"2024-03-11T18:36:06.352930+0000","last_peered":"2024-03-11T18:36:06.352930+0000","last_clean":"2024-03-11T18:36:06.352930+0000","last_became_active":"2024-03-11T18:35:18.197797+0000","last_became_peered":"2024-03-11T18:35:18.197797+0000","last_unstale":"2024-03-11T18:36:06.352930+0000","last_undegraded":"2024-03-11T18:36:06.352930+0000","last_fullsized":"2024-03-11T18:36:06.352930+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:16.207500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:16.207500+0000","last_clean_scrub_stamp":"2024-03-11T18:35:16.207500+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:29:00.357496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.411807+0000","last_change":"2024-03-11T18:36:04.599815+0000","last_active":"2024-03-11T18:36:06.411807+0000","last_peered":"2024-03-11T18:36:06.411807+0000","last_clean":"2024-03-11T18:36:06.411807+0000","last_became_active":"2024-03-11T18:36:04.598777+0000","last_became_peered":"2024-03-11T18:36:04.598777+0000","last_unstale":"2024-03-11T18:36:06.411807+0000","last_undegraded":"2024-03-11T18:36:06.411807+0000","last_fullsized":"2024-03-11T18:36:06.411807+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:39:32.875112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.354816+0000","last_change":"2024-03-11T18:35:57.687766+0000","last_active":"2024-03-11T18:36:06.354816+0000","last_peered":"2024-03-11T18:36:06.354816+0000","last_clean":"2024-03-11T18:36:06.354816+0000","last_became_active":"2024-03-11T18:35:57.687059+0000","last_became_peered":"2024-03-11T18:35:57.687059+0000","last_unstale":"2024-03-11T18:36:06.354816+0000","last_undegraded":"2024-03-11T18:36:06.354816+0000","last_fullsized":"2024-03-11T18:36:06.354816+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:39:25.767184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.402184+0000","last_change":"2024-03-11T18:35:56.294626+0000","last_active":"2024-03-11T18:36:06.402184+0000","last_peered":"2024-03-11T18:36:06.402184+0000","last_clean":"2024-03-11T18:36:06.402184+0000","last_became_active":"2024-03-11T18:35:56.293961+0000","last_became_peered":"2024-03-11T18:35:56.293961+0000","last_unstale":"2024-03-11T18:36:06.402184+0000","last_undegraded":"2024-03-11T18:36:06.402184+0000","last_fullsized":"2024-03-11T18:36:06.402184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:50:38.709532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.409872+0000","last_change":"2024-03-11T18:36:04.619862+0000","last_active":"2024-03-11T18:36:06.409872+0000","last_peered":"2024-03-11T18:36:06.409872+0000","last_clean":"2024-03-11T18:36:06.409872+0000","last_became_active":"2024-03-11T18:36:04.618908+0000","last_became_peered":"2024-03-11T18:36:04.618908+0000","last_unstale":"2024-03-11T18:36:06.409872+0000","last_undegraded":"2024-03-11T18:36:06.409872+0000","last_fullsized":"2024-03-11T18:36:06.409872+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:34:46.433500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.410912+0000","last_change":"2024-03-11T18:35:57.686416+0000","last_active":"2024-03-11T18:36:06.410912+0000","last_peered":"2024-03-11T18:36:06.410912+0000","last_clean":"2024-03-11T18:36:06.410912+0000","last_became_active":"2024-03-11T18:35:57.685828+0000","last_became_peered":"2024-03-11T18:35:57.685828+0000","last_unstale":"2024-03-11T18:36:06.410912+0000","last_undegraded":"2024-03-11T18:36:06.410912+0000","last_fullsized":"2024-03-11T18:36:06.410912+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:41:42.263652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"23'1","reported_seq":28,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.359093+0000","last_change":"2024-03-11T18:35:56.295539+0000","last_active":"2024-03-11T18:36:06.359093+0000","last_peered":"2024-03-11T18:36:06.359093+0000","last_clean":"2024-03-11T18:36:06.359093+0000","last_became_active":"2024-03-11T18:35:56.295123+0000","last_became_peered":"2024-03-11T18:35:56.295123+0000","last_unstale":"2024-03-11T18:36:06.359093+0000","last_undegraded":"2024-03-11T18:36:06.359093+0000","last_fullsized":"2024-03-11T18:36:06.359093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:03:06.585728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.406058+0000","last_change":"2024-03-11T18:36:04.616750+0000","last_active":"2024-03-11T18:36:06.406058+0000","last_peered":"2024-03-11T18:36:06.406058+0000","last_clean":"2024-03-11T18:36:06.406058+0000","last_became_active":"2024-03-11T18:36:04.616338+0000","last_became_peered":"2024-03-11T18:36:04.616338+0000","last_unstale":"2024-03-11T18:36:06.406058+0000","last_undegraded":"2024-03-11T18:36:06.406058+0000","last_fullsized":"2024-03-11T18:36:06.406058+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:14:42.558491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.354272+0000","last_change":"2024-03-11T18:35:57.689888+0000","last_active":"2024-03-11T18:36:06.354272+0000","last_peered":"2024-03-11T18:36:06.354272+0000","last_clean":"2024-03-11T18:36:06.354272+0000","last_became_active":"2024-03-11T18:35:57.689424+0000","last_became_peered":"2024-03-11T18:35:57.689424+0000","last_unstale":"2024-03-11T18:36:06.354272+0000","last_undegraded":"2024-03-11T18:36:06.354272+0000","last_fullsized":"2024-03-11T18:36:06.354272+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:25:16.564261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"23'2","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.402166+0000","last_change":"2024-03-11T18:35:56.293862+0000","last_active":"2024-03-11T18:36:06.402166+0000","last_peered":"2024-03-11T18:36:06.402166+0000","last_clean":"2024-03-11T18:36:06.402166+0000","last_became_active":"2024-03-11T18:35:56.293585+0000","last_became_peered":"2024-03-11T18:35:56.293585+0000","last_unstale":"2024-03-11T18:36:06.402166+0000","last_undegraded":"2024-03-11T18:36:06.402166+0000","last_fullsized":"2024-03-11T18:36:06.402166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:45:27.329274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.408514+0000","last_change":"2024-03-11T18:36:04.627729+0000","last_active":"2024-03-11T18:36:06.408514+0000","last_peered":"2024-03-11T18:36:06.408514+0000","last_clean":"2024-03-11T18:36:06.408514+0000","last_became_active":"2024-03-11T18:36:04.626596+0000","last_became_peered":"2024-03-11T18:36:04.626596+0000","last_unstale":"2024-03-11T18:36:06.408514+0000","last_undegraded":"2024-03-11T18:36:06.408514+0000","last_fullsized":"2024-03-11T18:36:06.408514+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:51:28.017349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.353860+0000","last_change":"2024-03-11T18:35:57.690506+0000","last_active":"2024-03-11T18:36:06.353860+0000","last_peered":"2024-03-11T18:36:06.353860+0000","last_clean":"2024-03-11T18:36:06.353860+0000","last_became_active":"2024-03-11T18:35:57.690223+0000","last_became_peered":"2024-03-11T18:35:57.690223+0000","last_unstale":"2024-03-11T18:36:06.353860+0000","last_undegraded":"2024-03-11T18:36:06.353860+0000","last_fullsized":"2024-03-11T18:36:06.353860+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:58:13.389498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"23'1","reported_seq":28,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.414711+0000","last_change":"2024-03-11T18:35:56.257209+0000","last_active":"2024-03-11T18:36:06.414711+0000","last_peered":"2024-03-11T18:36:06.414711+0000","last_clean":"2024-03-11T18:36:06.414711+0000","last_became_active":"2024-03-11T18:35:56.256454+0000","last_became_peered":"2024-03-11T18:35:56.256454+0000","last_unstale":"2024-03-11T18:36:06.414711+0000","last_undegraded":"2024-03-11T18:36:06.414711+0000","last_fullsized":"2024-03-11T18:36:06.414711+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:32:06.369248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.408104+0000","last_change":"2024-03-11T18:36:04.644184+0000","last_active":"2024-03-11T18:36:06.408104+0000","last_peered":"2024-03-11T18:36:06.408104+0000","last_clean":"2024-03-11T18:36:06.408104+0000","last_became_active":"2024-03-11T18:36:04.642079+0000","last_became_peered":"2024-03-11T18:36:04.642079+0000","last_unstale":"2024-03-11T18:36:06.408104+0000","last_undegraded":"2024-03-11T18:36:06.408104+0000","last_fullsized":"2024-03-11T18:36:06.408104+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:36:26.132020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.353155+0000","last_change":"2024-03-11T18:35:57.687311+0000","last_active":"2024-03-11T18:36:06.353155+0000","last_peered":"2024-03-11T18:36:06.353155+0000","last_clean":"2024-03-11T18:36:06.353155+0000","last_became_active":"2024-03-11T18:35:57.686850+0000","last_became_peered":"2024-03-11T18:35:57.686850+0000","last_unstale":"2024-03-11T18:36:06.353155+0000","last_undegraded":"2024-03-11T18:36:06.353155+0000","last_fullsized":"2024-03-11T18:36:06.353155+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:29:45.662029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"28'10","reported_seq":37,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:37.713866+0000","last_change":"2024-03-11T18:35:56.291232+0000","last_active":"2024-03-11T18:36:37.713866+0000","last_peered":"2024-03-11T18:36:37.713866+0000","last_clean":"2024-03-11T18:36:37.713866+0000","last_became_active":"2024-03-11T18:35:56.290944+0000","last_became_peered":"2024-03-11T18:35:56.290944+0000","last_unstale":"2024-03-11T18:36:37.713866+0000","last_undegraded":"2024-03-11T18:36:37.713866+0000","last_fullsized":"2024-03-11T18:36:37.713866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:49:27.827765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.405992+0000","last_change":"2024-03-11T18:36:04.602142+0000","last_active":"2024-03-11T18:36:06.405992+0000","last_peered":"2024-03-11T18:36:06.405992+0000","last_clean":"2024-03-11T18:36:06.405992+0000","last_became_active":"2024-03-11T18:36:04.601224+0000","last_became_peered":"2024-03-11T18:36:04.601224+0000","last_unstale":"2024-03-11T18:36:06.405992+0000","last_undegraded":"2024-03-11T18:36:06.405992+0000","last_fullsized":"2024-03-11T18:36:06.405992+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:17:42.075355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.409133+0000","last_change":"2024-03-11T18:35:57.611305+0000","last_active":"2024-03-11T18:36:06.409133+0000","last_peered":"2024-03-11T18:36:06.409133+0000","last_clean":"2024-03-11T18:36:06.409133+0000","last_became_active":"2024-03-11T18:35:57.610955+0000","last_became_peered":"2024-03-11T18:35:57.610955+0000","last_unstale":"2024-03-11T18:36:06.409133+0000","last_undegraded":"2024-03-11T18:36:06.409133+0000","last_fullsized":"2024-03-11T18:36:06.409133+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:31:19.941345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.410833+0000","last_change":"2024-03-11T18:35:56.361952+0000","last_active":"2024-03-11T18:36:06.410833+0000","last_peered":"2024-03-11T18:36:06.410833+0000","last_clean":"2024-03-11T18:36:06.410833+0000","last_became_active":"2024-03-11T18:35:56.361141+0000","last_became_peered":"2024-03-11T18:35:56.361141+0000","last_unstale":"2024-03-11T18:36:06.410833+0000","last_undegraded":"2024-03-11T18:36:06.410833+0000","last_fullsized":"2024-03-11T18:36:06.410833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:04:13.101160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.405270+0000","last_change":"2024-03-11T18:36:04.593604+0000","last_active":"2024-03-11T18:36:06.405270+0000","last_peered":"2024-03-11T18:36:06.405270+0000","last_clean":"2024-03-11T18:36:06.405270+0000","last_became_active":"2024-03-11T18:36:04.592304+0000","last_became_peered":"2024-03-11T18:36:04.592304+0000","last_unstale":"2024-03-11T18:36:06.405270+0000","last_undegraded":"2024-03-11T18:36:06.405270+0000","last_fullsized":"2024-03-11T18:36:06.405270+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:31:04.738940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.353525+0000","last_change":"2024-03-11T18:35:57.689404+0000","last_active":"2024-03-11T18:36:06.353525+0000","last_peered":"2024-03-11T18:36:06.353525+0000","last_clean":"2024-03-11T18:36:06.353525+0000","last_became_active":"2024-03-11T18:35:57.689139+0000","last_became_peered":"2024-03-11T18:35:57.689139+0000","last_unstale":"2024-03-11T18:36:06.353525+0000","last_undegraded":"2024-03-11T18:36:06.353525+0000","last_fullsized":"2024-03-11T18:36:06.353525+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:54:09.699371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.409872+0000","last_change":"2024-03-11T18:35:56.297682+0000","last_active":"2024-03-11T18:36:06.409872+0000","last_peered":"2024-03-11T18:36:06.409872+0000","last_clean":"2024-03-11T18:36:06.409872+0000","last_became_active":"2024-03-11T18:35:56.296426+0000","last_became_peered":"2024-03-11T18:35:56.296426+0000","last_unstale":"2024-03-11T18:36:06.409872+0000","last_undegraded":"2024-03-11T18:36:06.409872+0000","last_fullsized":"2024-03-11T18:36:06.409872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:28:50.050942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.403363+0000","last_change":"2024-03-11T18:36:04.590780+0000","last_active":"2024-03-11T18:36:06.403363+0000","last_peered":"2024-03-11T18:36:06.403363+0000","last_clean":"2024-03-11T18:36:06.403363+0000","last_became_active":"2024-03-11T18:36:04.590014+0000","last_became_peered":"2024-03-11T18:36:04.590014+0000","last_unstale":"2024-03-11T18:36:06.403363+0000","last_undegraded":"2024-03-11T18:36:06.403363+0000","last_fullsized":"2024-03-11T18:36:06.403363+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:51:13.205384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.407150+0000","last_change":"2024-03-11T18:35:57.671638+0000","last_active":"2024-03-11T18:36:06.407150+0000","last_peered":"2024-03-11T18:36:06.407150+0000","last_clean":"2024-03-11T18:36:06.407150+0000","last_became_active":"2024-03-11T18:35:57.670893+0000","last_became_peered":"2024-03-11T18:35:57.670893+0000","last_unstale":"2024-03-11T18:36:06.407150+0000","last_undegraded":"2024-03-11T18:36:06.407150+0000","last_fullsized":"2024-03-11T18:36:06.407150+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:31:21.395133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"28'4","reported_seq":31,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:37.719433+0000","last_change":"2024-03-11T18:35:56.251580+0000","last_active":"2024-03-11T18:36:37.719433+0000","last_peered":"2024-03-11T18:36:37.719433+0000","last_clean":"2024-03-11T18:36:37.719433+0000","last_became_active":"2024-03-11T18:35:56.251287+0000","last_became_peered":"2024-03-11T18:35:56.251287+0000","last_unstale":"2024-03-11T18:36:37.719433+0000","last_undegraded":"2024-03-11T18:36:37.719433+0000","last_fullsized":"2024-03-11T18:36:37.719433+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:30:39.907360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.412899+0000","last_change":"2024-03-11T18:36:04.625563+0000","last_active":"2024-03-11T18:36:06.412899+0000","last_peered":"2024-03-11T18:36:06.412899+0000","last_clean":"2024-03-11T18:36:06.412899+0000","last_became_active":"2024-03-11T18:36:04.618911+0000","last_became_peered":"2024-03-11T18:36:04.618911+0000","last_unstale":"2024-03-11T18:36:06.412899+0000","last_undegraded":"2024-03-11T18:36:06.412899+0000","last_fullsized":"2024-03-11T18:36:06.412899+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:31:04.738940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.354510+0000","last_change":"2024-03-11T18:35:57.697641+0000","last_active":"2024-03-11T18:36:06.354510+0000","last_peered":"2024-03-11T18:36:06.354510+0000","last_clean":"2024-03-11T18:36:06.354510+0000","last_became_active":"2024-03-11T18:35:57.697263+0000","last_became_peered":"2024-03-11T18:35:57.697263+0000","last_unstale":"2024-03-11T18:36:06.354510+0000","last_undegraded":"2024-03-11T18:36:06.354510+0000","last_fullsized":"2024-03-11T18:36:06.354510+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:34:39.325572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"28'10","reported_seq":37,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:15.644151+0000","last_change":"2024-03-11T18:35:56.355612+0000","last_active":"2024-03-11T18:36:15.644151+0000","last_peered":"2024-03-11T18:36:15.644151+0000","last_clean":"2024-03-11T18:36:15.644151+0000","last_became_active":"2024-03-11T18:35:56.354827+0000","last_became_peered":"2024-03-11T18:35:56.354827+0000","last_unstale":"2024-03-11T18:36:15.644151+0000","last_undegraded":"2024-03-11T18:36:15.644151+0000","last_fullsized":"2024-03-11T18:36:15.644151+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:06:02.844765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.358459+0000","last_change":"2024-03-11T18:36:04.617561+0000","last_active":"2024-03-11T18:36:06.358459+0000","last_peered":"2024-03-11T18:36:06.358459+0000","last_clean":"2024-03-11T18:36:06.358459+0000","last_became_active":"2024-03-11T18:36:04.616938+0000","last_became_peered":"2024-03-11T18:36:04.616938+0000","last_unstale":"2024-03-11T18:36:06.358459+0000","last_undegraded":"2024-03-11T18:36:06.358459+0000","last_fullsized":"2024-03-11T18:36:06.358459+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:16:10.508913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.412500+0000","last_change":"2024-03-11T18:35:57.679035+0000","last_active":"2024-03-11T18:36:06.412500+0000","last_peered":"2024-03-11T18:36:06.412500+0000","last_clean":"2024-03-11T18:36:06.412500+0000","last_became_active":"2024-03-11T18:35:57.678565+0000","last_became_peered":"2024-03-11T18:35:57.678565+0000","last_unstale":"2024-03-11T18:36:06.412500+0000","last_undegraded":"2024-03-11T18:36:06.412500+0000","last_fullsized":"2024-03-11T18:36:06.412500+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:18:05.980207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"28'4","reported_seq":31,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.358900+0000","last_change":"2024-03-11T18:35:56.288132+0000","last_active":"2024-03-11T18:36:06.358900+0000","last_peered":"2024-03-11T18:36:06.358900+0000","last_clean":"2024-03-11T18:36:06.358900+0000","last_became_active":"2024-03-11T18:35:56.287846+0000","last_became_peered":"2024-03-11T18:35:56.287846+0000","last_unstale":"2024-03-11T18:36:06.358900+0000","last_undegraded":"2024-03-11T18:36:06.358900+0000","last_fullsized":"2024-03-11T18:36:06.358900+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T18:51:00.795706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.411589+0000","last_change":"2024-03-11T18:36:04.643164+0000","last_active":"2024-03-11T18:36:06.411589+0000","last_peered":"2024-03-11T18:36:06.411589+0000","last_clean":"2024-03-11T18:36:06.411589+0000","last_became_active":"2024-03-11T18:36:04.642374+0000","last_became_peered":"2024-03-11T18:36:04.642374+0000","last_unstale":"2024-03-11T18:36:06.411589+0000","last_undegraded":"2024-03-11T18:36:06.411589+0000","last_fullsized":"2024-03-11T18:36:06.411589+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:25:56.815496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.405796+0000","last_change":"2024-03-11T18:35:57.636550+0000","last_active":"2024-03-11T18:36:06.405796+0000","last_peered":"2024-03-11T18:36:06.405796+0000","last_clean":"2024-03-11T18:36:06.405796+0000","last_became_active":"2024-03-11T18:35:57.636093+0000","last_became_peered":"2024-03-11T18:35:57.636093+0000","last_unstale":"2024-03-11T18:36:06.405796+0000","last_undegraded":"2024-03-11T18:36:06.405796+0000","last_fullsized":"2024-03-11T18:36:06.405796+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:33:31.344465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.413301+0000","last_change":"2024-03-11T18:35:56.355480+0000","last_active":"2024-03-11T18:36:06.413301+0000","last_peered":"2024-03-11T18:36:06.413301+0000","last_clean":"2024-03-11T18:36:06.413301+0000","last_became_active":"2024-03-11T18:35:56.354759+0000","last_became_peered":"2024-03-11T18:35:56.354759+0000","last_unstale":"2024-03-11T18:36:06.413301+0000","last_undegraded":"2024-03-11T18:36:06.413301+0000","last_fullsized":"2024-03-11T18:36:06.413301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:45:37.842725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.400682+0000","last_change":"2024-03-11T18:36:04.617364+0000","last_active":"2024-03-11T18:36:06.400682+0000","last_peered":"2024-03-11T18:36:06.400682+0000","last_clean":"2024-03-11T18:36:06.400682+0000","last_became_active":"2024-03-11T18:36:04.613159+0000","last_became_peered":"2024-03-11T18:36:04.613159+0000","last_unstale":"2024-03-11T18:36:06.400682+0000","last_undegraded":"2024-03-11T18:36:06.400682+0000","last_fullsized":"2024-03-11T18:36:06.400682+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:25:23.672189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.412882+0000","last_change":"2024-03-11T18:35:57.683426+0000","last_active":"2024-03-11T18:36:06.412882+0000","last_peered":"2024-03-11T18:36:06.412882+0000","last_clean":"2024-03-11T18:36:06.412882+0000","last_became_active":"2024-03-11T18:35:57.683150+0000","last_became_peered":"2024-03-11T18:35:57.683150+0000","last_unstale":"2024-03-11T18:36:06.412882+0000","last_undegraded":"2024-03-11T18:36:06.412882+0000","last_fullsized":"2024-03-11T18:36:06.412882+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:36:00.079075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.411507+0000","last_change":"2024-03-11T18:35:56.361613+0000","last_active":"2024-03-11T18:36:06.411507+0000","last_peered":"2024-03-11T18:36:06.411507+0000","last_clean":"2024-03-11T18:36:06.411507+0000","last_became_active":"2024-03-11T18:35:56.359629+0000","last_became_peered":"2024-03-11T18:35:56.359629+0000","last_unstale":"2024-03-11T18:36:06.411507+0000","last_undegraded":"2024-03-11T18:36:06.411507+0000","last_fullsized":"2024-03-11T18:36:06.411507+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:03:06.585728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.401756+0000","last_change":"2024-03-11T18:36:04.608168+0000","last_active":"2024-03-11T18:36:06.401756+0000","last_peered":"2024-03-11T18:36:06.401756+0000","last_clean":"2024-03-11T18:36:06.401756+0000","last_became_active":"2024-03-11T18:36:04.607063+0000","last_became_peered":"2024-03-11T18:36:04.607063+0000","last_unstale":"2024-03-11T18:36:06.401756+0000","last_undegraded":"2024-03-11T18:36:06.401756+0000","last_fullsized":"2024-03-11T18:36:06.401756+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:53:41.996024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.405114+0000","last_change":"2024-03-11T18:36:04.607992+0000","last_active":"2024-03-11T18:36:06.405114+0000","last_peered":"2024-03-11T18:36:06.405114+0000","last_clean":"2024-03-11T18:36:06.405114+0000","last_became_active":"2024-03-11T18:36:04.607109+0000","last_became_peered":"2024-03-11T18:36:04.607109+0000","last_unstale":"2024-03-11T18:36:06.405114+0000","last_undegraded":"2024-03-11T18:36:06.405114+0000","last_fullsized":"2024-03-11T18:36:06.405114+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:14:37.678854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.357424+0000","last_change":"2024-03-11T18:35:57.680534+0000","last_active":"2024-03-11T18:36:06.357424+0000","last_peered":"2024-03-11T18:36:06.357424+0000","last_clean":"2024-03-11T18:36:06.357424+0000","last_became_active":"2024-03-11T18:35:57.679642+0000","last_became_peered":"2024-03-11T18:35:57.679642+0000","last_unstale":"2024-03-11T18:36:06.357424+0000","last_undegraded":"2024-03-11T18:36:06.357424+0000","last_fullsized":"2024-03-11T18:36:06.357424+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:25:49.707568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.409300+0000","last_change":"2024-03-11T18:35:56.303510+0000","last_active":"2024-03-11T18:36:06.409300+0000","last_peered":"2024-03-11T18:36:06.409300+0000","last_clean":"2024-03-11T18:36:06.409300+0000","last_became_active":"2024-03-11T18:35:56.302758+0000","last_became_peered":"2024-03-11T18:35:56.302758+0000","last_unstale":"2024-03-11T18:36:06.409300+0000","last_undegraded":"2024-03-11T18:36:06.409300+0000","last_fullsized":"2024-03-11T18:36:06.409300+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:03:32.297342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.412847+0000","last_change":"2024-03-11T18:36:04.618818+0000","last_active":"2024-03-11T18:36:06.412847+0000","last_peered":"2024-03-11T18:36:06.412847+0000","last_clean":"2024-03-11T18:36:06.412847+0000","last_became_active":"2024-03-11T18:36:04.617868+0000","last_became_peered":"2024-03-11T18:36:04.617868+0000","last_unstale":"2024-03-11T18:36:06.412847+0000","last_undegraded":"2024-03-11T18:36:06.412847+0000","last_fullsized":"2024-03-11T18:36:06.412847+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:16:10.508913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.400811+0000","last_change":"2024-03-11T18:35:57.690347+0000","last_active":"2024-03-11T18:36:06.400811+0000","last_peered":"2024-03-11T18:36:06.400811+0000","last_clean":"2024-03-11T18:36:06.400811+0000","last_became_active":"2024-03-11T18:35:57.689146+0000","last_became_peered":"2024-03-11T18:35:57.689146+0000","last_unstale":"2024-03-11T18:36:06.400811+0000","last_undegraded":"2024-03-11T18:36:06.400811+0000","last_fullsized":"2024-03-11T18:36:06.400811+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:03:56.260394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.402147+0000","last_change":"2024-03-11T18:35:56.348355+0000","last_active":"2024-03-11T18:36:06.402147+0000","last_peered":"2024-03-11T18:36:06.402147+0000","last_clean":"2024-03-11T18:36:06.402147+0000","last_became_active":"2024-03-11T18:35:56.347631+0000","last_became_peered":"2024-03-11T18:35:56.347631+0000","last_unstale":"2024-03-11T18:36:06.402147+0000","last_undegraded":"2024-03-11T18:36:06.402147+0000","last_fullsized":"2024-03-11T18:36:06.402147+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:06:02.844765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"23'1","reported_seq":28,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.410764+0000","last_change":"2024-03-11T18:35:56.361254+0000","last_active":"2024-03-11T18:36:06.410764+0000","last_peered":"2024-03-11T18:36:06.410764+0000","last_clean":"2024-03-11T18:36:06.410764+0000","last_became_active":"2024-03-11T18:35:56.360580+0000","last_became_peered":"2024-03-11T18:35:56.360580+0000","last_unstale":"2024-03-11T18:36:06.410764+0000","last_undegraded":"2024-03-11T18:36:06.410764+0000","last_fullsized":"2024-03-11T18:36:06.410764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:27:26.471550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.406163+0000","last_change":"2024-03-11T18:35:57.602778+0000","last_active":"2024-03-11T18:36:06.406163+0000","last_peered":"2024-03-11T18:36:06.406163+0000","last_clean":"2024-03-11T18:36:06.406163+0000","last_became_active":"2024-03-11T18:35:57.601990+0000","last_became_peered":"2024-03-11T18:35:57.601990+0000","last_unstale":"2024-03-11T18:36:06.406163+0000","last_undegraded":"2024-03-11T18:36:06.406163+0000","last_fullsized":"2024-03-11T18:36:06.406163+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:56:59.014871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.401528+0000","last_change":"2024-03-11T18:36:04.606098+0000","last_active":"2024-03-11T18:36:06.401528+0000","last_peered":"2024-03-11T18:36:06.401528+0000","last_clean":"2024-03-11T18:36:06.401528+0000","last_became_active":"2024-03-11T18:36:04.604823+0000","last_became_peered":"2024-03-11T18:36:04.604823+0000","last_unstale":"2024-03-11T18:36:06.401528+0000","last_undegraded":"2024-03-11T18:36:06.401528+0000","last_fullsized":"2024-03-11T18:36:06.401528+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:52:04.862452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"28'5","reported_seq":32,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:37.941879+0000","last_change":"2024-03-11T18:35:56.296312+0000","last_active":"2024-03-11T18:36:37.941879+0000","last_peered":"2024-03-11T18:36:37.941879+0000","last_clean":"2024-03-11T18:36:37.941879+0000","last_became_active":"2024-03-11T18:35:56.296090+0000","last_became_peered":"2024-03-11T18:35:56.296090+0000","last_unstale":"2024-03-11T18:36:37.941879+0000","last_undegraded":"2024-03-11T18:36:37.941879+0000","last_fullsized":"2024-03-11T18:36:37.941879+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:03:06.585728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.410741+0000","last_change":"2024-03-11T18:35:57.611809+0000","last_active":"2024-03-11T18:36:06.410741+0000","last_peered":"2024-03-11T18:36:06.410741+0000","last_clean":"2024-03-11T18:36:06.410741+0000","last_became_active":"2024-03-11T18:35:57.611097+0000","last_became_peered":"2024-03-11T18:35:57.611097+0000","last_unstale":"2024-03-11T18:36:06.410741+0000","last_undegraded":"2024-03-11T18:36:06.410741+0000","last_fullsized":"2024-03-11T18:36:06.410741+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:49:29.042114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.403540+0000","last_change":"2024-03-11T18:36:04.590064+0000","last_active":"2024-03-11T18:36:06.403540+0000","last_peered":"2024-03-11T18:36:06.403540+0000","last_clean":"2024-03-11T18:36:06.403540+0000","last_became_active":"2024-03-11T18:36:04.589037+0000","last_became_peered":"2024-03-11T18:36:04.589037+0000","last_unstale":"2024-03-11T18:36:06.403540+0000","last_undegraded":"2024-03-11T18:36:06.403540+0000","last_fullsized":"2024-03-11T18:36:06.403540+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:51:28.017349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.407866+0000","last_change":"2024-03-11T18:35:57.614944+0000","last_active":"2024-03-11T18:36:06.407866+0000","last_peered":"2024-03-11T18:36:06.407866+0000","last_clean":"2024-03-11T18:36:06.407866+0000","last_became_active":"2024-03-11T18:35:57.614144+0000","last_became_peered":"2024-03-11T18:35:57.614144+0000","last_unstale":"2024-03-11T18:36:06.407866+0000","last_undegraded":"2024-03-11T18:36:06.407866+0000","last_fullsized":"2024-03-11T18:36:06.407866+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:53:26.493265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.400811+0000","last_change":"2024-03-11T18:35:57.630211+0000","last_active":"2024-03-11T18:36:06.400811+0000","last_peered":"2024-03-11T18:36:06.400811+0000","last_clean":"2024-03-11T18:36:06.400811+0000","last_became_active":"2024-03-11T18:35:57.629599+0000","last_became_peered":"2024-03-11T18:35:57.629599+0000","last_unstale":"2024-03-11T18:36:06.400811+0000","last_undegraded":"2024-03-11T18:36:06.400811+0000","last_fullsized":"2024-03-11T18:36:06.400811+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:18:05.980207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.409629+0000","last_change":"2024-03-11T18:35:57.688700+0000","last_active":"2024-03-11T18:36:06.409629+0000","last_peered":"2024-03-11T18:36:06.409629+0000","last_clean":"2024-03-11T18:36:06.409629+0000","last_became_active":"2024-03-11T18:35:57.687968+0000","last_became_peered":"2024-03-11T18:35:57.687968+0000","last_unstale":"2024-03-11T18:36:06.409629+0000","last_undegraded":"2024-03-11T18:36:06.409629+0000","last_fullsized":"2024-03-11T18:36:06.409629+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:25:16.564261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.405998+0000","last_change":"2024-03-11T18:35:57.680730+0000","last_active":"2024-03-11T18:36:06.405998+0000","last_peered":"2024-03-11T18:36:06.405998+0000","last_clean":"2024-03-11T18:36:06.405998+0000","last_became_active":"2024-03-11T18:35:57.679935+0000","last_became_peered":"2024-03-11T18:35:57.679935+0000","last_unstale":"2024-03-11T18:36:06.405998+0000","last_undegraded":"2024-03-11T18:36:06.405998+0000","last_fullsized":"2024-03-11T18:36:06.405998+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:21:21.894805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.403523+0000","last_change":"2024-03-11T18:35:57.690799+0000","last_active":"2024-03-11T18:36:06.403523+0000","last_peered":"2024-03-11T18:36:06.403523+0000","last_clean":"2024-03-11T18:36:06.403523+0000","last_became_active":"2024-03-11T18:35:57.686623+0000","last_became_peered":"2024-03-11T18:35:57.686623+0000","last_unstale":"2024-03-11T18:36:06.403523+0000","last_undegraded":"2024-03-11T18:36:06.403523+0000","last_fullsized":"2024-03-11T18:36:06.403523+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:53:30.324898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.358254+0000","last_change":"2024-03-11T18:35:57.691955+0000","last_active":"2024-03-11T18:36:06.358254+0000","last_peered":"2024-03-11T18:36:06.358254+0000","last_clean":"2024-03-11T18:36:06.358254+0000","last_became_active":"2024-03-11T18:35:57.691413+0000","last_became_peered":"2024-03-11T18:35:57.691413+0000","last_unstale":"2024-03-11T18:36:06.358254+0000","last_undegraded":"2024-03-11T18:36:06.358254+0000","last_fullsized":"2024-03-11T18:36:06.358254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:31:19.941345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.410285+0000","last_change":"2024-03-11T18:35:57.684695+0000","last_active":"2024-03-11T18:36:06.410285+0000","last_peered":"2024-03-11T18:36:06.410285+0000","last_clean":"2024-03-11T18:36:06.410285+0000","last_became_active":"2024-03-11T18:35:57.684227+0000","last_became_peered":"2024-03-11T18:35:57.684227+0000","last_unstale":"2024-03-11T18:36:06.410285+0000","last_undegraded":"2024-03-11T18:36:06.410285+0000","last_fullsized":"2024-03-11T18:36:06.410285+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:28:51.265291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.405217+0000","last_change":"2024-03-11T18:35:57.683549+0000","last_active":"2024-03-11T18:36:06.405217+0000","last_peered":"2024-03-11T18:36:06.405217+0000","last_clean":"2024-03-11T18:36:06.405217+0000","last_became_active":"2024-03-11T18:35:57.680610+0000","last_became_peered":"2024-03-11T18:35:57.680610+0000","last_unstale":"2024-03-11T18:36:06.405217+0000","last_undegraded":"2024-03-11T18:36:06.405217+0000","last_fullsized":"2024-03-11T18:36:06.405217+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:25:52.716744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.409054+0000","last_change":"2024-03-11T18:35:57.686833+0000","last_active":"2024-03-11T18:36:06.409054+0000","last_peered":"2024-03-11T18:36:06.409054+0000","last_clean":"2024-03-11T18:36:06.409054+0000","last_became_active":"2024-03-11T18:35:57.686144+0000","last_became_peered":"2024-03-11T18:35:57.686144+0000","last_unstale":"2024-03-11T18:36:06.409054+0000","last_undegraded":"2024-03-11T18:36:06.409054+0000","last_fullsized":"2024-03-11T18:36:06.409054+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:16:03.400985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.401952+0000","last_change":"2024-03-11T18:35:57.681630+0000","last_active":"2024-03-11T18:36:06.401952+0000","last_peered":"2024-03-11T18:36:06.401952+0000","last_clean":"2024-03-11T18:36:06.401952+0000","last_became_active":"2024-03-11T18:35:57.678876+0000","last_became_peered":"2024-03-11T18:35:57.678876+0000","last_unstale":"2024-03-11T18:36:06.401952+0000","last_undegraded":"2024-03-11T18:36:06.401952+0000","last_fullsized":"2024-03-11T18:36:06.401952+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:03:33.511691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.409370+0000","last_change":"2024-03-11T18:35:57.679589+0000","last_active":"2024-03-11T18:36:06.409370+0000","last_peered":"2024-03-11T18:36:06.409370+0000","last_clean":"2024-03-11T18:36:06.409370+0000","last_became_active":"2024-03-11T18:35:57.678838+0000","last_became_peered":"2024-03-11T18:35:57.678838+0000","last_unstale":"2024-03-11T18:36:06.409370+0000","last_undegraded":"2024-03-11T18:36:06.409370+0000","last_fullsized":"2024-03-11T18:36:06.409370+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:54:09.699371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.404736+0000","last_change":"2024-03-11T18:35:57.674571+0000","last_active":"2024-03-11T18:36:06.404736+0000","last_peered":"2024-03-11T18:36:06.404736+0000","last_clean":"2024-03-11T18:36:06.404736+0000","last_became_active":"2024-03-11T18:35:57.671573+0000","last_became_peered":"2024-03-11T18:35:57.671573+0000","last_unstale":"2024-03-11T18:36:06.404736+0000","last_undegraded":"2024-03-11T18:36:06.404736+0000","last_fullsized":"2024-03-11T18:36:06.404736+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:35:10.403666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.410712+0000","last_change":"2024-03-11T18:35:57.615939+0000","last_active":"2024-03-11T18:36:06.410712+0000","last_peered":"2024-03-11T18:36:06.410712+0000","last_clean":"2024-03-11T18:36:06.410712+0000","last_became_active":"2024-03-11T18:35:57.615579+0000","last_became_peered":"2024-03-11T18:35:57.615579+0000","last_unstale":"2024-03-11T18:36:06.410712+0000","last_undegraded":"2024-03-11T18:36:06.410712+0000","last_fullsized":"2024-03-11T18:36:06.410712+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:03:33.511691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.355121+0000","last_change":"2024-03-11T18:35:57.688955+0000","last_active":"2024-03-11T18:36:06.355121+0000","last_peered":"2024-03-11T18:36:06.355121+0000","last_clean":"2024-03-11T18:36:06.355121+0000","last_became_active":"2024-03-11T18:35:57.688365+0000","last_became_peered":"2024-03-11T18:35:57.688365+0000","last_unstale":"2024-03-11T18:36:06.355121+0000","last_undegraded":"2024-03-11T18:36:06.355121+0000","last_fullsized":"2024-03-11T18:36:06.355121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:44:11.877541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.401903+0000","last_change":"2024-03-11T18:35:57.602044+0000","last_active":"2024-03-11T18:36:06.401903+0000","last_peered":"2024-03-11T18:36:06.401903+0000","last_clean":"2024-03-11T18:36:06.401903+0000","last_became_active":"2024-03-11T18:35:57.601334+0000","last_became_peered":"2024-03-11T18:35:57.601334+0000","last_unstale":"2024-03-11T18:36:06.401903+0000","last_undegraded":"2024-03-11T18:36:06.401903+0000","last_fullsized":"2024-03-11T18:36:06.401903+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:39:12.693619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.413873+0000","last_change":"2024-03-11T18:35:57.677114+0000","last_active":"2024-03-11T18:36:06.413873+0000","last_peered":"2024-03-11T18:36:06.413873+0000","last_clean":"2024-03-11T18:36:06.413873+0000","last_became_active":"2024-03-11T18:35:57.676075+0000","last_became_peered":"2024-03-11T18:35:57.676075+0000","last_unstale":"2024-03-11T18:36:06.413873+0000","last_undegraded":"2024-03-11T18:36:06.413873+0000","last_fullsized":"2024-03-11T18:36:06.413873+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:44:11.877541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.408799+0000","last_change":"2024-03-11T18:35:57.641392+0000","last_active":"2024-03-11T18:36:06.408799+0000","last_peered":"2024-03-11T18:36:06.408799+0000","last_clean":"2024-03-11T18:36:06.408799+0000","last_became_active":"2024-03-11T18:35:57.640934+0000","last_became_peered":"2024-03-11T18:35:57.640934+0000","last_unstale":"2024-03-11T18:36:06.408799+0000","last_undegraded":"2024-03-11T18:36:06.408799+0000","last_fullsized":"2024-03-11T18:36:06.408799+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:25:52.716744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.357659+0000","last_change":"2024-03-11T18:35:57.605291+0000","last_active":"2024-03-11T18:36:06.357659+0000","last_peered":"2024-03-11T18:36:06.357659+0000","last_clean":"2024-03-11T18:36:06.357659+0000","last_became_active":"2024-03-11T18:35:57.604645+0000","last_became_peered":"2024-03-11T18:35:57.604645+0000","last_unstale":"2024-03-11T18:36:06.357659+0000","last_undegraded":"2024-03-11T18:36:06.357659+0000","last_fullsized":"2024-03-11T18:36:06.357659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:03:33.511691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.404894+0000","last_change":"2024-03-11T18:35:57.503994+0000","last_active":"2024-03-11T18:36:06.404894+0000","last_peered":"2024-03-11T18:36:06.404894+0000","last_clean":"2024-03-11T18:36:06.404894+0000","last_became_active":"2024-03-11T18:35:57.503362+0000","last_became_peered":"2024-03-11T18:35:57.503362+0000","last_unstale":"2024-03-11T18:36:06.404894+0000","last_undegraded":"2024-03-11T18:36:06.404894+0000","last_fullsized":"2024-03-11T18:36:06.404894+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:03:58.875164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.402309+0000","last_change":"2024-03-11T18:35:57.679571+0000","last_active":"2024-03-11T18:36:06.402309+0000","last_peered":"2024-03-11T18:36:06.402309+0000","last_clean":"2024-03-11T18:36:06.402309+0000","last_became_active":"2024-03-11T18:35:57.678787+0000","last_became_peered":"2024-03-11T18:35:57.678787+0000","last_unstale":"2024-03-11T18:36:06.402309+0000","last_undegraded":"2024-03-11T18:36:06.402309+0000","last_fullsized":"2024-03-11T18:36:06.402309+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:13:40.392038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.412355+0000","last_change":"2024-03-11T18:35:57.613374+0000","last_active":"2024-03-11T18:36:06.412355+0000","last_peered":"2024-03-11T18:36:06.412355+0000","last_clean":"2024-03-11T18:36:06.412355+0000","last_became_active":"2024-03-11T18:35:57.613050+0000","last_became_peered":"2024-03-11T18:35:57.613050+0000","last_unstale":"2024-03-11T18:36:06.412355+0000","last_undegraded":"2024-03-11T18:36:06.412355+0000","last_fullsized":"2024-03-11T18:36:06.412355+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:50:39.923881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.411942+0000","last_change":"2024-03-11T18:35:57.636565+0000","last_active":"2024-03-11T18:36:06.411942+0000","last_peered":"2024-03-11T18:36:06.411942+0000","last_clean":"2024-03-11T18:36:06.411942+0000","last_became_active":"2024-03-11T18:35:57.635978+0000","last_became_peered":"2024-03-11T18:35:57.635978+0000","last_unstale":"2024-03-11T18:36:06.411942+0000","last_undegraded":"2024-03-11T18:36:06.411942+0000","last_fullsized":"2024-03-11T18:36:06.411942+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:03:56.260394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.410011+0000","last_change":"2024-03-11T18:35:57.641714+0000","last_active":"2024-03-11T18:36:06.410011+0000","last_peered":"2024-03-11T18:36:06.410011+0000","last_clean":"2024-03-11T18:36:06.410011+0000","last_became_active":"2024-03-11T18:35:57.641417+0000","last_became_peered":"2024-03-11T18:35:57.641417+0000","last_unstale":"2024-03-11T18:36:06.410011+0000","last_undegraded":"2024-03-11T18:36:06.410011+0000","last_fullsized":"2024-03-11T18:36:06.410011+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:25:49.707568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.354140+0000","last_change":"2024-03-11T18:35:57.689703+0000","last_active":"2024-03-11T18:36:06.354140+0000","last_peered":"2024-03-11T18:36:06.354140+0000","last_clean":"2024-03-11T18:36:06.354140+0000","last_became_active":"2024-03-11T18:35:57.689056+0000","last_became_peered":"2024-03-11T18:35:57.689056+0000","last_unstale":"2024-03-11T18:36:06.354140+0000","last_undegraded":"2024-03-11T18:36:06.354140+0000","last_fullsized":"2024-03-11T18:36:06.354140+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:41:42.263652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.400467+0000","last_change":"2024-03-11T18:35:57.684305+0000","last_active":"2024-03-11T18:36:06.400467+0000","last_peered":"2024-03-11T18:36:06.400467+0000","last_clean":"2024-03-11T18:36:06.400467+0000","last_became_active":"2024-03-11T18:35:57.682947+0000","last_became_peered":"2024-03-11T18:35:57.682947+0000","last_unstale":"2024-03-11T18:36:06.400467+0000","last_undegraded":"2024-03-11T18:36:06.400467+0000","last_fullsized":"2024-03-11T18:36:06.400467+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:53:26.493265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.352832+0000","last_change":"2024-03-11T18:35:57.692717+0000","last_active":"2024-03-11T18:36:06.352832+0000","last_peered":"2024-03-11T18:36:06.352832+0000","last_clean":"2024-03-11T18:36:06.352832+0000","last_became_active":"2024-03-11T18:35:57.692504+0000","last_became_peered":"2024-03-11T18:35:57.692504+0000","last_unstale":"2024-03-11T18:36:06.352832+0000","last_undegraded":"2024-03-11T18:36:06.352832+0000","last_fullsized":"2024-03-11T18:36:06.352832+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:25:52.716744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.409110+0000","last_change":"2024-03-11T18:35:57.640483+0000","last_active":"2024-03-11T18:36:06.409110+0000","last_peered":"2024-03-11T18:36:06.409110+0000","last_clean":"2024-03-11T18:36:06.409110+0000","last_became_active":"2024-03-11T18:35:57.639886+0000","last_became_peered":"2024-03-11T18:35:57.639886+0000","last_unstale":"2024-03-11T18:36:06.409110+0000","last_undegraded":"2024-03-11T18:36:06.409110+0000","last_fullsized":"2024-03-11T18:36:06.409110+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:34:39.325572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.352837+0000","last_change":"2024-03-11T18:35:57.687096+0000","last_active":"2024-03-11T18:36:06.352837+0000","last_peered":"2024-03-11T18:36:06.352837+0000","last_clean":"2024-03-11T18:36:06.352837+0000","last_became_active":"2024-03-11T18:35:57.686771+0000","last_became_peered":"2024-03-11T18:35:57.686771+0000","last_unstale":"2024-03-11T18:36:06.352837+0000","last_undegraded":"2024-03-11T18:36:06.352837+0000","last_fullsized":"2024-03-11T18:36:06.352837+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:16:03.400985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.409284+0000","last_change":"2024-03-11T18:35:57.694707+0000","last_active":"2024-03-11T18:36:06.409284+0000","last_peered":"2024-03-11T18:36:06.409284+0000","last_clean":"2024-03-11T18:36:06.409284+0000","last_became_active":"2024-03-11T18:35:57.694104+0000","last_became_peered":"2024-03-11T18:35:57.694104+0000","last_unstale":"2024-03-11T18:36:06.409284+0000","last_undegraded":"2024-03-11T18:36:06.409284+0000","last_fullsized":"2024-03-11T18:36:06.409284+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:47:28.579467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.405456+0000","last_change":"2024-03-11T18:35:57.538548+0000","last_active":"2024-03-11T18:36:06.405456+0000","last_peered":"2024-03-11T18:36:06.405456+0000","last_clean":"2024-03-11T18:36:06.405456+0000","last_became_active":"2024-03-11T18:35:57.537704+0000","last_became_peered":"2024-03-11T18:35:57.537704+0000","last_unstale":"2024-03-11T18:36:06.405456+0000","last_undegraded":"2024-03-11T18:36:06.405456+0000","last_fullsized":"2024-03-11T18:36:06.405456+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:40:52.251981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.359086+0000","last_change":"2024-03-11T18:35:57.507126+0000","last_active":"2024-03-11T18:36:06.359086+0000","last_peered":"2024-03-11T18:36:06.359086+0000","last_clean":"2024-03-11T18:36:06.359086+0000","last_became_active":"2024-03-11T18:35:57.506429+0000","last_became_peered":"2024-03-11T18:35:57.506429+0000","last_unstale":"2024-03-11T18:36:06.359086+0000","last_undegraded":"2024-03-11T18:36:06.359086+0000","last_fullsized":"2024-03-11T18:36:06.359086+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:28:51.265291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.353858+0000","last_change":"2024-03-11T18:35:57.687817+0000","last_active":"2024-03-11T18:36:06.353858+0000","last_peered":"2024-03-11T18:36:06.353858+0000","last_clean":"2024-03-11T18:36:06.353858+0000","last_became_active":"2024-03-11T18:35:57.687532+0000","last_became_peered":"2024-03-11T18:35:57.687532+0000","last_unstale":"2024-03-11T18:36:06.353858+0000","last_undegraded":"2024-03-11T18:36:06.353858+0000","last_fullsized":"2024-03-11T18:36:06.353858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:47:28.579467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5934,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":62,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":10596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":56,"ondisk_log_size":56,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476758,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":888832,"data_stored":707497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476758,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":303104,"data_stored":120389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476756,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":294912,"data_stored":116637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476757,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":303104,"data_stored":116673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509461,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":311296,"data_stored":121011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509461,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":116547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509462,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":307200,"data_stored":117379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542163,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27896,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343944,"statfs":{"total":96636764160,"available":96608198656,"internally_reserved":0,"allocated":901120,"data_stored":707575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-11T18:36:43.306 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-11T18:36:43.306 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-11T18:36:43.306 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-03-11T18:36:43.307 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-03-11T18:36:43.307 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-03-11T18:36:43.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.460+0000 7f48b334a640 1 Processor -- start 2024-03-11T18:36:43.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.464+0000 7f48b334a640 1 -- start start 2024-03-11T18:36:43.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.464+0000 7f48b334a640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f48ac1509f0 0x7f48ac157390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:43.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.464+0000 7f48b334a640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f48ac150dd0 0x7f48ac0c28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:43.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.464+0000 7f48b334a640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f48ac070b00 0x7f48ac0c4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:43.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.464+0000 7f48b334a640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48ac0c5de0 con 0x7f48ac070b00 2024-03-11T18:36:43.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.464+0000 7f48b334a640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48ac156910 con 0x7f48ac1509f0 2024-03-11T18:36:43.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.464+0000 7f48b334a640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48ac1578d0 con 0x7f48ac150dd0 2024-03-11T18:36:43.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.464+0000 7f48b18e7640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f48ac070b00 0x7f48ac0c4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:43.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.464+0000 7f48b08e5640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f48ac150dd0 0x7f48ac0c28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:43.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.464+0000 7f48b18e7640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f48ac070b00 0x7f48ac0c4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50330/0 (socket says 172.21.15.90:50330) 2024-03-11T18:36:43.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.464+0000 7f48b18e7640 1 -- 172.21.15.90:0/1437665300 learned_addr learned my addr 172.21.15.90:0/1437665300 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:43.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.464+0000 7f48b10e6640 1 --2- 172.21.15.90:0/1437665300 >> v2:172.21.15.92:3300/0 conn(0x7f48ac1509f0 0x7f48ac157390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:43.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.468+0000 7f48b18e7640 1 -- 172.21.15.90:0/1437665300 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f48ac150dd0 msgr2=0x7f48ac0c28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:43.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.468+0000 7f48b18e7640 1 --2- 172.21.15.90:0/1437665300 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f48ac150dd0 0x7f48ac0c28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:43.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.468+0000 7f48b18e7640 1 -- 172.21.15.90:0/1437665300 >> v2:172.21.15.92:3300/0 conn(0x7f48ac1509f0 msgr2=0x7f48ac157390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:43.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.468+0000 7f48b18e7640 1 --2- 172.21.15.90:0/1437665300 >> v2:172.21.15.92:3300/0 conn(0x7f48ac1509f0 0x7f48ac157390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:43.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.468+0000 7f48b18e7640 1 -- 172.21.15.90:0/1437665300 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48ac0c6030 con 0x7f48ac070b00 2024-03-11T18:36:43.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.472+0000 7f48b18e7640 1 --2- 172.21.15.90:0/1437665300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48ac070b00 0x7f48ac0c4580 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f48ac0c8b00 tx=0x7f48a40213e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c71b811521046bcb server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:43.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.476+0000 7f48a3fff640 1 -- 172.21.15.90:0/1437665300 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a40aaea0 con 0x7f48ac070b00 2024-03-11T18:36:43.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.480+0000 7f48a3fff640 1 -- 172.21.15.90:0/1437665300 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a40ad050 con 0x7f48ac070b00 2024-03-11T18:36:43.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.480+0000 7f48a3fff640 1 -- 172.21.15.90:0/1437665300 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a40b1820 con 0x7f48ac070b00 2024-03-11T18:36:43.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.480+0000 7f48b334a640 1 -- 172.21.15.90:0/1437665300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48ac070b00 msgr2=0x7f48ac0c4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:43.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.480+0000 7f48b334a640 1 --2- 172.21.15.90:0/1437665300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48ac070b00 0x7f48ac0c4580 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f48ac0c8b00 tx=0x7f48a40213e0 comp rx=0 tx=0).stop 2024-03-11T18:36:43.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.480+0000 7f48b334a640 1 -- 172.21.15.90:0/1437665300 shutdown_connections 2024-03-11T18:36:43.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.480+0000 7f48b334a640 1 --2- 172.21.15.90:0/1437665300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48ac070b00 0x7f48ac0c4580 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:43.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.480+0000 7f48b334a640 1 --2- 172.21.15.90:0/1437665300 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f48ac150dd0 0x7f48ac0c28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:43.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.480+0000 7f48b334a640 1 --2- 172.21.15.90:0/1437665300 >> v2:172.21.15.92:3300/0 conn(0x7f48ac1509f0 0x7f48ac157390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:43.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.480+0000 7f48b334a640 1 -- 172.21.15.90:0/1437665300 >> 172.21.15.90:0/1437665300 conn(0x7f48ac06ac00 msgr2=0x7f48ac06b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:43.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.480+0000 7f48b334a640 1 -- 172.21.15.90:0/1437665300 shutdown_connections 2024-03-11T18:36:43.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.480+0000 7f48b334a640 1 -- 172.21.15.90:0/1437665300 wait complete. 2024-03-11T18:36:43.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.480+0000 7f48b334a640 1 Processor -- start 2024-03-11T18:36:43.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.480+0000 7f48b334a640 1 -- start start 2024-03-11T18:36:43.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.480+0000 7f48b334a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48ac070b00 0x7f48ac0cac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:43.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.480+0000 7f48b334a640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f48ac1509f0 0x7f48ac0c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:43.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.480+0000 7f48b334a640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48ac150dd0 0x7f48ac140120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:43.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.480+0000 7f48b334a640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ac0caa00 con 0x7f48ac150dd0 2024-03-11T18:36:43.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.480+0000 7f48b334a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ac154a60 con 0x7f48ac070b00 2024-03-11T18:36:43.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.480+0000 7f48b334a640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48ac0cba50 con 0x7f48ac1509f0 2024-03-11T18:36:43.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.480+0000 7f48b18e7640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48ac150dd0 0x7f48ac140120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:43.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.480+0000 7f48b08e5640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f48ac1509f0 0x7f48ac0c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:43.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.480+0000 7f48b18e7640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48ac150dd0 0x7f48ac140120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50334/0 (socket says 172.21.15.90:50334) 2024-03-11T18:36:43.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.480+0000 7f48b18e7640 1 -- 172.21.15.90:0/305480014 learned_addr learned my addr 172.21.15.90:0/305480014 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:43.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.484+0000 7f48b10e6640 1 --2- 172.21.15.90:0/305480014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48ac070b00 0x7f48ac0cac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:43.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.484+0000 7f48b18e7640 1 -- 172.21.15.90:0/305480014 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f48ac1509f0 msgr2=0x7f48ac0c7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:43.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.484+0000 7f48b18e7640 1 --2- 172.21.15.90:0/305480014 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f48ac1509f0 0x7f48ac0c7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:43.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.484+0000 7f48b18e7640 1 -- 172.21.15.90:0/305480014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48ac070b00 msgr2=0x7f48ac0cac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:43.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.484+0000 7f48b18e7640 1 --2- 172.21.15.90:0/305480014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48ac070b00 0x7f48ac0cac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:43.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.484+0000 7f48b18e7640 1 -- 172.21.15.90:0/305480014 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a400b770 con 0x7f48ac150dd0 2024-03-11T18:36:43.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.488+0000 7f48b18e7640 1 --2- 172.21.15.90:0/305480014 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48ac150dd0 0x7f48ac140120 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f48a401f010 tx=0x7f48a401e180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:43.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.492+0000 7f48a1ffb640 1 -- 172.21.15.90:0/305480014 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a40aaea0 con 0x7f48ac150dd0 2024-03-11T18:36:43.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.492+0000 7f48b334a640 1 -- 172.21.15.90:0/305480014 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48ac0cb4d0 con 0x7f48ac150dd0 2024-03-11T18:36:43.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.492+0000 7f48b334a640 1 -- 172.21.15.90:0/305480014 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48ac0cb770 con 0x7f48ac150dd0 2024-03-11T18:36:43.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.492+0000 7f48b334a640 1 -- 172.21.15.90:0/305480014 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48ac04eef0 con 0x7f48ac150dd0 2024-03-11T18:36:43.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.504+0000 7f48a1ffb640 1 -- 172.21.15.90:0/305480014 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a40d0430 con 0x7f48ac150dd0 2024-03-11T18:36:43.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.504+0000 7f48a1ffb640 1 -- 172.21.15.90:0/305480014 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a40d07a0 con 0x7f48ac150dd0 2024-03-11T18:36:43.893 INFO:tasks.ceph.mds.e.smithi090.stderr:2024-03-11T18:36:43.880+0000 f9e1640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-11T18:36:43.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.904+0000 7f48a1ffb640 1 -- 172.21.15.90:0/305480014 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f48a40b8070 con 0x7f48ac150dd0 2024-03-11T18:36:43.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.904+0000 7f48a1ffb640 1 --2- 172.21.15.90:0/305480014 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f4890066e70 0x7f4890069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:43.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.904+0000 7f48b10e6640 1 --2- 172.21.15.90:0/305480014 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f4890066e70 0x7f4890069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:43.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.908+0000 7f48b10e6640 1 --2- 172.21.15.90:0/305480014 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f4890066e70 0x7f4890069330 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f48a8003ab0 tx=0x7f48a8009830 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:43.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.932+0000 7f48a1ffb640 1 -- 172.21.15.90:0/305480014 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f48a40f5fb0 con 0x7f48ac150dd0 2024-03-11T18:36:43.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:43.936+0000 7f48a1ffb640 1 -- 172.21.15.90:0/305480014 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f48a40f2250 con 0x7f48ac150dd0 2024-03-11T18:36:44.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.144+0000 7f48b334a640 1 -- 172.21.15.90:0/305480014 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f48ac141680 con 0x7f48ac150dd0 2024-03-11T18:36:44.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.320+0000 7f48a1ffb640 1 -- 172.21.15.90:0/305480014 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+165194 (secure 0 0 0) 0x7f48a40b6070 con 0x7f48ac150dd0 2024-03-11T18:36:44.330 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:36:44.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.324+0000 7f48b334a640 1 -- 172.21.15.90:0/305480014 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f4890066e70 msgr2=0x7f4890069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:44.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.324+0000 7f48b334a640 1 --2- 172.21.15.90:0/305480014 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f4890066e70 0x7f4890069330 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f48a8003ab0 tx=0x7f48a8009830 comp rx=0 tx=0).stop 2024-03-11T18:36:44.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.324+0000 7f48b334a640 1 -- 172.21.15.90:0/305480014 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48ac150dd0 msgr2=0x7f48ac140120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:44.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.324+0000 7f48b334a640 1 --2- 172.21.15.90:0/305480014 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48ac150dd0 0x7f48ac140120 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f48a401f010 tx=0x7f48a401e180 comp rx=0 tx=0).stop 2024-03-11T18:36:44.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.324+0000 7f48b334a640 1 -- 172.21.15.90:0/305480014 shutdown_connections 2024-03-11T18:36:44.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.324+0000 7f48b334a640 1 --2- 172.21.15.90:0/305480014 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f4890066e70 0x7f4890069330 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:44.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.324+0000 7f48b334a640 1 --2- 172.21.15.90:0/305480014 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48ac150dd0 0x7f48ac140120 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:44.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.324+0000 7f48b334a640 1 --2- 172.21.15.90:0/305480014 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f48ac1509f0 0x7f48ac0c7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:44.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.324+0000 7f48b334a640 1 --2- 172.21.15.90:0/305480014 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f48ac070b00 0x7f48ac0cac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:44.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.324+0000 7f48b334a640 1 -- 172.21.15.90:0/305480014 >> 172.21.15.90:0/305480014 conn(0x7f48ac06ac00 msgr2=0x7f48ac0c5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:44.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.324+0000 7f48b334a640 1 -- 172.21.15.90:0/305480014 shutdown_connections 2024-03-11T18:36:44.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.324+0000 7f48b334a640 1 -- 172.21.15.90:0/305480014 wait complete. 2024-03-11T18:36:44.349 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":5,"flags":0,"active_gid":4101,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":1999720161},{"type":"v1","addr":"172.21.15.90:6801","nonce":1999720161}]},"active_addr":"172.21.15.90:6801/1999720161","active_change":"2024-03-11T18:35:01.381249+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4100,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.90:0","nonce":2631367673}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.90:0","nonce":878381270}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.90:0","nonce":2225708074}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.90:0","nonce":3378305285}]}]} 2024-03-11T18:36:44.353 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-03-11T18:36:44.354 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-03-11T18:36:44.354 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T18:36:44.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.468+0000 7fa7f380b640 1 Processor -- start 2024-03-11T18:36:44.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.468+0000 7fa7f380b640 1 -- start start 2024-03-11T18:36:44.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.472+0000 7fa7f380b640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fa7ec1598c0 0x7fa7ec06e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:44.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.472+0000 7fa7f380b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa7ec0c3d30 0x7fa7ec0c4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:44.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.472+0000 7fa7f380b640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7ec070b00 con 0x7fa7ec06f220 2024-03-11T18:36:44.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.472+0000 7fa7f380b640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7ec06eef0 con 0x7fa7ec1598c0 2024-03-11T18:36:44.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.472+0000 7fa7f380b640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7ec071df0 con 0x7fa7ec0c3d30 2024-03-11T18:36:44.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.472+0000 7fa7f15a7640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fa7ec1598c0 0x7fa7ec06e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:44.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.472+0000 7fa7f15a7640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fa7ec1598c0 0x7fa7ec06e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48008/0 (socket says 172.21.15.90:48008) 2024-03-11T18:36:44.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.472+0000 7fa7f15a7640 1 -- 172.21.15.90:0/4058532962 learned_addr learned my addr 172.21.15.90:0/4058532962 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:44.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.472+0000 7fa7f1da8640 1 --2- 172.21.15.90:0/4058532962 >> v2:172.21.15.92:3300/0 conn(0x7fa7ec0c3d30 0x7fa7ec0c4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:44.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.476+0000 7fa7f15a7640 1 -- 172.21.15.90:0/4058532962 >> v2:172.21.15.92:3300/0 conn(0x7fa7ec0c3d30 msgr2=0x7fa7ec0c4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:44.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.476+0000 7fa7f15a7640 1 --2- 172.21.15.90:0/4058532962 >> v2:172.21.15.92:3300/0 conn(0x7fa7ec0c3d30 0x7fa7ec0c4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:44.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.476+0000 7fa7f15a7640 1 -- 172.21.15.90:0/4058532962 >> v1:172.21.15.92:6789/0 conn(0x7fa7ec06f220 legacy=0x7fa7ec159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:44.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.476+0000 7fa7f15a7640 1 -- 172.21.15.90:0/4058532962 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7ec15a3b0 con 0x7fa7ec1598c0 2024-03-11T18:36:44.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.476+0000 7fa7f15a7640 1 --2- 172.21.15.90:0/4058532962 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7ec1598c0 0x7fa7ec06e9b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec06c8d0 tx=0x7fa7e80202f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc1eca0f22502bbb server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:44.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.480+0000 7fa7e3fff640 1 -- 172.21.15.90:0/4058532962 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e800a2d0 con 0x7fa7ec1598c0 2024-03-11T18:36:44.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.480+0000 7fa7e3fff640 1 -- 172.21.15.90:0/4058532962 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7e80ac050 con 0x7fa7ec1598c0 2024-03-11T18:36:44.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.480+0000 7fa7e3fff640 1 -- 172.21.15.90:0/4058532962 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e8008720 con 0x7fa7ec1598c0 2024-03-11T18:36:44.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.484+0000 7fa7f380b640 1 -- 172.21.15.90:0/4058532962 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7ec1598c0 msgr2=0x7fa7ec06e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:44.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.484+0000 7fa7f380b640 1 --2- 172.21.15.90:0/4058532962 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7ec1598c0 0x7fa7ec06e9b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec06c8d0 tx=0x7fa7e80202f0 comp rx=0 tx=0).stop 2024-03-11T18:36:44.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.484+0000 7fa7f380b640 1 -- 172.21.15.90:0/4058532962 shutdown_connections 2024-03-11T18:36:44.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.484+0000 7fa7f380b640 1 --2- 172.21.15.90:0/4058532962 >> v2:172.21.15.92:3300/0 conn(0x7fa7ec0c3d30 0x7fa7ec0c4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:44.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.484+0000 7fa7f380b640 1 --2- 172.21.15.90:0/4058532962 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7ec1598c0 0x7fa7ec06e9b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:44.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.484+0000 7fa7f380b640 1 -- 172.21.15.90:0/4058532962 >> 172.21.15.90:0/4058532962 conn(0x7fa7ec06ac00 msgr2=0x7fa7ec06b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:44.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.484+0000 7fa7f380b640 1 -- 172.21.15.90:0/4058532962 shutdown_connections 2024-03-11T18:36:44.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.484+0000 7fa7f380b640 1 -- 172.21.15.90:0/4058532962 wait complete. 2024-03-11T18:36:44.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.484+0000 7fa7f380b640 1 Processor -- start 2024-03-11T18:36:44.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.484+0000 7fa7f380b640 1 -- start start 2024-03-11T18:36:44.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.484+0000 7fa7f380b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7ec06f220 0x7fa7ec1e2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:44.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.484+0000 7fa7f380b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7ec0c3d30 0x7fa7ec1f3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:44.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.484+0000 7fa7f380b640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa7ec1598c0 0x7fa7ec1e4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:44.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.484+0000 7fa7f380b640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ec1e4400 con 0x7fa7ec0c3d30 2024-03-11T18:36:44.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.484+0000 7fa7f380b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ec1e3500 con 0x7fa7ec06f220 2024-03-11T18:36:44.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.484+0000 7fa7f380b640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ec1e3b40 con 0x7fa7ec1598c0 2024-03-11T18:36:44.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.484+0000 7fa7f0da6640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7ec0c3d30 0x7fa7ec1f3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:44.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.484+0000 7fa7f0da6640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7ec0c3d30 0x7fa7ec1f3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48012/0 (socket says 172.21.15.90:48012) 2024-03-11T18:36:44.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.484+0000 7fa7f0da6640 1 -- 172.21.15.90:0/647439241 learned_addr learned my addr 172.21.15.90:0/647439241 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:44.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.488+0000 7fa7f1da8640 1 --2- 172.21.15.90:0/647439241 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa7ec1598c0 0x7fa7ec1e4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:44.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.488+0000 7fa7f15a7640 1 --2- 172.21.15.90:0/647439241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7ec06f220 0x7fa7ec1e2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:44.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.488+0000 7fa7f0da6640 1 -- 172.21.15.90:0/647439241 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa7ec1598c0 msgr2=0x7fa7ec1e4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:44.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.488+0000 7fa7f0da6640 1 --2- 172.21.15.90:0/647439241 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa7ec1598c0 0x7fa7ec1e4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:44.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.488+0000 7fa7f0da6640 1 -- 172.21.15.90:0/647439241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7ec06f220 msgr2=0x7fa7ec1e2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:44.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.488+0000 7fa7f0da6640 1 --2- 172.21.15.90:0/647439241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7ec06f220 0x7fa7ec1e2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:44.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.488+0000 7fa7f0da6640 1 -- 172.21.15.90:0/647439241 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e800ac40 con 0x7fa7ec0c3d30 2024-03-11T18:36:44.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.488+0000 7fa7f1da8640 1 --2- 172.21.15.90:0/647439241 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa7ec1598c0 0x7fa7ec1e4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T18:36:44.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.492+0000 7fa7f0da6640 1 --2- 172.21.15.90:0/647439241 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7ec0c3d30 0x7fa7ec1f3080 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc00c370 tx=0x7fa7dc00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:44.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.496+0000 7fa7e1ffb640 1 -- 172.21.15.90:0/647439241 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7dc0052d0 con 0x7fa7ec0c3d30 2024-03-11T18:36:44.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.496+0000 7fa7f380b640 1 -- 172.21.15.90:0/647439241 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7ec1f3950 con 0x7fa7ec0c3d30 2024-03-11T18:36:44.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.496+0000 7fa7f380b640 1 -- 172.21.15.90:0/647439241 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7ec1e3d40 con 0x7fa7ec0c3d30 2024-03-11T18:36:44.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.496+0000 7fa7f380b640 1 -- 172.21.15.90:0/647439241 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7ec1e4540 con 0x7fa7ec0c3d30 2024-03-11T18:36:44.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.496+0000 7fa7e1ffb640 1 -- 172.21.15.90:0/647439241 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7dc005430 con 0x7fa7ec0c3d30 2024-03-11T18:36:44.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.496+0000 7fa7e1ffb640 1 -- 172.21.15.90:0/647439241 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7dc001110 con 0x7fa7ec0c3d30 2024-03-11T18:36:44.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.772+0000 7fa7e1ffb640 1 -- 172.21.15.90:0/647439241 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fa7dc007240 con 0x7fa7ec0c3d30 2024-03-11T18:36:44.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.772+0000 7fa7e1ffb640 1 --2- 172.21.15.90:0/647439241 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fa7d0066e70 0x7fa7d0069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:44.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.772+0000 7fa7f15a7640 1 --2- 172.21.15.90:0/647439241 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fa7d0066e70 0x7fa7d0069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:44.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.772+0000 7fa7f15a7640 1 --2- 172.21.15.90:0/647439241 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fa7d0066e70 0x7fa7d0069330 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa7e80b2000 tx=0x7fa7e80ab000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:44.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.784+0000 7fa7e1ffb640 1 -- 172.21.15.90:0/647439241 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa7dc054c90 con 0x7fa7ec0c3d30 2024-03-11T18:36:44.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.784+0000 7fa7e1ffb640 1 -- 172.21.15.90:0/647439241 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa7dc050f30 con 0x7fa7ec0c3d30 2024-03-11T18:36:44.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.948+0000 7fa7f380b640 1 -- 172.21.15.90:0/647439241 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa7ec1e4540 con 0x7fa7ec0c3d30 2024-03-11T18:36:44.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.988+0000 7fa7e1ffb640 1 -- 172.21.15.90:0/647439241 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18430 (secure 0 0 0) 0x7fa7ec1e4540 con 0x7fa7ec0c3d30 2024-03-11T18:36:44.996 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:36:44.996 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":28,"fsid":"76d56b10-6c29-476a-804b-8482ea90f9e2","created":"2024-03-11T18:34:54.521741+0000","modified":"2024-03-11T18:36:05.526044+0000","last_up_change":"2024-03-11T18:35:16.207500+0000","last_in_change":"2024-03-11T18:35:05.213634+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T18:35:16.193842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T18:35:54.001469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T18:35:55.917321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T18:36:02.210943+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"34decff9-7280-4d84-9665-d6af8ea6de84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6819","nonce":1755667921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6821","nonce":1755667921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6825","nonce":1755667921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6823","nonce":1755667921}]},"public_addr":"172.21.15.90:6819/1755667921","cluster_addr":"172.21.15.90:6821/1755667921","heartbeat_back_addr":"172.21.15.90:6825/1755667921","heartbeat_front_addr":"172.21.15.90:6823/1755667921","state":["exists","up"]},{"osd":1,"uuid":"8ecdce9e-5486-415a-a609-dd16e945440e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6807","nonce":438682081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6809","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6811","nonce":438682081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6817","nonce":438682081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6815","nonce":438682081}]},"public_addr":"172.21.15.90:6807/438682081","cluster_addr":"172.21.15.90:6811/438682081","heartbeat_back_addr":"172.21.15.90:6817/438682081","heartbeat_front_addr":"172.21.15.90:6815/438682081","state":["exists","up"]},{"osd":2,"uuid":"737486d4-0e90-4c82-9e79-bf37c812e5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6827","nonce":2837704738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6829","nonce":2837704738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6833","nonce":2837704738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6831","nonce":2837704738}]},"public_addr":"172.21.15.90:6827/2837704738","cluster_addr":"172.21.15.90:6829/2837704738","heartbeat_back_addr":"172.21.15.90:6833/2837704738","heartbeat_front_addr":"172.21.15.90:6831/2837704738","state":["exists","up"]},{"osd":3,"uuid":"698bd661-bbef-4699-a619-f7bfe23ce12c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6803","nonce":240987971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6805","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6806","nonce":240987971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6813","nonce":240987971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6810","nonce":240987971}]},"public_addr":"172.21.15.90:6803/240987971","cluster_addr":"172.21.15.90:6806/240987971","heartbeat_back_addr":"172.21.15.90:6813/240987971","heartbeat_front_addr":"172.21.15.90:6810/240987971","state":["exists","up"]},{"osd":4,"uuid":"aba9e31f-73d9-43fe-a10b-2dc5832968b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6801","nonce":502685275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6803","nonce":502685275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6807","nonce":502685275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6805","nonce":502685275}]},"public_addr":"172.21.15.92:6801/502685275","cluster_addr":"172.21.15.92:6803/502685275","heartbeat_back_addr":"172.21.15.92:6807/502685275","heartbeat_front_addr":"172.21.15.92:6805/502685275","state":["exists","up"]},{"osd":5,"uuid":"bdd56766-4b28-4007-8de3-b5b08cf335d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6809","nonce":200142070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6811","nonce":200142070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6815","nonce":200142070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6813","nonce":200142070}]},"public_addr":"172.21.15.92:6809/200142070","cluster_addr":"172.21.15.92:6811/200142070","heartbeat_back_addr":"172.21.15.92:6815/200142070","heartbeat_front_addr":"172.21.15.92:6813/200142070","state":["exists","up"]},{"osd":6,"uuid":"6c580a59-610f-455e-a96b-033299dcc118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6817","nonce":1919010204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6819","nonce":1919010204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6823","nonce":1919010204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6821","nonce":1919010204}]},"public_addr":"172.21.15.92:6817/1919010204","cluster_addr":"172.21.15.92:6819/1919010204","heartbeat_back_addr":"172.21.15.92:6823/1919010204","heartbeat_front_addr":"172.21.15.92:6821/1919010204","state":["exists","up"]},{"osd":7,"uuid":"958184cc-aebf-4c5b-91a0-2a3e38a6b42b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6825","nonce":3475925719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6827","nonce":3475925719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6831","nonce":3475925719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6829","nonce":3475925719}]},"public_addr":"172.21.15.92:6825/3475925719","cluster_addr":"172.21.15.92:6827/3475925719","heartbeat_back_addr":"172.21.15.92:6831/3475925719","heartbeat_front_addr":"172.21.15.92:6829/3475925719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.816294+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.048084+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.137975+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.968246+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.016829+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.931676+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.041673+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.079404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T18:36:44.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.992+0000 7fa7f380b640 1 -- 172.21.15.90:0/647439241 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fa7d0066e70 msgr2=0x7fa7d0069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:44.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.992+0000 7fa7f380b640 1 --2- 172.21.15.90:0/647439241 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fa7d0066e70 0x7fa7d0069330 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa7e80b2000 tx=0x7fa7e80ab000 comp rx=0 tx=0).stop 2024-03-11T18:36:44.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.992+0000 7fa7f380b640 1 -- 172.21.15.90:0/647439241 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7ec0c3d30 msgr2=0x7fa7ec1f3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:44.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.992+0000 7fa7f380b640 1 --2- 172.21.15.90:0/647439241 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7ec0c3d30 0x7fa7ec1f3080 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc00c370 tx=0x7fa7dc00c830 comp rx=0 tx=0).stop 2024-03-11T18:36:44.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.992+0000 7fa7f380b640 1 -- 172.21.15.90:0/647439241 shutdown_connections 2024-03-11T18:36:44.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.992+0000 7fa7f380b640 1 --2- 172.21.15.90:0/647439241 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fa7d0066e70 0x7fa7d0069330 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:44.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.992+0000 7fa7f380b640 1 --2- 172.21.15.90:0/647439241 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa7ec1598c0 0x7fa7ec1e4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:44.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.992+0000 7fa7f380b640 1 --2- 172.21.15.90:0/647439241 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7ec0c3d30 0x7fa7ec1f3080 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:44.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.992+0000 7fa7f380b640 1 --2- 172.21.15.90:0/647439241 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa7ec06f220 0x7fa7ec1e2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:45.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.992+0000 7fa7f380b640 1 -- 172.21.15.90:0/647439241 >> 172.21.15.90:0/647439241 conn(0x7fa7ec06ac00 msgr2=0x7fa7ec0c5ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:45.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.992+0000 7fa7f380b640 1 -- 172.21.15.90:0/647439241 shutdown_connections 2024-03-11T18:36:45.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:44.992+0000 7fa7f380b640 1 -- 172.21.15.90:0/647439241 wait complete. 2024-03-11T18:36:45.017 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-03-11T18:36:45.017 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T18:36:45.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.132+0000 7f7a1c9d7640 1 Processor -- start 2024-03-11T18:36:45.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.132+0000 7f7a1c9d7640 1 -- start start 2024-03-11T18:36:45.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.132+0000 7f7a1c9d7640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f7a180cb7d0 0x7f7a180cf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:45.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.132+0000 7f7a1c9d7640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a1806d000 con 0x7f7a18153fe0 2024-03-11T18:36:45.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.132+0000 7f7a1c9d7640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a180cad00 con 0x7f7a180cfd40 2024-03-11T18:36:45.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.132+0000 7f7a1c9d7640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a181543c0 con 0x7f7a180cb7d0 2024-03-11T18:36:45.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.136+0000 7f7a15d9b640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f7a180cb7d0 0x7f7a180cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:45.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.136+0000 7f7a15d9b640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f7a180cb7d0 0x7f7a180cf800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48028/0 (socket says 172.21.15.90:48028) 2024-03-11T18:36:45.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.136+0000 7f7a15d9b640 1 -- 172.21.15.90:0/1943092409 learned_addr learned my addr 172.21.15.90:0/1943092409 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:45.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.140+0000 7f7a15d9b640 1 -- 172.21.15.90:0/1943092409 >> v1:172.21.15.92:6789/0 conn(0x7f7a180cfd40 legacy=0x7f7a1806bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:45.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.140+0000 7f7a15d9b640 1 -- 172.21.15.90:0/1943092409 >> v1:172.21.15.90:6789/0 conn(0x7f7a18153fe0 legacy=0x7f7a180ca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:45.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.140+0000 7f7a15d9b640 1 -- 172.21.15.90:0/1943092409 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a18159ca0 con 0x7f7a180cb7d0 2024-03-11T18:36:45.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.148+0000 7f7a15d9b640 1 --2- 172.21.15.90:0/1943092409 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a180cb7d0 0x7f7a180cf800 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c009870 tx=0x7f7a0c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e14b8089c29d3d7a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:45.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.152+0000 7f7a1559a640 1 -- 172.21.15.90:0/1943092409 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c0ab610 con 0x7f7a180cb7d0 2024-03-11T18:36:45.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.152+0000 7f7a1559a640 1 -- 172.21.15.90:0/1943092409 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a0c0ab770 con 0x7f7a180cb7d0 2024-03-11T18:36:45.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.152+0000 7f7a1559a640 1 -- 172.21.15.90:0/1943092409 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c0abb00 con 0x7f7a180cb7d0 2024-03-11T18:36:45.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.156+0000 7f7a1c9d7640 1 -- 172.21.15.90:0/1943092409 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a180cb7d0 msgr2=0x7f7a180cf800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:45.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.156+0000 7f7a1c9d7640 1 --2- 172.21.15.90:0/1943092409 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a180cb7d0 0x7f7a180cf800 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c009870 tx=0x7f7a0c01c520 comp rx=0 tx=0).stop 2024-03-11T18:36:45.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.156+0000 7f7a1c9d7640 1 -- 172.21.15.90:0/1943092409 shutdown_connections 2024-03-11T18:36:45.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.156+0000 7f7a1c9d7640 1 --2- 172.21.15.90:0/1943092409 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a180cb7d0 0x7f7a180cf800 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:45.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.156+0000 7f7a1c9d7640 1 -- 172.21.15.90:0/1943092409 >> 172.21.15.90:0/1943092409 conn(0x7f7a18072500 msgr2=0x7f7a18072910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:45.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.156+0000 7f7a1c9d7640 1 -- 172.21.15.90:0/1943092409 shutdown_connections 2024-03-11T18:36:45.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.156+0000 7f7a1c9d7640 1 -- 172.21.15.90:0/1943092409 wait complete. 2024-03-11T18:36:45.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.156+0000 7f7a1c9d7640 1 Processor -- start 2024-03-11T18:36:45.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.156+0000 7f7a1c9d7640 1 -- start start 2024-03-11T18:36:45.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.156+0000 7f7a1c9d7640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a180cb7d0 0x7f7a180b3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:45.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.156+0000 7f7a1c9d7640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a180cfd40 0x7f7a180b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:45.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.156+0000 7f7a1c9d7640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f7a18153fe0 0x7f7a180c5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:45.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.156+0000 7f7a1c9d7640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a180d0150 con 0x7f7a180cfd40 2024-03-11T18:36:45.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.156+0000 7f7a1c9d7640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a18141f40 con 0x7f7a180cb7d0 2024-03-11T18:36:45.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.156+0000 7f7a1c9d7640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a180b4740 con 0x7f7a18153fe0 2024-03-11T18:36:45.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.156+0000 7f7a15d9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a180cfd40 0x7f7a180b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:45.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.160+0000 7f7a15d9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a180cfd40 0x7f7a180b4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48040/0 (socket says 172.21.15.90:48040) 2024-03-11T18:36:45.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.160+0000 7f7a15d9b640 1 -- 172.21.15.90:0/4250174375 learned_addr learned my addr 172.21.15.90:0/4250174375 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:45.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.160+0000 7f7a1659c640 1 --2- 172.21.15.90:0/4250174375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a180cb7d0 0x7f7a180b3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:45.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.160+0000 7f7a16d9d640 1 --2- 172.21.15.90:0/4250174375 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f7a18153fe0 0x7f7a180c5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:45.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.164+0000 7f7a15d9b640 1 -- 172.21.15.90:0/4250174375 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f7a18153fe0 msgr2=0x7f7a180c5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:45.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.164+0000 7f7a15d9b640 1 --2- 172.21.15.90:0/4250174375 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f7a18153fe0 0x7f7a180c5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:45.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.164+0000 7f7a15d9b640 1 -- 172.21.15.90:0/4250174375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a180cb7d0 msgr2=0x7f7a180b3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:45.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.164+0000 7f7a15d9b640 1 --2- 172.21.15.90:0/4250174375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a180cb7d0 0x7f7a180b3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:45.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.164+0000 7f7a15d9b640 1 -- 172.21.15.90:0/4250174375 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a0c0094d0 con 0x7f7a180cfd40 2024-03-11T18:36:45.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.164+0000 7f7a15d9b640 1 --2- 172.21.15.90:0/4250174375 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a180cfd40 0x7f7a180b4c80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c019130 tx=0x7f7a0c01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:45.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.172+0000 7f7a06ffd640 1 -- 172.21.15.90:0/4250174375 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c0b4800 con 0x7f7a180cfd40 2024-03-11T18:36:45.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.172+0000 7f7a06ffd640 1 -- 172.21.15.90:0/4250174375 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a0c0b4980 con 0x7f7a180cfd40 2024-03-11T18:36:45.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.172+0000 7f7a06ffd640 1 -- 172.21.15.90:0/4250174375 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c0b4d30 con 0x7f7a180cfd40 2024-03-11T18:36:45.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.172+0000 7f7a1c9d7640 1 -- 172.21.15.90:0/4250174375 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a180c5450 con 0x7f7a180cfd40 2024-03-11T18:36:45.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.172+0000 7f7a1c9d7640 1 -- 172.21.15.90:0/4250174375 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a180c4520 con 0x7f7a180cfd40 2024-03-11T18:36:45.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.172+0000 7f7a1c9d7640 1 -- 172.21.15.90:0/4250174375 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a1804eef0 con 0x7f7a180cfd40 2024-03-11T18:36:45.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.192+0000 7f7a06ffd640 1 -- 172.21.15.90:0/4250174375 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f7a0c0c8420 con 0x7f7a180cfd40 2024-03-11T18:36:45.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.192+0000 7f7a06ffd640 1 --2- 172.21.15.90:0/4250174375 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f79f8066e90 0x7f79f8069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:45.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.192+0000 7f7a1659c640 1 --2- 172.21.15.90:0/4250174375 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f79f8066e90 0x7f79f8069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:45.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.192+0000 7f7a1659c640 1 --2- 172.21.15.90:0/4250174375 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f79f8066e90 0x7f79f8069350 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7a08001770 tx=0x7f7a08009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:45.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.444+0000 7f7a06ffd640 1 -- 172.21.15.90:0/4250174375 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f7a0c0f0230 con 0x7f7a180cfd40 2024-03-11T18:36:45.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.444+0000 7f7a06ffd640 1 -- 172.21.15.90:0/4250174375 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7a0c0ec4d0 con 0x7f7a180cfd40 2024-03-11T18:36:45.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.612+0000 7f7a1c9d7640 1 -- 172.21.15.90:0/4250174375 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7a180caf30 con 0x7f7a180cfd40 2024-03-11T18:36:45.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.660+0000 7f7a06ffd640 1 -- 172.21.15.90:0/4250174375 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18430 (secure 0 0 0) 0x7f7a0c0efb90 con 0x7f7a180cfd40 2024-03-11T18:36:45.666 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:36:45.666 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":28,"fsid":"76d56b10-6c29-476a-804b-8482ea90f9e2","created":"2024-03-11T18:34:54.521741+0000","modified":"2024-03-11T18:36:05.526044+0000","last_up_change":"2024-03-11T18:35:16.207500+0000","last_in_change":"2024-03-11T18:35:05.213634+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T18:35:16.193842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T18:35:54.001469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T18:35:55.917321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T18:36:02.210943+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"34decff9-7280-4d84-9665-d6af8ea6de84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6819","nonce":1755667921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6821","nonce":1755667921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6825","nonce":1755667921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6823","nonce":1755667921}]},"public_addr":"172.21.15.90:6819/1755667921","cluster_addr":"172.21.15.90:6821/1755667921","heartbeat_back_addr":"172.21.15.90:6825/1755667921","heartbeat_front_addr":"172.21.15.90:6823/1755667921","state":["exists","up"]},{"osd":1,"uuid":"8ecdce9e-5486-415a-a609-dd16e945440e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6807","nonce":438682081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6809","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6811","nonce":438682081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6817","nonce":438682081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6815","nonce":438682081}]},"public_addr":"172.21.15.90:6807/438682081","cluster_addr":"172.21.15.90:6811/438682081","heartbeat_back_addr":"172.21.15.90:6817/438682081","heartbeat_front_addr":"172.21.15.90:6815/438682081","state":["exists","up"]},{"osd":2,"uuid":"737486d4-0e90-4c82-9e79-bf37c812e5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6827","nonce":2837704738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6829","nonce":2837704738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6833","nonce":2837704738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6831","nonce":2837704738}]},"public_addr":"172.21.15.90:6827/2837704738","cluster_addr":"172.21.15.90:6829/2837704738","heartbeat_back_addr":"172.21.15.90:6833/2837704738","heartbeat_front_addr":"172.21.15.90:6831/2837704738","state":["exists","up"]},{"osd":3,"uuid":"698bd661-bbef-4699-a619-f7bfe23ce12c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6803","nonce":240987971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6805","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6806","nonce":240987971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6813","nonce":240987971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6810","nonce":240987971}]},"public_addr":"172.21.15.90:6803/240987971","cluster_addr":"172.21.15.90:6806/240987971","heartbeat_back_addr":"172.21.15.90:6813/240987971","heartbeat_front_addr":"172.21.15.90:6810/240987971","state":["exists","up"]},{"osd":4,"uuid":"aba9e31f-73d9-43fe-a10b-2dc5832968b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6801","nonce":502685275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6803","nonce":502685275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6807","nonce":502685275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6805","nonce":502685275}]},"public_addr":"172.21.15.92:6801/502685275","cluster_addr":"172.21.15.92:6803/502685275","heartbeat_back_addr":"172.21.15.92:6807/502685275","heartbeat_front_addr":"172.21.15.92:6805/502685275","state":["exists","up"]},{"osd":5,"uuid":"bdd56766-4b28-4007-8de3-b5b08cf335d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6809","nonce":200142070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6811","nonce":200142070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6815","nonce":200142070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6813","nonce":200142070}]},"public_addr":"172.21.15.92:6809/200142070","cluster_addr":"172.21.15.92:6811/200142070","heartbeat_back_addr":"172.21.15.92:6815/200142070","heartbeat_front_addr":"172.21.15.92:6813/200142070","state":["exists","up"]},{"osd":6,"uuid":"6c580a59-610f-455e-a96b-033299dcc118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6817","nonce":1919010204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6819","nonce":1919010204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6823","nonce":1919010204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6821","nonce":1919010204}]},"public_addr":"172.21.15.92:6817/1919010204","cluster_addr":"172.21.15.92:6819/1919010204","heartbeat_back_addr":"172.21.15.92:6823/1919010204","heartbeat_front_addr":"172.21.15.92:6821/1919010204","state":["exists","up"]},{"osd":7,"uuid":"958184cc-aebf-4c5b-91a0-2a3e38a6b42b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6825","nonce":3475925719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6827","nonce":3475925719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6831","nonce":3475925719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6829","nonce":3475925719}]},"public_addr":"172.21.15.92:6825/3475925719","cluster_addr":"172.21.15.92:6827/3475925719","heartbeat_back_addr":"172.21.15.92:6831/3475925719","heartbeat_front_addr":"172.21.15.92:6829/3475925719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.816294+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.048084+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.137975+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.968246+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.016829+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.931676+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.041673+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.079404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T18:36:45.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.660+0000 7f7a1c9d7640 1 -- 172.21.15.90:0/4250174375 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f79f8066e90 msgr2=0x7f79f8069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:45.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.660+0000 7f7a1c9d7640 1 --2- 172.21.15.90:0/4250174375 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f79f8066e90 0x7f79f8069350 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7a08001770 tx=0x7f7a08009040 comp rx=0 tx=0).stop 2024-03-11T18:36:45.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.660+0000 7f7a1c9d7640 1 -- 172.21.15.90:0/4250174375 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a180cfd40 msgr2=0x7f7a180b4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:45.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.660+0000 7f7a1c9d7640 1 --2- 172.21.15.90:0/4250174375 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a180cfd40 0x7f7a180b4c80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c019130 tx=0x7f7a0c01ad60 comp rx=0 tx=0).stop 2024-03-11T18:36:45.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.660+0000 7f7a1c9d7640 1 -- 172.21.15.90:0/4250174375 shutdown_connections 2024-03-11T18:36:45.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.660+0000 7f7a1c9d7640 1 --2- 172.21.15.90:0/4250174375 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f79f8066e90 0x7f79f8069350 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:45.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.660+0000 7f7a1c9d7640 1 --2- 172.21.15.90:0/4250174375 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f7a18153fe0 0x7f7a180c5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:45.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.660+0000 7f7a1c9d7640 1 --2- 172.21.15.90:0/4250174375 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a180cfd40 0x7f7a180b4c80 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:45.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.660+0000 7f7a1c9d7640 1 --2- 172.21.15.90:0/4250174375 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7a180cb7d0 0x7f7a180b3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:45.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.660+0000 7f7a1c9d7640 1 -- 172.21.15.90:0/4250174375 >> 172.21.15.90:0/4250174375 conn(0x7f7a18072500 msgr2=0x7f7a1814e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:45.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.660+0000 7f7a1c9d7640 1 -- 172.21.15.90:0/4250174375 shutdown_connections 2024-03-11T18:36:45.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.660+0000 7f7a1c9d7640 1 -- 172.21.15.90:0/4250174375 wait complete. 2024-03-11T18:36:45.688 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-03-11T18:36:45.689 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-03-11T18:36:45.689 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-03-11T18:36:45.690 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-03-11T18:36:45.690 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-03-11T18:36:45.690 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-03-11T18:36:45.691 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-03-11T18:36:45.691 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-03-11T18:36:45.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.872+0000 7f1541eed640 1 Processor -- start 2024-03-11T18:36:45.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.872+0000 7f1541eed640 1 -- start start 2024-03-11T18:36:45.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.872+0000 7fe1af501640 1 Processor -- start 2024-03-11T18:36:45.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.872+0000 7fe04c1de640 1 Processor -- start 2024-03-11T18:36:45.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.872+0000 7fe1af501640 1 -- start start 2024-03-11T18:36:45.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.872+0000 7fe1af501640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fe1a8070c80 0x7fe1a8071bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:45.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.872+0000 7fe1af501640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe1a80c5400 0x7fe1a80c6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:45.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.872+0000 7fe04c1de640 1 -- start start 2024-03-11T18:36:45.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.872+0000 7fe1af501640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1a8169ad0 con 0x7fe1a8167240 2024-03-11T18:36:45.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.872+0000 7fe1af501640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1a8169d20 con 0x7fe1a8070c80 2024-03-11T18:36:45.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.872+0000 7fe1af501640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a816a000 con 0x7fe1a80c5400 2024-03-11T18:36:45.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.872+0000 7f1541eed640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f153c14fc90 0x7f153c071c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:45.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.872+0000 7f1541eed640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f153c070920 0x7f153c071120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:45.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.872+0000 7f1541eed640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f153c071660 0x7f153c1591b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:45.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.872+0000 7f1541eed640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f153c150070 con 0x7f153c071660 2024-03-11T18:36:45.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.872+0000 7f1541eed640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f153c06e110 con 0x7f153c14fc90 2024-03-11T18:36:45.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.872+0000 7f1541eed640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f153c0721a0 con 0x7f153c070920 2024-03-11T18:36:45.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.872+0000 7fe04c1de640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe03c0d6c60 0x7fe03c0dd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:45.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.872+0000 7fe04c1de640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe03c04af10 0x7fe03c04b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:45.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.872+0000 7fe04c1de640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe03c0339d0 con 0x7fe03c04c480 2024-03-11T18:36:45.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.872+0000 7fe04c1de640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe03c0dda80 con 0x7fe03c04af10 2024-03-11T18:36:45.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.872+0000 7fe04c1de640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c006310 con 0x7fe03c0d6c60 2024-03-11T18:36:45.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.872+0000 7fd696ce2640 1 Processor -- start 2024-03-11T18:36:45.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.872+0000 7fd696ce2640 1 -- start start 2024-03-11T18:36:45.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.872+0000 7fe1ada9e640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe1a80c5400 0x7fe1a80c6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:45.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.876+0000 7fd696ce2640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fd6900c8b00 0x7fd6900c8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:45.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.876+0000 7fd696ce2640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd69015a8d0 con 0x7fd690153d20 2024-03-11T18:36:45.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.876+0000 7fd696ce2640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd690159a00 con 0x7fd69006d9c0 2024-03-11T18:36:45.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.876+0000 7fd696ce2640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd690159cc0 con 0x7fd6900c8b00 2024-03-11T18:36:45.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.876+0000 7fe1ad29d640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe1a8167240 0x7fe1a80cb470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.90:47548/0 (socket says 172.21.15.90:47548) 2024-03-11T18:36:45.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.876+0000 7fe1ad29d640 1 -- 172.21.15.90:0/899926707 learned_addr learned my addr 172.21.15.90:0/899926707 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T18:36:45.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.876+0000 7f153b7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f153c14fc90 0x7f153c071c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:45.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.880+0000 7f153affd640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f153c070920 0x7f153c071120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:45.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.880+0000 7f153b7fe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f153c14fc90 0x7f153c071c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.90:60274/0 (socket says 172.21.15.90:60274) 2024-03-11T18:36:45.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.880+0000 7f153b7fe640 1 -- 172.21.15.90:0/88274557 learned_addr learned my addr 172.21.15.90:0/88274557 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:45.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.880+0000 7fa052b26640 1 Processor -- start 2024-03-11T18:36:45.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.880+0000 7fa052b26640 1 -- start start 2024-03-11T18:36:45.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.880+0000 7fa052b26640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fa044046200 0x7fa04404c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:45.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.880+0000 7fa052b26640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fa0440d5be0 0x7fa0440def90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:45.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.880+0000 7fa052b26640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa044035290 con 0x7fa0440d5800 2024-03-11T18:36:45.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.880+0000 7fa052b26640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0440dfef0 con 0x7fa0440d5be0 2024-03-11T18:36:45.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.880+0000 7fa052b26640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa044006010 con 0x7fa044046200 2024-03-11T18:36:45.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.880+0000 7fe049779640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe03c04c480 0x7fe03c0332c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.90:47564/0 (socket says 172.21.15.90:47564) 2024-03-11T18:36:45.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.880+0000 7fe049779640 1 -- 172.21.15.90:0/2757442604 learned_addr learned my addr 172.21.15.90:0/2757442604 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T18:36:45.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.884+0000 7fe170a3f640 1 Processor -- start 2024-03-11T18:36:45.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.884+0000 7fe170a3f640 1 -- start start 2024-03-11T18:36:45.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.884+0000 7f8e3fe2a640 1 Processor -- start 2024-03-11T18:36:45.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.884+0000 7f8e3fe2a640 1 -- start start 2024-03-11T18:36:45.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.884+0000 7fe1ada9e640 1 -- 172.21.15.90:0/899926707 >> v2:172.21.15.90:3300/0 conn(0x7fe1a8070c80 msgr2=0x7fe1a8071bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:45.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.884+0000 7fe1ada9e640 1 --2- 172.21.15.90:0/899926707 >> v2:172.21.15.90:3300/0 conn(0x7fe1a8070c80 0x7fe1a8071bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:45.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.884+0000 7fe1ada9e640 1 -- 172.21.15.90:0/899926707 >> v1:172.21.15.92:6789/0 conn(0x7fe1a8167240 legacy=0x7fe1a80cb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:45.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.884+0000 7fe1ada9e640 1 -- 172.21.15.90:0/899926707 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1a816a2e0 con 0x7fe1a80c5400 2024-03-11T18:36:45.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.884+0000 7fe170a3f640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe16c151a50 0x7fe16c0c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:45.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.884+0000 7fe170a3f640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fe16c0c3f30 0x7fe16c06e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:45.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.884+0000 7fe170a3f640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe16c0c8c40 con 0x7fe16c06eb70 2024-03-11T18:36:45.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.884+0000 7fe170a3f640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe16c0c7630 con 0x7fe16c0c3f30 2024-03-11T18:36:45.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.884+0000 7fe170a3f640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe16c0c24f0 con 0x7fe16c151a50 2024-03-11T18:36:45.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.884+0000 7f8e3fe2a640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f8e300d5540 0x7f8e30045ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:45.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.884+0000 7f8e3fe2a640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8e3004b950 0x7f8e30046020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:45.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.884+0000 7f8e3fe2a640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e3004c330 con 0x7f8e3004cf50 2024-03-11T18:36:45.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.884+0000 7f8e3fe2a640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e300343e0 con 0x7f8e300d5540 2024-03-11T18:36:45.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.884+0000 7f8e3fe2a640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e30008240 con 0x7f8e3004b950 2024-03-11T18:36:45.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.884+0000 7f0d966b2640 1 Processor -- start 2024-03-11T18:36:45.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.884+0000 7f0d966b2640 1 -- start start 2024-03-11T18:36:45.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.888+0000 7f0d966b2640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0d900c8e20 0x7f0d9015a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:45.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.888+0000 7f0d966b2640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f0d900afb60 0x7f0d9006c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:45.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.888+0000 7f0d966b2640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d9015abb0 con 0x7f0d90153f10 2024-03-11T18:36:45.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.888+0000 7f0d966b2640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d9006c2d0 con 0x7f0d900afb60 2024-03-11T18:36:45.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.888+0000 7f0d966b2640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d900c81e0 con 0x7f0d900c8e20 2024-03-11T18:36:45.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.888+0000 7f153b7fe640 1 -- 172.21.15.90:0/88274557 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f153c070920 msgr2=0x7f153c071120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:45.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.888+0000 7f153b7fe640 1 --2- 172.21.15.90:0/88274557 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f153c070920 0x7f153c071120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:45.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.888+0000 7f153b7fe640 1 -- 172.21.15.90:0/88274557 >> v2:172.21.15.90:3300/0 conn(0x7f153c071660 msgr2=0x7f153c1591b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:45.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.888+0000 7f153b7fe640 1 --2- 172.21.15.90:0/88274557 >> v2:172.21.15.90:3300/0 conn(0x7f153c071660 0x7f153c1591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:45.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.888+0000 7f153b7fe640 1 -- 172.21.15.90:0/88274557 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f153c155950 con 0x7f153c14fc90 2024-03-11T18:36:45.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.900+0000 7fe1ada9e640 1 --2- 172.21.15.90:0/899926707 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe1a80c5400 0x7fe1a80c6070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0006140 tx=0x7fe1a0021720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11c42691826fee6f server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:45.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.900+0000 7fe16bfff640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fe16c06eb70 0x7fe16c1577d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.90:47598/0 (socket says 172.21.15.90:47598) 2024-03-11T18:36:45.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.900+0000 7fe16bfff640 1 -- 172.21.15.90:0/2187146598 learned_addr learned my addr 172.21.15.90:0/2187146598 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T18:36:45.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.904+0000 7f153b7fe640 1 --2- 172.21.15.90:0/88274557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f153c14fc90 0x7f153c071c60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f153c0cbd60 tx=0x7f1530021230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=195a62c414ab3450 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:45.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.908+0000 7fe049f7a640 1 --2- 172.21.15.90:0/2757442604 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe03c0d6c60 0x7fe03c0dd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:45.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.912+0000 7fe16b7fe640 1 --2- 172.21.15.90:0/2187146598 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe16c151a50 0x7fe16c0c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:45.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.912+0000 7f8e3e3c7640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8e3004b950 0x7f8e30046020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:45.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.912+0000 7fd6954df640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fd69006d9c0 0x7fd69006de10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.90:47580/0 (socket says 172.21.15.90:47580) 2024-03-11T18:36:45.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.912+0000 7fd6954df640 1 -- 172.21.15.90:0/3344537292 learned_addr learned my addr 172.21.15.90:0/3344537292 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T18:36:45.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.912+0000 7f8e3e3c7640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8e3004b950 0x7f8e30046020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.90:59088/0 (socket says 172.21.15.90:59088) 2024-03-11T18:36:45.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.912+0000 7f8e3e3c7640 1 -- 172.21.15.90:0/407410412 learned_addr learned my addr 172.21.15.90:0/407410412 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:45.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.916+0000 7fe049f7a640 1 -- 172.21.15.90:0/2757442604 >> v2:172.21.15.92:3300/0 conn(0x7fe03c04af10 msgr2=0x7fe03c04b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:45.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.916+0000 7fe049f7a640 1 --2- 172.21.15.90:0/2757442604 >> v2:172.21.15.92:3300/0 conn(0x7fe03c04af10 0x7fe03c04b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:45.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.916+0000 7fe049f7a640 1 -- 172.21.15.90:0/2757442604 >> v1:172.21.15.92:6789/0 conn(0x7fe03c04c480 legacy=0x7fe03c0332c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:45.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.916+0000 7fe049f7a640 1 -- 172.21.15.90:0/2757442604 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe03c0065f0 con 0x7fe03c0d6c60 2024-03-11T18:36:45.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.924+0000 7f153a7fc640 1 -- 172.21.15.90:0/88274557 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15300b3410 con 0x7f153c14fc90 2024-03-11T18:36:45.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.924+0000 7f153a7fc640 1 -- 172.21.15.90:0/88274557 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15300ad030 con 0x7f153c14fc90 2024-03-11T18:36:45.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.924+0000 7f153a7fc640 1 -- 172.21.15.90:0/88274557 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15300b3790 con 0x7f153c14fc90 2024-03-11T18:36:45.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.928+0000 7f1541eed640 1 -- 172.21.15.90:0/88274557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f153c14fc90 msgr2=0x7f153c071c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:45.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.928+0000 7f1541eed640 1 --2- 172.21.15.90:0/88274557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f153c14fc90 0x7f153c071c60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f153c0cbd60 tx=0x7f1530021230 comp rx=0 tx=0).stop 2024-03-11T18:36:45.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.928+0000 7f1541eed640 1 -- 172.21.15.90:0/88274557 shutdown_connections 2024-03-11T18:36:45.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.928+0000 7f1541eed640 1 --2- 172.21.15.90:0/88274557 >> v2:172.21.15.90:3300/0 conn(0x7f153c071660 0x7f153c1591b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:45.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.928+0000 7f1541eed640 1 --2- 172.21.15.90:0/88274557 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f153c070920 0x7f153c071120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:45.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.928+0000 7f1541eed640 1 --2- 172.21.15.90:0/88274557 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f153c14fc90 0x7f153c071c60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:45.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.928+0000 7f1541eed640 1 -- 172.21.15.90:0/88274557 >> 172.21.15.90:0/88274557 conn(0x7f153c06e680 msgr2=0x7f153c06ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:45.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.928+0000 7f1541eed640 1 -- 172.21.15.90:0/88274557 shutdown_connections 2024-03-11T18:36:45.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.928+0000 7f1541eed640 1 -- 172.21.15.90:0/88274557 wait complete. 2024-03-11T18:36:45.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.928+0000 7fe16b7fe640 1 -- 172.21.15.90:0/2187146598 >> v2:172.21.15.90:3300/0 conn(0x7fe16c0c3f30 msgr2=0x7fe16c06e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:45.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.928+0000 7fe16b7fe640 1 --2- 172.21.15.90:0/2187146598 >> v2:172.21.15.90:3300/0 conn(0x7fe16c0c3f30 0x7fe16c06e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:45.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.928+0000 7fe16b7fe640 1 -- 172.21.15.90:0/2187146598 >> v1:172.21.15.92:6789/0 conn(0x7fe16c06eb70 legacy=0x7fe16c1577d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:45.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.928+0000 7fe16b7fe640 1 -- 172.21.15.90:0/2187146598 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe16c0c8330 con 0x7fe16c151a50 2024-03-11T18:36:45.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.928+0000 7f1541eed640 1 Processor -- start 2024-03-11T18:36:45.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.928+0000 7f1541eed640 1 -- start start 2024-03-11T18:36:45.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.928+0000 7f1541eed640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f153c070920 0x7f153c1502b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:45.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.928+0000 7f1541eed640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f153c071660 0x7f153c157fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:45.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.928+0000 7f1541eed640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f153c14fc90 0x7f153c14ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:45.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.928+0000 7f1541eed640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153c1536b0 con 0x7f153c070920 2024-03-11T18:36:45.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.928+0000 7f1541eed640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153c14f4d0 con 0x7f153c071660 2024-03-11T18:36:45.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.928+0000 7f1541eed640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f153c14e270 con 0x7f153c14fc90 2024-03-11T18:36:45.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.928+0000 7f0d8ffff640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f0d90153f10 0x7f0d9015db00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.90:47624/0 (socket says 172.21.15.90:47624) 2024-03-11T18:36:45.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.928+0000 7f0d8ffff640 1 -- 172.21.15.90:0/2547699308 learned_addr learned my addr 172.21.15.90:0/2547699308 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T18:36:45.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.932+0000 7fe049f7a640 1 --2- 172.21.15.90:0/2757442604 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe03c0d6c60 0x7fe03c0dd540 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe044050200 tx=0x7fe0440979a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de1aea24f99c1202 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:45.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.936+0000 7fe16b7fe640 1 --2- 172.21.15.90:0/2187146598 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe16c151a50 0x7fe16c0c7b90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe15c0097c0 tx=0x7fe15c01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf92c1e499a75ef5 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:45.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.940+0000 7f8e3e3c7640 1 -- 172.21.15.90:0/407410412 >> v2:172.21.15.90:3300/0 conn(0x7f8e300d5540 msgr2=0x7f8e30045ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:45.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.940+0000 7f8e3e3c7640 1 --2- 172.21.15.90:0/407410412 >> v2:172.21.15.90:3300/0 conn(0x7f8e300d5540 0x7f8e30045ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:45.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.940+0000 7f8e3e3c7640 1 -- 172.21.15.90:0/407410412 >> v1:172.21.15.92:6789/0 conn(0x7f8e3004cf50 legacy=0x7f8e300de7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:45.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.940+0000 7f8e3e3c7640 1 -- 172.21.15.90:0/407410412 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e300d5920 con 0x7f8e3004b950 2024-03-11T18:36:45.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.940+0000 7fa051b24640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fa0440d5800 0x7fa044045af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.90:47596/0 (socket says 172.21.15.90:47596) 2024-03-11T18:36:45.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.940+0000 7fa051b24640 1 -- 172.21.15.90:0/2946479609 learned_addr learned my addr 172.21.15.90:0/2946479609 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T18:36:45.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.940+0000 7fd6964e1640 1 --2- 172.21.15.90:0/3344537292 >> v2:172.21.15.90:3300/0 conn(0x7fd6900c8b00 0x7fd6900c8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:45.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.948+0000 7fd6964e1640 1 -- 172.21.15.90:0/3344537292 >> v1:172.21.15.92:6789/0 conn(0x7fd69006d9c0 legacy=0x7fd69006de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:45.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.948+0000 7fd6964e1640 1 -- 172.21.15.90:0/3344537292 >> v1:172.21.15.90:6789/0 conn(0x7fd690153d20 legacy=0x7fd6900c6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:45.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.948+0000 7fd6964e1640 1 -- 172.21.15.90:0/3344537292 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd690159fa0 con 0x7fd6900c8b00 2024-03-11T18:36:45.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.952+0000 7fa052325640 1 --2- 172.21.15.90:0/2946479609 >> v2:172.21.15.90:3300/0 conn(0x7fa0440d5be0 0x7fa0440def90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:45.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.960+0000 7fe19ffff640 1 -- 172.21.15.90:0/899926707 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a0003ba0 con 0x7fe1a80c5400 2024-03-11T18:36:45.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.960+0000 7fe19ffff640 1 -- 172.21.15.90:0/899926707 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1a00ad050 con 0x7fe1a80c5400 2024-03-11T18:36:45.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.960+0000 7fe19ffff640 1 -- 172.21.15.90:0/899926707 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a0004030 con 0x7fe1a80c5400 2024-03-11T18:36:45.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.960+0000 7fe1af501640 1 -- 172.21.15.90:0/899926707 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe1a80c5400 msgr2=0x7fe1a80c6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:45.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.960+0000 7fe1af501640 1 --2- 172.21.15.90:0/899926707 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe1a80c5400 0x7fe1a80c6070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0006140 tx=0x7fe1a0021720 comp rx=0 tx=0).stop 2024-03-11T18:36:45.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.960+0000 7fe1af501640 1 -- 172.21.15.90:0/899926707 shutdown_connections 2024-03-11T18:36:45.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.960+0000 7fe1af501640 1 --2- 172.21.15.90:0/899926707 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe1a80c5400 0x7fe1a80c6070 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:45.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.960+0000 7fe1af501640 1 --2- 172.21.15.90:0/899926707 >> v2:172.21.15.90:3300/0 conn(0x7fe1a8070c80 0x7fe1a8071bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:45.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.960+0000 7fe1af501640 1 -- 172.21.15.90:0/899926707 >> 172.21.15.90:0/899926707 conn(0x7fe1a806e680 msgr2=0x7fe1a806ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:45.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.960+0000 7fe1af501640 1 -- 172.21.15.90:0/899926707 shutdown_connections 2024-03-11T18:36:45.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.960+0000 7fe1af501640 1 -- 172.21.15.90:0/899926707 wait complete. 2024-03-11T18:36:45.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.960+0000 7fa052325640 1 -- 172.21.15.90:0/2946479609 >> v2:172.21.15.92:3300/0 conn(0x7fa044046200 msgr2=0x7fa04404c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:45.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.960+0000 7fa052325640 1 --2- 172.21.15.90:0/2946479609 >> v2:172.21.15.92:3300/0 conn(0x7fa044046200 0x7fa04404c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:45.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.960+0000 7fa052325640 1 -- 172.21.15.90:0/2946479609 >> v1:172.21.15.92:6789/0 conn(0x7fa0440d5800 legacy=0x7fa044045af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:45.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.960+0000 7fa052325640 1 -- 172.21.15.90:0/2946479609 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa044035530 con 0x7fa0440d5be0 2024-03-11T18:36:45.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.960+0000 7fd6964e1640 1 --2- 172.21.15.90:0/3344537292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6900c8b00 0x7fd6900c8f10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd68c009d90 tx=0x7fd68c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab6699108bfe6d46 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:45.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.964+0000 7fe1af501640 1 Processor -- start 2024-03-11T18:36:45.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.964+0000 7fe1af501640 1 -- start start 2024-03-11T18:36:45.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.964+0000 7fe1af501640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1a8070c80 0x7fe1a82003f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:45.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.964+0000 7fe1af501640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe1a80c5400 0x7fe1a81f2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:45.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.964+0000 7fe1af501640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe1a8167240 0x7fe1a80c9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:45.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.964+0000 7fe1af501640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a81f1dc0 con 0x7fe1a8167240 2024-03-11T18:36:45.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.964+0000 7fe1af501640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a80c5210 con 0x7fe1a8070c80 2024-03-11T18:36:45.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.964+0000 7fe1af501640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a81f1500 con 0x7fe1a80c5400 2024-03-11T18:36:45.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.968+0000 7f8e3e3c7640 1 --2- 172.21.15.90:0/407410412 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8e3004b950 0x7f8e30046020 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8e38073760 tx=0x7f8e3809b5e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5dc535810598b4b server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:45.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.968+0000 7f0d94c4f640 1 --2- 172.21.15.90:0/2547699308 >> v2:172.21.15.90:3300/0 conn(0x7f0d900afb60 0x7f0d9006c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:45.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.972+0000 7f153b7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f153c070920 0x7f153c1502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:45.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.972+0000 7fd694cde640 1 -- 172.21.15.90:0/3344537292 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68c0a58e0 con 0x7fd6900c8b00 2024-03-11T18:36:45.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.972+0000 7fd694cde640 1 -- 172.21.15.90:0/3344537292 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd68c0a5a40 con 0x7fd6900c8b00 2024-03-11T18:36:45.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.972+0000 7fd694cde640 1 -- 172.21.15.90:0/3344537292 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68c0a5dd0 con 0x7fd6900c8b00 2024-03-11T18:36:45.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.972+0000 7f153b7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f153c070920 0x7f153c1502b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48094/0 (socket says 172.21.15.90:48094) 2024-03-11T18:36:45.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.972+0000 7f153b7fe640 1 -- 172.21.15.90:0/1132917042 learned_addr learned my addr 172.21.15.90:0/1132917042 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:45.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.972+0000 7fe048f78640 1 -- 172.21.15.90:0/2757442604 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe044121730 con 0x7fe03c0d6c60 2024-03-11T18:36:45.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.972+0000 7fe048f78640 1 -- 172.21.15.90:0/2757442604 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe044121890 con 0x7fe03c0d6c60 2024-03-11T18:36:45.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.972+0000 7fe048f78640 1 -- 172.21.15.90:0/2757442604 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe044121cc0 con 0x7fe03c0d6c60 2024-03-11T18:36:45.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.972+0000 7fd696ce2640 1 -- 172.21.15.90:0/3344537292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6900c8b00 msgr2=0x7fd6900c8f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:45.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.972+0000 7fd696ce2640 1 --2- 172.21.15.90:0/3344537292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6900c8b00 0x7fd6900c8f10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd68c009d90 tx=0x7fd68c01c930 comp rx=0 tx=0).stop 2024-03-11T18:36:45.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.972+0000 7fd696ce2640 1 -- 172.21.15.90:0/3344537292 shutdown_connections 2024-03-11T18:36:45.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.972+0000 7fd696ce2640 1 --2- 172.21.15.90:0/3344537292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6900c8b00 0x7fd6900c8f10 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:45.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.972+0000 7fd696ce2640 1 -- 172.21.15.90:0/3344537292 >> 172.21.15.90:0/3344537292 conn(0x7fd69006f420 msgr2=0x7fd69006f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:45.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.972+0000 7fd696ce2640 1 -- 172.21.15.90:0/3344537292 shutdown_connections 2024-03-11T18:36:45.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.972+0000 7fd696ce2640 1 -- 172.21.15.90:0/3344537292 wait complete. 2024-03-11T18:36:45.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.972+0000 7fe04c1de640 1 -- 172.21.15.90:0/2757442604 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe03c0d6c60 msgr2=0x7fe03c0dd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:45.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.972+0000 7fe04c1de640 1 --2- 172.21.15.90:0/2757442604 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe03c0d6c60 0x7fe03c0dd540 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe044050200 tx=0x7fe0440979a0 comp rx=0 tx=0).stop 2024-03-11T18:36:45.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.972+0000 7fe04c1de640 1 -- 172.21.15.90:0/2757442604 shutdown_connections 2024-03-11T18:36:45.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.972+0000 7fe04c1de640 1 --2- 172.21.15.90:0/2757442604 >> v2:172.21.15.92:3300/0 conn(0x7fe03c04af10 0x7fe03c04b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:45.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.972+0000 7fe04c1de640 1 --2- 172.21.15.90:0/2757442604 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe03c0d6c60 0x7fe03c0dd540 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:45.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.972+0000 7fe04c1de640 1 -- 172.21.15.90:0/2757442604 >> 172.21.15.90:0/2757442604 conn(0x7fe03c050190 msgr2=0x7fe03c0505a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:45.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.972+0000 7fe1ada9e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe1a8167240 0x7fe1a80c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:45.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.972+0000 7fe04c1de640 1 -- 172.21.15.90:0/2757442604 shutdown_connections 2024-03-11T18:36:45.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.972+0000 7fe04c1de640 1 -- 172.21.15.90:0/2757442604 wait complete. 2024-03-11T18:36:45.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.976+0000 7fd696ce2640 1 Processor -- start 2024-03-11T18:36:45.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.976+0000 7fd696ce2640 1 -- start start 2024-03-11T18:36:45.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.976+0000 7fd696ce2640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd69006d9c0 0x7fd690155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:45.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.976+0000 7fd696ce2640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6900c8b00 0x7fd69014c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:45.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.976+0000 7fd696ce2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd690153d20 0x7fd69014c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:45.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.976+0000 7fe04c1de640 1 Processor -- start 2024-03-11T18:36:45.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.976+0000 7fd696ce2640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6901449a0 con 0x7fd6900c8b00 2024-03-11T18:36:45.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.976+0000 7fd696ce2640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69014e090 con 0x7fd690153d20 2024-03-11T18:36:45.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.976+0000 7fd696ce2640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6900c5240 con 0x7fd69006d9c0 2024-03-11T18:36:45.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.976+0000 7fe04c1de640 1 -- start start 2024-03-11T18:36:45.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.976+0000 7fe04c1de640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe03c04af10 0x7fe03c0cbf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:45.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.976+0000 7fe04c1de640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe03c04c480 0x7fe03c0d31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:45.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.976+0000 7fe04c1de640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe03c0d6c60 0x7fe03c0d36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:45.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.976+0000 7fe04c1de640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c178a30 con 0x7fe03c04c480 2024-03-11T18:36:45.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.976+0000 7fe04c1de640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c0467f0 con 0x7fe03c04af10 2024-03-11T18:36:45.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.976+0000 7fe04c1de640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c046ad0 con 0x7fe03c0d6c60 2024-03-11T18:36:45.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.976+0000 7fe16affd640 1 -- 172.21.15.90:0/2187146598 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15c0ab600 con 0x7fe16c151a50 2024-03-11T18:36:45.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.976+0000 7fe16affd640 1 -- 172.21.15.90:0/2187146598 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe15c0a7050 con 0x7fe16c151a50 2024-03-11T18:36:45.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.976+0000 7fe16affd640 1 -- 172.21.15.90:0/2187146598 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15c0ab980 con 0x7fe16c151a50 2024-03-11T18:36:45.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.976+0000 7f153b7fe640 1 -- 172.21.15.90:0/1132917042 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f153c14fc90 msgr2=0x7f153c14ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:45.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.976+0000 7f153b7fe640 1 --2- 172.21.15.90:0/1132917042 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f153c14fc90 0x7f153c14ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:45.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.976+0000 7f153b7fe640 1 -- 172.21.15.90:0/1132917042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f153c071660 msgr2=0x7f153c157fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:45.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.976+0000 7f153b7fe640 1 --2- 172.21.15.90:0/1132917042 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f153c071660 0x7f153c157fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:45.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.976+0000 7f153b7fe640 1 -- 172.21.15.90:0/1132917042 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1530008550 con 0x7f153c070920 2024-03-11T18:36:45.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.976+0000 7fe170a3f640 1 -- 172.21.15.90:0/2187146598 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe16c151a50 msgr2=0x7fe16c0c7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:45.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.976+0000 7fe170a3f640 1 --2- 172.21.15.90:0/2187146598 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe16c151a50 0x7fe16c0c7b90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe15c0097c0 tx=0x7fe15c01c4a0 comp rx=0 tx=0).stop 2024-03-11T18:36:45.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.976+0000 7fe170a3f640 1 -- 172.21.15.90:0/2187146598 shutdown_connections 2024-03-11T18:36:45.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.976+0000 7fe170a3f640 1 --2- 172.21.15.90:0/2187146598 >> v2:172.21.15.90:3300/0 conn(0x7fe16c0c3f30 0x7fe16c06e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:45.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.976+0000 7fe170a3f640 1 --2- 172.21.15.90:0/2187146598 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe16c151a50 0x7fe16c0c7b90 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:45.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.980+0000 7fe170a3f640 1 -- 172.21.15.90:0/2187146598 >> 172.21.15.90:0/2187146598 conn(0x7fe16c066990 msgr2=0x7fe16c066da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:45.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.980+0000 7fe170a3f640 1 -- 172.21.15.90:0/2187146598 shutdown_connections 2024-03-11T18:36:45.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.980+0000 7fe170a3f640 1 -- 172.21.15.90:0/2187146598 wait complete. 2024-03-11T18:36:45.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.980+0000 7fe170a3f640 1 Processor -- start 2024-03-11T18:36:45.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.980+0000 7fe170a3f640 1 -- start start 2024-03-11T18:36:45.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.980+0000 7fe1ada9e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe1a8167240 0x7fe1a80c9380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48098/0 (socket says 172.21.15.90:48098) 2024-03-11T18:36:45.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.980+0000 7fe1ada9e640 1 -- 172.21.15.90:0/3948357089 learned_addr learned my addr 172.21.15.90:0/3948357089 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:45.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.980+0000 7fe170a3f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe16c06eb70 0x7fe16c152360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:45.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.980+0000 7fe170a3f640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe16c0c3f30 0x7fe16c145530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:45.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.980+0000 7fe170a3f640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe16c151a50 0x7fe16c143c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:45.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.980+0000 7fe170a3f640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16c152f00 con 0x7fe16c151a50 2024-03-11T18:36:45.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.980+0000 7fe170a3f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16c1450a0 con 0x7fe16c06eb70 2024-03-11T18:36:45.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.980+0000 7fe170a3f640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe16c157c00 con 0x7fe16c0c3f30 2024-03-11T18:36:45.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.980+0000 7f153b7fe640 1 --2- 172.21.15.90:0/1132917042 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f153c070920 0x7f153c1502b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f153001ee60 tx=0x7f153001dfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:45.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.984+0000 7fe1ada9e640 1 -- 172.21.15.90:0/3948357089 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe1a80c5400 msgr2=0x7fe1a81f2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:45.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.984+0000 7fe1ada9e640 1 --2- 172.21.15.90:0/3948357089 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe1a80c5400 0x7fe1a81f2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:45.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.984+0000 7fe1ada9e640 1 -- 172.21.15.90:0/3948357089 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1a8070c80 msgr2=0x7fe1a82003f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:45.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.984+0000 7fe1ada9e640 1 --2- 172.21.15.90:0/3948357089 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe1a8070c80 0x7fe1a82003f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:45.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.984+0000 7fe1ada9e640 1 -- 172.21.15.90:0/3948357089 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1a00077d0 con 0x7fe1a8167240 2024-03-11T18:36:45.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.988+0000 7fe1ada9e640 1 --2- 172.21.15.90:0/3948357089 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe1a8167240 0x7fe1a80c9380 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0007170 tx=0x7fe1a001e4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:45.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.988+0000 7f0d87fff640 1 --2- 172.21.15.90:0/2547699308 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0d900c8e20 0x7f0d9015a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:46.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.992+0000 7f0d94c4f640 1 -- 172.21.15.90:0/2547699308 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0d900c8e20 msgr2=0x7f0d9015a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:46.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.992+0000 7f0d94c4f640 1 --2- 172.21.15.90:0/2547699308 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0d900c8e20 0x7f0d9015a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.992+0000 7f0d94c4f640 1 -- 172.21.15.90:0/2547699308 >> v1:172.21.15.92:6789/0 conn(0x7f0d90153f10 legacy=0x7f0d9015db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.992+0000 7f0d94c4f640 1 -- 172.21.15.90:0/2547699308 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d9006bd10 con 0x7f0d900afb60 2024-03-11T18:36:46.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.996+0000 7fd6954df640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6900c8b00 0x7fd69014c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:46.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.996+0000 7fd6954df640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6900c8b00 0x7fd69014c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48100/0 (socket says 172.21.15.90:48100) 2024-03-11T18:36:46.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.996+0000 7fd6954df640 1 -- 172.21.15.90:0/3395884597 learned_addr learned my addr 172.21.15.90:0/3395884597 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:46.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.996+0000 7f1523fff640 1 -- 172.21.15.90:0/1132917042 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153000b580 con 0x7f153c070920 2024-03-11T18:36:46.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.996+0000 7f1523fff640 1 -- 172.21.15.90:0/1132917042 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f153000b720 con 0x7f153c070920 2024-03-11T18:36:46.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.996+0000 7f1523fff640 1 -- 172.21.15.90:0/1132917042 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15300be480 con 0x7f153c070920 2024-03-11T18:36:46.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.996+0000 7f1541eed640 1 -- 172.21.15.90:0/1132917042 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f153c0c6940 con 0x7f153c070920 2024-03-11T18:36:46.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:45.996+0000 7f1541eed640 1 -- 172.21.15.90:0/1132917042 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f153c14e510 con 0x7f153c070920 2024-03-11T18:36:46.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.000+0000 7f1541eed640 1 -- 172.21.15.90:0/1132917042 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f153c1561f0 con 0x7f153c070920 2024-03-11T18:36:46.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.000+0000 7fa052325640 1 --2- 172.21.15.90:0/2946479609 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0440d5be0 0x7fa0440def90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa04c073680 tx=0x7fa04c09a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da9073ec99173a31 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:46.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.000+0000 7fe19dffb640 1 -- 172.21.15.90:0/3948357089 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a0004440 con 0x7fe1a8167240 2024-03-11T18:36:46.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.000+0000 7fe19dffb640 1 -- 172.21.15.90:0/3948357089 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1a00045a0 con 0x7fe1a8167240 2024-03-11T18:36:46.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.000+0000 7fe19dffb640 1 -- 172.21.15.90:0/3948357089 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a00b1440 con 0x7fe1a8167240 2024-03-11T18:36:46.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.000+0000 7fe1af501640 1 -- 172.21.15.90:0/3948357089 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1a80c4c50 con 0x7fe1a8167240 2024-03-11T18:36:46.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.000+0000 7fe1af501640 1 -- 172.21.15.90:0/3948357089 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1a81f1a50 con 0x7fe1a8167240 2024-03-11T18:36:46.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.000+0000 7fe1af501640 1 -- 172.21.15.90:0/3948357089 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe1a80c74e0 con 0x7fe1a8167240 2024-03-11T18:36:46.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.004+0000 7fd6954df640 1 -- 172.21.15.90:0/3395884597 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd69006d9c0 msgr2=0x7fd690155820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.004+0000 7fd6954df640 1 --2- 172.21.15.90:0/3395884597 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd69006d9c0 0x7fd690155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.004+0000 7fd6954df640 1 -- 172.21.15.90:0/3395884597 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd690153d20 msgr2=0x7fd69014c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.004+0000 7fd6954df640 1 --2- 172.21.15.90:0/3395884597 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd690153d20 0x7fd69014c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.004+0000 7fd6954df640 1 -- 172.21.15.90:0/3395884597 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd68c009a70 con 0x7fd6900c8b00 2024-03-11T18:36:46.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.004+0000 7fe049779640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe03c04c480 0x7fe03c0d31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:46.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.008+0000 7fe049779640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe03c04c480 0x7fe03c0d31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48112/0 (socket says 172.21.15.90:48112) 2024-03-11T18:36:46.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.008+0000 7fe049779640 1 -- 172.21.15.90:0/4119030134 learned_addr learned my addr 172.21.15.90:0/4119030134 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:46.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.008+0000 7fd6954df640 1 --2- 172.21.15.90:0/3395884597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6900c8b00 0x7fd69014c440 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd68800c400 tx=0x7fd68800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:46.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.012+0000 7fe16bfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe16c151a50 0x7fe16c143c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:46.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.016+0000 7f0d94c4f640 1 --2- 172.21.15.90:0/2547699308 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d900afb60 0x7f0d9006c520 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0d88009d90 tx=0x7f0d8801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83dfa8fc12221fa8 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:46.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.016+0000 7fe16bfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe16c151a50 0x7fe16c143c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48126/0 (socket says 172.21.15.90:48126) 2024-03-11T18:36:46.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.016+0000 7fe16bfff640 1 -- 172.21.15.90:0/3393719004 learned_addr learned my addr 172.21.15.90:0/3393719004 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:46.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.024+0000 7fe049779640 1 -- 172.21.15.90:0/4119030134 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe03c0d6c60 msgr2=0x7fe03c0d36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.024+0000 7fe049779640 1 --2- 172.21.15.90:0/4119030134 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe03c0d6c60 0x7fe03c0d36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.024+0000 7fe049779640 1 -- 172.21.15.90:0/4119030134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe03c04af10 msgr2=0x7fe03c0cbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.024+0000 7fe049779640 1 --2- 172.21.15.90:0/4119030134 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe03c04af10 0x7fe03c0cbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.024+0000 7fe049779640 1 -- 172.21.15.90:0/4119030134 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe03c046db0 con 0x7fe03c04c480 2024-03-11T18:36:46.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.044+0000 7fe16bfff640 1 -- 172.21.15.90:0/3393719004 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe16c0c3f30 msgr2=0x7fe16c145530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.044+0000 7fe16bfff640 1 --2- 172.21.15.90:0/3393719004 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe16c0c3f30 0x7fe16c145530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.044+0000 7fe16bfff640 1 -- 172.21.15.90:0/3393719004 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe16c06eb70 msgr2=0x7fe16c152360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.044+0000 7fe16bfff640 1 --2- 172.21.15.90:0/3393719004 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe16c06eb70 0x7fe16c152360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.044+0000 7fe16bfff640 1 -- 172.21.15.90:0/3393719004 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe15c009420 con 0x7fe16c151a50 2024-03-11T18:36:46.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.048+0000 7fe049779640 1 --2- 172.21.15.90:0/4119030134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe03c04c480 0x7fe03c0d31a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe04000c420 tx=0x7fe04000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:46.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.048+0000 7fa050b22640 1 -- 172.21.15.90:0/2946479609 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04c123e70 con 0x7fa0440d5be0 2024-03-11T18:36:46.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.048+0000 7fa050b22640 1 -- 172.21.15.90:0/2946479609 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa04c12b440 con 0x7fa0440d5be0 2024-03-11T18:36:46.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.048+0000 7fa050b22640 1 -- 172.21.15.90:0/2946479609 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04c12b870 con 0x7fa0440d5be0 2024-03-11T18:36:46.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.048+0000 7fa052b26640 1 -- 172.21.15.90:0/2946479609 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0440d5be0 msgr2=0x7fa0440def90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.048+0000 7fa052b26640 1 --2- 172.21.15.90:0/2946479609 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0440d5be0 0x7fa0440def90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa04c073680 tx=0x7fa04c09a1d0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.048+0000 7fe16bfff640 1 --2- 172.21.15.90:0/3393719004 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe16c151a50 0x7fe16c143c60 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe16400cad0 tx=0x7fe16400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:46.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.048+0000 7fa052b26640 1 -- 172.21.15.90:0/2946479609 shutdown_connections 2024-03-11T18:36:46.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.048+0000 7fa052b26640 1 --2- 172.21.15.90:0/2946479609 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0440d5be0 0x7fa0440def90 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.048+0000 7fa052b26640 1 --2- 172.21.15.90:0/2946479609 >> v2:172.21.15.92:3300/0 conn(0x7fa044046200 0x7fa04404c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.048+0000 7fa052b26640 1 -- 172.21.15.90:0/2946479609 >> 172.21.15.90:0/2946479609 conn(0x7fa044050190 msgr2=0x7fa0440505a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:46.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.048+0000 7fa052b26640 1 -- 172.21.15.90:0/2946479609 shutdown_connections 2024-03-11T18:36:46.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.048+0000 7fa052b26640 1 -- 172.21.15.90:0/2946479609 wait complete. 2024-03-11T18:36:46.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.052+0000 7fa052b26640 1 Processor -- start 2024-03-11T18:36:46.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.052+0000 7fa052b26640 1 -- start start 2024-03-11T18:36:46.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.052+0000 7fa052b26640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa0440d5800 0x7fa0440ce410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.052+0000 7fa052b26640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0440d5be0 0x7fa0440d0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.052+0000 7fa052b26640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0440496f0 0x7fa0440de620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.052+0000 7fa052b26640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa04404a5b0 con 0x7fa0440496f0 2024-03-11T18:36:46.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.052+0000 7fa052b26640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0440cd610 con 0x7fa0440d5be0 2024-03-11T18:36:46.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.052+0000 7fa052b26640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0440ce950 con 0x7fa0440d5800 2024-03-11T18:36:46.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.064+0000 7fe19dffb640 1 -- 172.21.15.90:0/3948357089 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fe1a00b15a0 con 0x7fe1a8167240 2024-03-11T18:36:46.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.064+0000 7fe19dffb640 1 --2- 172.21.15.90:0/3948357089 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe18c066e90 0x7fe18c069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.064+0000 7fe1ad29d640 1 --2- 172.21.15.90:0/3948357089 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe18c066e90 0x7fe18c069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:46.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.068+0000 7fe1ad29d640 1 --2- 172.21.15.90:0/3948357089 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe18c066e90 0x7fe18c069350 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4001560 tx=0x7fe1a4007a50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:46.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.068+0000 7fe19dffb640 1 -- 172.21.15.90:0/3948357089 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe1a00f6ec0 con 0x7fe1a8167240 2024-03-11T18:36:46.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.068+0000 7fe19dffb640 1 --2- 172.21.15.90:0/3948357089 >> [v2:172.21.15.90:6804/438682081,v1:172.21.15.90:6807/438682081] conn(0x7fe18c070580 0x7fe18c0729c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.068+0000 7fe19dffb640 1 -- 172.21.15.90:0/3948357089 --> [v2:172.21.15.90:6804/438682081,v1:172.21.15.90:6807/438682081] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe18c073090 con 0x7fe18c070580 2024-03-11T18:36:46.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.068+0000 7fe19dffb640 1 -- 172.21.15.90:0/3948357089 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1a00073f0 con 0x7fe1a8167240 2024-03-11T18:36:46.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.068+0000 7fe1aca9c640 1 --2- 172.21.15.90:0/3948357089 >> [v2:172.21.15.90:6804/438682081,v1:172.21.15.90:6807/438682081] conn(0x7fe18c070580 0x7fe18c0729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:46.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.068+0000 7fe1aca9c640 1 --2- 172.21.15.90:0/3948357089 >> [v2:172.21.15.90:6804/438682081,v1:172.21.15.90:6807/438682081] conn(0x7fe18c070580 0x7fe18c0729c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:46.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.072+0000 7fe19dffb640 1 -- 172.21.15.90:0/3948357089 <== osd.1 v2:172.21.15.90:6804/438682081 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fe18c073090 con 0x7fe18c070580 2024-03-11T18:36:46.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.076+0000 7fd6867fc640 1 -- 172.21.15.90:0/3395884597 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd688005cf0 con 0x7fd6900c8b00 2024-03-11T18:36:46.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.076+0000 7fd6867fc640 1 -- 172.21.15.90:0/3395884597 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd688005e50 con 0x7fd6900c8b00 2024-03-11T18:36:46.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.076+0000 7fd6867fc640 1 -- 172.21.15.90:0/3395884597 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6880072d0 con 0x7fd6900c8b00 2024-03-11T18:36:46.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.076+0000 7fd696ce2640 1 -- 172.21.15.90:0/3395884597 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd69014e2e0 con 0x7fd6900c8b00 2024-03-11T18:36:46.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.076+0000 7fd696ce2640 1 -- 172.21.15.90:0/3395884597 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6900c4ed0 con 0x7fd6900c8b00 2024-03-11T18:36:46.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.076+0000 7fd696ce2640 1 -- 172.21.15.90:0/3395884597 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd6900c7180 con 0x7fd6900c8b00 2024-03-11T18:36:46.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.076+0000 7f0d8f7fe640 1 -- 172.21.15.90:0/2547699308 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d880a5a30 con 0x7f0d900afb60 2024-03-11T18:36:46.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.076+0000 7f0d8f7fe640 1 -- 172.21.15.90:0/2547699308 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d880a5b90 con 0x7f0d900afb60 2024-03-11T18:36:46.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.076+0000 7f0d8f7fe640 1 -- 172.21.15.90:0/2547699308 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d880ad440 con 0x7f0d900afb60 2024-03-11T18:36:46.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.080+0000 7fe168ff9640 1 -- 172.21.15.90:0/3393719004 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16400ac30 con 0x7fe16c151a50 2024-03-11T18:36:46.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.080+0000 7fe168ff9640 1 -- 172.21.15.90:0/3393719004 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe16400ad90 con 0x7fe16c151a50 2024-03-11T18:36:46.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.080+0000 7fe168ff9640 1 -- 172.21.15.90:0/3393719004 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe164012620 con 0x7fe16c151a50 2024-03-11T18:36:46.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.080+0000 7fe170a3f640 1 -- 172.21.15.90:0/3393719004 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe16c157640 con 0x7fe16c151a50 2024-03-11T18:36:46.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.080+0000 7fe170a3f640 1 -- 172.21.15.90:0/3393719004 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe16c144a50 con 0x7fe16c151a50 2024-03-11T18:36:46.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.080+0000 7f0d966b2640 1 -- 172.21.15.90:0/2547699308 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d900afb60 msgr2=0x7f0d9006c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.080+0000 7f0d966b2640 1 --2- 172.21.15.90:0/2547699308 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d900afb60 0x7f0d9006c520 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0d88009d90 tx=0x7f0d8801c970 comp rx=0 tx=0).stop 2024-03-11T18:36:46.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.080+0000 7f0d966b2640 1 -- 172.21.15.90:0/2547699308 shutdown_connections 2024-03-11T18:36:46.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.080+0000 7f0d966b2640 1 --2- 172.21.15.90:0/2547699308 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d900afb60 0x7f0d9006c520 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.080+0000 7f0d966b2640 1 --2- 172.21.15.90:0/2547699308 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0d900c8e20 0x7f0d9015a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.080+0000 7f0d966b2640 1 -- 172.21.15.90:0/2547699308 >> 172.21.15.90:0/2547699308 conn(0x7f0d90070e80 msgr2=0x7f0d90066990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:46.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.080+0000 7f0d966b2640 1 -- 172.21.15.90:0/2547699308 shutdown_connections 2024-03-11T18:36:46.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.080+0000 7f0d966b2640 1 -- 172.21.15.90:0/2547699308 wait complete. 2024-03-11T18:36:46.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.080+0000 7fe170a3f640 1 -- 172.21.15.90:0/3393719004 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe16c151e30 con 0x7fe16c151a50 2024-03-11T18:36:46.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.080+0000 7f0d966b2640 1 Processor -- start 2024-03-11T18:36:46.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.080+0000 7f0d966b2640 1 -- start start 2024-03-11T18:36:46.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.080+0000 7f0d966b2640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d900afb60 0x7f0d901f5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.080+0000 7f0d966b2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d900c8e20 0x7f0d900cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.080+0000 7f0d966b2640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0d90153f10 0x7f0d9014ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.080+0000 7f0d966b2640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d900ce160 con 0x7f0d900afb60 2024-03-11T18:36:46.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.080+0000 7f0d966b2640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d900cc230 con 0x7f0d900c8e20 2024-03-11T18:36:46.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.080+0000 7f0d966b2640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d901f56a0 con 0x7f0d90153f10 2024-03-11T18:36:46.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.088+0000 7fe1af501640 1 -- 172.21.15.90:0/3948357089 --> [v2:172.21.15.90:6804/438682081,v1:172.21.15.90:6807/438682081] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe1a80cae70 con 0x7fe18c070580 2024-03-11T18:36:46.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.088+0000 7fe19dffb640 1 -- 172.21.15.90:0/3948357089 <== osd.1 v2:172.21.15.90:6804/438682081 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe1a80cae70 con 0x7fe18c070580 2024-03-11T18:36:46.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.088+0000 7fe1af501640 1 -- 172.21.15.90:0/3948357089 >> [v2:172.21.15.90:6804/438682081,v1:172.21.15.90:6807/438682081] conn(0x7fe18c070580 msgr2=0x7fe18c0729c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.088+0000 7fe1af501640 1 --2- 172.21.15.90:0/3948357089 >> [v2:172.21.15.90:6804/438682081,v1:172.21.15.90:6807/438682081] conn(0x7fe18c070580 0x7fe18c0729c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.088+0000 7fe1af501640 1 -- 172.21.15.90:0/3948357089 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe18c066e90 msgr2=0x7fe18c069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.088+0000 7fe1af501640 1 --2- 172.21.15.90:0/3948357089 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe18c066e90 0x7fe18c069350 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4001560 tx=0x7fe1a4007a50 comp rx=0 tx=0).stop 2024-03-11T18:36:46.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.088+0000 7fe1af501640 1 -- 172.21.15.90:0/3948357089 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe1a8167240 msgr2=0x7fe1a80c9380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.088+0000 7fe1af501640 1 --2- 172.21.15.90:0/3948357089 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe1a8167240 0x7fe1a80c9380 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0007170 tx=0x7fe1a001e4c0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.088+0000 7fe1ada9e640 1 -- 172.21.15.90:0/3948357089 reap_dead start 2024-03-11T18:36:46.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.088+0000 7fe1af501640 1 -- 172.21.15.90:0/3948357089 shutdown_connections 2024-03-11T18:36:46.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.088+0000 7fe1af501640 1 -- 172.21.15.90:0/3948357089 >> 172.21.15.90:0/3948357089 conn(0x7fe1a806e680 msgr2=0x7fe1a80cb440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:46.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.088+0000 7fe1af501640 1 -- 172.21.15.90:0/3948357089 shutdown_connections 2024-03-11T18:36:46.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.088+0000 7fe1af501640 1 -- 172.21.15.90:0/3948357089 wait complete. 2024-03-11T18:36:46.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.096+0000 7fe03a7fc640 1 -- 172.21.15.90:0/4119030134 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe040005cf0 con 0x7fe03c04c480 2024-03-11T18:36:46.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.096+0000 7fa052325640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0440496f0 0x7fa0440de620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:46.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.096+0000 7fe03a7fc640 1 -- 172.21.15.90:0/4119030134 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe040005e50 con 0x7fe03c04c480 2024-03-11T18:36:46.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.096+0000 7fe04c1de640 1 -- 172.21.15.90:0/4119030134 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe03c047090 con 0x7fe03c04c480 2024-03-11T18:36:46.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.096+0000 7fe03a7fc640 1 -- 172.21.15.90:0/4119030134 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0400072f0 con 0x7fe03c04c480 2024-03-11T18:36:46.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.096+0000 7fe04c1de640 1 -- 172.21.15.90:0/4119030134 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe03c0da660 con 0x7fe03c04c480 2024-03-11T18:36:46.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.096+0000 7fe04c1de640 1 -- 172.21.15.90:0/4119030134 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe03c007690 con 0x7fe03c04c480 2024-03-11T18:36:46.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.100+0000 7fa051323640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0440d5be0 0x7fa0440d0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:46.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.100+0000 7fa051323640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0440d5be0 0x7fa0440d0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.90:60364/0 (socket says 172.21.15.90:60364) 2024-03-11T18:36:46.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.100+0000 7fa051323640 1 -- 172.21.15.90:0/1596626827 learned_addr learned my addr 172.21.15.90:0/1596626827 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:46.111 INFO:teuthology.orchestra.run.smithi090.stdout:64424509463 2024-03-11T18:36:46.112 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-11T18:36:46.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.112+0000 7f1523fff640 1 -- 172.21.15.90:0/1132917042 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f15300be5e0 con 0x7f153c070920 2024-03-11T18:36:46.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.112+0000 7f1523fff640 1 --2- 172.21.15.90:0/1132917042 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1528066e40 0x7f1528069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.112+0000 7f153affd640 1 --2- 172.21.15.90:0/1132917042 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1528066e40 0x7f1528069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:46.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.112+0000 7f153affd640 1 --2- 172.21.15.90:0/1132917042 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1528066e40 0x7f1528069300 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f153c0c1d40 tx=0x7f153400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:46.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.112+0000 7fa051323640 1 -- 172.21.15.90:0/1596626827 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa0440d5800 msgr2=0x7fa0440ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.112+0000 7fa051323640 1 --2- 172.21.15.90:0/1596626827 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fa0440d5800 0x7fa0440ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.112+0000 7fa051323640 1 -- 172.21.15.90:0/1596626827 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0440496f0 msgr2=0x7fa0440de620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.112+0000 7fa051323640 1 --2- 172.21.15.90:0/1596626827 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0440496f0 0x7fa0440de620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.112+0000 7fa051323640 1 -- 172.21.15.90:0/1596626827 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa044007480 con 0x7fa0440d5be0 2024-03-11T18:36:46.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.124+0000 7fd6867fc640 1 -- 172.21.15.90:0/3395884597 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fd688007430 con 0x7fd6900c8b00 2024-03-11T18:36:46.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.124+0000 7fd6867fc640 1 --2- 172.21.15.90:0/3395884597 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fd674066e90 0x7fd674069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.124+0000 7fd695ce0640 1 --2- 172.21.15.90:0/3395884597 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fd674066e90 0x7fd674069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:46.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.128+0000 7fd695ce0640 1 --2- 172.21.15.90:0/3395884597 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fd674066e90 0x7fd674069350 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd680007fd0 tx=0x7fd6800079a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:46.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.128+0000 7fa051323640 1 --2- 172.21.15.90:0/1596626827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0440d5be0 0x7fa0440d0220 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa03c00b5b0 tx=0x7fa03c00ba70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:46.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.136+0000 7f0d87fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d900c8e20 0x7f0d900cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:46.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.136+0000 7f1523fff640 1 -- 172.21.15.90:0/1132917042 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f15300f4bd0 con 0x7f153c070920 2024-03-11T18:36:46.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.136+0000 7f1523fff640 1 --2- 172.21.15.90:0/1132917042 >> [v2:172.21.15.92:6800/502685275,v1:172.21.15.92:6801/502685275] conn(0x7f1528070430 0x7f1528072870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.136+0000 7f1523fff640 1 -- 172.21.15.90:0/1132917042 --> [v2:172.21.15.92:6800/502685275,v1:172.21.15.92:6801/502685275] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1528072f40 con 0x7f1528070430 2024-03-11T18:36:46.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.136+0000 7f1523fff640 1 -- 172.21.15.90:0/1132917042 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15300f0770 con 0x7f153c070920 2024-03-11T18:36:46.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.136+0000 7f153bfff640 1 --2- 172.21.15.90:0/1132917042 >> [v2:172.21.15.92:6800/502685275,v1:172.21.15.92:6801/502685275] conn(0x7f1528070430 0x7f1528072870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:46.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.136+0000 7f153bfff640 1 --2- 172.21.15.90:0/1132917042 >> [v2:172.21.15.92:6800/502685275,v1:172.21.15.92:6801/502685275] conn(0x7f1528070430 0x7f1528072870 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:46.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.140+0000 7fd6867fc640 1 -- 172.21.15.90:0/3395884597 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd688055340 con 0x7fd6900c8b00 2024-03-11T18:36:46.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.140+0000 7f0d8ffff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d900afb60 0x7f0d901f5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:46.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.140+0000 7fd6867fc640 1 --2- 172.21.15.90:0/3395884597 >> [v2:172.21.15.92:6824/3475925719,v1:172.21.15.92:6825/3475925719] conn(0x7fd674070590 0x7fd6740729d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.140+0000 7fd6867fc640 1 -- 172.21.15.90:0/3395884597 --> [v2:172.21.15.92:6824/3475925719,v1:172.21.15.92:6825/3475925719] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd6740730a0 con 0x7fd674070590 2024-03-11T18:36:46.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.140+0000 7fd6867fc640 1 -- 172.21.15.90:0/3395884597 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd6880515e0 con 0x7fd6900c8b00 2024-03-11T18:36:46.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.140+0000 7fd6964e1640 1 --2- 172.21.15.90:0/3395884597 >> [v2:172.21.15.92:6824/3475925719,v1:172.21.15.92:6825/3475925719] conn(0x7fd674070590 0x7fd6740729d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:46.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.140+0000 7f1523fff640 1 -- 172.21.15.90:0/1132917042 <== osd.4 v2:172.21.15.92:6800/502685275 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f1528072f40 con 0x7f1528070430 2024-03-11T18:36:46.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.140+0000 7fd6964e1640 1 --2- 172.21.15.90:0/3395884597 >> [v2:172.21.15.92:6824/3475925719,v1:172.21.15.92:6825/3475925719] conn(0x7fd674070590 0x7fd6740729d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:46.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.140+0000 7f0d8ffff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d900afb60 0x7f0d901f5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48140/0 (socket says 172.21.15.90:48140) 2024-03-11T18:36:46.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.140+0000 7f0d8ffff640 1 -- 172.21.15.90:0/2551566929 learned_addr learned my addr 172.21.15.90:0/2551566929 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:46.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.144+0000 7fd6867fc640 1 -- 172.21.15.90:0/3395884597 <== osd.7 v2:172.21.15.92:6824/3475925719 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fd6740730a0 con 0x7fd674070590 2024-03-11T18:36:46.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.144+0000 7fa0427fc640 1 -- 172.21.15.90:0/1596626827 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03c010890 con 0x7fa0440d5be0 2024-03-11T18:36:46.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.144+0000 7fa0427fc640 1 -- 172.21.15.90:0/1596626827 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa03c010a10 con 0x7fa0440d5be0 2024-03-11T18:36:46.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.144+0000 7fa052b26640 1 -- 172.21.15.90:0/1596626827 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0440071e0 con 0x7fa0440d5be0 2024-03-11T18:36:46.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.148+0000 7fa0427fc640 1 -- 172.21.15.90:0/1596626827 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03c010dc0 con 0x7fa0440d5be0 2024-03-11T18:36:46.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.148+0000 7fa052b26640 1 -- 172.21.15.90:0/1596626827 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0441764a0 con 0x7fa0440d5be0 2024-03-11T18:36:46.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.148+0000 7fa052b26640 1 -- 172.21.15.90:0/1596626827 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa044035dd0 con 0x7fa0440d5be0 2024-03-11T18:36:46.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.156+0000 7f0d87fff640 1 -- 172.21.15.90:0/2551566929 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0d90153f10 msgr2=0x7f0d9014ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.156+0000 7f0d87fff640 1 --2- 172.21.15.90:0/2551566929 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0d90153f10 0x7f0d9014ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.156+0000 7f0d87fff640 1 -- 172.21.15.90:0/2551566929 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d900afb60 msgr2=0x7f0d901f5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.156+0000 7f0d87fff640 1 --2- 172.21.15.90:0/2551566929 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d900afb60 0x7f0d901f5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.156+0000 7f0d87fff640 1 -- 172.21.15.90:0/2551566929 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d88009a70 con 0x7f0d900c8e20 2024-03-11T18:36:46.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.168+0000 7f1541eed640 1 -- 172.21.15.90:0/1132917042 --> [v2:172.21.15.92:6800/502685275,v1:172.21.15.92:6801/502685275] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f153c155f10 con 0x7f1528070430 2024-03-11T18:36:46.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.168+0000 7f1523fff640 1 -- 172.21.15.90:0/1132917042 <== osd.4 v2:172.21.15.92:6800/502685275 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f152c0097e0 con 0x7f1528070430 2024-03-11T18:36:46.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.168+0000 7f1541eed640 1 -- 172.21.15.90:0/1132917042 >> [v2:172.21.15.92:6800/502685275,v1:172.21.15.92:6801/502685275] conn(0x7f1528070430 msgr2=0x7f1528072870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.168+0000 7f1541eed640 1 --2- 172.21.15.90:0/1132917042 >> [v2:172.21.15.92:6800/502685275,v1:172.21.15.92:6801/502685275] conn(0x7f1528070430 0x7f1528072870 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.168+0000 7f1541eed640 1 -- 172.21.15.90:0/1132917042 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1528066e40 msgr2=0x7f1528069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.168+0000 7f1541eed640 1 --2- 172.21.15.90:0/1132917042 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1528066e40 0x7f1528069300 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f153c0c1d40 tx=0x7f153400b040 comp rx=0 tx=0).stop 2024-03-11T18:36:46.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.168+0000 7f1541eed640 1 -- 172.21.15.90:0/1132917042 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f153c070920 msgr2=0x7f153c1502b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.168+0000 7f1541eed640 1 --2- 172.21.15.90:0/1132917042 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f153c070920 0x7f153c1502b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f153001ee60 tx=0x7f153001dfd0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.168+0000 7f153bfff640 1 -- 172.21.15.90:0/1132917042 reap_dead start 2024-03-11T18:36:46.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.168+0000 7f1541eed640 1 -- 172.21.15.90:0/1132917042 shutdown_connections 2024-03-11T18:36:46.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.168+0000 7f1541eed640 1 -- 172.21.15.90:0/1132917042 >> 172.21.15.90:0/1132917042 conn(0x7f153c06e680 msgr2=0x7f153c06ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:46.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.168+0000 7f1541eed640 1 -- 172.21.15.90:0/1132917042 shutdown_connections 2024-03-11T18:36:46.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.168+0000 7f1541eed640 1 -- 172.21.15.90:0/1132917042 wait complete. 2024-03-11T18:36:46.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.168+0000 7fd696ce2640 1 -- 172.21.15.90:0/3395884597 --> [v2:172.21.15.92:6824/3475925719,v1:172.21.15.92:6825/3475925719] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd69014e800 con 0x7fd674070590 2024-03-11T18:36:46.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.168+0000 7fd6867fc640 1 -- 172.21.15.90:0/3395884597 <== osd.7 v2:172.21.15.92:6824/3475925719 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd69014e800 con 0x7fd674070590 2024-03-11T18:36:46.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.168+0000 7fd696ce2640 1 -- 172.21.15.90:0/3395884597 >> [v2:172.21.15.92:6824/3475925719,v1:172.21.15.92:6825/3475925719] conn(0x7fd674070590 msgr2=0x7fd6740729d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.168+0000 7fd696ce2640 1 --2- 172.21.15.90:0/3395884597 >> [v2:172.21.15.92:6824/3475925719,v1:172.21.15.92:6825/3475925719] conn(0x7fd674070590 0x7fd6740729d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.168+0000 7fd696ce2640 1 -- 172.21.15.90:0/3395884597 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fd674066e90 msgr2=0x7fd674069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.168+0000 7fd696ce2640 1 --2- 172.21.15.90:0/3395884597 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fd674066e90 0x7fd674069350 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd680007fd0 tx=0x7fd6800079a0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.168+0000 7fd696ce2640 1 -- 172.21.15.90:0/3395884597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6900c8b00 msgr2=0x7fd69014c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.168+0000 7fd696ce2640 1 --2- 172.21.15.90:0/3395884597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6900c8b00 0x7fd69014c440 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd68800c400 tx=0x7fd68800c8c0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.168+0000 7fd6964e1640 1 -- 172.21.15.90:0/3395884597 reap_dead start 2024-03-11T18:36:46.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.172+0000 7fd696ce2640 1 -- 172.21.15.90:0/3395884597 shutdown_connections 2024-03-11T18:36:46.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.172+0000 7fd696ce2640 1 -- 172.21.15.90:0/3395884597 >> 172.21.15.90:0/3395884597 conn(0x7fd69006f420 msgr2=0x7fd69006f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:46.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.172+0000 7fd696ce2640 1 -- 172.21.15.90:0/3395884597 shutdown_connections 2024-03-11T18:36:46.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.172+0000 7fd696ce2640 1 -- 172.21.15.90:0/3395884597 wait complete. 2024-03-11T18:36:46.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.172+0000 7fe168ff9640 1 -- 172.21.15.90:0/3393719004 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fe164007070 con 0x7fe16c151a50 2024-03-11T18:36:46.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.172+0000 7fe168ff9640 1 --2- 172.21.15.90:0/3393719004 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe14c066ec0 0x7fe14c069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.172+0000 7fe16b7fe640 1 --2- 172.21.15.90:0/3393719004 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe14c066ec0 0x7fe14c069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:46.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.172+0000 7fe16b7fe640 1 --2- 172.21.15.90:0/3393719004 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe14c066ec0 0x7fe14c069380 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe15c0ad000 tx=0x7fe15c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:46.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.176+0000 7f8e3cbc4640 1 -- 172.21.15.90:0/407410412 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3812d660 con 0x7f8e3004b950 2024-03-11T18:36:46.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.176+0000 7f8e3cbc4640 1 -- 172.21.15.90:0/407410412 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e3812d7c0 con 0x7f8e3004b950 2024-03-11T18:36:46.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.176+0000 7f8e3cbc4640 1 -- 172.21.15.90:0/407410412 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3812dbf0 con 0x7f8e3004b950 2024-03-11T18:36:46.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.176+0000 7f8e3fe2a640 1 -- 172.21.15.90:0/407410412 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8e3004b950 msgr2=0x7f8e30046020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.176+0000 7f8e3fe2a640 1 --2- 172.21.15.90:0/407410412 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8e3004b950 0x7f8e30046020 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8e38073760 tx=0x7f8e3809b5e0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.176+0000 7f8e3fe2a640 1 -- 172.21.15.90:0/407410412 shutdown_connections 2024-03-11T18:36:46.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.176+0000 7f8e3fe2a640 1 --2- 172.21.15.90:0/407410412 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8e3004b950 0x7f8e30046020 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.176+0000 7f8e3fe2a640 1 --2- 172.21.15.90:0/407410412 >> v2:172.21.15.90:3300/0 conn(0x7f8e300d5540 0x7f8e30045ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.176+0000 7f8e3fe2a640 1 -- 172.21.15.90:0/407410412 >> 172.21.15.90:0/407410412 conn(0x7f8e3004d4c0 msgr2=0x7f8e3004d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:46.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.176+0000 7f8e3fe2a640 1 -- 172.21.15.90:0/407410412 shutdown_connections 2024-03-11T18:36:46.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.176+0000 7f8e3fe2a640 1 -- 172.21.15.90:0/407410412 wait complete. 2024-03-11T18:36:46.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.180+0000 7f8e3fe2a640 1 Processor -- start 2024-03-11T18:36:46.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.180+0000 7f8e3fe2a640 1 -- start start 2024-03-11T18:36:46.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.180+0000 7f8e3fe2a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e3004b950 0x7f8e300c4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.180+0000 7f8e3fe2a640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e3004cf50 0x7f8e300dae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.180+0000 7f8e3fe2a640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8e300d5540 0x7f8e30050440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.180+0000 7f8e3fe2a640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e300c4670 con 0x7f8e3004cf50 2024-03-11T18:36:46.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.180+0000 7f8e3fe2a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e300c48e0 con 0x7f8e3004b950 2024-03-11T18:36:46.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.180+0000 7f8e3fe2a640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3004ffb0 con 0x7f8e300d5540 2024-03-11T18:36:46.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.180+0000 7fe168ff9640 1 -- 172.21.15.90:0/3393719004 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe164056390 con 0x7fe16c151a50 2024-03-11T18:36:46.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.180+0000 7fe168ff9640 1 --2- 172.21.15.90:0/3393719004 >> [v2:172.21.15.90:6826/2837704738,v1:172.21.15.90:6827/2837704738] conn(0x7fe14c0705b0 0x7fe14c0729f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.180+0000 7fe168ff9640 1 -- 172.21.15.90:0/3393719004 --> [v2:172.21.15.90:6826/2837704738,v1:172.21.15.90:6827/2837704738] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe14c0730c0 con 0x7fe14c0705b0 2024-03-11T18:36:46.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.180+0000 7fe168ff9640 1 -- 172.21.15.90:0/3393719004 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe164051f30 con 0x7fe16c151a50 2024-03-11T18:36:46.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.180+0000 7fe163fff640 1 --2- 172.21.15.90:0/3393719004 >> [v2:172.21.15.90:6826/2837704738,v1:172.21.15.90:6827/2837704738] conn(0x7fe14c0705b0 0x7fe14c0729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:46.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.184+0000 7fe163fff640 1 --2- 172.21.15.90:0/3393719004 >> [v2:172.21.15.90:6826/2837704738,v1:172.21.15.90:6827/2837704738] conn(0x7fe14c0705b0 0x7fe14c0729f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:46.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.184+0000 7fe168ff9640 1 -- 172.21.15.90:0/3393719004 <== osd.2 v2:172.21.15.90:6826/2837704738 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fe14c0730c0 con 0x7fe14c0705b0 2024-03-11T18:36:46.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.192+0000 7f8e3e3c7640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8e300d5540 0x7f8e30050440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:46.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.192+0000 7f8e3e3c7640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8e300d5540 0x7f8e30050440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.90:59162/0 (socket says 172.21.15.90:59162) 2024-03-11T18:36:46.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.192+0000 7f8e3e3c7640 1 -- 172.21.15.90:0/1866734328 learned_addr learned my addr 172.21.15.90:0/1866734328 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:46.198 INFO:teuthology.orchestra.run.smithi090.stdout:68719476760 2024-03-11T18:36:46.199 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-11T18:36:46.199 INFO:teuthology.orchestra.run.smithi090.stdout:68719476759 2024-03-11T18:36:46.200 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-11T18:36:46.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.200+0000 7f8e3e3c7640 1 -- 172.21.15.90:0/1866734328 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e3004b950 msgr2=0x7f8e300c4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.200+0000 7f8e3e3c7640 1 --2- 172.21.15.90:0/1866734328 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8e3004b950 0x7f8e300c4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.200+0000 7f8e3e3c7640 1 -- 172.21.15.90:0/1866734328 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e3004cf50 msgr2=0x7f8e300dae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.200+0000 7f8e3e3c7640 1 --2- 172.21.15.90:0/1866734328 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e3004cf50 0x7f8e300dae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.200+0000 7f8e3e3c7640 1 -- 172.21.15.90:0/1866734328 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e30036550 con 0x7f8e300d5540 2024-03-11T18:36:46.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.200+0000 7fe170a3f640 1 -- 172.21.15.90:0/3393719004 --> [v2:172.21.15.90:6826/2837704738,v1:172.21.15.90:6827/2837704738] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe16c0c21b0 con 0x7fe14c0705b0 2024-03-11T18:36:46.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.200+0000 7fe168ff9640 1 -- 172.21.15.90:0/3393719004 <== osd.2 v2:172.21.15.90:6826/2837704738 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe154009860 con 0x7fe14c0705b0 2024-03-11T18:36:46.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.204+0000 7fe170a3f640 1 -- 172.21.15.90:0/3393719004 >> [v2:172.21.15.90:6826/2837704738,v1:172.21.15.90:6827/2837704738] conn(0x7fe14c0705b0 msgr2=0x7fe14c0729f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.204+0000 7fe170a3f640 1 --2- 172.21.15.90:0/3393719004 >> [v2:172.21.15.90:6826/2837704738,v1:172.21.15.90:6827/2837704738] conn(0x7fe14c0705b0 0x7fe14c0729f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.204+0000 7fe170a3f640 1 -- 172.21.15.90:0/3393719004 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe14c066ec0 msgr2=0x7fe14c069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.204+0000 7fe170a3f640 1 --2- 172.21.15.90:0/3393719004 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe14c066ec0 0x7fe14c069380 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe15c0ad000 tx=0x7fe15c008040 comp rx=0 tx=0).stop 2024-03-11T18:36:46.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.204+0000 7fe170a3f640 1 -- 172.21.15.90:0/3393719004 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe16c151a50 msgr2=0x7fe16c143c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.204+0000 7fe170a3f640 1 --2- 172.21.15.90:0/3393719004 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe16c151a50 0x7fe16c143c60 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe16400cad0 tx=0x7fe16400cf90 comp rx=0 tx=0).stop 2024-03-11T18:36:46.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.204+0000 7fe16bfff640 1 -- 172.21.15.90:0/3393719004 reap_dead start 2024-03-11T18:36:46.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.204+0000 7fe170a3f640 1 -- 172.21.15.90:0/3393719004 shutdown_connections 2024-03-11T18:36:46.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.204+0000 7fe170a3f640 1 -- 172.21.15.90:0/3393719004 >> 172.21.15.90:0/3393719004 conn(0x7fe16c066990 msgr2=0x7fe16c0ccc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:46.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.204+0000 7fe170a3f640 1 -- 172.21.15.90:0/3393719004 shutdown_connections 2024-03-11T18:36:46.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.204+0000 7fe170a3f640 1 -- 172.21.15.90:0/3393719004 wait complete. 2024-03-11T18:36:46.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.208+0000 7f8e3e3c7640 1 --2- 172.21.15.90:0/1866734328 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8e300d5540 0x7f8e30050440 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8e38073760 tx=0x7f8e38098420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:46.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.216+0000 7fa0427fc640 1 -- 172.21.15.90:0/1596626827 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fa03c00f3e0 con 0x7fa0440d5be0 2024-03-11T18:36:46.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.216+0000 7fa0427fc640 1 --2- 172.21.15.90:0/1596626827 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fa03006b2a0 0x7fa03006d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.216+0000 7fa051b24640 1 --2- 172.21.15.90:0/1596626827 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fa03006b2a0 0x7fa03006d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:46.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.216+0000 7f8e2e7fc640 1 -- 172.21.15.90:0/1866734328 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3813b070 con 0x7f8e300d5540 2024-03-11T18:36:46.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.216+0000 7f8e2e7fc640 1 -- 172.21.15.90:0/1866734328 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e38136430 con 0x7f8e300d5540 2024-03-11T18:36:46.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.216+0000 7fa051b24640 1 --2- 172.21.15.90:0/1596626827 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fa03006b2a0 0x7fa03006d760 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa048001600 tx=0x7fa048007a50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:46.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.216+0000 7f8e3fe2a640 1 -- 172.21.15.90:0/1866734328 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e30178920 con 0x7f8e300d5540 2024-03-11T18:36:46.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.216+0000 7f8e2e7fc640 1 -- 172.21.15.90:0/1866734328 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e381367a0 con 0x7f8e300d5540 2024-03-11T18:36:46.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.216+0000 7f8e3fe2a640 1 -- 172.21.15.90:0/1866734328 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e300c4b30 con 0x7f8e300d5540 2024-03-11T18:36:46.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.220+0000 7f8e3fe2a640 1 -- 172.21.15.90:0/1866734328 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8e30007b80 con 0x7f8e300d5540 2024-03-11T18:36:46.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.224+0000 7fa0427fc640 1 -- 172.21.15.90:0/1596626827 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa03c024e30 con 0x7fa0440d5be0 2024-03-11T18:36:46.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.224+0000 7fa0427fc640 1 --2- 172.21.15.90:0/1596626827 >> [v2:172.21.15.90:6818/1755667921,v1:172.21.15.90:6819/1755667921] conn(0x7fa030074910 0x7fa030076d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.224+0000 7fa0427fc640 1 -- 172.21.15.90:0/1596626827 --> [v2:172.21.15.90:6818/1755667921,v1:172.21.15.90:6819/1755667921] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa030077420 con 0x7fa030074910 2024-03-11T18:36:46.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.224+0000 7fa052325640 1 --2- 172.21.15.90:0/1596626827 >> [v2:172.21.15.90:6818/1755667921,v1:172.21.15.90:6819/1755667921] conn(0x7fa030074910 0x7fa030076d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:46.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.224+0000 7fa052325640 1 --2- 172.21.15.90:0/1596626827 >> [v2:172.21.15.90:6818/1755667921,v1:172.21.15.90:6819/1755667921] conn(0x7fa030074910 0x7fa030076d50 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:46.232 INFO:teuthology.orchestra.run.smithi090.stdout:64424509462 2024-03-11T18:36:46.232 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-11T18:36:46.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.224+0000 7f0d87fff640 1 --2- 172.21.15.90:0/2551566929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d900c8e20 0x7f0d900cc6c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0d7800c7c0 tx=0x7f0d7800cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:46.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.224+0000 7fa0427fc640 1 -- 172.21.15.90:0/1596626827 <== osd.0 v2:172.21.15.90:6818/1755667921 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fa030077420 con 0x7fa030074910 2024-03-11T18:36:46.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.228+0000 7f29984ac640 1 Processor -- start 2024-03-11T18:36:46.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.228+0000 7f29984ac640 1 -- start start 2024-03-11T18:36:46.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.232+0000 7f29984ac640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f2990159c80 0x7f299006eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.232+0000 7f29984ac640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f299006f0a0 0x7f29900cb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.232+0000 7f29984ac640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29900c7d80 con 0x7f2990153f00 2024-03-11T18:36:46.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.232+0000 7f29984ac640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2990071430 con 0x7f2990159c80 2024-03-11T18:36:46.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.232+0000 7f29984ac640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f299006e3f0 con 0x7f299006f0a0 2024-03-11T18:36:46.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.240+0000 7f0d8d7fa640 1 -- 172.21.15.90:0/2551566929 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d78004220 con 0x7f0d900c8e20 2024-03-11T18:36:46.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.240+0000 7f0d8d7fa640 1 -- 172.21.15.90:0/2551566929 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d78004380 con 0x7f0d900c8e20 2024-03-11T18:36:46.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.240+0000 7f0d8d7fa640 1 -- 172.21.15.90:0/2551566929 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d78003840 con 0x7f0d900c8e20 2024-03-11T18:36:46.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.240+0000 7f0d966b2640 1 -- 172.21.15.90:0/2551566929 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d900cb340 con 0x7f0d900c8e20 2024-03-11T18:36:46.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.240+0000 7f0d966b2640 1 -- 172.21.15.90:0/2551566929 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d900cb900 con 0x7f0d900c8e20 2024-03-11T18:36:46.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.240+0000 7f0d966b2640 1 -- 172.21.15.90:0/2551566929 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0d9006ce70 con 0x7f0d900c8e20 2024-03-11T18:36:46.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.248+0000 7f2996a49640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f299006f0a0 0x7f29900cb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:46.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.248+0000 7f2996a49640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f299006f0a0 0x7f29900cb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.90:60378/0 (socket says 172.21.15.90:60378) 2024-03-11T18:36:46.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.248+0000 7f2996a49640 1 -- 172.21.15.90:0/2118123973 learned_addr learned my addr 172.21.15.90:0/2118123973 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:46.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.256+0000 7fa052b26640 1 -- 172.21.15.90:0/1596626827 --> [v2:172.21.15.90:6818/1755667921,v1:172.21.15.90:6819/1755667921] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa044035af0 con 0x7fa030074910 2024-03-11T18:36:46.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.256+0000 7fa0427fc640 1 -- 172.21.15.90:0/1596626827 <== osd.0 v2:172.21.15.90:6818/1755667921 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa04c062160 con 0x7fa030074910 2024-03-11T18:36:46.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.256+0000 7fa052b26640 1 -- 172.21.15.90:0/1596626827 >> [v2:172.21.15.90:6818/1755667921,v1:172.21.15.90:6819/1755667921] conn(0x7fa030074910 msgr2=0x7fa030076d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.256+0000 7fa052b26640 1 --2- 172.21.15.90:0/1596626827 >> [v2:172.21.15.90:6818/1755667921,v1:172.21.15.90:6819/1755667921] conn(0x7fa030074910 0x7fa030076d50 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.256+0000 7fa052b26640 1 -- 172.21.15.90:0/1596626827 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fa03006b2a0 msgr2=0x7fa03006d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.256+0000 7fa052b26640 1 --2- 172.21.15.90:0/1596626827 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fa03006b2a0 0x7fa03006d760 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa048001600 tx=0x7fa048007a50 comp rx=0 tx=0).stop 2024-03-11T18:36:46.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.256+0000 7fa052b26640 1 -- 172.21.15.90:0/1596626827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0440d5be0 msgr2=0x7fa0440d0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.256+0000 7fa052b26640 1 --2- 172.21.15.90:0/1596626827 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fa0440d5be0 0x7fa0440d0220 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa03c00b5b0 tx=0x7fa03c00ba70 comp rx=0 tx=0).stop 2024-03-11T18:36:46.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.256+0000 7fa052325640 1 -- 172.21.15.90:0/1596626827 reap_dead start 2024-03-11T18:36:46.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.256+0000 7fa052b26640 1 -- 172.21.15.90:0/1596626827 shutdown_connections 2024-03-11T18:36:46.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.256+0000 7fa052b26640 1 -- 172.21.15.90:0/1596626827 >> 172.21.15.90:0/1596626827 conn(0x7fa044050190 msgr2=0x7fa044033450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:46.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.256+0000 7fa052b26640 1 -- 172.21.15.90:0/1596626827 shutdown_connections 2024-03-11T18:36:46.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.256+0000 7fa052b26640 1 -- 172.21.15.90:0/1596626827 wait complete. 2024-03-11T18:36:46.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.276+0000 7f8e2e7fc640 1 -- 172.21.15.90:0/1866734328 <== mon.2 v2:172.21.15.92:3301/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f8e38134040 con 0x7f8e300d5540 2024-03-11T18:36:46.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.276+0000 7f8e2e7fc640 1 --2- 172.21.15.90:0/1866734328 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f8e1c066e40 0x7f8e1c069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.276+0000 7f8e3dbc6640 1 --2- 172.21.15.90:0/1866734328 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f8e1c066e40 0x7f8e1c069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:46.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.276+0000 7f8e3dbc6640 1 --2- 172.21.15.90:0/1866734328 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f8e1c066e40 0x7f8e1c069300 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8e34009730 tx=0x7f8e34006ca0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:46.290 INFO:teuthology.orchestra.run.smithi090.stdout:60129542165 2024-03-11T18:36:46.290 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-11T18:36:46.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.292+0000 7f0d8d7fa640 1 -- 172.21.15.90:0/2551566929 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f0d780039a0 con 0x7f0d900c8e20 2024-03-11T18:36:46.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.292+0000 7f0d8d7fa640 1 --2- 172.21.15.90:0/2551566929 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0d70066e90 0x7f0d70069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.292+0000 7f0d8ffff640 1 --2- 172.21.15.90:0/2551566929 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0d70066e90 0x7f0d70069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:46.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.292+0000 7f0d8ffff640 1 --2- 172.21.15.90:0/2551566929 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0d70066e90 0x7f0d70069350 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0d80007fd0 tx=0x7f0d80007f20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:46.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.296+0000 7f2996a49640 1 -- 172.21.15.90:0/2118123973 >> v2:172.21.15.90:3300/0 conn(0x7f2990159c80 msgr2=0x7f299006eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:46.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.296+0000 7f2996a49640 1 --2- 172.21.15.90:0/2118123973 >> v2:172.21.15.90:3300/0 conn(0x7f2990159c80 0x7f299006eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.296+0000 7f2996a49640 1 -- 172.21.15.90:0/2118123973 >> v1:172.21.15.90:6789/0 conn(0x7f2990153f00 legacy=0x7f29900c7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:46.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.296+0000 7f2996a49640 1 -- 172.21.15.90:0/2118123973 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f299006e6d0 con 0x7f299006f0a0 2024-03-11T18:36:46.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.296+0000 7fe03a7fc640 1 -- 172.21.15.90:0/4119030134 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fe040007450 con 0x7fe03c04c480 2024-03-11T18:36:46.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.296+0000 7fe03a7fc640 1 --2- 172.21.15.90:0/4119030134 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe028066ec0 0x7fe028069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.296+0000 7fe049f7a640 1 --2- 172.21.15.90:0/4119030134 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe028066ec0 0x7fe028069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:46.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.296+0000 7fe049f7a640 1 --2- 172.21.15.90:0/4119030134 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe028066ec0 0x7fe028069380 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe03c006450 tx=0x7fe044074ec0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:46.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.300+0000 7f8e2e7fc640 1 -- 172.21.15.90:0/1866734328 <== mon.2 v2:172.21.15.92:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f8e3816ec10 con 0x7f8e300d5540 2024-03-11T18:36:46.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.300+0000 7f8e2e7fc640 1 --2- 172.21.15.90:0/1866734328 >> [v2:172.21.15.90:6802/240987971,v1:172.21.15.90:6803/240987971] conn(0x7f8e1c070430 0x7f8e1c072870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.300+0000 7f8e2e7fc640 1 -- 172.21.15.90:0/1866734328 --> [v2:172.21.15.90:6802/240987971,v1:172.21.15.90:6803/240987971] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8e1c072f40 con 0x7f8e1c070430 2024-03-11T18:36:46.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.300+0000 7f8e3d3c5640 1 --2- 172.21.15.90:0/1866734328 >> [v2:172.21.15.90:6802/240987971,v1:172.21.15.90:6803/240987971] conn(0x7f8e1c070430 0x7f8e1c072870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:46.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.300+0000 7f8e3d3c5640 1 --2- 172.21.15.90:0/1866734328 >> [v2:172.21.15.90:6802/240987971,v1:172.21.15.90:6803/240987971] conn(0x7f8e1c070430 0x7f8e1c072870 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:46.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.300+0000 7f8e2e7fc640 1 -- 172.21.15.90:0/1866734328 <== osd.3 v2:172.21.15.90:6802/240987971 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f8e1c072f40 con 0x7f8e1c070430 2024-03-11T18:36:46.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.304+0000 7f0d8d7fa640 1 -- 172.21.15.90:0/2551566929 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0d78024e30 con 0x7f0d900c8e20 2024-03-11T18:36:46.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.304+0000 7f0d8d7fa640 1 --2- 172.21.15.90:0/2551566929 >> [v2:172.21.15.92:6816/1919010204,v1:172.21.15.92:6817/1919010204] conn(0x7f0d70070500 0x7f0d70072940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.304+0000 7f0d8d7fa640 1 -- 172.21.15.90:0/2551566929 --> [v2:172.21.15.92:6816/1919010204,v1:172.21.15.92:6817/1919010204] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0d70073010 con 0x7f0d70070500 2024-03-11T18:36:46.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.304+0000 7fe03a7fc640 1 -- 172.21.15.90:0/4119030134 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe040055970 con 0x7fe03c04c480 2024-03-11T18:36:46.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.304+0000 7fe03a7fc640 1 --2- 172.21.15.90:0/4119030134 >> [v2:172.21.15.92:6808/200142070,v1:172.21.15.92:6809/200142070] conn(0x7fe0280705c0 0x7fe028072a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.304+0000 7fe03a7fc640 1 -- 172.21.15.90:0/4119030134 --> [v2:172.21.15.92:6808/200142070,v1:172.21.15.92:6809/200142070] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe0280730d0 con 0x7fe0280705c0 2024-03-11T18:36:46.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.304+0000 7fe03a7fc640 1 -- 172.21.15.90:0/4119030134 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe040051510 con 0x7fe03c04c480 2024-03-11T18:36:46.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.304+0000 7fe04a77b640 1 --2- 172.21.15.90:0/4119030134 >> [v2:172.21.15.92:6808/200142070,v1:172.21.15.92:6809/200142070] conn(0x7fe0280705c0 0x7fe028072a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:46.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.304+0000 7f0d94c4f640 1 --2- 172.21.15.90:0/2551566929 >> [v2:172.21.15.92:6816/1919010204,v1:172.21.15.92:6817/1919010204] conn(0x7f0d70070500 0x7f0d70072940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:46.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.304+0000 7fe04a77b640 1 --2- 172.21.15.90:0/4119030134 >> [v2:172.21.15.92:6808/200142070,v1:172.21.15.92:6809/200142070] conn(0x7fe0280705c0 0x7fe028072a00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:46.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.304+0000 7f0d94c4f640 1 --2- 172.21.15.90:0/2551566929 >> [v2:172.21.15.92:6816/1919010204,v1:172.21.15.92:6817/1919010204] conn(0x7f0d70070500 0x7f0d70072940 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:46.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.308+0000 7f0d8d7fa640 1 -- 172.21.15.90:0/2551566929 <== osd.6 v2:172.21.15.92:6816/1919010204 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f0d70073010 con 0x7f0d70070500 2024-03-11T18:36:46.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.308+0000 7f2996a49640 1 --2- 172.21.15.90:0/2118123973 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f299006f0a0 0x7f29900cb760 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f29900c5960 tx=0x7f298801c9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3360f82d7907bda7 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:46.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.312+0000 7fe03a7fc640 1 -- 172.21.15.90:0/4119030134 <== osd.5 v2:172.21.15.92:6808/200142070 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fe0280730d0 con 0x7fe0280705c0 2024-03-11T18:36:46.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.328+0000 7f0d966b2640 1 -- 172.21.15.90:0/2551566929 --> [v2:172.21.15.92:6816/1919010204,v1:172.21.15.92:6817/1919010204] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0d901542f0 con 0x7f0d70070500 2024-03-11T18:36:46.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.328+0000 7f0d8d7fa640 1 -- 172.21.15.90:0/2551566929 <== osd.6 v2:172.21.15.92:6816/1919010204 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0d880a59d0 con 0x7f0d70070500 2024-03-11T18:36:46.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.328+0000 7f8e3fe2a640 1 -- 172.21.15.90:0/1866734328 --> [v2:172.21.15.90:6802/240987971,v1:172.21.15.90:6803/240987971] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8e300078a0 con 0x7f8e1c070430 2024-03-11T18:36:46.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.328+0000 7f0d966b2640 1 -- 172.21.15.90:0/2551566929 >> [v2:172.21.15.92:6816/1919010204,v1:172.21.15.92:6817/1919010204] conn(0x7f0d70070500 msgr2=0x7f0d70072940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.328+0000 7f0d966b2640 1 --2- 172.21.15.90:0/2551566929 >> [v2:172.21.15.92:6816/1919010204,v1:172.21.15.92:6817/1919010204] conn(0x7f0d70070500 0x7f0d70072940 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.328+0000 7f0d966b2640 1 -- 172.21.15.90:0/2551566929 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0d70066e90 msgr2=0x7f0d70069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.328+0000 7f0d966b2640 1 --2- 172.21.15.90:0/2551566929 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0d70066e90 0x7f0d70069350 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0d80007fd0 tx=0x7f0d80007f20 comp rx=0 tx=0).stop 2024-03-11T18:36:46.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.328+0000 7f0d966b2640 1 -- 172.21.15.90:0/2551566929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d900c8e20 msgr2=0x7f0d900cc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.328+0000 7f0d966b2640 1 --2- 172.21.15.90:0/2551566929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0d900c8e20 0x7f0d900cc6c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0d7800c7c0 tx=0x7f0d7800cc80 comp rx=0 tx=0).stop 2024-03-11T18:36:46.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.328+0000 7f0d94c4f640 1 -- 172.21.15.90:0/2551566929 reap_dead start 2024-03-11T18:36:46.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.332+0000 7f0d966b2640 1 -- 172.21.15.90:0/2551566929 shutdown_connections 2024-03-11T18:36:46.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.332+0000 7f8e2e7fc640 1 -- 172.21.15.90:0/1866734328 <== osd.3 v2:172.21.15.90:6802/240987971 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8e28001240 con 0x7f8e1c070430 2024-03-11T18:36:46.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.332+0000 7f0d966b2640 1 -- 172.21.15.90:0/2551566929 >> 172.21.15.90:0/2551566929 conn(0x7f0d90070e80 msgr2=0x7f0d9015da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:46.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.332+0000 7f0d966b2640 1 -- 172.21.15.90:0/2551566929 shutdown_connections 2024-03-11T18:36:46.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.332+0000 7f8e3fe2a640 1 -- 172.21.15.90:0/1866734328 >> [v2:172.21.15.90:6802/240987971,v1:172.21.15.90:6803/240987971] conn(0x7f8e1c070430 msgr2=0x7f8e1c072870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.332+0000 7f8e3fe2a640 1 --2- 172.21.15.90:0/1866734328 >> [v2:172.21.15.90:6802/240987971,v1:172.21.15.90:6803/240987971] conn(0x7f8e1c070430 0x7f8e1c072870 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.332+0000 7f0d966b2640 1 -- 172.21.15.90:0/2551566929 wait complete. 2024-03-11T18:36:46.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.332+0000 7f06c0c28640 1 Processor -- start 2024-03-11T18:36:46.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.332+0000 7f06c0c28640 1 -- start start 2024-03-11T18:36:46.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.336+0000 7f8e3fe2a640 1 -- 172.21.15.90:0/1866734328 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f8e1c066e40 msgr2=0x7f8e1c069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.336+0000 7f8e3fe2a640 1 --2- 172.21.15.90:0/1866734328 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f8e1c066e40 0x7f8e1c069300 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8e34009730 tx=0x7f8e34006ca0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.336+0000 7f8e3fe2a640 1 -- 172.21.15.90:0/1866734328 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8e300d5540 msgr2=0x7f8e30050440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.336+0000 7f8e3fe2a640 1 --2- 172.21.15.90:0/1866734328 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8e300d5540 0x7f8e30050440 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8e38073760 tx=0x7f8e38098420 comp rx=0 tx=0).stop 2024-03-11T18:36:46.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.336+0000 7f8e3e3c7640 1 -- 172.21.15.90:0/1866734328 reap_dead start 2024-03-11T18:36:46.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.336+0000 7f8e3fe2a640 1 -- 172.21.15.90:0/1866734328 shutdown_connections 2024-03-11T18:36:46.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.336+0000 7f8e3fe2a640 1 -- 172.21.15.90:0/1866734328 >> 172.21.15.90:0/1866734328 conn(0x7f8e3004d4c0 msgr2=0x7f8e30035ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:46.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.336+0000 7f06c0c28640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f06bc150a90 0x7f06bc157220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.336+0000 7f06c0c28640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06bc0c5e60 con 0x7f06bc0cc280 2024-03-11T18:36:46.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.336+0000 7f06c0c28640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06bc156a60 con 0x7f06bc0c65b0 2024-03-11T18:36:46.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.336+0000 7f06c0c28640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06bc0c55c0 con 0x7f06bc150a90 2024-03-11T18:36:46.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.340+0000 7fe04c1de640 1 -- 172.21.15.90:0/4119030134 --> [v2:172.21.15.92:6808/200142070,v1:172.21.15.92:6809/200142070] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe03c007170 con 0x7fe0280705c0 2024-03-11T18:36:46.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.340+0000 7fe03a7fc640 1 -- 172.21.15.90:0/4119030134 <== osd.5 v2:172.21.15.92:6808/200142070 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe034009280 con 0x7fe0280705c0 2024-03-11T18:36:46.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.344+0000 7fe04c1de640 1 -- 172.21.15.90:0/4119030134 >> [v2:172.21.15.92:6808/200142070,v1:172.21.15.92:6809/200142070] conn(0x7fe0280705c0 msgr2=0x7fe028072a00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.344+0000 7fe04c1de640 1 --2- 172.21.15.90:0/4119030134 >> [v2:172.21.15.92:6808/200142070,v1:172.21.15.92:6809/200142070] conn(0x7fe0280705c0 0x7fe028072a00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.344+0000 7f8e3fe2a640 1 -- 172.21.15.90:0/1866734328 shutdown_connections 2024-03-11T18:36:46.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.344+0000 7fe04c1de640 1 -- 172.21.15.90:0/4119030134 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe028066ec0 msgr2=0x7fe028069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.344+0000 7fe04c1de640 1 --2- 172.21.15.90:0/4119030134 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe028066ec0 0x7fe028069380 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe03c006450 tx=0x7fe044074ec0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.344+0000 7fe04c1de640 1 -- 172.21.15.90:0/4119030134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe03c04c480 msgr2=0x7fe03c0d31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.344+0000 7fe04c1de640 1 --2- 172.21.15.90:0/4119030134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe03c04c480 0x7fe03c0d31a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe04000c420 tx=0x7fe04000c8e0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.344+0000 7f8e3fe2a640 1 -- 172.21.15.90:0/1866734328 wait complete. 2024-03-11T18:36:46.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.344+0000 7fe04a77b640 1 -- 172.21.15.90:0/4119030134 reap_dead start 2024-03-11T18:36:46.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.344+0000 7fe04c1de640 1 -- 172.21.15.90:0/4119030134 shutdown_connections 2024-03-11T18:36:46.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.344+0000 7fe04c1de640 1 -- 172.21.15.90:0/4119030134 >> 172.21.15.90:0/4119030134 conn(0x7fe03c050190 msgr2=0x7fe03c050570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:46.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.344+0000 7fe04c1de640 1 -- 172.21.15.90:0/4119030134 shutdown_connections 2024-03-11T18:36:46.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.344+0000 7fe04c1de640 1 -- 172.21.15.90:0/4119030134 wait complete. 2024-03-11T18:36:46.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.352+0000 7f06bad9d640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f06bc0cc280 0x7f06bc161240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:42890/0 (socket says 172.21.15.90:42890) 2024-03-11T18:36:46.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.352+0000 7f06bad9d640 1 -- 172.21.15.90:0/1769373713 learned_addr learned my addr 172.21.15.90:0/1769373713 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T18:36:46.364 INFO:teuthology.orchestra.run.smithi090.stdout:68719476760 2024-03-11T18:36:46.364 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-11T18:36:46.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.360+0000 7f06b959a640 1 -- 172.21.15.90:0/1769373713 <== mon.0 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2336059867 0 0) 0x7f06bc0c5e60 con 0x7f06bc0cc280 2024-03-11T18:36:46.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.360+0000 7f06b959a640 1 -- 172.21.15.90:0/1769373713 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06a4003530 con 0x7f06bc0cc280 2024-03-11T18:36:46.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.360+0000 7fb8e6d7f640 1 Processor -- start 2024-03-11T18:36:46.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.360+0000 7fb8e6d7f640 1 -- start start 2024-03-11T18:36:46.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.360+0000 7fb8e6d7f640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fb8e0150a90 0x7fb8e0157220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.360+0000 7fb8e6d7f640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb8e00c65b0 0x7fb8e00c37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.360+0000 7fb8e6d7f640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb8e00c3d30 0x7fb8e00cc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.360+0000 7fb8e6d7f640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8e00c5e60 con 0x7fb8e0150a90 2024-03-11T18:36:46.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.360+0000 7fb8e6d7f640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8e0156a60 con 0x7fb8e00c3d30 2024-03-11T18:36:46.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.360+0000 7fb8e6d7f640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e00c55c0 con 0x7fb8e00c65b0 2024-03-11T18:36:46.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.364+0000 7fb8e4b1b640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fb8e0150a90 0x7fb8e0157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:46.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.364+0000 7f06b959a640 1 -- 172.21.15.90:0/1769373713 <== mon.0 v1:172.21.15.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4235701896 0 0) 0x7f06a4003530 con 0x7f06bc0cc280 2024-03-11T18:36:46.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.364+0000 7f06b959a640 1 -- 172.21.15.90:0/1769373713 >> v2:172.21.15.92:3300/0 conn(0x7f06bc150a90 msgr2=0x7f06bc157220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:46.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.364+0000 7f06b959a640 1 --2- 172.21.15.90:0/1769373713 >> v2:172.21.15.92:3300/0 conn(0x7f06bc150a90 0x7f06bc157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.364+0000 7f06b959a640 1 -- 172.21.15.90:0/1769373713 >> v1:172.21.15.92:6789/0 conn(0x7f06bc0c65b0 legacy=0x7f06bc0c37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:46.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.364+0000 7f06b959a640 1 -- 172.21.15.90:0/1769373713 --> v1:172.21.15.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06bc0c58a0 con 0x7f06bc0cc280 2024-03-11T18:36:46.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.364+0000 7f06b959a640 1 -- 172.21.15.90:0/1769373713 <== mon.0 v1:172.21.15.90:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 691735435 0 0) 0x7f06b4001a00 con 0x7f06bc0cc280 2024-03-11T18:36:46.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.364+0000 7f06b959a640 1 -- 172.21.15.90:0/1769373713 >> v1:172.21.15.90:6789/0 conn(0x7f06bc0cc280 legacy=0x7f06bc161240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.364+0000 7f06b959a640 1 --2- 172.21.15.90:0/1769373713 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06a4003980 0x7f06a4003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.364+0000 7f06b959a640 1 --2- 172.21.15.90:0/1769373713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06a40047d0 0x7f06a4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.364+0000 7f06b959a640 1 --2- 172.21.15.90:0/1769373713 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f06a40052b0 0x7f06a400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.364+0000 7f06b959a640 1 -- 172.21.15.90:0/1769373713 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc0c55c0 con 0x7f06a4003980 2024-03-11T18:36:46.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.364+0000 7f06b959a640 1 -- 172.21.15.90:0/1769373713 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a40149d0 con 0x7f06a40047d0 2024-03-11T18:36:46.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.364+0000 7f06b959a640 1 -- 172.21.15.90:0/1769373713 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06a4014b90 con 0x7f06a40052b0 2024-03-11T18:36:46.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.364+0000 7fb8e4b1b640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fb8e0150a90 0x7fb8e0157220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48166/0 (socket says 172.21.15.90:48166) 2024-03-11T18:36:46.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.364+0000 7fb8e4b1b640 1 -- 172.21.15.90:0/3498285080 learned_addr learned my addr 172.21.15.90:0/3498285080 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:46.376 INFO:teuthology.orchestra.run.smithi090.stdout:64424509463 2024-03-11T18:36:46.376 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-11T18:36:46.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.368+0000 7fb8e4b1b640 1 -- 172.21.15.90:0/3498285080 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb8e00c65b0 msgr2=0x7fb8e00c37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:46.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.368+0000 7fb8e4b1b640 1 --2- 172.21.15.90:0/3498285080 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb8e00c65b0 0x7fb8e00c37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.368+0000 7fb8e4b1b640 1 -- 172.21.15.90:0/3498285080 >> v2:172.21.15.92:3300/0 conn(0x7fb8e00c3d30 msgr2=0x7fb8e00cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:46.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.368+0000 7fb8e4b1b640 1 --2- 172.21.15.90:0/3498285080 >> v2:172.21.15.92:3300/0 conn(0x7fb8e00c3d30 0x7fb8e00cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.368+0000 7fb8e4b1b640 1 -- 172.21.15.90:0/3498285080 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e00c58a0 con 0x7fb8e0150a90 2024-03-11T18:36:46.380 INFO:teuthology.orchestra.run.smithi090.stdout:68719476758 2024-03-11T18:36:46.381 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-11T18:36:46.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.372+0000 7f06ba59c640 1 --2- 172.21.15.90:0/1769373713 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06a4003980 0x7f06a4003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:46.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.376+0000 7f16f46b9640 1 Processor -- start 2024-03-11T18:36:46.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.376+0000 7f16f46b9640 1 -- start start 2024-03-11T18:36:46.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.376+0000 7f06ba59c640 1 -- 172.21.15.90:0/1769373713 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f06a40052b0 msgr2=0x7f06a400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:46.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.376+0000 7f06ba59c640 1 --2- 172.21.15.90:0/1769373713 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f06a40052b0 0x7f06a400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.376+0000 7f06ba59c640 1 -- 172.21.15.90:0/1769373713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06a40047d0 msgr2=0x7f06a4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:46.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.376+0000 7f06ba59c640 1 --2- 172.21.15.90:0/1769373713 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06a40047d0 0x7f06a4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.376+0000 7f06ba59c640 1 -- 172.21.15.90:0/1769373713 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06bc0c58a0 con 0x7f06a4003980 2024-03-11T18:36:46.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.380+0000 7f06bad9d640 1 -- 172.21.15.90:0/1769373713 reap_dead start 2024-03-11T18:36:46.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.380+0000 7fb8e4b1b640 1 --2- 172.21.15.90:0/3498285080 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8e0150a90 0x7fb8e0157220 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0156c30 tx=0x7fb8d4020de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a91942b8ce970d server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:46.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.380+0000 7f16f46b9640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f16ec1550c0 0x7f16ec0c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.380+0000 7f16f46b9640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f16ec06fe60 0x7f16ec0c2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.380+0000 7f16f46b9640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16ec0c2ae0 con 0x7f16ec1554a0 2024-03-11T18:36:46.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.380+0000 7f16f46b9640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16ec0c2560 con 0x7f16ec06fe60 2024-03-11T18:36:46.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.380+0000 7f16f46b9640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16ec157cc0 con 0x7f16ec1550c0 2024-03-11T18:36:46.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.380+0000 7f16f2c56640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f16ec06fe60 0x7f16ec0c2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:46.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.380+0000 7f16f2c56640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f16ec06fe60 0x7f16ec0c2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48186/0 (socket says 172.21.15.90:48186) 2024-03-11T18:36:46.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.380+0000 7f16f2c56640 1 -- 172.21.15.90:0/4190295525 learned_addr learned my addr 172.21.15.90:0/4190295525 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:46.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.388+0000 7f06ba59c640 1 --2- 172.21.15.90:0/1769373713 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06a4003980 0x7f06a4003d80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f06ac009860 tx=0x7f06ac01c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a2b0198324383a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:46.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.392+0000 7f16f2c56640 1 -- 172.21.15.90:0/4190295525 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f16ec1550c0 msgr2=0x7f16ec0c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:46.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.392+0000 7f16f2c56640 1 --2- 172.21.15.90:0/4190295525 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f16ec1550c0 0x7f16ec0c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.392+0000 7f16f2c56640 1 -- 172.21.15.90:0/4190295525 >> v1:172.21.15.90:6789/0 conn(0x7f16ec1554a0 legacy=0x7f16ec15ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:46.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.392+0000 7f16f2c56640 1 -- 172.21.15.90:0/4190295525 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16ec0c2800 con 0x7f16ec06fe60 2024-03-11T18:36:46.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.392+0000 7fb8df7fe640 1 -- 172.21.15.90:0/3498285080 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d40a9b60 con 0x7fb8e0150a90 2024-03-11T18:36:46.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.392+0000 7fb8df7fe640 1 -- 172.21.15.90:0/3498285080 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8d40a9cc0 con 0x7fb8e0150a90 2024-03-11T18:36:46.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.392+0000 7fb8df7fe640 1 -- 172.21.15.90:0/3498285080 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d40b1660 con 0x7fb8e0150a90 2024-03-11T18:36:46.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.396+0000 7fb8e6d7f640 1 -- 172.21.15.90:0/3498285080 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8e0150a90 msgr2=0x7fb8e0157220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.396+0000 7fb8e6d7f640 1 --2- 172.21.15.90:0/3498285080 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8e0150a90 0x7fb8e0157220 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0156c30 tx=0x7fb8d4020de0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.396+0000 7fb8e6d7f640 1 -- 172.21.15.90:0/3498285080 shutdown_connections 2024-03-11T18:36:46.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.396+0000 7fb8e6d7f640 1 --2- 172.21.15.90:0/3498285080 >> v2:172.21.15.92:3300/0 conn(0x7fb8e00c3d30 0x7fb8e00cc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.396+0000 7fb8e6d7f640 1 --2- 172.21.15.90:0/3498285080 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb8e00c65b0 0x7fb8e00c37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.396+0000 7fb8e6d7f640 1 --2- 172.21.15.90:0/3498285080 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8e0150a90 0x7fb8e0157220 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.396+0000 7fb8e6d7f640 1 -- 172.21.15.90:0/3498285080 >> 172.21.15.90:0/3498285080 conn(0x7fb8e0071aa0 msgr2=0x7fb8e0071eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:46.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.396+0000 7fb8e6d7f640 1 -- 172.21.15.90:0/3498285080 shutdown_connections 2024-03-11T18:36:46.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.396+0000 7fb8e6d7f640 1 -- 172.21.15.90:0/3498285080 wait complete. 2024-03-11T18:36:46.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.396+0000 7fb8e6d7f640 1 Processor -- start 2024-03-11T18:36:46.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.396+0000 7fb8e6d7f640 1 -- start start 2024-03-11T18:36:46.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.396+0000 7fb8e6d7f640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb8e00c3d30 0x7fb8e0155580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.396+0000 7fb8e6d7f640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8e00c65b0 0x7fb8e02001c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.396+0000 7fb8e6d7f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8e0150a90 0x7fb8e01594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.396+0000 7fb8e6d7f640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e01ffda0 con 0x7fb8e00c65b0 2024-03-11T18:36:46.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.396+0000 7fb8e6d7f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e0155ac0 con 0x7fb8e0150a90 2024-03-11T18:36:46.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.396+0000 7fb8e6d7f640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e0149cb0 con 0x7fb8e00c3d30 2024-03-11T18:36:46.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.400+0000 7f16f2c56640 1 --2- 172.21.15.90:0/4190295525 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16ec06fe60 0x7f16ec0c2dd0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f16e4009e50 tx=0x7f16e401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=438fd8272331bc2c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:46.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.400+0000 7f2995246640 1 -- 172.21.15.90:0/2118123973 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29880a5670 con 0x7f299006f0a0 2024-03-11T18:36:46.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.400+0000 7f2995246640 1 -- 172.21.15.90:0/2118123973 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29880a57d0 con 0x7f299006f0a0 2024-03-11T18:36:46.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.400+0000 7f2995246640 1 -- 172.21.15.90:0/2118123973 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29880a5b60 con 0x7f299006f0a0 2024-03-11T18:36:46.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.400+0000 7f29984ac640 1 -- 172.21.15.90:0/2118123973 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f299006f0a0 msgr2=0x7f29900cb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.400+0000 7f29984ac640 1 --2- 172.21.15.90:0/2118123973 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f299006f0a0 0x7f29900cb760 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f29900c5960 tx=0x7f298801c9b0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.400+0000 7f29984ac640 1 -- 172.21.15.90:0/2118123973 shutdown_connections 2024-03-11T18:36:46.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.400+0000 7f29984ac640 1 --2- 172.21.15.90:0/2118123973 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f299006f0a0 0x7f29900cb760 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.400+0000 7f29984ac640 1 --2- 172.21.15.90:0/2118123973 >> v2:172.21.15.90:3300/0 conn(0x7f2990159c80 0x7f299006eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.400+0000 7f29984ac640 1 -- 172.21.15.90:0/2118123973 >> 172.21.15.90:0/2118123973 conn(0x7f299006add0 msgr2=0x7f299006b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:46.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.400+0000 7f29984ac640 1 -- 172.21.15.90:0/2118123973 shutdown_connections 2024-03-11T18:36:46.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.400+0000 7f06b959a640 1 -- 172.21.15.90:0/1769373713 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ac0ac440 con 0x7f06a4003980 2024-03-11T18:36:46.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.400+0000 7f06b959a640 1 -- 172.21.15.90:0/1769373713 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06ac0ac5c0 con 0x7f06a4003980 2024-03-11T18:36:46.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.400+0000 7f06b959a640 1 -- 172.21.15.90:0/1769373713 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ac0ac970 con 0x7f06a4003980 2024-03-11T18:36:46.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.400+0000 7f06c0c28640 1 -- 172.21.15.90:0/1769373713 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06a4003980 msgr2=0x7f06a4003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.400+0000 7f06c0c28640 1 --2- 172.21.15.90:0/1769373713 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06a4003980 0x7f06a4003d80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f06ac009860 tx=0x7f06ac01c380 comp rx=0 tx=0).stop 2024-03-11T18:36:46.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.400+0000 7f06c0c28640 1 -- 172.21.15.90:0/1769373713 shutdown_connections 2024-03-11T18:36:46.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.400+0000 7f06c0c28640 1 --2- 172.21.15.90:0/1769373713 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06a4003980 0x7f06a4003d80 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.400+0000 7f06c0c28640 1 -- 172.21.15.90:0/1769373713 >> 172.21.15.90:0/1769373713 conn(0x7f06bc071aa0 msgr2=0x7f06bc071eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:46.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.400+0000 7f06c0c28640 1 -- 172.21.15.90:0/1769373713 shutdown_connections 2024-03-11T18:36:46.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.400+0000 7f29984ac640 1 -- 172.21.15.90:0/2118123973 wait complete. 2024-03-11T18:36:46.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.400+0000 7f29984ac640 1 Processor -- start 2024-03-11T18:36:46.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.400+0000 7f29984ac640 1 -- start start 2024-03-11T18:36:46.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.404+0000 7f29984ac640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f299006f0a0 0x7f2990149980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.404+0000 7f29984ac640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2990153f00 0x7f29901554c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.404+0000 7f29984ac640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f2990159c80 0x7f29901f5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.404+0000 7f29984ac640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2990147900 con 0x7f299006f0a0 2024-03-11T18:36:46.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.404+0000 7f29984ac640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2990145d70 con 0x7f2990153f00 2024-03-11T18:36:46.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.404+0000 7f29984ac640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2990155030 con 0x7f2990159c80 2024-03-11T18:36:46.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.400+0000 7f06c0c28640 1 -- 172.21.15.90:0/1769373713 wait complete. 2024-03-11T18:36:46.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.404+0000 7f06c0c28640 1 Processor -- start 2024-03-11T18:36:46.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.404+0000 7f06c0c28640 1 -- start start 2024-03-11T18:36:46.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.404+0000 7f06c0c28640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f06a4003980 0x7f06bc14f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.404+0000 7f06c0c28640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06bc144e10 0x7f06bc2003b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.404+0000 7f06c0c28640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06bc2008f0 0x7f06bc1fcb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.404+0000 7f06c0c28640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc14cfa0 con 0x7f06bc2008f0 2024-03-11T18:36:46.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.404+0000 7f06c0c28640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc14ad40 con 0x7f06bc144e10 2024-03-11T18:36:46.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.404+0000 7f06c0c28640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc1ffbd0 con 0x7f06a4003980 2024-03-11T18:36:46.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.408+0000 7fb8dffff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8e00c65b0 0x7fb8e02001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:46.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.408+0000 7f16f1453640 1 -- 172.21.15.90:0/4190295525 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e40a56d0 con 0x7f16ec06fe60 2024-03-11T18:36:46.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.408+0000 7f16f1453640 1 -- 172.21.15.90:0/4190295525 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16e40a5830 con 0x7f16ec06fe60 2024-03-11T18:36:46.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.408+0000 7f16f1453640 1 -- 172.21.15.90:0/4190295525 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e40a5bc0 con 0x7f16ec06fe60 2024-03-11T18:36:46.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.408+0000 7fb8dffff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8e00c65b0 0x7fb8e02001c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48192/0 (socket says 172.21.15.90:48192) 2024-03-11T18:36:46.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.408+0000 7fb8dffff640 1 -- 172.21.15.90:0/1008985873 learned_addr learned my addr 172.21.15.90:0/1008985873 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:46.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.412+0000 7f2996248640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f299006f0a0 0x7f2990149980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:46.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.412+0000 7f2996248640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f299006f0a0 0x7f2990149980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48196/0 (socket says 172.21.15.90:48196) 2024-03-11T18:36:46.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.412+0000 7f2996248640 1 -- 172.21.15.90:0/3198063468 learned_addr learned my addr 172.21.15.90:0/3198063468 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:46.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.412+0000 7f16f46b9640 1 -- 172.21.15.90:0/4190295525 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16ec06fe60 msgr2=0x7f16ec0c2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.412+0000 7f16f46b9640 1 --2- 172.21.15.90:0/4190295525 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16ec06fe60 0x7f16ec0c2dd0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f16e4009e50 tx=0x7f16e401c960 comp rx=0 tx=0).stop 2024-03-11T18:36:46.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.412+0000 7f16f46b9640 1 -- 172.21.15.90:0/4190295525 shutdown_connections 2024-03-11T18:36:46.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.412+0000 7f16f46b9640 1 --2- 172.21.15.90:0/4190295525 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16ec06fe60 0x7f16ec0c2dd0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.412+0000 7f16f46b9640 1 --2- 172.21.15.90:0/4190295525 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f16ec1550c0 0x7f16ec0c8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.412+0000 7f16f46b9640 1 -- 172.21.15.90:0/4190295525 >> 172.21.15.90:0/4190295525 conn(0x7f16ec070e80 msgr2=0x7f16ec066990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:46.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.412+0000 7f16f46b9640 1 -- 172.21.15.90:0/4190295525 shutdown_connections 2024-03-11T18:36:46.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.412+0000 7f16f46b9640 1 -- 172.21.15.90:0/4190295525 wait complete. 2024-03-11T18:36:46.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.412+0000 7f06bad9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06bc2008f0 0x7f06bc1fcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:46.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.412+0000 7f06bad9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06bc2008f0 0x7f06bc1fcb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48202/0 (socket says 172.21.15.90:48202) 2024-03-11T18:36:46.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.412+0000 7f06bad9d640 1 -- 172.21.15.90:0/4259698342 learned_addr learned my addr 172.21.15.90:0/4259698342 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:46.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.412+0000 7fb8e531c640 1 --2- 172.21.15.90:0/1008985873 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8e0150a90 0x7fb8e01594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:46.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.412+0000 7f16f46b9640 1 Processor -- start 2024-03-11T18:36:46.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.412+0000 7f16f46b9640 1 -- start start 2024-03-11T18:36:46.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.416+0000 7f16f46b9640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f16ec06fe60 0x7f16ec1f3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.416+0000 7f16f46b9640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16ec1550c0 0x7f16ec155880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.416+0000 7f16f46b9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16ec1554a0 0x7f16ec155dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.416+0000 7f16f46b9640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ec1f2f90 con 0x7f16ec1550c0 2024-03-11T18:36:46.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.416+0000 7f16f46b9640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ec06d510 con 0x7f16ec1554a0 2024-03-11T18:36:46.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.416+0000 7f16f46b9640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16ec1f3ec0 con 0x7f16ec06fe60 2024-03-11T18:36:46.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.416+0000 7fb8dffff640 1 -- 172.21.15.90:0/1008985873 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb8e00c3d30 msgr2=0x7fb8e0155580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.416+0000 7fb8dffff640 1 --2- 172.21.15.90:0/1008985873 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb8e00c3d30 0x7fb8e0155580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.416+0000 7fb8dffff640 1 -- 172.21.15.90:0/1008985873 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8e0150a90 msgr2=0x7fb8e01594e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.416+0000 7fb8dffff640 1 --2- 172.21.15.90:0/1008985873 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8e0150a90 0x7fb8e01594e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.416+0000 7fb8dffff640 1 -- 172.21.15.90:0/1008985873 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d400b770 con 0x7fb8e00c65b0 2024-03-11T18:36:46.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.420+0000 7f2995a47640 1 --2- 172.21.15.90:0/3198063468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2990153f00 0x7f29901554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:46.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.428+0000 7f2996248640 1 -- 172.21.15.90:0/3198063468 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f2990159c80 msgr2=0x7f29901f5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.428+0000 7f2996248640 1 --2- 172.21.15.90:0/3198063468 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f2990159c80 0x7f29901f5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.428+0000 7f2996248640 1 -- 172.21.15.90:0/3198063468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2990153f00 msgr2=0x7f29901554c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.428+0000 7f2996248640 1 --2- 172.21.15.90:0/3198063468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2990153f00 0x7f29901554c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.428+0000 7f2996248640 1 -- 172.21.15.90:0/3198063468 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2988009b30 con 0x7f299006f0a0 2024-03-11T18:36:46.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.428+0000 7fb8dffff640 1 --2- 172.21.15.90:0/1008985873 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8e00c65b0 0x7fb8e02001c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc00b4b0 tx=0x7fb8cc00b970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:46.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.428+0000 7f06b9d9b640 1 --2- 172.21.15.90:0/4259698342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06bc144e10 0x7f06bc2003b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:46.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.432+0000 7f2996248640 1 --2- 172.21.15.90:0/3198063468 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f299006f0a0 0x7f2990149980 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f298c00c4c0 tx=0x7f298c001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:46.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.432+0000 7f16f2455640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f16ec06fe60 0x7f16ec1f3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:46.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.432+0000 7f06ba59c640 1 --2- 172.21.15.90:0/4259698342 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f06a4003980 0x7f06bc14f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:46.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.432+0000 7f16f2455640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f16ec06fe60 0x7f16ec1f3980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.90:59244/0 (socket says 172.21.15.90:59244) 2024-03-11T18:36:46.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.432+0000 7f16f2455640 1 -- 172.21.15.90:0/1988464808 learned_addr learned my addr 172.21.15.90:0/1988464808 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:46.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.436+0000 7f713ffff640 1 Processor -- start 2024-03-11T18:36:46.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.436+0000 7f713ffff640 1 -- start start 2024-03-11T18:36:46.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.436+0000 7f06bad9d640 1 -- 172.21.15.90:0/4259698342 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f06a4003980 msgr2=0x7f06bc14f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.436+0000 7f06bad9d640 1 --2- 172.21.15.90:0/4259698342 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f06a4003980 0x7f06bc14f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.436+0000 7f06bad9d640 1 -- 172.21.15.90:0/4259698342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06bc144e10 msgr2=0x7f06bc2003b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.436+0000 7f06bad9d640 1 --2- 172.21.15.90:0/4259698342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06bc144e10 0x7f06bc2003b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.436+0000 7f06bad9d640 1 -- 172.21.15.90:0/4259698342 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06bc0cbe20 con 0x7f06bc2008f0 2024-03-11T18:36:46.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.436+0000 7f713ffff640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f71380d9020 0x7f7138046180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.436+0000 7f713ffff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7138007e10 0x7f71380466c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.436+0000 7f713ffff640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71380dfa20 con 0x7f71380471d0 2024-03-11T18:36:46.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.436+0000 7f713ffff640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7138047bd0 con 0x7f7138007e10 2024-03-11T18:36:46.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.436+0000 7f713ffff640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71380df0f0 con 0x7f71380d9020 2024-03-11T18:36:46.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.436+0000 7f16f2455640 1 -- 172.21.15.90:0/1988464808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16ec1554a0 msgr2=0x7f16ec155dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.440+0000 7f16f2455640 1 --2- 172.21.15.90:0/1988464808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16ec1554a0 0x7f16ec155dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.440+0000 7f16f2455640 1 -- 172.21.15.90:0/1988464808 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16ec1550c0 msgr2=0x7f16ec155880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.440+0000 7f16f2455640 1 --2- 172.21.15.90:0/1988464808 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16ec1550c0 0x7f16ec155880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.440+0000 7f16f2455640 1 -- 172.21.15.90:0/1988464808 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16e4009b30 con 0x7f16ec06fe60 2024-03-11T18:36:46.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.444+0000 7f06bad9d640 1 --2- 172.21.15.90:0/4259698342 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06bc2008f0 0x7f06bc1fcb00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f06bc06ffe0 tx=0x7f06b400ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:46.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.448+0000 7f713e7fc640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7138007e10 0x7f71380466c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:46.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.448+0000 7f16f2455640 1 --2- 172.21.15.90:0/1988464808 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f16ec06fe60 0x7f16ec1f3980 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f16e800b410 tx=0x7f16e800b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:46.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.448+0000 7f713e7fc640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f7138007e10 0x7f71380466c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.90:60430/0 (socket says 172.21.15.90:60430) 2024-03-11T18:36:46.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.448+0000 7f713e7fc640 1 -- 172.21.15.90:0/3020956706 learned_addr learned my addr 172.21.15.90:0/3020956706 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:46.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.448+0000 7f713effd640 1 --2- 172.21.15.90:0/3020956706 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f71380d9020 0x7f7138046180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:46.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.448+0000 7fb8dd7fa640 1 -- 172.21.15.90:0/1008985873 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc010610 con 0x7fb8e00c65b0 2024-03-11T18:36:46.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.448+0000 7fb8dd7fa640 1 -- 172.21.15.90:0/1008985873 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8cc010770 con 0x7fb8e00c65b0 2024-03-11T18:36:46.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.448+0000 7fb8dd7fa640 1 -- 172.21.15.90:0/1008985873 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc010b20 con 0x7fb8e00c65b0 2024-03-11T18:36:46.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.448+0000 7f713e7fc640 1 -- 172.21.15.90:0/3020956706 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f71380d9020 msgr2=0x7f7138046180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:46.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.448+0000 7f713e7fc640 1 --2- 172.21.15.90:0/3020956706 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f71380d9020 0x7f7138046180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.448+0000 7f713e7fc640 1 -- 172.21.15.90:0/3020956706 >> v1:172.21.15.90:6789/0 conn(0x7f71380471d0 legacy=0x7f713804bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:46.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.448+0000 7f713e7fc640 1 -- 172.21.15.90:0/3020956706 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71380df3d0 con 0x7f7138007e10 2024-03-11T18:36:46.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.448+0000 7fb8e6d7f640 1 -- 172.21.15.90:0/1008985873 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8e01501c0 con 0x7fb8e00c65b0 2024-03-11T18:36:46.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.448+0000 7fb8e6d7f640 1 -- 172.21.15.90:0/1008985873 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8e01ff9c0 con 0x7fb8e00c65b0 2024-03-11T18:36:46.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.452+0000 7fb8e6d7f640 1 -- 172.21.15.90:0/1008985873 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8e004eef0 con 0x7fb8e00c65b0 2024-03-11T18:36:46.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.452+0000 7f16e2ffd640 1 -- 172.21.15.90:0/1988464808 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e8011830 con 0x7f16ec06fe60 2024-03-11T18:36:46.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.452+0000 7f16e2ffd640 1 -- 172.21.15.90:0/1988464808 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16e80119b0 con 0x7f16ec06fe60 2024-03-11T18:36:46.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.452+0000 7f16e2ffd640 1 -- 172.21.15.90:0/1988464808 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e8011d60 con 0x7f16ec06fe60 2024-03-11T18:36:46.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.452+0000 7f16f46b9640 1 -- 172.21.15.90:0/1988464808 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16ec06d7d0 con 0x7f16ec06fe60 2024-03-11T18:36:46.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.452+0000 7f16f46b9640 1 -- 172.21.15.90:0/1988464808 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16ec1f3180 con 0x7f16ec06fe60 2024-03-11T18:36:46.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.452+0000 7f16f46b9640 1 -- 172.21.15.90:0/1988464808 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16d40052a0 con 0x7f16ec06fe60 2024-03-11T18:36:46.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.452+0000 7f2986ffd640 1 -- 172.21.15.90:0/3198063468 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298c005480 con 0x7f299006f0a0 2024-03-11T18:36:46.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.452+0000 7f2986ffd640 1 -- 172.21.15.90:0/3198063468 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f298c003600 con 0x7f299006f0a0 2024-03-11T18:36:46.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.452+0000 7f2986ffd640 1 -- 172.21.15.90:0/3198063468 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298c003990 con 0x7f299006f0a0 2024-03-11T18:36:46.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.456+0000 7f29984ac640 1 -- 172.21.15.90:0/3198063468 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2990145b60 con 0x7f299006f0a0 2024-03-11T18:36:46.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.456+0000 7f29984ac640 1 -- 172.21.15.90:0/3198063468 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2990145880 con 0x7f299006f0a0 2024-03-11T18:36:46.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.456+0000 7f06aaffd640 1 -- 172.21.15.90:0/4259698342 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b400a610 con 0x7f06bc2008f0 2024-03-11T18:36:46.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.456+0000 7f06aaffd640 1 -- 172.21.15.90:0/4259698342 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06b400a770 con 0x7f06bc2008f0 2024-03-11T18:36:46.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.456+0000 7f06c0c28640 1 -- 172.21.15.90:0/4259698342 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06bc14a4c0 con 0x7f06bc2008f0 2024-03-11T18:36:46.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.456+0000 7f06aaffd640 1 -- 172.21.15.90:0/4259698342 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b400ab00 con 0x7f06bc2008f0 2024-03-11T18:36:46.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.456+0000 7f06c0c28640 1 -- 172.21.15.90:0/4259698342 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06bc1ffe90 con 0x7f06bc2008f0 2024-03-11T18:36:46.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.456+0000 7f713e7fc640 1 --2- 172.21.15.90:0/3020956706 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7138007e10 0x7f71380466c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7134009870 tx=0x7f713401ed10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0b172d66111b7f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:46.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.456+0000 7f29984ac640 1 -- 172.21.15.90:0/3198063468 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f299004eef0 con 0x7f299006f0a0 2024-03-11T18:36:46.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.456+0000 7f06c0c28640 1 -- 172.21.15.90:0/4259698342 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06bc0c8350 con 0x7f06bc2008f0 2024-03-11T18:36:46.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.460+0000 7f713dffb640 1 -- 172.21.15.90:0/3020956706 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71340a89d0 con 0x7f7138007e10 2024-03-11T18:36:46.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.460+0000 7f713dffb640 1 -- 172.21.15.90:0/3020956706 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71340a8b30 con 0x7f7138007e10 2024-03-11T18:36:46.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.460+0000 7f713dffb640 1 -- 172.21.15.90:0/3020956706 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71340b0440 con 0x7f7138007e10 2024-03-11T18:36:46.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.460+0000 7f713ffff640 1 -- 172.21.15.90:0/3020956706 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7138007e10 msgr2=0x7f71380466c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.460+0000 7f713ffff640 1 --2- 172.21.15.90:0/3020956706 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7138007e10 0x7f71380466c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7134009870 tx=0x7f713401ed10 comp rx=0 tx=0).stop 2024-03-11T18:36:46.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.464+0000 7f713ffff640 1 -- 172.21.15.90:0/3020956706 shutdown_connections 2024-03-11T18:36:46.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.464+0000 7f713ffff640 1 --2- 172.21.15.90:0/3020956706 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f7138007e10 0x7f71380466c0 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7134009870 tx=0x7f713401ed10 comp rx=0 tx=0).stop 2024-03-11T18:36:46.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.464+0000 7f713ffff640 1 --2- 172.21.15.90:0/3020956706 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f71380d9020 0x7f7138046180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.464+0000 7f713ffff640 1 -- 172.21.15.90:0/3020956706 >> 172.21.15.90:0/3020956706 conn(0x7f71380512a0 msgr2=0x7f71380516b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:46.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.464+0000 7f713ffff640 1 -- 172.21.15.90:0/3020956706 shutdown_connections 2024-03-11T18:36:46.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.464+0000 7f713ffff640 1 -- 172.21.15.90:0/3020956706 wait complete. 2024-03-11T18:36:46.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.464+0000 7f713ffff640 1 Processor -- start 2024-03-11T18:36:46.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.464+0000 7f713ffff640 1 -- start start 2024-03-11T18:36:46.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.464+0000 7f713ffff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71380471d0 0x7f7138034210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.464+0000 7f713ffff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71380d9020 0x7f7138034750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.464+0000 7f713ffff640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f713804a080 0x7f713804a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.464+0000 7f713ffff640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7138033fb0 con 0x7f71380471d0 2024-03-11T18:36:46.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.464+0000 7f713ffff640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713816b4c0 con 0x7f71380d9020 2024-03-11T18:36:46.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.464+0000 7f713ffff640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f713816ba80 con 0x7f713804a080 2024-03-11T18:36:46.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.468+0000 7f713e7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71380d9020 0x7f7138034750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:46.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.468+0000 7f713e7fc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71380d9020 0x7f7138034750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.90:60434/0 (socket says 172.21.15.90:60434) 2024-03-11T18:36:46.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.468+0000 7f713e7fc640 1 -- 172.21.15.90:0/1279689292 learned_addr learned my addr 172.21.15.90:0/1279689292 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:46.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.472+0000 7f713e7fc640 1 -- 172.21.15.90:0/1279689292 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f713804a080 msgr2=0x7f713804a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.472+0000 7f713e7fc640 1 --2- 172.21.15.90:0/1279689292 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f713804a080 0x7f713804a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.472+0000 7f713e7fc640 1 -- 172.21.15.90:0/1279689292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71380471d0 msgr2=0x7f7138034210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.472+0000 7f713e7fc640 1 --2- 172.21.15.90:0/1279689292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71380471d0 0x7f7138034210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.472+0000 7f713e7fc640 1 -- 172.21.15.90:0/1279689292 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71340094d0 con 0x7f71380d9020 2024-03-11T18:36:46.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.472+0000 7f713e7fc640 1 --2- 172.21.15.90:0/1279689292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71380d9020 0x7f7138034750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f713401d3c0 tx=0x7f713401bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:46.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.476+0000 7f71277fe640 1 -- 172.21.15.90:0/1279689292 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71340b0e60 con 0x7f71380d9020 2024-03-11T18:36:46.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.476+0000 7f713ffff640 1 -- 172.21.15.90:0/1279689292 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f713804aa70 con 0x7f71380d9020 2024-03-11T18:36:46.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.476+0000 7f713ffff640 1 -- 172.21.15.90:0/1279689292 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f713804b360 con 0x7f71380d9020 2024-03-11T18:36:46.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.476+0000 7f713ffff640 1 -- 172.21.15.90:0/1279689292 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f713816bbf0 con 0x7f71380d9020 2024-03-11T18:36:46.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.476+0000 7f71277fe640 1 -- 172.21.15.90:0/1279689292 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71340c5460 con 0x7f71380d9020 2024-03-11T18:36:46.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.476+0000 7f71277fe640 1 -- 172.21.15.90:0/1279689292 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71340c57d0 con 0x7f71380d9020 2024-03-11T18:36:46.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.484+0000 7fe3bef90640 1 Processor -- start 2024-03-11T18:36:46.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.484+0000 7fe3bef90640 1 -- start start 2024-03-11T18:36:46.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.484+0000 7fe3bef90640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe3b80c34e0 0x7fe3b80c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.488+0000 7fe3bef90640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3b80c2ae0 con 0x7fe3b81550c0 2024-03-11T18:36:46.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.488+0000 7fe3bef90640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3b80c2560 con 0x7fe3b81554a0 2024-03-11T18:36:46.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.488+0000 7fe3bef90640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3b8157cc0 con 0x7fe3b80c34e0 2024-03-11T18:36:46.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.532+0000 7fec3cee5640 1 Processor -- start 2024-03-11T18:36:46.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.532+0000 7fec3cee5640 1 -- start start 2024-03-11T18:36:46.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.532+0000 7f4e988d2640 1 Processor -- start 2024-03-11T18:36:46.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.532+0000 7f4e988d2640 1 -- start start 2024-03-11T18:36:46.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.532+0000 7fec3cee5640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fec38151d50 0x7fec380c3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.532+0000 7fec3cee5640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fec3806b860 0x7fec380c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.532+0000 7fec3cee5640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec380c7430 con 0x7fec38157970 2024-03-11T18:36:46.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.532+0000 7fec3cee5640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec38152130 con 0x7fec38151d50 2024-03-11T18:36:46.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.532+0000 7fec3cee5640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec380c7a40 con 0x7fec3806b860 2024-03-11T18:36:46.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.532+0000 7f4e988d2640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4e940c5b50 0x7f4e940c4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.532+0000 7f4e988d2640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e940c7980 con 0x7f4e94066d60 2024-03-11T18:36:46.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.532+0000 7f4e988d2640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e940c2ad0 con 0x7f4e94151000 2024-03-11T18:36:46.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.532+0000 7f4e988d2640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e940cc820 con 0x7f4e940c5b50 2024-03-11T18:36:46.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.744+0000 7f16e2ffd640 1 -- 172.21.15.90:0/1988464808 <== mon.2 v2:172.21.15.92:3301/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f16e801a400 con 0x7f16ec06fe60 2024-03-11T18:36:46.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.744+0000 7f16e2ffd640 1 --2- 172.21.15.90:0/1988464808 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f16d0066ec0 0x7f16d0069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.744+0000 7f16f1c54640 1 --2- 172.21.15.90:0/1988464808 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f16d0066ec0 0x7f16d0069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:46.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.744+0000 7f16f1c54640 1 --2- 172.21.15.90:0/1988464808 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f16d0066ec0 0x7f16d0069380 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f16dc007fd0 tx=0x7f16dc0079a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:46.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.748+0000 7f16e2ffd640 1 -- 172.21.15.90:0/1988464808 <== mon.2 v2:172.21.15.92:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f16e8025e30 con 0x7f16ec06fe60 2024-03-11T18:36:46.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.752+0000 7fe3bd52d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe3b80c34e0 0x7fe3b80c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:46.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.756+0000 7f71277fe640 1 -- 172.21.15.90:0/1279689292 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f71340c5930 con 0x7f71380d9020 2024-03-11T18:36:46.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.756+0000 7f71277fe640 1 --2- 172.21.15.90:0/1279689292 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f712c066e70 0x7f712c069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.756+0000 7f713effd640 1 --2- 172.21.15.90:0/1279689292 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f712c066e70 0x7f712c069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:46.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.756+0000 7fe3bd52d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe3b80c34e0 0x7fe3b80c8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.90:60436/0 (socket says 172.21.15.90:60436) 2024-03-11T18:36:46.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.756+0000 7fe3bd52d640 1 -- 172.21.15.90:0/3524616908 learned_addr learned my addr 172.21.15.90:0/3524616908 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:46.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.756+0000 7f713effd640 1 --2- 172.21.15.90:0/1279689292 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f712c066e70 0x7f712c069330 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f713000a8b0 tx=0x7f7130008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:46.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.760+0000 7f16e2ffd640 1 -- 172.21.15.90:0/1988464808 <== mon.2 v2:172.21.15.92:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f16e8051f00 con 0x7f16ec06fe60 2024-03-11T18:36:46.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.764+0000 7f71277fe640 1 -- 172.21.15.90:0/1279689292 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f71340f49c0 con 0x7f71380d9020 2024-03-11T18:36:46.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.764+0000 7fec35d9b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fec3806b860 0x7fec380c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:46.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.764+0000 7f4e8bfff640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4e940c5b50 0x7f4e940c4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:46.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.764+0000 7f4e8bfff640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4e940c5b50 0x7f4e940c4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.90:59264/0 (socket says 172.21.15.90:59264) 2024-03-11T18:36:46.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.764+0000 7f4e8bfff640 1 -- 172.21.15.90:0/2757752471 learned_addr learned my addr 172.21.15.90:0/2757752471 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:46.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.764+0000 7fec35d9b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fec3806b860 0x7fec380c95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.90:60446/0 (socket says 172.21.15.90:60446) 2024-03-11T18:36:46.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.764+0000 7fec35d9b640 1 -- 172.21.15.90:0/2534379822 learned_addr learned my addr 172.21.15.90:0/2534379822 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:46.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.764+0000 7f4e8bfff640 1 -- 172.21.15.90:0/2757752471 >> v1:172.21.15.92:6789/0 conn(0x7f4e94151000 legacy=0x7f4e940c2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:46.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.764+0000 7f4e8bfff640 1 -- 172.21.15.90:0/2757752471 >> v1:172.21.15.90:6789/0 conn(0x7f4e94066d60 legacy=0x7f4e941613a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:46.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.764+0000 7f4e8bfff640 1 -- 172.21.15.90:0/2757752471 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e940ca9c0 con 0x7f4e940c5b50 2024-03-11T18:36:46.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.772+0000 7f4e8bfff640 1 --2- 172.21.15.90:0/2757752471 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4e940c5b50 0x7f4e940c4ad0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c00bb40 tx=0x7f4e7c01eab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb094f89f467fca5 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:46.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.776+0000 7f4e91d9b640 1 -- 172.21.15.90:0/2757752471 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c0a7930 con 0x7f4e940c5b50 2024-03-11T18:36:46.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.776+0000 7f4e91d9b640 1 -- 172.21.15.90:0/2757752471 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e7c0a7a90 con 0x7f4e940c5b50 2024-03-11T18:36:46.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.776+0000 7f4e91d9b640 1 -- 172.21.15.90:0/2757752471 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c0a7e20 con 0x7f4e940c5b50 2024-03-11T18:36:46.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.780+0000 7f4e988d2640 1 -- 172.21.15.90:0/2757752471 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4e940c5b50 msgr2=0x7f4e940c4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.780+0000 7f4e988d2640 1 --2- 172.21.15.90:0/2757752471 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4e940c5b50 0x7f4e940c4ad0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c00bb40 tx=0x7f4e7c01eab0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.780+0000 7f4e988d2640 1 -- 172.21.15.90:0/2757752471 shutdown_connections 2024-03-11T18:36:46.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.780+0000 7f4e988d2640 1 --2- 172.21.15.90:0/2757752471 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4e940c5b50 0x7f4e940c4ad0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.780+0000 7f4e988d2640 1 -- 172.21.15.90:0/2757752471 >> 172.21.15.90:0/2757752471 conn(0x7f4e940cb880 msgr2=0x7f4e940cbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:46.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.780+0000 7f4e988d2640 1 -- 172.21.15.90:0/2757752471 shutdown_connections 2024-03-11T18:36:46.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.780+0000 7f4e988d2640 1 -- 172.21.15.90:0/2757752471 wait complete. 2024-03-11T18:36:46.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.780+0000 7fe3bd52d640 1 -- 172.21.15.90:0/3524616908 >> v1:172.21.15.92:6789/0 conn(0x7fe3b81554a0 legacy=0x7fe3b80c2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:46.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.780+0000 7fe3bd52d640 1 -- 172.21.15.90:0/3524616908 >> v1:172.21.15.90:6789/0 conn(0x7fe3b81550c0 legacy=0x7fe3b815ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:46.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.780+0000 7fe3bd52d640 1 -- 172.21.15.90:0/3524616908 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b80c2800 con 0x7fe3b80c34e0 2024-03-11T18:36:46.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.780+0000 7f71277fe640 1 -- 172.21.15.90:0/1279689292 <== mon.1 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f71340f0560 con 0x7f71380d9020 2024-03-11T18:36:46.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.784+0000 7fec35d9b640 1 -- 172.21.15.90:0/2534379822 >> v2:172.21.15.90:3300/0 conn(0x7fec38151d50 msgr2=0x7fec380c3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:46.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.784+0000 7fec35d9b640 1 --2- 172.21.15.90:0/2534379822 >> v2:172.21.15.90:3300/0 conn(0x7fec38151d50 0x7fec380c3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.784+0000 7fec35d9b640 1 -- 172.21.15.90:0/2534379822 >> v1:172.21.15.92:6789/0 conn(0x7fec38157970 legacy=0x7fec380c5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:46.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.784+0000 7fec35d9b640 1 -- 172.21.15.90:0/2534379822 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec380c6e70 con 0x7fec3806b860 2024-03-11T18:36:46.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.784+0000 7f4e988d2640 1 Processor -- start 2024-03-11T18:36:46.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.784+0000 7f4e988d2640 1 -- start start 2024-03-11T18:36:46.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.788+0000 7f4e988d2640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e94066d60 0x7f4e94145f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.788+0000 7f4e988d2640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4e940c5b50 0x7f4e94144bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.788+0000 7f4e988d2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e94151000 0x7f4e9414b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.788+0000 7f4e988d2640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e941ef010 con 0x7f4e94066d60 2024-03-11T18:36:46.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.788+0000 7f4e988d2640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9414aa80 con 0x7f4e94151000 2024-03-11T18:36:46.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.788+0000 7f4e988d2640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e94146470 con 0x7f4e940c5b50 2024-03-11T18:36:46.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.788+0000 7f4e8bfff640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4e940c5b50 0x7f4e94144bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:46.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.788+0000 7f4e8bfff640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4e940c5b50 0x7f4e94144bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.90:59270/0 (socket says 172.21.15.90:59270) 2024-03-11T18:36:46.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.788+0000 7f4e8bfff640 1 -- 172.21.15.90:0/3925709223 learned_addr learned my addr 172.21.15.90:0/3925709223 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:46.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.792+0000 7fec35d9b640 1 --2- 172.21.15.90:0/2534379822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec3806b860 0x7fec380c95f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fec38074ac0 tx=0x7fec2c01ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee433685445797fa server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:46.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.796+0000 7f4e8bfff640 1 -- 172.21.15.90:0/3925709223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e94151000 msgr2=0x7f4e9414b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.796+0000 7f4e8bfff640 1 --2- 172.21.15.90:0/3925709223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e94151000 0x7f4e9414b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.796+0000 7f4e8bfff640 1 -- 172.21.15.90:0/3925709223 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e94066d60 msgr2=0x7f4e94145f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.796+0000 7f4e8bfff640 1 --2- 172.21.15.90:0/3925709223 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e94066d60 0x7f4e94145f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.796+0000 7f4e8bfff640 1 -- 172.21.15.90:0/3925709223 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e7c00b770 con 0x7f4e940c5b50 2024-03-11T18:36:46.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.800+0000 7fec3559a640 1 -- 172.21.15.90:0/2534379822 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2c0a89d0 con 0x7fec3806b860 2024-03-11T18:36:46.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.800+0000 7fec3559a640 1 -- 172.21.15.90:0/2534379822 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec2c0a8b30 con 0x7fec3806b860 2024-03-11T18:36:46.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.800+0000 7fec3559a640 1 -- 172.21.15.90:0/2534379822 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2c0b0440 con 0x7fec3806b860 2024-03-11T18:36:46.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.800+0000 7fec3cee5640 1 -- 172.21.15.90:0/2534379822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec3806b860 msgr2=0x7fec380c95f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.800+0000 7fec3cee5640 1 --2- 172.21.15.90:0/2534379822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec3806b860 0x7fec380c95f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fec38074ac0 tx=0x7fec2c01ecd0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.800+0000 7f4e8bfff640 1 --2- 172.21.15.90:0/3925709223 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4e940c5b50 0x7f4e94144bb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c01d160 tx=0x7f4e7c01b850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:46.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.800+0000 7fec3cee5640 1 -- 172.21.15.90:0/2534379822 shutdown_connections 2024-03-11T18:36:46.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.800+0000 7fec3cee5640 1 --2- 172.21.15.90:0/2534379822 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec3806b860 0x7fec380c95f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.800+0000 7fec3cee5640 1 --2- 172.21.15.90:0/2534379822 >> v2:172.21.15.90:3300/0 conn(0x7fec38151d50 0x7fec380c3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.800+0000 7fec3cee5640 1 -- 172.21.15.90:0/2534379822 >> 172.21.15.90:0/2534379822 conn(0x7fec38071aa0 msgr2=0x7fec38071eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:46.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.800+0000 7fec3cee5640 1 -- 172.21.15.90:0/2534379822 shutdown_connections 2024-03-11T18:36:46.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.800+0000 7fec3cee5640 1 -- 172.21.15.90:0/2534379822 wait complete. 2024-03-11T18:36:46.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.800+0000 7fe3bd52d640 1 --2- 172.21.15.90:0/3524616908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3b80c34e0 0x7fe3b80c8fb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0009e50 tx=0x7fe3b001c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc2e4eb717ad75c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:46.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.804+0000 7fec3cee5640 1 Processor -- start 2024-03-11T18:36:46.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.804+0000 7fec3cee5640 1 -- start start 2024-03-11T18:36:46.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.804+0000 7fec3cee5640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fec3806b860 0x7fec380c4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.804+0000 7fec3cee5640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec38151d50 0x7fec38150de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.804+0000 7fec3cee5640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec38157970 0x7fec3813eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.804+0000 7fec3cee5640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec380c4d40 con 0x7fec38151d50 2024-03-11T18:36:46.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.804+0000 7fec3cee5640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec380c52f0 con 0x7fec38157970 2024-03-11T18:36:46.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.804+0000 7fec3cee5640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec38154e10 con 0x7fec3806b860 2024-03-11T18:36:46.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.808+0000 7f4e8affd640 1 -- 172.21.15.90:0/3925709223 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c0a7930 con 0x7f4e940c5b50 2024-03-11T18:36:46.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.808+0000 7f4e8affd640 1 -- 172.21.15.90:0/3925709223 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e7c0b8400 con 0x7f4e940c5b50 2024-03-11T18:36:46.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.808+0000 7f4e8affd640 1 -- 172.21.15.90:0/3925709223 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c0b87d0 con 0x7f4e940c5b50 2024-03-11T18:36:46.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.808+0000 7f4e988d2640 1 -- 172.21.15.90:0/3925709223 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e941ef870 con 0x7f4e940c5b50 2024-03-11T18:36:46.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.808+0000 7f4e988d2640 1 -- 172.21.15.90:0/3925709223 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e9414a7a0 con 0x7f4e940c5b50 2024-03-11T18:36:46.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.808+0000 7fec3659c640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fec3806b860 0x7fec380c4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:46.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.808+0000 7fec3659c640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fec3806b860 0x7fec380c4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.90:59280/0 (socket says 172.21.15.90:59280) 2024-03-11T18:36:46.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.808+0000 7fec3659c640 1 -- 172.21.15.90:0/1455271732 learned_addr learned my addr 172.21.15.90:0/1455271732 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:46.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.808+0000 7f4e988d2640 1 -- 172.21.15.90:0/3925709223 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e9404eef0 con 0x7f4e940c5b50 2024-03-11T18:36:46.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.812+0000 7fec36d9d640 1 --2- 172.21.15.90:0/1455271732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec38157970 0x7fec3813eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:46.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.816+0000 7fe3b77fe640 1 -- 172.21.15.90:0/3524616908 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b00a56d0 con 0x7fe3b80c34e0 2024-03-11T18:36:46.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.816+0000 7fe3b77fe640 1 -- 172.21.15.90:0/3524616908 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3b00a5830 con 0x7fe3b80c34e0 2024-03-11T18:36:46.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.816+0000 7fe3b77fe640 1 -- 172.21.15.90:0/3524616908 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b00a5bc0 con 0x7fe3b80c34e0 2024-03-11T18:36:46.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.816+0000 7fe3bef90640 1 -- 172.21.15.90:0/3524616908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3b80c34e0 msgr2=0x7fe3b80c8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.816+0000 7fe3bef90640 1 --2- 172.21.15.90:0/3524616908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3b80c34e0 0x7fe3b80c8fb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0009e50 tx=0x7fe3b001c920 comp rx=0 tx=0).stop 2024-03-11T18:36:46.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.816+0000 7fe3bef90640 1 -- 172.21.15.90:0/3524616908 shutdown_connections 2024-03-11T18:36:46.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.816+0000 7fe3bef90640 1 --2- 172.21.15.90:0/3524616908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3b80c34e0 0x7fe3b80c8fb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.816+0000 7fe3bef90640 1 -- 172.21.15.90:0/3524616908 >> 172.21.15.90:0/3524616908 conn(0x7fe3b8070e80 msgr2=0x7fe3b8066990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:46.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.816+0000 7fe3bef90640 1 -- 172.21.15.90:0/3524616908 shutdown_connections 2024-03-11T18:36:46.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.816+0000 7fe3bef90640 1 -- 172.21.15.90:0/3524616908 wait complete. 2024-03-11T18:36:46.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.816+0000 7fe3bef90640 1 Processor -- start 2024-03-11T18:36:46.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.816+0000 7fe3bef90640 1 -- start start 2024-03-11T18:36:46.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.820+0000 7fe3bef90640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe3b81550c0 0x7fe3b80b0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.820+0000 7fe3bef90640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe3b81554a0 0x7fe3b806cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.820+0000 7fe3bef90640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3b80b28f0 0x7fe3b81562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.820+0000 7fe3bef90640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b8151c30 con 0x7fe3b81554a0 2024-03-11T18:36:46.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.820+0000 7fe3bef90640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b80b12a0 con 0x7fe3b80b28f0 2024-03-11T18:36:46.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.820+0000 7fe3bef90640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b81f65e0 con 0x7fe3b81550c0 2024-03-11T18:36:46.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.820+0000 7fec36d9d640 1 -- 172.21.15.90:0/1455271732 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fec3806b860 msgr2=0x7fec380c4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.820+0000 7fec36d9d640 1 --2- 172.21.15.90:0/1455271732 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fec3806b860 0x7fec380c4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.820+0000 7fec36d9d640 1 -- 172.21.15.90:0/1455271732 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec38151d50 msgr2=0x7fec38150de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:46.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.820+0000 7fec36d9d640 1 --2- 172.21.15.90:0/1455271732 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec38151d50 0x7fec38150de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:46.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.820+0000 7fec36d9d640 1 -- 172.21.15.90:0/1455271732 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec2c0094d0 con 0x7fec38157970 2024-03-11T18:36:46.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.824+0000 7fec36d9d640 1 --2- 172.21.15.90:0/1455271732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec38157970 0x7fec3813eb50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fec3000be20 tx=0x7fec3000bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:46.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.832+0000 7fe3bd52d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3b80b28f0 0x7fe3b81562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:46.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.836+0000 7fec26ffd640 1 -- 172.21.15.90:0/1455271732 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3000ce70 con 0x7fec38157970 2024-03-11T18:36:46.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.836+0000 7fec26ffd640 1 -- 172.21.15.90:0/1455271732 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec30014440 con 0x7fec38157970 2024-03-11T18:36:46.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.836+0000 7fec26ffd640 1 -- 172.21.15.90:0/1455271732 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec30014830 con 0x7fec38157970 2024-03-11T18:36:46.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.836+0000 7fec3cee5640 1 -- 172.21.15.90:0/1455271732 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec381550f0 con 0x7fec38157970 2024-03-11T18:36:46.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.836+0000 7fec3cee5640 1 -- 172.21.15.90:0/1455271732 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec380c5890 con 0x7fec38157970 2024-03-11T18:36:46.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.836+0000 7fec3cee5640 1 -- 172.21.15.90:0/1455271732 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec3804eef0 con 0x7fec38157970 2024-03-11T18:36:46.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.948+0000 7f16f46b9640 1 -- 172.21.15.90:0/1988464808 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f16d4005820 con 0x7f16ec06fe60 2024-03-11T18:36:46.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.948+0000 7f2986ffd640 1 -- 172.21.15.90:0/3198063468 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f298c01c440 con 0x7f299006f0a0 2024-03-11T18:36:46.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.952+0000 7f2986ffd640 1 --2- 172.21.15.90:0/3198063468 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f2974066e90 0x7f2974069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:46.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.952+0000 7f2995a47640 1 --2- 172.21.15.90:0/3198063468 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f2974066e90 0x7f2974069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:46.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.952+0000 7f2995a47640 1 --2- 172.21.15.90:0/3198063468 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f2974066e90 0x7f2974069350 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2990147db0 tx=0x7f2980009110 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:46.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.972+0000 7f713ffff640 1 -- 172.21.15.90:0/1279689292 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f713816bbf0 con 0x7f71380d9020 2024-03-11T18:36:46.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:46.992+0000 7f2986ffd640 1 -- 172.21.15.90:0/3198063468 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f298c0552b0 con 0x7f299006f0a0 2024-03-11T18:36:47.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.092+0000 7f4e8affd640 1 -- 172.21.15.90:0/3925709223 <== mon.2 v2:172.21.15.92:3301/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f4e7c0b6040 con 0x7f4e940c5b50 2024-03-11T18:36:47.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.096+0000 7f4e8affd640 1 --2- 172.21.15.90:0/3925709223 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f4e74066e40 0x7f4e74069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:47.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.096+0000 7f4e9259c640 1 --2- 172.21.15.90:0/3925709223 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f4e74066e40 0x7f4e74069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:47.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.096+0000 7f4e9259c640 1 --2- 172.21.15.90:0/3925709223 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f4e74066e40 0x7f4e74069300 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4e84009650 tx=0x7f4e84006bb0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:47.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.104+0000 7f4e8affd640 1 -- 172.21.15.90:0/3925709223 <== mon.2 v2:172.21.15.92:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4e7c0f3160 con 0x7f4e940c5b50 2024-03-11T18:36:47.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.128+0000 7f16e2ffd640 1 -- 172.21.15.90:0/1988464808 <== mon.2 v2:172.21.15.92:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f16e80555c0 con 0x7f16ec06fe60 2024-03-11T18:36:47.135 INFO:teuthology.orchestra.run.smithi090.stdout:64424509461 2024-03-11T18:36:47.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.128+0000 7f4e8affd640 1 -- 172.21.15.90:0/3925709223 <== mon.2 v2:172.21.15.92:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4e7c0eed00 con 0x7f4e940c5b50 2024-03-11T18:36:47.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.128+0000 7f16f46b9640 1 -- 172.21.15.90:0/1988464808 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f16d0066ec0 msgr2=0x7f16d0069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:47.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.128+0000 7f16f46b9640 1 --2- 172.21.15.90:0/1988464808 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f16d0066ec0 0x7f16d0069380 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f16dc007fd0 tx=0x7f16dc0079a0 comp rx=0 tx=0).stop 2024-03-11T18:36:47.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.128+0000 7f16f46b9640 1 -- 172.21.15.90:0/1988464808 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f16ec06fe60 msgr2=0x7f16ec1f3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:47.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.128+0000 7f16f46b9640 1 --2- 172.21.15.90:0/1988464808 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f16ec06fe60 0x7f16ec1f3980 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f16e800b410 tx=0x7f16e800b8d0 comp rx=0 tx=0).stop 2024-03-11T18:36:47.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.132+0000 7f16f46b9640 1 -- 172.21.15.90:0/1988464808 shutdown_connections 2024-03-11T18:36:47.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.132+0000 7f16f46b9640 1 --2- 172.21.15.90:0/1988464808 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f16d0066ec0 0x7f16d0069380 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:47.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.132+0000 7f16f46b9640 1 --2- 172.21.15.90:0/1988464808 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f16ec1554a0 0x7f16ec155dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:47.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.132+0000 7f16f46b9640 1 --2- 172.21.15.90:0/1988464808 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16ec1550c0 0x7f16ec155880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:47.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.132+0000 7f16f46b9640 1 --2- 172.21.15.90:0/1988464808 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f16ec06fe60 0x7f16ec1f3980 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:47.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.132+0000 7f16f46b9640 1 -- 172.21.15.90:0/1988464808 >> 172.21.15.90:0/1988464808 conn(0x7f16ec070e80 msgr2=0x7f16ec0c8d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:47.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.132+0000 7f16f46b9640 1 -- 172.21.15.90:0/1988464808 shutdown_connections 2024-03-11T18:36:47.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.132+0000 7f16f46b9640 1 -- 172.21.15.90:0/1988464808 wait complete. 2024-03-11T18:36:47.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.132+0000 7fe3bcd2c640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe3b81550c0 0x7fe3b80b0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:47.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.136+0000 7fe3bcd2c640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe3b81550c0 0x7fe3b80b0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.90:59292/0 (socket says 172.21.15.90:59292) 2024-03-11T18:36:47.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.136+0000 7fe3bcd2c640 1 -- 172.21.15.90:0/2397849799 learned_addr learned my addr 172.21.15.90:0/2397849799 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:47.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.144+0000 7fe3bcd2c640 1 -- 172.21.15.90:0/2397849799 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3b80b28f0 msgr2=0x7fe3b81562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:47.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.144+0000 7fe3bcd2c640 1 --2- 172.21.15.90:0/2397849799 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3b80b28f0 0x7fe3b81562c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:47.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.144+0000 7fe3bcd2c640 1 -- 172.21.15.90:0/2397849799 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe3b81554a0 msgr2=0x7fe3b806cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:47.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.144+0000 7fe3bcd2c640 1 --2- 172.21.15.90:0/2397849799 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe3b81554a0 0x7fe3b806cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:47.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.144+0000 7fe3bcd2c640 1 -- 172.21.15.90:0/2397849799 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b0009b30 con 0x7fe3b81550c0 2024-03-11T18:36:47.152 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509462 got 64424509461 for osd.2 2024-03-11T18:36:47.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.148+0000 7fe3bcd2c640 1 --2- 172.21.15.90:0/2397849799 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe3b81550c0 0x7fe3b80b0c10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac00c490 tx=0x7fe3ac001e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:47.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.148+0000 7fec26ffd640 1 -- 172.21.15.90:0/1455271732 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fec3001d3b0 con 0x7fec38157970 2024-03-11T18:36:47.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.148+0000 7fec26ffd640 1 --2- 172.21.15.90:0/1455271732 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fec18066e90 0x7fec18069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:47.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.148+0000 7fec3659c640 1 --2- 172.21.15.90:0/1455271732 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fec18066e90 0x7fec18069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:47.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.152+0000 7fec3659c640 1 --2- 172.21.15.90:0/1455271732 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fec18066e90 0x7fec18069350 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fec28006fd0 tx=0x7fec28008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:47.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.156+0000 7fec26ffd640 1 -- 172.21.15.90:0/1455271732 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fec300562f0 con 0x7fec38157970 2024-03-11T18:36:47.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.156+0000 7fe3b57fa640 1 -- 172.21.15.90:0/2397849799 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac003600 con 0x7fe3b81550c0 2024-03-11T18:36:47.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.156+0000 7fe3b57fa640 1 -- 172.21.15.90:0/2397849799 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3ac003780 con 0x7fe3b81550c0 2024-03-11T18:36:47.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.156+0000 7fe3bef90640 1 -- 172.21.15.90:0/2397849799 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3b8158af0 con 0x7fe3b81550c0 2024-03-11T18:36:47.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.156+0000 7fe3b57fa640 1 -- 172.21.15.90:0/2397849799 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac003b70 con 0x7fe3b81550c0 2024-03-11T18:36:47.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.156+0000 7fe3bef90640 1 -- 172.21.15.90:0/2397849799 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3b81f6ba0 con 0x7fe3b81550c0 2024-03-11T18:36:47.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.160+0000 7fe3bef90640 1 -- 172.21.15.90:0/2397849799 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3b80c96c0 con 0x7fe3b81550c0 2024-03-11T18:36:47.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.172+0000 7fec26ffd640 1 -- 172.21.15.90:0/1455271732 <== mon.1 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fec30051e90 con 0x7fec38157970 2024-03-11T18:36:47.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.188+0000 7f71277fe640 1 -- 172.21.15.90:0/1279689292 <== mon.1 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f71340b5030 con 0x7f71380d9020 2024-03-11T18:36:47.195 INFO:teuthology.orchestra.run.smithi090.stdout:60129542163 2024-03-11T18:36:47.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.192+0000 7f713ffff640 1 -- 172.21.15.90:0/1279689292 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f712c066e70 msgr2=0x7f712c069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:47.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.192+0000 7f713ffff640 1 --2- 172.21.15.90:0/1279689292 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f712c066e70 0x7f712c069330 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f713000a8b0 tx=0x7f7130008040 comp rx=0 tx=0).stop 2024-03-11T18:36:47.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.192+0000 7f713ffff640 1 -- 172.21.15.90:0/1279689292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71380d9020 msgr2=0x7f7138034750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:47.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.192+0000 7f713ffff640 1 --2- 172.21.15.90:0/1279689292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71380d9020 0x7f7138034750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f713401d3c0 tx=0x7f713401bab0 comp rx=0 tx=0).stop 2024-03-11T18:36:47.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.192+0000 7f713ffff640 1 -- 172.21.15.90:0/1279689292 shutdown_connections 2024-03-11T18:36:47.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.192+0000 7f713ffff640 1 --2- 172.21.15.90:0/1279689292 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f712c066e70 0x7f712c069330 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:47.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.192+0000 7f713ffff640 1 --2- 172.21.15.90:0/1279689292 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f713804a080 0x7f713804a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:47.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.192+0000 7f713ffff640 1 --2- 172.21.15.90:0/1279689292 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f71380d9020 0x7f7138034750 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:47.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.192+0000 7f713ffff640 1 --2- 172.21.15.90:0/1279689292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71380471d0 0x7f7138034210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:47.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.192+0000 7f713ffff640 1 -- 172.21.15.90:0/1279689292 >> 172.21.15.90:0/1279689292 conn(0x7f71380512a0 msgr2=0x7f71380dde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:47.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.192+0000 7f713ffff640 1 -- 172.21.15.90:0/1279689292 shutdown_connections 2024-03-11T18:36:47.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.192+0000 7f713ffff640 1 -- 172.21.15.90:0/1279689292 wait complete. 2024-03-11T18:36:47.218 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542165 got 60129542163 for osd.0 2024-03-11T18:36:47.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.232+0000 7f2986ffd640 1 -- 172.21.15.90:0/3198063468 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f298c051550 con 0x7f299006f0a0 2024-03-11T18:36:47.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.272+0000 7fb8dd7fa640 1 -- 172.21.15.90:0/1008985873 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fb8cc010c80 con 0x7fb8e00c65b0 2024-03-11T18:36:47.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.272+0000 7fb8dd7fa640 1 --2- 172.21.15.90:0/1008985873 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fb8c4066e90 0x7fb8c4069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:47.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.272+0000 7fb8e4b1b640 1 --2- 172.21.15.90:0/1008985873 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fb8c4066e90 0x7fb8c4069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:47.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.272+0000 7fb8e4b1b640 1 --2- 172.21.15.90:0/1008985873 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fb8c4066e90 0x7fb8c4069350 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb8d40b3000 tx=0x7fb8d4004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:47.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.292+0000 7fb8dd7fa640 1 -- 172.21.15.90:0/1008985873 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb8cc0556e0 con 0x7fb8e00c65b0 2024-03-11T18:36:47.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.292+0000 7fb8dd7fa640 1 -- 172.21.15.90:0/1008985873 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb8cc051980 con 0x7fb8e00c65b0 2024-03-11T18:36:47.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.320+0000 7f06aaffd640 1 -- 172.21.15.90:0/4259698342 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f06b400ac60 con 0x7f06bc2008f0 2024-03-11T18:36:47.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.320+0000 7f06aaffd640 1 --2- 172.21.15.90:0/4259698342 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f069c066e90 0x7f069c069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:47.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.320+0000 7f06ba59c640 1 --2- 172.21.15.90:0/4259698342 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f069c066e90 0x7f069c069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:47.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.320+0000 7f06ba59c640 1 --2- 172.21.15.90:0/4259698342 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f069c066e90 0x7f069c069350 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f06ac0ae000 tx=0x7f06ac004850 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:47.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.332+0000 7f4e988d2640 1 -- 172.21.15.90:0/3925709223 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f4e9404eef0 con 0x7f4e940c5b50 2024-03-11T18:36:47.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.336+0000 7f06aaffd640 1 -- 172.21.15.90:0/4259698342 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f06b40551d0 con 0x7f06bc2008f0 2024-03-11T18:36:47.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.336+0000 7f06aaffd640 1 -- 172.21.15.90:0/4259698342 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f06b4051470 con 0x7f06bc2008f0 2024-03-11T18:36:47.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.340+0000 7fec3cee5640 1 -- 172.21.15.90:0/1455271732 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fec3806f5e0 con 0x7fec38157970 2024-03-11T18:36:47.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.344+0000 7fec26ffd640 1 -- 172.21.15.90:0/1455271732 <== mon.1 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fec30019070 con 0x7fec38157970 2024-03-11T18:36:47.352 INFO:teuthology.orchestra.run.smithi090.stdout:64424509461 2024-03-11T18:36:47.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.348+0000 7fec3cee5640 1 -- 172.21.15.90:0/1455271732 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fec18066e90 msgr2=0x7fec18069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:47.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.348+0000 7fec3cee5640 1 --2- 172.21.15.90:0/1455271732 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fec18066e90 0x7fec18069350 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fec28006fd0 tx=0x7fec28008040 comp rx=0 tx=0).stop 2024-03-11T18:36:47.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.348+0000 7fec3cee5640 1 -- 172.21.15.90:0/1455271732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec38157970 msgr2=0x7fec3813eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:47.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.348+0000 7fec3cee5640 1 --2- 172.21.15.90:0/1455271732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec38157970 0x7fec3813eb50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fec3000be20 tx=0x7fec3000bf10 comp rx=0 tx=0).stop 2024-03-11T18:36:47.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.348+0000 7fec3cee5640 1 -- 172.21.15.90:0/1455271732 shutdown_connections 2024-03-11T18:36:47.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.348+0000 7fec3cee5640 1 --2- 172.21.15.90:0/1455271732 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fec18066e90 0x7fec18069350 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:47.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.348+0000 7fec3cee5640 1 --2- 172.21.15.90:0/1455271732 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec38157970 0x7fec3813eb50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:47.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.348+0000 7fec3cee5640 1 --2- 172.21.15.90:0/1455271732 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec38151d50 0x7fec38150de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:47.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.348+0000 7fec3cee5640 1 --2- 172.21.15.90:0/1455271732 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fec3806b860 0x7fec380c4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:47.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.348+0000 7fec3cee5640 1 -- 172.21.15.90:0/1455271732 >> 172.21.15.90:0/1455271732 conn(0x7fec38071aa0 msgr2=0x7fec380c9380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:47.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.348+0000 7fec3cee5640 1 -- 172.21.15.90:0/1455271732 shutdown_connections 2024-03-11T18:36:47.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.348+0000 7fec3cee5640 1 -- 172.21.15.90:0/1455271732 wait complete. 2024-03-11T18:36:47.370 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509463 got 64424509461 for osd.3 2024-03-11T18:36:47.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.408+0000 7f29984ac640 1 -- 172.21.15.90:0/3198063468 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f29900731c0 con 0x7f299006f0a0 2024-03-11T18:36:47.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.432+0000 7fe3b57fa640 1 -- 172.21.15.90:0/2397849799 <== mon.2 v2:172.21.15.92:3301/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fe3ac003cd0 con 0x7fe3b81550c0 2024-03-11T18:36:47.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.432+0000 7fe3b57fa640 1 --2- 172.21.15.90:0/2397849799 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe39c066e90 0x7fe39c069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:47.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.432+0000 7fe3b7fff640 1 --2- 172.21.15.90:0/2397849799 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe39c066e90 0x7fe39c069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:47.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.432+0000 7fe3b7fff640 1 --2- 172.21.15.90:0/2397849799 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe39c066e90 0x7fe39c069350 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe3b815b4b0 tx=0x7fe3a4006bb0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:47.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.436+0000 7fe3b57fa640 1 -- 172.21.15.90:0/2397849799 <== mon.2 v2:172.21.15.92:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe3ac055730 con 0x7fe3b81550c0 2024-03-11T18:36:47.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.460+0000 7fe3b57fa640 1 -- 172.21.15.90:0/2397849799 <== mon.2 v2:172.21.15.92:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe3ac0512d0 con 0x7fe3b81550c0 2024-03-11T18:36:47.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.464+0000 7f2986ffd640 1 -- 172.21.15.90:0/3198063468 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f298c017020 con 0x7f299006f0a0 2024-03-11T18:36:47.469 INFO:teuthology.orchestra.run.smithi090.stdout:64424509462 2024-03-11T18:36:47.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.464+0000 7f29984ac640 1 -- 172.21.15.90:0/3198063468 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f2974066e90 msgr2=0x7f2974069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:47.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.464+0000 7f29984ac640 1 --2- 172.21.15.90:0/3198063468 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f2974066e90 0x7f2974069350 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2990147db0 tx=0x7f2980009110 comp rx=0 tx=0).stop 2024-03-11T18:36:47.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.464+0000 7f29984ac640 1 -- 172.21.15.90:0/3198063468 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f299006f0a0 msgr2=0x7f2990149980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:47.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.464+0000 7f29984ac640 1 --2- 172.21.15.90:0/3198063468 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f299006f0a0 0x7f2990149980 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f298c00c4c0 tx=0x7f298c001e80 comp rx=0 tx=0).stop 2024-03-11T18:36:47.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.464+0000 7f29984ac640 1 -- 172.21.15.90:0/3198063468 shutdown_connections 2024-03-11T18:36:47.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.464+0000 7f29984ac640 1 --2- 172.21.15.90:0/3198063468 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f2974066e90 0x7f2974069350 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:47.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.464+0000 7f29984ac640 1 --2- 172.21.15.90:0/3198063468 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f2990159c80 0x7f29901f5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:47.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.464+0000 7f29984ac640 1 --2- 172.21.15.90:0/3198063468 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2990153f00 0x7f29901554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:47.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.464+0000 7f29984ac640 1 --2- 172.21.15.90:0/3198063468 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f299006f0a0 0x7f2990149980 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:47.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.464+0000 7f29984ac640 1 -- 172.21.15.90:0/3198063468 >> 172.21.15.90:0/3198063468 conn(0x7f299006add0 msgr2=0x7f2990152110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:47.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.464+0000 7f29984ac640 1 -- 172.21.15.90:0/3198063468 shutdown_connections 2024-03-11T18:36:47.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.464+0000 7f29984ac640 1 -- 172.21.15.90:0/3198063468 wait complete. 2024-03-11T18:36:47.493 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509463 got 64424509462 for osd.1 2024-03-11T18:36:47.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.504+0000 7f06c0c28640 1 -- 172.21.15.90:0/4259698342 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f06bc04eef0 con 0x7f06bc2008f0 2024-03-11T18:36:47.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.512+0000 7f06aaffd640 1 -- 172.21.15.90:0/4259698342 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f06b4054b30 con 0x7f06bc2008f0 2024-03-11T18:36:47.521 INFO:teuthology.orchestra.run.smithi090.stdout:68719476759 2024-03-11T18:36:47.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.516+0000 7f06c0c28640 1 -- 172.21.15.90:0/4259698342 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f069c066e90 msgr2=0x7f069c069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:47.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.516+0000 7f06c0c28640 1 --2- 172.21.15.90:0/4259698342 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f069c066e90 0x7f069c069350 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f06ac0ae000 tx=0x7f06ac004850 comp rx=0 tx=0).stop 2024-03-11T18:36:47.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.516+0000 7f06c0c28640 1 -- 172.21.15.90:0/4259698342 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06bc2008f0 msgr2=0x7f06bc1fcb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:47.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.516+0000 7f06c0c28640 1 --2- 172.21.15.90:0/4259698342 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06bc2008f0 0x7f06bc1fcb00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f06bc06ffe0 tx=0x7f06b400ce80 comp rx=0 tx=0).stop 2024-03-11T18:36:47.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.516+0000 7f06c0c28640 1 -- 172.21.15.90:0/4259698342 shutdown_connections 2024-03-11T18:36:47.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.516+0000 7f06c0c28640 1 --2- 172.21.15.90:0/4259698342 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f069c066e90 0x7f069c069350 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:47.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.516+0000 7f06c0c28640 1 --2- 172.21.15.90:0/4259698342 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06bc2008f0 0x7f06bc1fcb00 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:47.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.516+0000 7f06c0c28640 1 --2- 172.21.15.90:0/4259698342 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f06bc144e10 0x7f06bc2003b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:47.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.516+0000 7f06c0c28640 1 --2- 172.21.15.90:0/4259698342 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f06a4003980 0x7f06bc14f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:47.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.516+0000 7f06c0c28640 1 -- 172.21.15.90:0/4259698342 >> 172.21.15.90:0/4259698342 conn(0x7f06bc071aa0 msgr2=0x7f06bc0c8f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:47.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.516+0000 7f06c0c28640 1 -- 172.21.15.90:0/4259698342 shutdown_connections 2024-03-11T18:36:47.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.516+0000 7f06c0c28640 1 -- 172.21.15.90:0/4259698342 wait complete. 2024-03-11T18:36:47.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.520+0000 7fb8e6d7f640 1 -- 172.21.15.90:0/1008985873 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fb8e00c8640 con 0x7fb8e00c65b0 2024-03-11T18:36:47.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.524+0000 7fb8dd7fa640 1 -- 172.21.15.90:0/1008985873 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb8cc01e090 con 0x7fb8e00c65b0 2024-03-11T18:36:47.532 INFO:teuthology.orchestra.run.smithi090.stdout:68719476758 2024-03-11T18:36:47.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.528+0000 7fb8e6d7f640 1 -- 172.21.15.90:0/1008985873 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fb8c4066e90 msgr2=0x7fb8c4069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:47.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.528+0000 7fb8e6d7f640 1 --2- 172.21.15.90:0/1008985873 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fb8c4066e90 0x7fb8c4069350 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb8d40b3000 tx=0x7fb8d4004740 comp rx=0 tx=0).stop 2024-03-11T18:36:47.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.528+0000 7fb8e6d7f640 1 -- 172.21.15.90:0/1008985873 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8e00c65b0 msgr2=0x7fb8e02001c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:47.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.528+0000 7fb8e6d7f640 1 --2- 172.21.15.90:0/1008985873 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8e00c65b0 0x7fb8e02001c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc00b4b0 tx=0x7fb8cc00b970 comp rx=0 tx=0).stop 2024-03-11T18:36:47.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.528+0000 7fb8e6d7f640 1 -- 172.21.15.90:0/1008985873 shutdown_connections 2024-03-11T18:36:47.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.528+0000 7fb8e6d7f640 1 --2- 172.21.15.90:0/1008985873 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fb8c4066e90 0x7fb8c4069350 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:47.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.528+0000 7fb8e6d7f640 1 --2- 172.21.15.90:0/1008985873 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb8e0150a90 0x7fb8e01594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:47.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.528+0000 7fb8e6d7f640 1 --2- 172.21.15.90:0/1008985873 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8e00c65b0 0x7fb8e02001c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:47.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.528+0000 7fb8e6d7f640 1 --2- 172.21.15.90:0/1008985873 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb8e00c3d30 0x7fb8e0155580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:47.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.528+0000 7fb8e6d7f640 1 -- 172.21.15.90:0/1008985873 >> 172.21.15.90:0/1008985873 conn(0x7fb8e0071aa0 msgr2=0x7fb8e01607b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:47.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.528+0000 7fb8e6d7f640 1 -- 172.21.15.90:0/1008985873 shutdown_connections 2024-03-11T18:36:47.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.528+0000 7fb8e6d7f640 1 -- 172.21.15.90:0/1008985873 wait complete. 2024-03-11T18:36:47.540 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476760 got 68719476759 for osd.7 2024-03-11T18:36:47.551 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476759 got 68719476758 for osd.4 2024-03-11T18:36:47.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.620+0000 7f4e8affd640 1 -- 172.21.15.90:0/3925709223 <== mon.2 v2:172.21.15.92:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4e7c0b4020 con 0x7f4e940c5b50 2024-03-11T18:36:47.629 INFO:teuthology.orchestra.run.smithi090.stdout:68719476757 2024-03-11T18:36:47.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.624+0000 7fe3bef90640 1 -- 172.21.15.90:0/2397849799 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fe3b804eef0 con 0x7fe3b81550c0 2024-03-11T18:36:47.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.624+0000 7f4e988d2640 1 -- 172.21.15.90:0/3925709223 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f4e74066e40 msgr2=0x7f4e74069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:47.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.624+0000 7f4e988d2640 1 --2- 172.21.15.90:0/3925709223 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f4e74066e40 0x7f4e74069300 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4e84009650 tx=0x7f4e84006bb0 comp rx=0 tx=0).stop 2024-03-11T18:36:47.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.624+0000 7f4e988d2640 1 -- 172.21.15.90:0/3925709223 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4e940c5b50 msgr2=0x7f4e94144bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:47.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.624+0000 7f4e988d2640 1 --2- 172.21.15.90:0/3925709223 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4e940c5b50 0x7f4e94144bb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c01d160 tx=0x7f4e7c01b850 comp rx=0 tx=0).stop 2024-03-11T18:36:47.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.624+0000 7f4e988d2640 1 -- 172.21.15.90:0/3925709223 shutdown_connections 2024-03-11T18:36:47.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.624+0000 7f4e988d2640 1 --2- 172.21.15.90:0/3925709223 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f4e74066e40 0x7f4e74069300 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:47.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.624+0000 7f4e988d2640 1 --2- 172.21.15.90:0/3925709223 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4e94151000 0x7f4e9414b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:47.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.624+0000 7f4e988d2640 1 --2- 172.21.15.90:0/3925709223 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4e940c5b50 0x7f4e94144bb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:47.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.624+0000 7f4e988d2640 1 --2- 172.21.15.90:0/3925709223 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e94066d60 0x7f4e94145f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:47.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.624+0000 7f4e988d2640 1 -- 172.21.15.90:0/3925709223 >> 172.21.15.90:0/3925709223 conn(0x7f4e940cb880 msgr2=0x7f4e94146dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:47.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.624+0000 7f4e988d2640 1 -- 172.21.15.90:0/3925709223 shutdown_connections 2024-03-11T18:36:47.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.624+0000 7f4e988d2640 1 -- 172.21.15.90:0/3925709223 wait complete. 2024-03-11T18:36:47.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.632+0000 7fe3b57fa640 1 -- 172.21.15.90:0/2397849799 <== mon.2 v2:172.21.15.92:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe3ac017070 con 0x7fe3b81550c0 2024-03-11T18:36:47.637 INFO:teuthology.orchestra.run.smithi090.stdout:68719476759 2024-03-11T18:36:47.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.632+0000 7fe3bef90640 1 -- 172.21.15.90:0/2397849799 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe39c066e90 msgr2=0x7fe39c069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:47.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.632+0000 7fe3bef90640 1 --2- 172.21.15.90:0/2397849799 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe39c066e90 0x7fe39c069350 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe3b815b4b0 tx=0x7fe3a4006bb0 comp rx=0 tx=0).stop 2024-03-11T18:36:47.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.632+0000 7fe3bef90640 1 -- 172.21.15.90:0/2397849799 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe3b81550c0 msgr2=0x7fe3b80b0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:47.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.632+0000 7fe3bef90640 1 --2- 172.21.15.90:0/2397849799 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe3b81550c0 0x7fe3b80b0c10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac00c490 tx=0x7fe3ac001e80 comp rx=0 tx=0).stop 2024-03-11T18:36:47.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.632+0000 7fe3bef90640 1 -- 172.21.15.90:0/2397849799 shutdown_connections 2024-03-11T18:36:47.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.632+0000 7fe3bef90640 1 --2- 172.21.15.90:0/2397849799 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe39c066e90 0x7fe39c069350 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:47.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.632+0000 7fe3bef90640 1 --2- 172.21.15.90:0/2397849799 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe3b80b28f0 0x7fe3b81562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:47.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.632+0000 7fe3bef90640 1 --2- 172.21.15.90:0/2397849799 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe3b81554a0 0x7fe3b806cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:47.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.632+0000 7fe3bef90640 1 --2- 172.21.15.90:0/2397849799 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe3b81550c0 0x7fe3b80b0c10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:47.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.632+0000 7fe3bef90640 1 -- 172.21.15.90:0/2397849799 >> 172.21.15.90:0/2397849799 conn(0x7fe3b8070e80 msgr2=0x7fe3b80c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:47.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.632+0000 7fe3bef90640 1 -- 172.21.15.90:0/2397849799 shutdown_connections 2024-03-11T18:36:47.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:47.632+0000 7fe3bef90640 1 -- 172.21.15.90:0/2397849799 wait complete. 2024-03-11T18:36:47.649 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476758 got 68719476757 for osd.5 2024-03-11T18:36:47.658 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476760 got 68719476759 for osd.6 2024-03-11T18:36:48.153 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-11T18:36:48.218 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-11T18:36:48.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.268+0000 7f5fecfd0640 1 Processor -- start 2024-03-11T18:36:48.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.268+0000 7f5fecfd0640 1 -- start start 2024-03-11T18:36:48.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.272+0000 7f5fecfd0640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f5fe81540c0 0x7f5fe80c6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:48.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.272+0000 7f5fecfd0640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f5fe806f400 0x7f5fe815d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:48.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.272+0000 7f5fecfd0640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5fe80cc280 0x7f5fe80cc710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:48.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.272+0000 7f5fecfd0640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fe80c7d50 con 0x7f5fe806f400 2024-03-11T18:36:48.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.272+0000 7f5fecfd0640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fe80703f0 con 0x7f5fe80cc280 2024-03-11T18:36:48.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.272+0000 7f5fecfd0640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe806b480 con 0x7f5fe81540c0 2024-03-11T18:36:48.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.272+0000 7f5fe659c640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f5fe81540c0 0x7f5fe80c6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:48.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.272+0000 7f5fe6d9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5fe80cc280 0x7f5fe80cc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:48.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.276+0000 7f5fe6d9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f5fe80cc280 0x7f5fe80cc710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.90:60476/0 (socket says 172.21.15.90:60476) 2024-03-11T18:36:48.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.276+0000 7f5fe6d9d640 1 -- 172.21.15.90:0/504840016 learned_addr learned my addr 172.21.15.90:0/504840016 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:48.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.284+0000 7f5fe6d9d640 1 -- 172.21.15.90:0/504840016 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f5fe81540c0 msgr2=0x7f5fe80c6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:48.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.284+0000 7f5fe6d9d640 1 --2- 172.21.15.90:0/504840016 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f5fe81540c0 0x7f5fe80c6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:48.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.284+0000 7f5fe6d9d640 1 -- 172.21.15.90:0/504840016 >> v2:172.21.15.90:3300/0 conn(0x7f5fe806f400 msgr2=0x7f5fe815d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:48.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.284+0000 7f5fe6d9d640 1 --2- 172.21.15.90:0/504840016 >> v2:172.21.15.90:3300/0 conn(0x7f5fe806f400 0x7f5fe815d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:48.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.284+0000 7f5fe6d9d640 1 -- 172.21.15.90:0/504840016 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe806b760 con 0x7f5fe80cc280 2024-03-11T18:36:48.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.292+0000 7f5fe6d9d640 1 --2- 172.21.15.90:0/504840016 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fe80cc280 0x7f5fe80cc710 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8073340 tx=0x7f5fe001f1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f8ff05327974933 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:48.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.344+0000 7f0a8f6bb640 1 Processor -- start 2024-03-11T18:36:48.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.344+0000 7f0a8f6bb640 1 -- start start 2024-03-11T18:36:48.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.344+0000 7f0a8f6bb640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0a88066d60 0x7f0a881623b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:48.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.344+0000 7f0a8f6bb640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a880c7980 con 0x7f0a88151000 2024-03-11T18:36:48.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.344+0000 7f0a8f6bb640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a880c2ad0 con 0x7f0a880c5b50 2024-03-11T18:36:48.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.344+0000 7f0a8f6bb640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a880cc820 con 0x7f0a88066d60 2024-03-11T18:36:48.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.344+0000 7f0a8d457640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f0a88151000 0x7f0a880c2d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:42952/0 (socket says 172.21.15.90:42952) 2024-03-11T18:36:48.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.344+0000 7f0a8d457640 1 -- 172.21.15.90:0/1478647866 learned_addr learned my addr 172.21.15.90:0/1478647866 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T18:36:48.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.348+0000 7f0a7ffff640 1 -- 172.21.15.90:0/1478647866 <== mon.0 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4010110867 0 0) 0x7f0a880c7980 con 0x7f0a88151000 2024-03-11T18:36:48.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.348+0000 7f0a7ffff640 1 -- 172.21.15.90:0/1478647866 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a74003530 con 0x7f0a88151000 2024-03-11T18:36:48.371 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-11T18:36:48.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.368+0000 7f0a7ffff640 1 -- 172.21.15.90:0/1478647866 <== mon.0 v1:172.21.15.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2926429836 0 0) 0x7f0a74003530 con 0x7f0a88151000 2024-03-11T18:36:48.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.368+0000 7f0a7ffff640 1 -- 172.21.15.90:0/1478647866 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0a88066d60 msgr2=0x7f0a881623b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:48.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.368+0000 7f0a7ffff640 1 --2- 172.21.15.90:0/1478647866 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0a88066d60 0x7f0a881623b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:48.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.368+0000 7f0a7ffff640 1 -- 172.21.15.90:0/1478647866 >> v1:172.21.15.92:6789/0 conn(0x7f0a880c5b50 legacy=0x7f0a880c4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:48.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.368+0000 7f0a7ffff640 1 -- 172.21.15.90:0/1478647866 --> v1:172.21.15.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a880ca9c0 con 0x7f0a88151000 2024-03-11T18:36:48.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.368+0000 7f0a7ffff640 1 -- 172.21.15.90:0/1478647866 <== mon.0 v1:172.21.15.90:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 691735435 0 0) 0x7f0a80001a00 con 0x7f0a88151000 2024-03-11T18:36:48.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.368+0000 7f0a7ffff640 1 -- 172.21.15.90:0/1478647866 >> v1:172.21.15.90:6789/0 conn(0x7f0a88151000 legacy=0x7f0a880c2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:48.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.368+0000 7f0a7ffff640 1 --2- 172.21.15.90:0/1478647866 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0a88066d60 0x7f0a740039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:48.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.368+0000 7f0a7ffff640 1 --2- 172.21.15.90:0/1478647866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a740040f0 0x7f0a740047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:48.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.368+0000 7f0a7ffff640 1 --2- 172.21.15.90:0/1478647866 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a74004e60 0x7f0a7400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:48.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.368+0000 7f0a7ffff640 1 -- 172.21.15.90:0/1478647866 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a880cc820 con 0x7f0a74004e60 2024-03-11T18:36:48.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.368+0000 7f0a7ffff640 1 -- 172.21.15.90:0/1478647866 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a74005680 con 0x7f0a740040f0 2024-03-11T18:36:48.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.368+0000 7f0a7ffff640 1 -- 172.21.15.90:0/1478647866 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7400bf50 con 0x7f0a88066d60 2024-03-11T18:36:48.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.372+0000 7f0a8d457640 1 --2- 172.21.15.90:0/1478647866 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0a88066d60 0x7f0a740039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:48.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.380+0000 7f0a8dc58640 1 --2- 172.21.15.90:0/1478647866 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a74004e60 0x7f0a7400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:48.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.384+0000 7f0a8dc58640 1 -- 172.21.15.90:0/1478647866 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0a88066d60 msgr2=0x7f0a740039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:48.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.384+0000 7f0a8dc58640 1 --2- 172.21.15.90:0/1478647866 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0a88066d60 0x7f0a740039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:48.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.384+0000 7f0a8dc58640 1 -- 172.21.15.90:0/1478647866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a740040f0 msgr2=0x7f0a740047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:48.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.384+0000 7f0a8dc58640 1 --2- 172.21.15.90:0/1478647866 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a740040f0 0x7f0a740047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:48.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.384+0000 7f0a8dc58640 1 -- 172.21.15.90:0/1478647866 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a74003530 con 0x7f0a74004e60 2024-03-11T18:36:48.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.384+0000 7f0a8dc58640 1 --2- 172.21.15.90:0/1478647866 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a74004e60 0x7f0a7400b990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0a84009890 tx=0x7f0a8401c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bb45f0d2ec97b9a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:48.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.392+0000 7f0a7ffff640 1 -- 172.21.15.90:0/1478647866 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a840ab600 con 0x7f0a74004e60 2024-03-11T18:36:48.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.392+0000 7f0a7ffff640 1 -- 172.21.15.90:0/1478647866 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a840a7050 con 0x7f0a74004e60 2024-03-11T18:36:48.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.392+0000 7f0a7ffff640 1 -- 172.21.15.90:0/1478647866 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a840aba00 con 0x7f0a74004e60 2024-03-11T18:36:48.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.392+0000 7f0a8f6bb640 1 -- 172.21.15.90:0/1478647866 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a74004e60 msgr2=0x7f0a7400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:48.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.392+0000 7f0a8f6bb640 1 --2- 172.21.15.90:0/1478647866 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a74004e60 0x7f0a7400b990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0a84009890 tx=0x7f0a8401c380 comp rx=0 tx=0).stop 2024-03-11T18:36:48.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.392+0000 7f0a8dc58640 1 -- 172.21.15.90:0/1478647866 reap_dead start 2024-03-11T18:36:48.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.392+0000 7f0a8f6bb640 1 -- 172.21.15.90:0/1478647866 shutdown_connections 2024-03-11T18:36:48.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.392+0000 7f0a8f6bb640 1 -- 172.21.15.90:0/1478647866 >> 172.21.15.90:0/1478647866 conn(0x7f0a880cb880 msgr2=0x7f0a880cbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:48.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.392+0000 7f0a8f6bb640 1 -- 172.21.15.90:0/1478647866 shutdown_connections 2024-03-11T18:36:48.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.392+0000 7f0a8f6bb640 1 -- 172.21.15.90:0/1478647866 wait complete. 2024-03-11T18:36:48.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.392+0000 7f0a8f6bb640 1 Processor -- start 2024-03-11T18:36:48.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.392+0000 7f0a8f6bb640 1 -- start start 2024-03-11T18:36:48.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.392+0000 7f0a8f6bb640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a88072900 0x7f0a881f36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:48.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.392+0000 7f0a8f6bb640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a88072ce0 0x7f0a8814b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:48.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.392+0000 7f0a8f6bb640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0a88145250 0x7f0a881456d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:48.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.392+0000 7f0a8f6bb640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a881462d0 con 0x7f0a88072ce0 2024-03-11T18:36:48.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.392+0000 7f0a8f6bb640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8814d490 con 0x7f0a88072900 2024-03-11T18:36:48.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.392+0000 7f0a8f6bb640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8814d730 con 0x7f0a88145250 2024-03-11T18:36:48.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.396+0000 7f5fe5d9b640 1 -- 172.21.15.90:0/504840016 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe00af400 con 0x7f5fe80cc280 2024-03-11T18:36:48.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.396+0000 7f5fe5d9b640 1 -- 172.21.15.90:0/504840016 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe00ab050 con 0x7f5fe80cc280 2024-03-11T18:36:48.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.396+0000 7f5fe5d9b640 1 -- 172.21.15.90:0/504840016 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe00af780 con 0x7f5fe80cc280 2024-03-11T18:36:48.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.400+0000 7f5fecfd0640 1 -- 172.21.15.90:0/504840016 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fe80cc280 msgr2=0x7f5fe80cc710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:48.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.400+0000 7f5fecfd0640 1 --2- 172.21.15.90:0/504840016 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fe80cc280 0x7f5fe80cc710 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8073340 tx=0x7f5fe001f1e0 comp rx=0 tx=0).stop 2024-03-11T18:36:48.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.400+0000 7f5fecfd0640 1 -- 172.21.15.90:0/504840016 shutdown_connections 2024-03-11T18:36:48.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.400+0000 7f5fecfd0640 1 --2- 172.21.15.90:0/504840016 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fe80cc280 0x7f5fe80cc710 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:48.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.400+0000 7f5fecfd0640 1 --2- 172.21.15.90:0/504840016 >> v2:172.21.15.90:3300/0 conn(0x7f5fe806f400 0x7f5fe815d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:48.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.400+0000 7f5fecfd0640 1 --2- 172.21.15.90:0/504840016 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f5fe81540c0 0x7f5fe80c6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:48.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.400+0000 7f5fecfd0640 1 -- 172.21.15.90:0/504840016 >> 172.21.15.90:0/504840016 conn(0x7f5fe8071aa0 msgr2=0x7f5fe8071eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:48.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.400+0000 7f5fecfd0640 1 -- 172.21.15.90:0/504840016 shutdown_connections 2024-03-11T18:36:48.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.400+0000 7f5fecfd0640 1 -- 172.21.15.90:0/504840016 wait complete. 2024-03-11T18:36:48.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.400+0000 7f5fecfd0640 1 Processor -- start 2024-03-11T18:36:48.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.400+0000 7f5fecfd0640 1 -- start start 2024-03-11T18:36:48.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.400+0000 7f5fecfd0640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f5fe806f400 0x7f5fe81f01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:48.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.400+0000 7f5fecfd0640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fe80cc280 0x7f5fe814c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:48.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.400+0000 7f5fecfd0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fe81540c0 0x7f5fe81f2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:48.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.400+0000 7f5fecfd0640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe814be40 con 0x7f5fe80cc280 2024-03-11T18:36:48.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.400+0000 7f5fecfd0640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe81efd50 con 0x7f5fe81540c0 2024-03-11T18:36:48.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.400+0000 7f5fecfd0640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe81ef740 con 0x7f5fe806f400 2024-03-11T18:36:48.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.400+0000 7f0a8cc56640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a88072ce0 0x7f0a8814b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:48.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.404+0000 7f0a8cc56640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a88072ce0 0x7f0a8814b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48268/0 (socket says 172.21.15.90:48268) 2024-03-11T18:36:48.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.404+0000 7f0a8cc56640 1 -- 172.21.15.90:0/1959989133 learned_addr learned my addr 172.21.15.90:0/1959989133 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:48.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.404+0000 7f5fdffff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fe80cc280 0x7f5fe814c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:48.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.404+0000 7f5fdffff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fe80cc280 0x7f5fe814c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48282/0 (socket says 172.21.15.90:48282) 2024-03-11T18:36:48.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.404+0000 7f5fdffff640 1 -- 172.21.15.90:0/4004048064 learned_addr learned my addr 172.21.15.90:0/4004048064 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:48.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.404+0000 7f5fe659c640 1 --2- 172.21.15.90:0/4004048064 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f5fe806f400 0x7f5fe81f01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:48.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.408+0000 7f0a8dc58640 1 --2- 172.21.15.90:0/1959989133 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0a88145250 0x7f0a881456d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:48.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.408+0000 7f0a8cc56640 1 -- 172.21.15.90:0/1959989133 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0a88145250 msgr2=0x7f0a881456d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:48.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.408+0000 7f0a8cc56640 1 --2- 172.21.15.90:0/1959989133 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0a88145250 0x7f0a881456d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:48.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.408+0000 7f0a8cc56640 1 -- 172.21.15.90:0/1959989133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a88072900 msgr2=0x7f0a881f36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:48.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.408+0000 7f0a8cc56640 1 --2- 172.21.15.90:0/1959989133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a88072900 0x7f0a881f36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:48.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.408+0000 7f0a8cc56640 1 -- 172.21.15.90:0/1959989133 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a881f2c00 con 0x7f0a88072ce0 2024-03-11T18:36:48.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.408+0000 7f5fe6d9d640 1 --2- 172.21.15.90:0/4004048064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fe81540c0 0x7f5fe81f2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:48.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.412+0000 7f5fe659c640 1 -- 172.21.15.90:0/4004048064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fe81540c0 msgr2=0x7f5fe81f2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:48.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.412+0000 7f5fe659c640 1 --2- 172.21.15.90:0/4004048064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fe81540c0 0x7f5fe81f2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:48.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.412+0000 7f5fe659c640 1 -- 172.21.15.90:0/4004048064 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fe80cc280 msgr2=0x7f5fe814c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:48.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.412+0000 7f5fe659c640 1 --2- 172.21.15.90:0/4004048064 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fe80cc280 0x7f5fe814c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:48.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.412+0000 7f5fe659c640 1 -- 172.21.15.90:0/4004048064 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe0009b30 con 0x7f5fe806f400 2024-03-11T18:36:48.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.412+0000 7f0a8cc56640 1 --2- 172.21.15.90:0/1959989133 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a88072ce0 0x7f0a8814b2b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0a7800c410 tx=0x7f0a7800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:48.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.416+0000 7f5fe659c640 1 --2- 172.21.15.90:0/4004048064 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f5fe806f400 0x7f5fe81f01e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5fd800c820 tx=0x7f5fd800cce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:48.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.424+0000 7f5fdeffd640 1 -- 172.21.15.90:0/4004048064 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd8004020 con 0x7f5fe806f400 2024-03-11T18:36:48.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.424+0000 7f5fdeffd640 1 -- 172.21.15.90:0/4004048064 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd8004180 con 0x7f5fe806f400 2024-03-11T18:36:48.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.424+0000 7f5fdeffd640 1 -- 172.21.15.90:0/4004048064 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd8004510 con 0x7f5fe806f400 2024-03-11T18:36:48.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.424+0000 7f0a7dffb640 1 -- 172.21.15.90:0/1959989133 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a78003600 con 0x7f0a88072ce0 2024-03-11T18:36:48.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.424+0000 7f0a7dffb640 1 -- 172.21.15.90:0/1959989133 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a78003760 con 0x7f0a88072ce0 2024-03-11T18:36:48.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.424+0000 7f0a8f6bb640 1 -- 172.21.15.90:0/1959989133 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a881f2ee0 con 0x7f0a88072ce0 2024-03-11T18:36:48.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.424+0000 7f5fecfd0640 1 -- 172.21.15.90:0/4004048064 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fe81f3450 con 0x7f5fe806f400 2024-03-11T18:36:48.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.424+0000 7f5fecfd0640 1 -- 172.21.15.90:0/4004048064 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fe81ef9e0 con 0x7f5fe806f400 2024-03-11T18:36:48.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.424+0000 7f0a8f6bb640 1 -- 172.21.15.90:0/1959989133 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a8815efd0 con 0x7f0a88072ce0 2024-03-11T18:36:48.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.424+0000 7f0a7dffb640 1 -- 172.21.15.90:0/1959989133 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a78003b50 con 0x7f0a88072ce0 2024-03-11T18:36:48.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.424+0000 7f5fecfd0640 1 -- 172.21.15.90:0/4004048064 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fe804eef0 con 0x7f5fe806f400 2024-03-11T18:36:48.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.424+0000 7f0a8f6bb640 1 -- 172.21.15.90:0/1959989133 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a680052a0 con 0x7f0a88072ce0 2024-03-11T18:36:48.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.480+0000 7f1757321640 1 Processor -- start 2024-03-11T18:36:48.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.480+0000 7f1757321640 1 -- start start 2024-03-11T18:36:48.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.480+0000 7f1757321640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f175006b860 0x7f17500c3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:48.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.480+0000 7f1757321640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f17500c95f0 0x7f17500c5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:48.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.480+0000 7f1757321640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17500c7430 con 0x7f1750151d50 2024-03-11T18:36:48.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.480+0000 7f1757321640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1750152130 con 0x7f175006b860 2024-03-11T18:36:48.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.480+0000 7f1757321640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17500c7a40 con 0x7f17500c95f0 2024-03-11T18:36:48.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.480+0000 7f17548bc640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f175006b860 0x7f17500c3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:48.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.480+0000 7f17548bc640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f175006b860 0x7f17500c3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.90:60520/0 (socket says 172.21.15.90:60520) 2024-03-11T18:36:48.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.480+0000 7f17548bc640 1 -- 172.21.15.90:0/329583357 learned_addr learned my addr 172.21.15.90:0/329583357 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:48.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.484+0000 7f17548bc640 1 -- 172.21.15.90:0/329583357 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f17500c95f0 msgr2=0x7f17500c5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:48.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.484+0000 7f17548bc640 1 --2- 172.21.15.90:0/329583357 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f17500c95f0 0x7f17500c5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:48.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.484+0000 7f17548bc640 1 -- 172.21.15.90:0/329583357 >> v1:172.21.15.90:6789/0 conn(0x7f1750151d50 legacy=0x7f1750157970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:48.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.484+0000 7f17548bc640 1 -- 172.21.15.90:0/329583357 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17500c9a40 con 0x7f175006b860 2024-03-11T18:36:48.494 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-11T18:36:48.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.492+0000 7f17548bc640 1 --2- 172.21.15.90:0/329583357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f175006b860 0x7f17500c3130 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1744009870 tx=0x7f174401ed10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c08fc9ab3dec167 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:48.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.496+0000 7f1743fff640 1 -- 172.21.15.90:0/329583357 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17440a89d0 con 0x7f175006b860 2024-03-11T18:36:48.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.496+0000 7f1743fff640 1 -- 172.21.15.90:0/329583357 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17440a8b30 con 0x7f175006b860 2024-03-11T18:36:48.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.496+0000 7f1743fff640 1 -- 172.21.15.90:0/329583357 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17440b0440 con 0x7f175006b860 2024-03-11T18:36:48.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.500+0000 7f1757321640 1 -- 172.21.15.90:0/329583357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f175006b860 msgr2=0x7f17500c3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:48.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.500+0000 7f1757321640 1 --2- 172.21.15.90:0/329583357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f175006b860 0x7f17500c3130 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1744009870 tx=0x7f174401ed10 comp rx=0 tx=0).stop 2024-03-11T18:36:48.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.500+0000 7f1757321640 1 -- 172.21.15.90:0/329583357 shutdown_connections 2024-03-11T18:36:48.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.500+0000 7f1757321640 1 --2- 172.21.15.90:0/329583357 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f17500c95f0 0x7f17500c5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:48.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.500+0000 7f1757321640 1 --2- 172.21.15.90:0/329583357 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f175006b860 0x7f17500c3130 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:48.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.500+0000 7f1757321640 1 -- 172.21.15.90:0/329583357 >> 172.21.15.90:0/329583357 conn(0x7f1750071aa0 msgr2=0x7f1750071eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:48.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.500+0000 7f1757321640 1 -- 172.21.15.90:0/329583357 shutdown_connections 2024-03-11T18:36:48.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.500+0000 7f1757321640 1 -- 172.21.15.90:0/329583357 wait complete. 2024-03-11T18:36:48.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.500+0000 7f1757321640 1 Processor -- start 2024-03-11T18:36:48.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.500+0000 7f1757321640 1 -- start start 2024-03-11T18:36:48.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.500+0000 7f1757321640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f175006b860 0x7f1750152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:48.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.500+0000 7f1757321640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17500c95f0 0x7f17501548c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:48.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.500+0000 7f1757321640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1750151d50 0x7f1750145ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:48.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.500+0000 7f1757321640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17501494c0 con 0x7f17500c95f0 2024-03-11T18:36:48.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.500+0000 7f1757321640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17501ed660 con 0x7f175006b860 2024-03-11T18:36:48.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.500+0000 7f1757321640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17501538f0 con 0x7f1750151d50 2024-03-11T18:36:48.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.504+0000 7f17550bd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f175006b860 0x7f1750152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:48.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.504+0000 7f17550bd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f175006b860 0x7f1750152520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.90:60528/0 (socket says 172.21.15.90:60528) 2024-03-11T18:36:48.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.504+0000 7f17550bd640 1 -- 172.21.15.90:0/3026481570 learned_addr learned my addr 172.21.15.90:0/3026481570 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:48.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.508+0000 7f17550bd640 1 -- 172.21.15.90:0/3026481570 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1750151d50 msgr2=0x7f1750145ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:48.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.508+0000 7f17550bd640 1 --2- 172.21.15.90:0/3026481570 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1750151d50 0x7f1750145ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:48.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.508+0000 7f17550bd640 1 -- 172.21.15.90:0/3026481570 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17500c95f0 msgr2=0x7f17501548c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:48.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.508+0000 7f17550bd640 1 --2- 172.21.15.90:0/3026481570 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17500c95f0 0x7f17501548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:48.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.508+0000 7f17550bd640 1 -- 172.21.15.90:0/3026481570 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17440094d0 con 0x7f175006b860 2024-03-11T18:36:48.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.512+0000 7f17550bd640 1 --2- 172.21.15.90:0/3026481570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f175006b860 0x7f1750152520 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f174c00b3e0 tx=0x7f174c00b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:48.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.520+0000 7f1741ffb640 1 -- 172.21.15.90:0/3026481570 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174c011830 con 0x7f175006b860 2024-03-11T18:36:48.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.520+0000 7f1741ffb640 1 -- 172.21.15.90:0/3026481570 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f174c0119b0 con 0x7f175006b860 2024-03-11T18:36:48.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.520+0000 7f1741ffb640 1 -- 172.21.15.90:0/3026481570 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174c011d60 con 0x7f175006b860 2024-03-11T18:36:48.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.520+0000 7f1757321640 1 -- 172.21.15.90:0/3026481570 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f175014a340 con 0x7f175006b860 2024-03-11T18:36:48.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.520+0000 7f1757321640 1 -- 172.21.15.90:0/3026481570 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1750153b00 con 0x7f175006b860 2024-03-11T18:36:48.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.520+0000 7f1757321640 1 -- 172.21.15.90:0/3026481570 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17500c8bf0 con 0x7f175006b860 2024-03-11T18:36:48.541 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-11T18:36:48.553 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-11T18:36:48.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.596+0000 7fae80aac640 1 Processor -- start 2024-03-11T18:36:48.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.596+0000 7fae80aac640 1 -- start start 2024-03-11T18:36:48.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.596+0000 7fae80aac640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fae7c151000 0x7fae7c0c2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:48.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.596+0000 7fae80aac640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae7c0c7980 con 0x7fae7c066d60 2024-03-11T18:36:48.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.596+0000 7fae80aac640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae7c0c2ad0 con 0x7fae7c0c5b50 2024-03-11T18:36:48.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.596+0000 7fae80aac640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae7c0cc820 con 0x7fae7c151000 2024-03-11T18:36:48.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.644+0000 7f02bb5e1640 1 Processor -- start 2024-03-11T18:36:48.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.644+0000 7f02bb5e1640 1 -- start start 2024-03-11T18:36:48.650 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-11T18:36:48.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.644+0000 7f02bb5e1640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f02b41540c0 0x7f02b40c6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:48.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.644+0000 7f02bb5e1640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f02b406f400 0x7f02b415d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:48.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.644+0000 7f02bb5e1640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02b40c7d50 con 0x7f02b40cc280 2024-03-11T18:36:48.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.644+0000 7f02bb5e1640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02b40703f0 con 0x7f02b406f400 2024-03-11T18:36:48.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.644+0000 7f02bb5e1640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02b406b480 con 0x7f02b41540c0 2024-03-11T18:36:48.659 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-11T18:36:48.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.688+0000 7fc4bd62a640 1 Processor -- start 2024-03-11T18:36:48.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.688+0000 7fc4bd62a640 1 -- start start 2024-03-11T18:36:48.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.688+0000 7fc4bd62a640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc4b8153f10 0x7fc4b815a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:48.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.688+0000 7fc4bd62a640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fc4b80afb60 0x7fc4b806c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:48.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.688+0000 7fc4bd62a640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4b815abb0 con 0x7fc4b80c8e20 2024-03-11T18:36:48.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.688+0000 7fc4bd62a640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4b806c2d0 con 0x7fc4b80afb60 2024-03-11T18:36:48.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.688+0000 7fc4bd62a640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b80c81e0 con 0x7fc4b8153f10 2024-03-11T18:36:48.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.720+0000 7f5fdeffd640 1 -- 172.21.15.90:0/4004048064 <== mon.2 v2:172.21.15.92:3301/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f5fd8007830 con 0x7f5fe806f400 2024-03-11T18:36:48.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.720+0000 7f5fdeffd640 1 --2- 172.21.15.90:0/4004048064 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f5fc8066ec0 0x7f5fc8069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:48.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.720+0000 7f5fdffff640 1 --2- 172.21.15.90:0/4004048064 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f5fc8066ec0 0x7f5fc8069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:48.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.720+0000 7f5fdffff640 1 --2- 172.21.15.90:0/4004048064 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f5fc8066ec0 0x7f5fc8069380 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5fe814db90 tx=0x7f5fd0009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:48.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.724+0000 7f5fdeffd640 1 -- 172.21.15.90:0/4004048064 <== mon.2 v2:172.21.15.92:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5fd80556e0 con 0x7f5fe806f400 2024-03-11T18:36:48.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.744+0000 7f5fdeffd640 1 -- 172.21.15.90:0/4004048064 <== mon.2 v2:172.21.15.92:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5fd8051980 con 0x7f5fe806f400 2024-03-11T18:36:48.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.752+0000 7f0a7dffb640 1 -- 172.21.15.90:0/1959989133 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f0a78003cb0 con 0x7f0a88072ce0 2024-03-11T18:36:48.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.752+0000 7f0a7dffb640 1 --2- 172.21.15.90:0/1959989133 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0a6c066e70 0x7f0a6c069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:48.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.752+0000 7f0a8d457640 1 --2- 172.21.15.90:0/1959989133 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0a6c066e70 0x7f0a6c069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:48.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.752+0000 7f0a8d457640 1 --2- 172.21.15.90:0/1959989133 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0a6c066e70 0x7f0a6c069330 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0a80007200 tx=0x7f0a80001e70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:48.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.756+0000 7f93cb913640 1 Processor -- start 2024-03-11T18:36:48.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.756+0000 7f93cb913640 1 -- start start 2024-03-11T18:36:48.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.756+0000 7f93cb913640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f93c4151000 0x7f93c40c2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:48.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.756+0000 7f93cb913640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f93c4066d60 0x7f93c41613a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:48.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.756+0000 7f93cb913640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93c40c7980 con 0x7f93c40c5b50 2024-03-11T18:36:48.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.756+0000 7f93cb913640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93c40cc820 con 0x7f93c4151000 2024-03-11T18:36:48.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.756+0000 7f93cb913640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93c40caca0 con 0x7f93c4066d60 2024-03-11T18:36:48.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.776+0000 7f0a7dffb640 1 -- 172.21.15.90:0/1959989133 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0a780556a0 con 0x7f0a88072ce0 2024-03-11T18:36:48.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.776+0000 7f0a7dffb640 1 -- 172.21.15.90:0/1959989133 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0a78051240 con 0x7f0a88072ce0 2024-03-11T18:36:48.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.776+0000 7f02b8b7c640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f02b406f400 0x7f02b415d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:48.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.780+0000 7f02b8b7c640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f02b406f400 0x7f02b415d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48298/0 (socket says 172.21.15.90:48298) 2024-03-11T18:36:48.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.780+0000 7f02b8b7c640 1 -- 172.21.15.90:0/3444711021 learned_addr learned my addr 172.21.15.90:0/3444711021 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:48.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.780+0000 7fc4b77fe640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fc4b80afb60 0x7fc4b806c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:48.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.780+0000 7f02b8b7c640 1 -- 172.21.15.90:0/3444711021 >> v2:172.21.15.92:3300/0 conn(0x7f02b41540c0 msgr2=0x7f02b40c6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:48.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.780+0000 7f02b8b7c640 1 --2- 172.21.15.90:0/3444711021 >> v2:172.21.15.92:3300/0 conn(0x7f02b41540c0 0x7f02b40c6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:48.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.780+0000 7f02b8b7c640 1 -- 172.21.15.90:0/3444711021 >> v1:172.21.15.90:6789/0 conn(0x7f02b40cc280 legacy=0x7f02b40c8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:48.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.780+0000 7f02b8b7c640 1 -- 172.21.15.90:0/3444711021 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b406b760 con 0x7f02b406f400 2024-03-11T18:36:48.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.784+0000 7fc4b77fe640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fc4b80afb60 0x7fc4b806c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48302/0 (socket says 172.21.15.90:48302) 2024-03-11T18:36:48.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.784+0000 7fc4b77fe640 1 -- 172.21.15.90:0/4044422969 learned_addr learned my addr 172.21.15.90:0/4044422969 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:48.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.784+0000 7f93c96af640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f93c4151000 0x7f93c40c2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:48.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.788+0000 7fc4b77fe640 1 -- 172.21.15.90:0/4044422969 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc4b8153f10 msgr2=0x7fc4b815a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:48.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.788+0000 7fc4b77fe640 1 --2- 172.21.15.90:0/4044422969 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc4b8153f10 0x7fc4b815a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:48.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.788+0000 7fc4b77fe640 1 -- 172.21.15.90:0/4044422969 >> v1:172.21.15.90:6789/0 conn(0x7fc4b80c8e20 legacy=0x7fc4b815db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:48.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.788+0000 7fc4b77fe640 1 -- 172.21.15.90:0/4044422969 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b806bd10 con 0x7fc4b80afb60 2024-03-11T18:36:48.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.792+0000 7f02b8b7c640 1 --2- 172.21.15.90:0/3444711021 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02b406f400 0x7f02b415d1f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f02b40c8d50 tx=0x7f02a401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdd3c4ecbd000a02 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:48.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.792+0000 7f93c96af640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f93c4151000 0x7f93c40c2d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48310/0 (socket says 172.21.15.90:48310) 2024-03-11T18:36:48.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.792+0000 7f93c96af640 1 -- 172.21.15.90:0/569398808 learned_addr learned my addr 172.21.15.90:0/569398808 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:48.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.796+0000 7fae7ad9d640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7fae7c066d60 0x7fae7c1613a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:42972/0 (socket says 172.21.15.90:42972) 2024-03-11T18:36:48.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.796+0000 7fae7ad9d640 1 -- 172.21.15.90:0/3410431354 learned_addr learned my addr 172.21.15.90:0/3410431354 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T18:36:48.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.800+0000 7f93c96af640 1 -- 172.21.15.90:0/569398808 >> v2:172.21.15.92:3300/0 conn(0x7f93c4066d60 msgr2=0x7f93c41613a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:48.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.800+0000 7f93c96af640 1 --2- 172.21.15.90:0/569398808 >> v2:172.21.15.92:3300/0 conn(0x7f93c4066d60 0x7f93c41613a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:48.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.800+0000 7f93c96af640 1 -- 172.21.15.90:0/569398808 >> v1:172.21.15.92:6789/0 conn(0x7f93c40c5b50 legacy=0x7f93c40c4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:48.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.800+0000 7f93c96af640 1 -- 172.21.15.90:0/569398808 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c40c2ad0 con 0x7f93c4151000 2024-03-11T18:36:48.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.804+0000 7f02abfff640 1 -- 172.21.15.90:0/3444711021 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a40ac620 con 0x7f02b406f400 2024-03-11T18:36:48.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.804+0000 7f02abfff640 1 -- 172.21.15.90:0/3444711021 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02a40ac780 con 0x7f02b406f400 2024-03-11T18:36:48.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.804+0000 7f02abfff640 1 -- 172.21.15.90:0/3444711021 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a40acb10 con 0x7f02b406f400 2024-03-11T18:36:48.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.804+0000 7f02bb5e1640 1 -- 172.21.15.90:0/3444711021 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02b406f400 msgr2=0x7f02b415d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:48.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.804+0000 7f02bb5e1640 1 --2- 172.21.15.90:0/3444711021 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02b406f400 0x7f02b415d1f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f02b40c8d50 tx=0x7f02a401c4b0 comp rx=0 tx=0).stop 2024-03-11T18:36:48.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.804+0000 7f02bb5e1640 1 -- 172.21.15.90:0/3444711021 shutdown_connections 2024-03-11T18:36:48.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.804+0000 7f02bb5e1640 1 --2- 172.21.15.90:0/3444711021 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02b406f400 0x7f02b415d1f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:48.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.804+0000 7f02bb5e1640 1 --2- 172.21.15.90:0/3444711021 >> v2:172.21.15.92:3300/0 conn(0x7f02b41540c0 0x7f02b40c6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:48.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.804+0000 7f02bb5e1640 1 -- 172.21.15.90:0/3444711021 >> 172.21.15.90:0/3444711021 conn(0x7f02b4071aa0 msgr2=0x7f02b4071eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:48.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.804+0000 7f02bb5e1640 1 -- 172.21.15.90:0/3444711021 shutdown_connections 2024-03-11T18:36:48.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.804+0000 7f02bb5e1640 1 -- 172.21.15.90:0/3444711021 wait complete. 2024-03-11T18:36:48.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.808+0000 7f02bb5e1640 1 Processor -- start 2024-03-11T18:36:48.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.808+0000 7f02bb5e1640 1 -- start start 2024-03-11T18:36:48.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.808+0000 7f02bb5e1640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02b406f400 0x7f02b4150d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:48.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.808+0000 7f02bb5e1640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f02b40cc280 0x7f02b41530d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:48.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.808+0000 7f02bb5e1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02b41540c0 0x7f02b4202af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:48.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.808+0000 7f02bb5e1640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b40caa60 con 0x7f02b406f400 2024-03-11T18:36:48.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.808+0000 7f02bb5e1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b415cab0 con 0x7f02b41540c0 2024-03-11T18:36:48.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.808+0000 7f02bb5e1640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02b414f1a0 con 0x7f02b40cc280 2024-03-11T18:36:48.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.808+0000 7fae7a59c640 1 --2- 172.21.15.90:0/3410431354 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fae7c151000 0x7fae7c0c2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:48.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.812+0000 7fc4b77fe640 1 --2- 172.21.15.90:0/4044422969 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4b80afb60 0x7fc4b806c520 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0009e50 tx=0x7fc4b001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4fd379b84aa0000 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:48.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.812+0000 7f93c96af640 1 --2- 172.21.15.90:0/569398808 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93c4151000 0x7f93c40c2d80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f93c40678c0 tx=0x7f93c001c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d7266ec0dbd25b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:48.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.812+0000 7f02b8b7c640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f02b40cc280 0x7f02b41530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:48.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.812+0000 7f02b8b7c640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f02b40cc280 0x7f02b41530d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.90:59382/0 (socket says 172.21.15.90:59382) 2024-03-11T18:36:48.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.812+0000 7f02b8b7c640 1 -- 172.21.15.90:0/3488954203 learned_addr learned my addr 172.21.15.90:0/3488954203 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:48.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.816+0000 7fae7959a640 1 -- 172.21.15.90:0/3410431354 <== mon.0 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 409815162 0 0) 0x7fae7c0c7980 con 0x7fae7c066d60 2024-03-11T18:36:48.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.816+0000 7fae7959a640 1 -- 172.21.15.90:0/3410431354 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae64005740 con 0x7fae7c066d60 2024-03-11T18:36:48.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.820+0000 7f02b8b7c640 1 -- 172.21.15.90:0/3488954203 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02b41540c0 msgr2=0x7f02b4202af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:48.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.820+0000 7f02b8b7c640 1 --2- 172.21.15.90:0/3488954203 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02b41540c0 0x7f02b4202af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:48.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.820+0000 7f02b8b7c640 1 -- 172.21.15.90:0/3488954203 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02b406f400 msgr2=0x7f02b4150d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:48.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.820+0000 7f02b8b7c640 1 --2- 172.21.15.90:0/3488954203 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02b406f400 0x7f02b4150d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:48.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.820+0000 7f02b8b7c640 1 -- 172.21.15.90:0/3488954203 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a40094d0 con 0x7f02b40cc280 2024-03-11T18:36:48.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.820+0000 7ff271c41640 1 Processor -- start 2024-03-11T18:36:48.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.820+0000 7ff271c41640 1 -- start start 2024-03-11T18:36:48.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.824+0000 7ff271c41640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff26c06f400 0x7ff26c0c6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:48.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.824+0000 7ff271c41640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff26c0cc280 0x7ff26c0cc710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:48.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.824+0000 7ff271c41640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff26c0c7d50 con 0x7ff26c1540c0 2024-03-11T18:36:48.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.824+0000 7ff271c41640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff26c15d900 con 0x7ff26c0cc280 2024-03-11T18:36:48.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.824+0000 7ff271c41640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c0703f0 con 0x7ff26c06f400 2024-03-11T18:36:48.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.824+0000 7f02b8b7c640 1 --2- 172.21.15.90:0/3488954203 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f02b40cc280 0x7f02b41530d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f02a40190c0 tx=0x7f02a4019250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:48.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.828+0000 7fc4b5ffb640 1 -- 172.21.15.90:0/4044422969 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b00a5870 con 0x7fc4b80afb60 2024-03-11T18:36:48.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.828+0000 7fc4b5ffb640 1 -- 172.21.15.90:0/4044422969 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b00a59d0 con 0x7fc4b80afb60 2024-03-11T18:36:48.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.828+0000 7fc4b5ffb640 1 -- 172.21.15.90:0/4044422969 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b00a5d60 con 0x7fc4b80afb60 2024-03-11T18:36:48.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.828+0000 7fae7959a640 1 -- 172.21.15.90:0/3410431354 <== mon.0 v1:172.21.15.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2148726147 0 0) 0x7fae64005740 con 0x7fae7c066d60 2024-03-11T18:36:48.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.828+0000 7fae7959a640 1 -- 172.21.15.90:0/3410431354 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fae7c151000 msgr2=0x7fae7c0c2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:48.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.828+0000 7fae7959a640 1 --2- 172.21.15.90:0/3410431354 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fae7c151000 0x7fae7c0c2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:48.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.828+0000 7fae7959a640 1 -- 172.21.15.90:0/3410431354 >> v1:172.21.15.92:6789/0 conn(0x7fae7c0c5b50 legacy=0x7fae7c0c4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:48.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.828+0000 7fae7959a640 1 -- 172.21.15.90:0/3410431354 --> v1:172.21.15.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae7c0ca9c0 con 0x7fae7c066d60 2024-03-11T18:36:48.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.828+0000 7fae7959a640 1 -- 172.21.15.90:0/3410431354 <== mon.0 v1:172.21.15.90:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 691735435 0 0) 0x7fae74001a00 con 0x7fae7c066d60 2024-03-11T18:36:48.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.828+0000 7fae7959a640 1 -- 172.21.15.90:0/3410431354 >> v1:172.21.15.90:6789/0 conn(0x7fae7c066d60 legacy=0x7fae7c1613a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:48.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.828+0000 7fae7959a640 1 --2- 172.21.15.90:0/3410431354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae64005bc0 0x7fae64005fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:48.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.828+0000 7fae7959a640 1 --2- 172.21.15.90:0/3410431354 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fae7c151000 0x7fae640069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:48.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.828+0000 7fae7959a640 1 --2- 172.21.15.90:0/3410431354 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae64006f70 0x7fae640073f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:48.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.828+0000 7fae7959a640 1 -- 172.21.15.90:0/3410431354 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7c0cc820 con 0x7fae64006f70 2024-03-11T18:36:48.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.828+0000 7fae7959a640 1 -- 172.21.15.90:0/3410431354 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae6400de00 con 0x7fae64005bc0 2024-03-11T18:36:48.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.828+0000 7fae7959a640 1 -- 172.21.15.90:0/3410431354 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae6400dfc0 con 0x7fae7c151000 2024-03-11T18:36:48.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.828+0000 7fc4bd62a640 1 -- 172.21.15.90:0/4044422969 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4b80afb60 msgr2=0x7fc4b806c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:48.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.828+0000 7fc4bd62a640 1 --2- 172.21.15.90:0/4044422969 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4b80afb60 0x7fc4b806c520 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0009e50 tx=0x7fc4b001c9f0 comp rx=0 tx=0).stop 2024-03-11T18:36:48.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.828+0000 7fc4bd62a640 1 -- 172.21.15.90:0/4044422969 shutdown_connections 2024-03-11T18:36:48.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.828+0000 7fc4bd62a640 1 --2- 172.21.15.90:0/4044422969 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4b80afb60 0x7fc4b806c520 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:48.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.828+0000 7fc4bd62a640 1 --2- 172.21.15.90:0/4044422969 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc4b8153f10 0x7fc4b815a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:48.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.828+0000 7fc4bd62a640 1 -- 172.21.15.90:0/4044422969 >> 172.21.15.90:0/4044422969 conn(0x7fc4b8070e80 msgr2=0x7fc4b8066990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:48.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.828+0000 7fc4bd62a640 1 -- 172.21.15.90:0/4044422969 shutdown_connections 2024-03-11T18:36:48.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.828+0000 7fc4bd62a640 1 -- 172.21.15.90:0/4044422969 wait complete. 2024-03-11T18:36:48.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.828+0000 7fc4bd62a640 1 Processor -- start 2024-03-11T18:36:48.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.828+0000 7fc4bd62a640 1 -- start start 2024-03-11T18:36:48.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.828+0000 7fc4bd62a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4b80afb60 0x7fc4b80cd960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:48.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.828+0000 7fc4bd62a640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc4b80c8e20 0x7fc4b814fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:48.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.828+0000 7fc4bd62a640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4b8153f10 0x7fc4b80cb210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:48.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.828+0000 7fc4bd62a640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b814e440 con 0x7fc4b8153f10 2024-03-11T18:36:48.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.828+0000 7fc4bd62a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b814ccd0 con 0x7fc4b80afb60 2024-03-11T18:36:48.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.828+0000 7fc4bd62a640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b814f260 con 0x7fc4b80c8e20 2024-03-11T18:36:48.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.832+0000 7f93bbfff640 1 -- 172.21.15.90:0/569398808 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c00ad640 con 0x7f93c4151000 2024-03-11T18:36:48.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.832+0000 7f93bbfff640 1 -- 172.21.15.90:0/569398808 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93c00ad7a0 con 0x7f93c4151000 2024-03-11T18:36:48.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.832+0000 7f93bbfff640 1 -- 172.21.15.90:0/569398808 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c00adb30 con 0x7f93c4151000 2024-03-11T18:36:48.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.832+0000 7f93cb913640 1 -- 172.21.15.90:0/569398808 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93c4151000 msgr2=0x7f93c40c2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:48.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.832+0000 7f93cb913640 1 --2- 172.21.15.90:0/569398808 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93c4151000 0x7f93c40c2d80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f93c40678c0 tx=0x7f93c001c390 comp rx=0 tx=0).stop 2024-03-11T18:36:48.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.832+0000 7f93cb913640 1 -- 172.21.15.90:0/569398808 shutdown_connections 2024-03-11T18:36:48.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.832+0000 7f93cb913640 1 --2- 172.21.15.90:0/569398808 >> v2:172.21.15.92:3300/0 conn(0x7f93c4066d60 0x7f93c41613a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:48.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.832+0000 7f93cb913640 1 --2- 172.21.15.90:0/569398808 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93c4151000 0x7f93c40c2d80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:48.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.832+0000 7f93cb913640 1 -- 172.21.15.90:0/569398808 >> 172.21.15.90:0/569398808 conn(0x7f93c40cb880 msgr2=0x7f93c40cbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:48.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.832+0000 7f93cb913640 1 -- 172.21.15.90:0/569398808 shutdown_connections 2024-03-11T18:36:48.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.832+0000 7f93cb913640 1 -- 172.21.15.90:0/569398808 wait complete. 2024-03-11T18:36:48.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.832+0000 7f02a9ffb640 1 -- 172.21.15.90:0/3488954203 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a40b6440 con 0x7f02b40cc280 2024-03-11T18:36:48.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.832+0000 7f02a9ffb640 1 -- 172.21.15.90:0/3488954203 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02a40b65a0 con 0x7f02b40cc280 2024-03-11T18:36:48.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.832+0000 7f02bb5e1640 1 -- 172.21.15.90:0/3488954203 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02b40c8360 con 0x7f02b40cc280 2024-03-11T18:36:48.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.832+0000 7f02bb5e1640 1 -- 172.21.15.90:0/3488954203 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02b414fed0 con 0x7f02b40cc280 2024-03-11T18:36:48.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.832+0000 7f02a9ffb640 1 -- 172.21.15.90:0/3488954203 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a40b6930 con 0x7f02b40cc280 2024-03-11T18:36:48.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.832+0000 7fae7ad9d640 1 --2- 172.21.15.90:0/3410431354 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae64006f70 0x7fae640073f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:48.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.832+0000 7f02bb5e1640 1 -- 172.21.15.90:0/3488954203 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02b404eef0 con 0x7f02b40cc280 2024-03-11T18:36:48.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.832+0000 7f93cb913640 1 Processor -- start 2024-03-11T18:36:48.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.832+0000 7f93cb913640 1 -- start start 2024-03-11T18:36:48.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.832+0000 7fc4b77fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4b8153f10 0x7fc4b80cb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:48.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.836+0000 7fc4b77fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4b8153f10 0x7fc4b80cb210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48342/0 (socket says 172.21.15.90:48342) 2024-03-11T18:36:48.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.836+0000 7fc4b77fe640 1 -- 172.21.15.90:0/1062043898 learned_addr learned my addr 172.21.15.90:0/1062043898 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:48.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.836+0000 7f93cb913640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f93c4066d60 0x7f93c4155210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:48.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.836+0000 7f93cb913640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93c40c5b50 0x7f93c4155750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:48.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.836+0000 7f93cb913640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93c4151000 0x7f93c414c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:48.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.836+0000 7f93cb913640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c406bed0 con 0x7f93c4151000 2024-03-11T18:36:48.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.836+0000 7f93cb913640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c41611f0 con 0x7f93c40c5b50 2024-03-11T18:36:48.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.836+0000 7f93cb913640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93c4160950 con 0x7f93c4066d60 2024-03-11T18:36:48.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.844+0000 7fae7ad9d640 1 -- 172.21.15.90:0/3410431354 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fae7c151000 msgr2=0x7fae640069e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:48.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.844+0000 7fae7ad9d640 1 --2- 172.21.15.90:0/3410431354 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fae7c151000 0x7fae640069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:48.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.844+0000 7fae7ad9d640 1 -- 172.21.15.90:0/3410431354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae64005bc0 msgr2=0x7fae64005fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:48.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.844+0000 7fae7ad9d640 1 --2- 172.21.15.90:0/3410431354 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae64005bc0 0x7fae64005fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:48.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.844+0000 7fae7ad9d640 1 -- 172.21.15.90:0/3410431354 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae7c0ca9c0 con 0x7fae64006f70 2024-03-11T18:36:48.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.844+0000 7fc4b77fe640 1 -- 172.21.15.90:0/1062043898 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc4b80c8e20 msgr2=0x7fc4b814fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:48.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.844+0000 7fc4b77fe640 1 --2- 172.21.15.90:0/1062043898 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc4b80c8e20 0x7fc4b814fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:48.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.844+0000 7fc4b77fe640 1 -- 172.21.15.90:0/1062043898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4b80afb60 msgr2=0x7fc4b80cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:48.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.844+0000 7fc4b77fe640 1 --2- 172.21.15.90:0/1062043898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4b80afb60 0x7fc4b80cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:48.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.844+0000 7fc4b77fe640 1 -- 172.21.15.90:0/1062043898 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b0009b30 con 0x7fc4b8153f10 2024-03-11T18:36:48.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.844+0000 7f93c9eb0640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93c4151000 0x7f93c414c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:48.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.844+0000 7f93c9eb0640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93c4151000 0x7f93c414c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48356/0 (socket says 172.21.15.90:48356) 2024-03-11T18:36:48.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.844+0000 7f93c9eb0640 1 -- 172.21.15.90:0/472993249 learned_addr learned my addr 172.21.15.90:0/472993249 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:48.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.848+0000 7fc4b77fe640 1 --2- 172.21.15.90:0/1062043898 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4b8153f10 0x7fc4b80cb210 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc4b00af040 tx=0x7fc4b0019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:48.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.852+0000 7f93c9eb0640 1 -- 172.21.15.90:0/472993249 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f93c4066d60 msgr2=0x7f93c4155210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:48.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.852+0000 7f93c9eb0640 1 --2- 172.21.15.90:0/472993249 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f93c4066d60 0x7f93c4155210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:48.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.852+0000 7f93c9eb0640 1 -- 172.21.15.90:0/472993249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93c40c5b50 msgr2=0x7f93c4155750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:48.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.852+0000 7f93c9eb0640 1 --2- 172.21.15.90:0/472993249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93c40c5b50 0x7f93c4155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:48.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.852+0000 7f93c9eb0640 1 -- 172.21.15.90:0/472993249 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c00094d0 con 0x7f93c4151000 2024-03-11T18:36:48.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.856+0000 7fae7ad9d640 1 --2- 172.21.15.90:0/3410431354 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae64006f70 0x7fae640073f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fae7400a880 tx=0x7fae7401d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=492fd80c2363e923 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:48.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.856+0000 7f93c9eb0640 1 --2- 172.21.15.90:0/472993249 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93c4151000 0x7f93c414c4a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f93bc00b9e0 tx=0x7f93bc00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:48.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.856+0000 7fc49f7fe640 1 -- 172.21.15.90:0/1062043898 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b00bd070 con 0x7fc4b8153f10 2024-03-11T18:36:48.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.856+0000 7fc49f7fe640 1 -- 172.21.15.90:0/1062043898 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b00b8470 con 0x7fc4b8153f10 2024-03-11T18:36:48.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.856+0000 7fc4bd62a640 1 -- 172.21.15.90:0/1062043898 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4b814e650 con 0x7fc4b8153f10 2024-03-11T18:36:48.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.856+0000 7fc49f7fe640 1 -- 172.21.15.90:0/1062043898 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b00b8860 con 0x7fc4b8153f10 2024-03-11T18:36:48.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.856+0000 7fc4bd62a640 1 -- 172.21.15.90:0/1062043898 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4b814ec10 con 0x7fc4b8153f10 2024-03-11T18:36:48.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.860+0000 7fc4bd62a640 1 -- 172.21.15.90:0/1062043898 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4b804eef0 con 0x7fc4b8153f10 2024-03-11T18:36:48.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.864+0000 7fae7959a640 1 -- 172.21.15.90:0/3410431354 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae740ad630 con 0x7fae64006f70 2024-03-11T18:36:48.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.864+0000 7fae7959a640 1 -- 172.21.15.90:0/3410431354 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae740a8070 con 0x7fae64006f70 2024-03-11T18:36:48.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.864+0000 7fae7959a640 1 -- 172.21.15.90:0/3410431354 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae740ad9d0 con 0x7fae64006f70 2024-03-11T18:36:48.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.864+0000 7fae80aac640 1 -- 172.21.15.90:0/3410431354 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae64006f70 msgr2=0x7fae640073f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:48.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.864+0000 7fae80aac640 1 --2- 172.21.15.90:0/3410431354 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae64006f70 0x7fae640073f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fae7400a880 tx=0x7fae7401d3a0 comp rx=0 tx=0).stop 2024-03-11T18:36:48.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.864+0000 7fae7ad9d640 1 -- 172.21.15.90:0/3410431354 reap_dead start 2024-03-11T18:36:48.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.864+0000 7fae80aac640 1 -- 172.21.15.90:0/3410431354 shutdown_connections 2024-03-11T18:36:48.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.864+0000 7fae80aac640 1 -- 172.21.15.90:0/3410431354 >> 172.21.15.90:0/3410431354 conn(0x7fae7c0cb880 msgr2=0x7fae7c0cbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:48.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.864+0000 7fae80aac640 1 -- 172.21.15.90:0/3410431354 shutdown_connections 2024-03-11T18:36:48.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.864+0000 7fae80aac640 1 -- 172.21.15.90:0/3410431354 wait complete. 2024-03-11T18:36:48.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.864+0000 7fae80aac640 1 Processor -- start 2024-03-11T18:36:48.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.864+0000 7fae80aac640 1 -- start start 2024-03-11T18:36:48.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.864+0000 7fae80aac640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fae7c1f1880 0x7fae7c158dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:48.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.864+0000 7fae80aac640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae7c157c60 0x7fae7c143840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:48.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.864+0000 7fae80aac640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae7c150570 0x7fae7c1f1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:48.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.864+0000 7fae80aac640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7c158650 con 0x7fae7c150570 2024-03-11T18:36:48.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.864+0000 7fae80aac640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7c1433b0 con 0x7fae7c157c60 2024-03-11T18:36:48.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.864+0000 7fae80aac640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae7c158830 con 0x7fae7c1f1880 2024-03-11T18:36:48.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.868+0000 7f93b9ffb640 1 -- 172.21.15.90:0/472993249 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93bc00caa0 con 0x7f93c4151000 2024-03-11T18:36:48.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.868+0000 7f93cb913640 1 -- 172.21.15.90:0/472993249 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93c4160c30 con 0x7f93c4151000 2024-03-11T18:36:48.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.868+0000 7f93cb913640 1 -- 172.21.15.90:0/472993249 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93c4154a10 con 0x7f93c4151000 2024-03-11T18:36:48.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.868+0000 7f93cb913640 1 -- 172.21.15.90:0/472993249 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93c404eef0 con 0x7f93c4151000 2024-03-11T18:36:48.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.868+0000 7fae7ad9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae7c150570 0x7fae7c1f1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:48.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.888+0000 7f1741ffb640 1 -- 172.21.15.90:0/3026481570 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f174c01a400 con 0x7f175006b860 2024-03-11T18:36:48.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.888+0000 7f1741ffb640 1 --2- 172.21.15.90:0/3026481570 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1734066e90 0x7f1734069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:48.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.888+0000 7f17548bc640 1 --2- 172.21.15.90:0/3026481570 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1734066e90 0x7f1734069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:48.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.888+0000 7f17548bc640 1 --2- 172.21.15.90:0/3026481570 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1734066e90 0x7f1734069350 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f17440b2000 tx=0x7f1744004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:48.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.896+0000 7f1741ffb640 1 -- 172.21.15.90:0/3026481570 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f174c055de0 con 0x7f175006b860 2024-03-11T18:36:48.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.928+0000 7f1741ffb640 1 -- 172.21.15.90:0/3026481570 <== mon.1 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f174c052080 con 0x7f175006b860 2024-03-11T18:36:48.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.928+0000 7ff26bfff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff26c0cc280 0x7ff26c0cc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:48.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.928+0000 7ff26bfff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff26c0cc280 0x7ff26c0cc710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.90:60562/0 (socket says 172.21.15.90:60562) 2024-03-11T18:36:48.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.928+0000 7ff26bfff640 1 -- 172.21.15.90:0/1029192164 learned_addr learned my addr 172.21.15.90:0/1029192164 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:48.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.932+0000 7ff26bfff640 1 -- 172.21.15.90:0/1029192164 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff26c06f400 msgr2=0x7ff26c0c6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:48.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.932+0000 7ff26bfff640 1 --2- 172.21.15.90:0/1029192164 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff26c06f400 0x7ff26c0c6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:48.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.932+0000 7ff26bfff640 1 -- 172.21.15.90:0/1029192164 >> v1:172.21.15.92:6789/0 conn(0x7ff26c1540c0 legacy=0x7ff26c15d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:48.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.932+0000 7ff26bfff640 1 -- 172.21.15.90:0/1029192164 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff26c06b480 con 0x7ff26c0cc280 2024-03-11T18:36:48.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.936+0000 7f5fecfd0640 1 -- 172.21.15.90:0/4004048064 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5fe806f1c0 con 0x7f5fe806f400 2024-03-11T18:36:48.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.944+0000 7ff26bfff640 1 --2- 172.21.15.90:0/1029192164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff26c0cc280 0x7ff26c0cc710 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff264009e50 tx=0x7ff26401f1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e051480c6f650bc server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:48.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:48.944+0000 7f0a8f6bb640 1 -- 172.21.15.90:0/1959989133 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0a68005820 con 0x7f0a88072ce0 2024-03-11T18:36:49.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.052+0000 7ff26affd640 1 -- 172.21.15.90:0/1029192164 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2640af400 con 0x7ff26c0cc280 2024-03-11T18:36:49.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.052+0000 7ff26affd640 1 -- 172.21.15.90:0/1029192164 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2640ab050 con 0x7ff26c0cc280 2024-03-11T18:36:49.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.052+0000 7ff26affd640 1 -- 172.21.15.90:0/1029192164 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2640af780 con 0x7ff26c0cc280 2024-03-11T18:36:49.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.052+0000 7ff271c41640 1 -- 172.21.15.90:0/1029192164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff26c0cc280 msgr2=0x7ff26c0cc710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:49.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.052+0000 7ff271c41640 1 --2- 172.21.15.90:0/1029192164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff26c0cc280 0x7ff26c0cc710 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff264009e50 tx=0x7ff26401f1e0 comp rx=0 tx=0).stop 2024-03-11T18:36:49.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.052+0000 7ff271c41640 1 -- 172.21.15.90:0/1029192164 shutdown_connections 2024-03-11T18:36:49.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.052+0000 7ff271c41640 1 --2- 172.21.15.90:0/1029192164 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff26c0cc280 0x7ff26c0cc710 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:49.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.052+0000 7ff271c41640 1 --2- 172.21.15.90:0/1029192164 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff26c06f400 0x7ff26c0c6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:49.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.052+0000 7ff271c41640 1 -- 172.21.15.90:0/1029192164 >> 172.21.15.90:0/1029192164 conn(0x7ff26c071aa0 msgr2=0x7ff26c071eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:49.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.052+0000 7ff271c41640 1 -- 172.21.15.90:0/1029192164 shutdown_connections 2024-03-11T18:36:49.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.052+0000 7ff271c41640 1 -- 172.21.15.90:0/1029192164 wait complete. 2024-03-11T18:36:49.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.056+0000 7fae79d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae7c157c60 0x7fae7c143840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:49.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.056+0000 7ff271c41640 1 Processor -- start 2024-03-11T18:36:49.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.056+0000 7ff271c41640 1 -- start start 2024-03-11T18:36:49.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.056+0000 7ff271c41640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff26c06f400 0x7ff26c14ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:49.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.060+0000 7ff271c41640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff26c0cc280 0x7ff26c0c8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:49.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.060+0000 7ff271c41640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff26c1540c0 0x7ff26c14acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:49.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.060+0000 7ff271c41640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c14cfa0 con 0x7ff26c0cc280 2024-03-11T18:36:49.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.060+0000 7ff271c41640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c14d320 con 0x7ff26c06f400 2024-03-11T18:36:49.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.060+0000 7ff271c41640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c14d690 con 0x7ff26c1540c0 2024-03-11T18:36:49.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.060+0000 7fae79d9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae7c157c60 0x7fae7c143840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.90:60582/0 (socket says 172.21.15.90:60582) 2024-03-11T18:36:49.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.060+0000 7fae79d9b640 1 -- 172.21.15.90:0/1040968884 learned_addr learned my addr 172.21.15.90:0/1040968884 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:49.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.068+0000 7ff26b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff26c06f400 0x7ff26c14ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:49.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.068+0000 7ff26b7fe640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff26c06f400 0x7ff26c14ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.90:60598/0 (socket says 172.21.15.90:60598) 2024-03-11T18:36:49.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.068+0000 7ff26b7fe640 1 -- 172.21.15.90:0/2509576962 learned_addr learned my addr 172.21.15.90:0/2509576962 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:49.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.072+0000 7ff26b7fe640 1 -- 172.21.15.90:0/2509576962 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff26c1540c0 msgr2=0x7ff26c14acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:49.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.072+0000 7ff26b7fe640 1 --2- 172.21.15.90:0/2509576962 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff26c1540c0 0x7ff26c14acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:49.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.072+0000 7ff26b7fe640 1 -- 172.21.15.90:0/2509576962 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff26c0cc280 msgr2=0x7ff26c0c8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:49.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.072+0000 7ff26b7fe640 1 --2- 172.21.15.90:0/2509576962 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff26c0cc280 0x7ff26c0c8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:49.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.072+0000 7ff26b7fe640 1 -- 172.21.15.90:0/2509576962 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff264009b30 con 0x7ff26c06f400 2024-03-11T18:36:49.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.080+0000 7fae79d9b640 1 -- 172.21.15.90:0/1040968884 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fae7c1f1880 msgr2=0x7fae7c158dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:49.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.080+0000 7fae79d9b640 1 --2- 172.21.15.90:0/1040968884 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fae7c1f1880 0x7fae7c158dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:49.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.080+0000 7fae79d9b640 1 -- 172.21.15.90:0/1040968884 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae7c150570 msgr2=0x7fae7c1f1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:49.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.080+0000 7fae79d9b640 1 --2- 172.21.15.90:0/1040968884 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae7c150570 0x7fae7c1f1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:49.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.080+0000 7fae79d9b640 1 -- 172.21.15.90:0/1040968884 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae7c158b10 con 0x7fae7c157c60 2024-03-11T18:36:49.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.092+0000 7ff26b7fe640 1 --2- 172.21.15.90:0/2509576962 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff26c06f400 0x7ff26c14ca60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff25c00b3e0 tx=0x7ff25c00b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:49.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.092+0000 7f1757321640 1 -- 172.21.15.90:0/3026481570 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f175004eef0 con 0x7f175006b860 2024-03-11T18:36:49.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.104+0000 7ff268ff9640 1 -- 172.21.15.90:0/2509576962 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25c011830 con 0x7ff26c06f400 2024-03-11T18:36:49.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.104+0000 7ff268ff9640 1 -- 172.21.15.90:0/2509576962 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff25c0119b0 con 0x7ff26c06f400 2024-03-11T18:36:49.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.104+0000 7ff271c41640 1 -- 172.21.15.90:0/2509576962 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff26c14bd20 con 0x7ff26c06f400 2024-03-11T18:36:49.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.104+0000 7ff268ff9640 1 -- 172.21.15.90:0/2509576962 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25c011d60 con 0x7ff26c06f400 2024-03-11T18:36:49.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.104+0000 7ff271c41640 1 -- 172.21.15.90:0/2509576962 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff26c14f6b0 con 0x7ff26c06f400 2024-03-11T18:36:49.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.108+0000 7ff271c41640 1 -- 172.21.15.90:0/2509576962 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff26c04eef0 con 0x7ff26c06f400 2024-03-11T18:36:49.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.132+0000 7fc49f7fe640 1 -- 172.21.15.90:0/1062043898 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fc4b00b89c0 con 0x7fc4b8153f10 2024-03-11T18:36:49.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.132+0000 7fc49f7fe640 1 --2- 172.21.15.90:0/1062043898 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fc4a4066e70 0x7fc4a4069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:49.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.132+0000 7fc4b6ffd640 1 --2- 172.21.15.90:0/1062043898 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fc4a4066e70 0x7fc4a4069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:49.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.132+0000 7fc4b6ffd640 1 --2- 172.21.15.90:0/1062043898 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fc4a4066e70 0x7fc4a4069330 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc4a8009730 tx=0x7fc4a8006ca0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:49.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.136+0000 7f02a9ffb640 1 -- 172.21.15.90:0/3488954203 <== mon.2 v2:172.21.15.92:3301/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f02a40aa070 con 0x7f02b40cc280 2024-03-11T18:36:49.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.136+0000 7f02a9ffb640 1 --2- 172.21.15.90:0/3488954203 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0298066e40 0x7f0298069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:49.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.136+0000 7f02b937d640 1 --2- 172.21.15.90:0/3488954203 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0298066e40 0x7f0298069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:49.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.136+0000 7f02b937d640 1 --2- 172.21.15.90:0/3488954203 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0298066e40 0x7f0298069300 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f02b0009690 tx=0x7f02b0006c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:49.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.140+0000 7f02a9ffb640 1 -- 172.21.15.90:0/3488954203 <== mon.2 v2:172.21.15.92:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f02a40f0800 con 0x7f02b40cc280 2024-03-11T18:36:49.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.156+0000 7fc49f7fe640 1 -- 172.21.15.90:0/1062043898 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc4b00f19d0 con 0x7fc4b8153f10 2024-03-11T18:36:49.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.156+0000 7fc49f7fe640 1 -- 172.21.15.90:0/1062043898 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc4b00ed570 con 0x7fc4b8153f10 2024-03-11T18:36:49.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.156+0000 7f02a9ffb640 1 -- 172.21.15.90:0/3488954203 <== mon.2 v2:172.21.15.92:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f02a40ec3a0 con 0x7f02b40cc280 2024-03-11T18:36:49.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.168+0000 7f5fdeffd640 1 -- 172.21.15.90:0/4004048064 <== mon.2 v2:172.21.15.92:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5fd801e070 con 0x7f5fe806f400 2024-03-11T18:36:49.175 INFO:teuthology.orchestra.run.smithi090.stdout:64424509463 2024-03-11T18:36:49.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.172+0000 7f5fecfd0640 1 -- 172.21.15.90:0/4004048064 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f5fc8066ec0 msgr2=0x7f5fc8069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:49.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.172+0000 7f5fecfd0640 1 --2- 172.21.15.90:0/4004048064 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f5fc8066ec0 0x7f5fc8069380 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5fe814db90 tx=0x7f5fd0009190 comp rx=0 tx=0).stop 2024-03-11T18:36:49.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.172+0000 7f5fecfd0640 1 -- 172.21.15.90:0/4004048064 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f5fe806f400 msgr2=0x7f5fe81f01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:49.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.172+0000 7f5fecfd0640 1 --2- 172.21.15.90:0/4004048064 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f5fe806f400 0x7f5fe81f01e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5fd800c820 tx=0x7f5fd800cce0 comp rx=0 tx=0).stop 2024-03-11T18:36:49.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.172+0000 7f5fecfd0640 1 -- 172.21.15.90:0/4004048064 shutdown_connections 2024-03-11T18:36:49.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.172+0000 7f5fecfd0640 1 --2- 172.21.15.90:0/4004048064 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f5fc8066ec0 0x7f5fc8069380 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:49.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.172+0000 7f5fecfd0640 1 --2- 172.21.15.90:0/4004048064 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f5fe81540c0 0x7f5fe81f2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:49.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.172+0000 7f5fecfd0640 1 --2- 172.21.15.90:0/4004048064 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fe80cc280 0x7f5fe814c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:49.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.172+0000 7f5fecfd0640 1 --2- 172.21.15.90:0/4004048064 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f5fe806f400 0x7f5fe81f01e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:49.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.172+0000 7f5fecfd0640 1 -- 172.21.15.90:0/4004048064 >> 172.21.15.90:0/4004048064 conn(0x7f5fe8071aa0 msgr2=0x7f5fe80c5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:49.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.172+0000 7f5fecfd0640 1 -- 172.21.15.90:0/4004048064 shutdown_connections 2024-03-11T18:36:49.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.172+0000 7f5fecfd0640 1 -- 172.21.15.90:0/4004048064 wait complete. 2024-03-11T18:36:49.196 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509462 got 64424509463 for osd.2 2024-03-11T18:36:49.197 DEBUG:teuthology.parallel:result is None 2024-03-11T18:36:49.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.320+0000 7f02bb5e1640 1 -- 172.21.15.90:0/3488954203 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f02b404eef0 con 0x7f02b40cc280 2024-03-11T18:36:49.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.324+0000 7fc4bd62a640 1 -- 172.21.15.90:0/1062043898 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fc4b815abb0 con 0x7fc4b8153f10 2024-03-11T18:36:49.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.328+0000 7f02a9ffb640 1 -- 172.21.15.90:0/3488954203 <== mon.2 v2:172.21.15.92:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f02a40b1030 con 0x7f02b40cc280 2024-03-11T18:36:49.334 INFO:teuthology.orchestra.run.smithi090.stdout:68719476761 2024-03-11T18:36:49.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.328+0000 7f02bb5e1640 1 -- 172.21.15.90:0/3488954203 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0298066e40 msgr2=0x7f0298069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:49.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.328+0000 7f02bb5e1640 1 --2- 172.21.15.90:0/3488954203 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0298066e40 0x7f0298069300 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f02b0009690 tx=0x7f02b0006c00 comp rx=0 tx=0).stop 2024-03-11T18:36:49.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.328+0000 7f02bb5e1640 1 -- 172.21.15.90:0/3488954203 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f02b40cc280 msgr2=0x7f02b41530d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:49.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.328+0000 7f02bb5e1640 1 --2- 172.21.15.90:0/3488954203 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f02b40cc280 0x7f02b41530d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f02a40190c0 tx=0x7f02a4019250 comp rx=0 tx=0).stop 2024-03-11T18:36:49.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.328+0000 7f02bb5e1640 1 -- 172.21.15.90:0/3488954203 shutdown_connections 2024-03-11T18:36:49.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.328+0000 7f02bb5e1640 1 --2- 172.21.15.90:0/3488954203 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0298066e40 0x7f0298069300 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:49.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.328+0000 7f02bb5e1640 1 --2- 172.21.15.90:0/3488954203 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02b41540c0 0x7f02b4202af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:49.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.328+0000 7f02bb5e1640 1 --2- 172.21.15.90:0/3488954203 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f02b40cc280 0x7f02b41530d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:49.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.328+0000 7f02bb5e1640 1 --2- 172.21.15.90:0/3488954203 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02b406f400 0x7f02b4150d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:49.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.328+0000 7f02bb5e1640 1 -- 172.21.15.90:0/3488954203 >> 172.21.15.90:0/3488954203 conn(0x7f02b4071aa0 msgr2=0x7f02b415a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:49.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.328+0000 7f02bb5e1640 1 -- 172.21.15.90:0/3488954203 shutdown_connections 2024-03-11T18:36:49.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.328+0000 7f02bb5e1640 1 -- 172.21.15.90:0/3488954203 wait complete. 2024-03-11T18:36:49.351 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476760 got 68719476761 for osd.7 2024-03-11T18:36:49.351 DEBUG:teuthology.parallel:result is None 2024-03-11T18:36:49.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.432+0000 7ff268ff9640 1 -- 172.21.15.90:0/2509576962 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7ff25c004020 con 0x7ff26c06f400 2024-03-11T18:36:49.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.432+0000 7ff268ff9640 1 --2- 172.21.15.90:0/2509576962 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7ff24c066ec0 0x7ff24c069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:49.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.432+0000 7ff263fff640 1 --2- 172.21.15.90:0/2509576962 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7ff24c066ec0 0x7ff24c069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:49.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.432+0000 7ff263fff640 1 --2- 172.21.15.90:0/2509576962 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7ff24c066ec0 0x7ff24c069380 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff26c1504e0 tx=0x7ff254006ca0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:49.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.436+0000 7ff268ff9640 1 -- 172.21.15.90:0/2509576962 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff25c055fc0 con 0x7ff26c06f400 2024-03-11T18:36:49.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.444+0000 7f93b9ffb640 1 -- 172.21.15.90:0/472993249 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93bc0097d0 con 0x7f93c4151000 2024-03-11T18:36:49.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.444+0000 7f93b9ffb640 1 -- 172.21.15.90:0/472993249 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93bc00ce00 con 0x7f93c4151000 2024-03-11T18:36:49.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.448+0000 7f93b9ffb640 1 -- 172.21.15.90:0/472993249 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f93bc014460 con 0x7f93c4151000 2024-03-11T18:36:49.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.448+0000 7f93b9ffb640 1 --2- 172.21.15.90:0/472993249 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f93a8066e40 0x7f93a8069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:49.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.448+0000 7f93c96af640 1 --2- 172.21.15.90:0/472993249 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f93a8066e40 0x7f93a8069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:49.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.448+0000 7f93c96af640 1 --2- 172.21.15.90:0/472993249 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f93a8066e40 0x7f93a8069300 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f93c00af000 tx=0x7f93c0004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:49.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.464+0000 7fae79d9b640 1 --2- 172.21.15.90:0/1040968884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae7c157c60 0x7fae7c143840 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fae7000c410 tx=0x7fae7000c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:49.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.464+0000 7ff268ff9640 1 -- 172.21.15.90:0/2509576962 <== mon.1 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff25c051b60 con 0x7ff26c06f400 2024-03-11T18:36:49.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.488+0000 7f93b9ffb640 1 -- 172.21.15.90:0/472993249 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f93bc01fe70 con 0x7f93c4151000 2024-03-11T18:36:49.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.488+0000 7f93b9ffb640 1 -- 172.21.15.90:0/472993249 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f93bc051f10 con 0x7f93c4151000 2024-03-11T18:36:49.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.508+0000 7fae6affd640 1 -- 172.21.15.90:0/1040968884 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae70003600 con 0x7fae7c157c60 2024-03-11T18:36:49.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.508+0000 7fae6affd640 1 -- 172.21.15.90:0/1040968884 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae70003760 con 0x7fae7c157c60 2024-03-11T18:36:49.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.508+0000 7fae6affd640 1 -- 172.21.15.90:0/1040968884 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae70003b50 con 0x7fae7c157c60 2024-03-11T18:36:49.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.508+0000 7fae80aac640 1 -- 172.21.15.90:0/1040968884 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae7c142a80 con 0x7fae7c157c60 2024-03-11T18:36:49.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.508+0000 7fae80aac640 1 -- 172.21.15.90:0/1040968884 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae7c143040 con 0x7fae7c157c60 2024-03-11T18:36:49.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.512+0000 7fae80aac640 1 -- 172.21.15.90:0/1040968884 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae7c04eef0 con 0x7fae7c157c60 2024-03-11T18:36:49.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.512+0000 7f1741ffb640 1 -- 172.21.15.90:0/3026481570 <== mon.1 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f174c015090 con 0x7f175006b860 2024-03-11T18:36:49.518 INFO:teuthology.orchestra.run.smithi090.stdout:64424509463 2024-03-11T18:36:49.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.512+0000 7f0a7dffb640 1 -- 172.21.15.90:0/1959989133 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0a78017070 con 0x7f0a88072ce0 2024-03-11T18:36:49.519 INFO:teuthology.orchestra.run.smithi090.stdout:60129542165 2024-03-11T18:36:49.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.512+0000 7fc49f7fe640 1 -- 172.21.15.90:0/1062043898 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc4b815abb0 con 0x7fc4b8153f10 2024-03-11T18:36:49.520 INFO:teuthology.orchestra.run.smithi090.stdout:68719476760 2024-03-11T18:36:49.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.512+0000 7f1757321640 1 -- 172.21.15.90:0/3026481570 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1734066e90 msgr2=0x7f1734069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:49.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.512+0000 7f1757321640 1 --2- 172.21.15.90:0/3026481570 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1734066e90 0x7f1734069350 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f17440b2000 tx=0x7f1744004740 comp rx=0 tx=0).stop 2024-03-11T18:36:49.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.512+0000 7f1757321640 1 -- 172.21.15.90:0/3026481570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f175006b860 msgr2=0x7f1750152520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:49.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.512+0000 7f1757321640 1 --2- 172.21.15.90:0/3026481570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f175006b860 0x7f1750152520 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f174c00b3e0 tx=0x7f174c00b8a0 comp rx=0 tx=0).stop 2024-03-11T18:36:49.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.516+0000 7f1757321640 1 -- 172.21.15.90:0/3026481570 shutdown_connections 2024-03-11T18:36:49.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.516+0000 7f1757321640 1 --2- 172.21.15.90:0/3026481570 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1734066e90 0x7f1734069350 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:49.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.516+0000 7f1757321640 1 --2- 172.21.15.90:0/3026481570 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1750151d50 0x7f1750145ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:49.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.516+0000 7f1757321640 1 --2- 172.21.15.90:0/3026481570 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17500c95f0 0x7f17501548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:49.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.516+0000 7f1757321640 1 --2- 172.21.15.90:0/3026481570 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f175006b860 0x7f1750152520 secure :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f174c00b3e0 tx=0x7f174c00b8a0 comp rx=0 tx=0).stop 2024-03-11T18:36:49.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.516+0000 7f1757321640 1 -- 172.21.15.90:0/3026481570 >> 172.21.15.90:0/3026481570 conn(0x7f1750071aa0 msgr2=0x7f1750147e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:49.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.516+0000 7f1757321640 1 -- 172.21.15.90:0/3026481570 shutdown_connections 2024-03-11T18:36:49.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.516+0000 7f1757321640 1 -- 172.21.15.90:0/3026481570 wait complete. 2024-03-11T18:36:49.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.516+0000 7f0a8f6bb640 1 -- 172.21.15.90:0/1959989133 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0a6c066e70 msgr2=0x7f0a6c069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:49.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.516+0000 7f0a8f6bb640 1 --2- 172.21.15.90:0/1959989133 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0a6c066e70 0x7f0a6c069330 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0a80007200 tx=0x7f0a80001e70 comp rx=0 tx=0).stop 2024-03-11T18:36:49.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.516+0000 7f0a8f6bb640 1 -- 172.21.15.90:0/1959989133 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a88072ce0 msgr2=0x7f0a8814b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:49.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.516+0000 7f0a8f6bb640 1 --2- 172.21.15.90:0/1959989133 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a88072ce0 0x7f0a8814b2b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0a7800c410 tx=0x7f0a7800c8d0 comp rx=0 tx=0).stop 2024-03-11T18:36:49.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.516+0000 7f0a8f6bb640 1 -- 172.21.15.90:0/1959989133 shutdown_connections 2024-03-11T18:36:49.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.516+0000 7f0a8f6bb640 1 --2- 172.21.15.90:0/1959989133 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0a6c066e70 0x7f0a6c069330 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:49.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.516+0000 7f0a8f6bb640 1 --2- 172.21.15.90:0/1959989133 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0a88145250 0x7f0a881456d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:49.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.516+0000 7f0a8f6bb640 1 --2- 172.21.15.90:0/1959989133 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a88072ce0 0x7f0a8814b2b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:49.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.516+0000 7f0a8f6bb640 1 --2- 172.21.15.90:0/1959989133 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a88072900 0x7f0a881f36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:49.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.516+0000 7f0a8f6bb640 1 -- 172.21.15.90:0/1959989133 >> 172.21.15.90:0/1959989133 conn(0x7f0a880cb880 msgr2=0x7f0a88149370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:49.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.516+0000 7fc4bd62a640 1 -- 172.21.15.90:0/1062043898 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fc4a4066e70 msgr2=0x7fc4a4069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:49.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.516+0000 7fc4bd62a640 1 --2- 172.21.15.90:0/1062043898 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fc4a4066e70 0x7fc4a4069330 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc4a8009730 tx=0x7fc4a8006ca0 comp rx=0 tx=0).stop 2024-03-11T18:36:49.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.516+0000 7fc4bd62a640 1 -- 172.21.15.90:0/1062043898 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4b8153f10 msgr2=0x7fc4b80cb210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:49.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.516+0000 7fc4bd62a640 1 --2- 172.21.15.90:0/1062043898 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4b8153f10 0x7fc4b80cb210 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc4b00af040 tx=0x7fc4b0019790 comp rx=0 tx=0).stop 2024-03-11T18:36:49.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.516+0000 7fc4bd62a640 1 -- 172.21.15.90:0/1062043898 shutdown_connections 2024-03-11T18:36:49.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.516+0000 7fc4bd62a640 1 --2- 172.21.15.90:0/1062043898 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fc4a4066e70 0x7fc4a4069330 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:49.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.516+0000 7fc4bd62a640 1 --2- 172.21.15.90:0/1062043898 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4b8153f10 0x7fc4b80cb210 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:49.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.516+0000 7fc4bd62a640 1 --2- 172.21.15.90:0/1062043898 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc4b80c8e20 0x7fc4b814fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:49.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.516+0000 7fc4bd62a640 1 --2- 172.21.15.90:0/1062043898 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc4b80afb60 0x7fc4b80cd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:49.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.516+0000 7fc4bd62a640 1 -- 172.21.15.90:0/1062043898 >> 172.21.15.90:0/1062043898 conn(0x7fc4b8070e80 msgr2=0x7fc4b815af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:49.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.516+0000 7fc4bd62a640 1 -- 172.21.15.90:0/1062043898 shutdown_connections 2024-03-11T18:36:49.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.516+0000 7fc4bd62a640 1 -- 172.21.15.90:0/1062043898 wait complete. 2024-03-11T18:36:49.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.520+0000 7f0a8f6bb640 1 -- 172.21.15.90:0/1959989133 shutdown_connections 2024-03-11T18:36:49.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.520+0000 7f0a8f6bb640 1 -- 172.21.15.90:0/1959989133 wait complete. 2024-03-11T18:36:49.550 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509463 got 64424509463 for osd.3 2024-03-11T18:36:49.551 DEBUG:teuthology.parallel:result is None 2024-03-11T18:36:49.552 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476759 got 68719476760 for osd.4 2024-03-11T18:36:49.552 DEBUG:teuthology.parallel:result is None 2024-03-11T18:36:49.554 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542165 got 60129542165 for osd.0 2024-03-11T18:36:49.554 DEBUG:teuthology.parallel:result is None 2024-03-11T18:36:49.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.660+0000 7ff271c41640 1 -- 172.21.15.90:0/2509576962 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7ff26c06f1c0 con 0x7ff26c06f400 2024-03-11T18:36:49.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.672+0000 7f93cb913640 1 -- 172.21.15.90:0/472993249 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f93c4066b40 con 0x7f93c4151000 2024-03-11T18:36:49.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.680+0000 7f93b9ffb640 1 -- 172.21.15.90:0/472993249 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f93bc023020 con 0x7f93c4151000 2024-03-11T18:36:49.685 INFO:teuthology.orchestra.run.smithi090.stdout:68719476759 2024-03-11T18:36:49.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.680+0000 7f93cb913640 1 -- 172.21.15.90:0/472993249 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f93a8066e40 msgr2=0x7f93a8069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:49.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.680+0000 7f93cb913640 1 --2- 172.21.15.90:0/472993249 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f93a8066e40 0x7f93a8069300 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f93c00af000 tx=0x7f93c0004740 comp rx=0 tx=0).stop 2024-03-11T18:36:49.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.680+0000 7f93cb913640 1 -- 172.21.15.90:0/472993249 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93c4151000 msgr2=0x7f93c414c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:49.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.680+0000 7f93cb913640 1 --2- 172.21.15.90:0/472993249 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93c4151000 0x7f93c414c4a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f93bc00b9e0 tx=0x7f93bc00bea0 comp rx=0 tx=0).stop 2024-03-11T18:36:49.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.680+0000 7f93cb913640 1 -- 172.21.15.90:0/472993249 shutdown_connections 2024-03-11T18:36:49.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.680+0000 7f93cb913640 1 --2- 172.21.15.90:0/472993249 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f93a8066e40 0x7f93a8069300 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:49.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.680+0000 7f93cb913640 1 --2- 172.21.15.90:0/472993249 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93c4151000 0x7f93c414c4a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:49.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.680+0000 7f93cb913640 1 --2- 172.21.15.90:0/472993249 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f93c40c5b50 0x7f93c4155750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:49.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.680+0000 7f93cb913640 1 --2- 172.21.15.90:0/472993249 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f93c4066d60 0x7f93c4155210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:49.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.680+0000 7f93cb913640 1 -- 172.21.15.90:0/472993249 >> 172.21.15.90:0/472993249 conn(0x7f93c40cb880 msgr2=0x7f93c40cbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:49.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.680+0000 7f93cb913640 1 -- 172.21.15.90:0/472993249 shutdown_connections 2024-03-11T18:36:49.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.680+0000 7f93cb913640 1 -- 172.21.15.90:0/472993249 wait complete. 2024-03-11T18:36:49.702 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476758 got 68719476759 for osd.5 2024-03-11T18:36:49.702 DEBUG:teuthology.parallel:result is None 2024-03-11T18:36:49.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.824+0000 7ff268ff9640 1 -- 172.21.15.90:0/2509576962 <== mon.1 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff25c015020 con 0x7ff26c06f400 2024-03-11T18:36:49.831 INFO:teuthology.orchestra.run.smithi090.stdout:68719476761 2024-03-11T18:36:49.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.828+0000 7ff271c41640 1 -- 172.21.15.90:0/2509576962 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7ff24c066ec0 msgr2=0x7ff24c069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:49.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.828+0000 7ff271c41640 1 --2- 172.21.15.90:0/2509576962 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7ff24c066ec0 0x7ff24c069380 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff26c1504e0 tx=0x7ff254006ca0 comp rx=0 tx=0).stop 2024-03-11T18:36:49.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.828+0000 7ff271c41640 1 -- 172.21.15.90:0/2509576962 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff26c06f400 msgr2=0x7ff26c14ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:49.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.828+0000 7ff271c41640 1 --2- 172.21.15.90:0/2509576962 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff26c06f400 0x7ff26c14ca60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff25c00b3e0 tx=0x7ff25c00b8a0 comp rx=0 tx=0).stop 2024-03-11T18:36:49.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.828+0000 7ff271c41640 1 -- 172.21.15.90:0/2509576962 shutdown_connections 2024-03-11T18:36:49.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.828+0000 7ff271c41640 1 --2- 172.21.15.90:0/2509576962 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7ff24c066ec0 0x7ff24c069380 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:49.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.828+0000 7ff271c41640 1 --2- 172.21.15.90:0/2509576962 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff26c1540c0 0x7ff26c14acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:49.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.828+0000 7ff271c41640 1 --2- 172.21.15.90:0/2509576962 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff26c0cc280 0x7ff26c0c8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:49.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.828+0000 7ff271c41640 1 --2- 172.21.15.90:0/2509576962 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff26c06f400 0x7ff26c14ca60 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:49.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.828+0000 7ff271c41640 1 -- 172.21.15.90:0/2509576962 >> 172.21.15.90:0/2509576962 conn(0x7ff26c071aa0 msgr2=0x7ff26c0c69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:49.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.828+0000 7ff271c41640 1 -- 172.21.15.90:0/2509576962 shutdown_connections 2024-03-11T18:36:49.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.828+0000 7ff271c41640 1 -- 172.21.15.90:0/2509576962 wait complete. 2024-03-11T18:36:49.851 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476760 got 68719476761 for osd.6 2024-03-11T18:36:49.851 DEBUG:teuthology.parallel:result is None 2024-03-11T18:36:49.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.868+0000 7fae6affd640 1 -- 172.21.15.90:0/1040968884 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fae70003cb0 con 0x7fae7c157c60 2024-03-11T18:36:49.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.868+0000 7fae6affd640 1 --2- 172.21.15.90:0/1040968884 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fae5c066e90 0x7fae5c069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:49.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.868+0000 7fae7a59c640 1 --2- 172.21.15.90:0/1040968884 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fae5c066e90 0x7fae5c069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:49.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.868+0000 7fae7a59c640 1 --2- 172.21.15.90:0/1040968884 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fae5c066e90 0x7fae5c069350 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fae6c0097f0 tx=0x7fae6c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:49.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.876+0000 7fae6affd640 1 -- 172.21.15.90:0/1040968884 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fae700556e0 con 0x7fae7c157c60 2024-03-11T18:36:49.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:49.976+0000 7fae6affd640 1 -- 172.21.15.90:0/1040968884 <== mon.1 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fae70051280 con 0x7fae7c157c60 2024-03-11T18:36:50.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.156+0000 7fae80aac640 1 -- 172.21.15.90:0/1040968884 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fae7c066b40 con 0x7fae7c157c60 2024-03-11T18:36:50.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.160+0000 7fae6affd640 1 -- 172.21.15.90:0/1040968884 <== mon.1 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fae70017070 con 0x7fae7c157c60 2024-03-11T18:36:50.168 INFO:teuthology.orchestra.run.smithi090.stdout:64424509464 2024-03-11T18:36:50.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.164+0000 7fae80aac640 1 -- 172.21.15.90:0/1040968884 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fae5c066e90 msgr2=0x7fae5c069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:50.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.164+0000 7fae80aac640 1 --2- 172.21.15.90:0/1040968884 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fae5c066e90 0x7fae5c069350 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fae6c0097f0 tx=0x7fae6c008040 comp rx=0 tx=0).stop 2024-03-11T18:36:50.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.164+0000 7fae80aac640 1 -- 172.21.15.90:0/1040968884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae7c157c60 msgr2=0x7fae7c143840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:50.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.164+0000 7fae80aac640 1 --2- 172.21.15.90:0/1040968884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae7c157c60 0x7fae7c143840 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fae7000c410 tx=0x7fae7000c8d0 comp rx=0 tx=0).stop 2024-03-11T18:36:50.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.164+0000 7fae80aac640 1 -- 172.21.15.90:0/1040968884 shutdown_connections 2024-03-11T18:36:50.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.164+0000 7fae80aac640 1 --2- 172.21.15.90:0/1040968884 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fae5c066e90 0x7fae5c069350 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:50.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.164+0000 7fae80aac640 1 --2- 172.21.15.90:0/1040968884 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae7c150570 0x7fae7c1f1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:50.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.164+0000 7fae80aac640 1 --2- 172.21.15.90:0/1040968884 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fae7c157c60 0x7fae7c143840 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:50.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.164+0000 7fae80aac640 1 --2- 172.21.15.90:0/1040968884 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fae7c1f1880 0x7fae7c158dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:50.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.164+0000 7fae80aac640 1 -- 172.21.15.90:0/1040968884 >> 172.21.15.90:0/1040968884 conn(0x7fae7c0cb880 msgr2=0x7fae7c0c7910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:50.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.164+0000 7fae80aac640 1 -- 172.21.15.90:0/1040968884 shutdown_connections 2024-03-11T18:36:50.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.164+0000 7fae80aac640 1 -- 172.21.15.90:0/1040968884 wait complete. 2024-03-11T18:36:50.188 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509463 got 64424509464 for osd.1 2024-03-11T18:36:50.188 DEBUG:teuthology.parallel:result is None 2024-03-11T18:36:50.189 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-11T18:36:50.189 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-11T18:36:50.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.344+0000 7f45d9a6e640 1 Processor -- start 2024-03-11T18:36:50.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.344+0000 7f45d9a6e640 1 -- start start 2024-03-11T18:36:50.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.344+0000 7f45d9a6e640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f45d4151000 0x7f45d40c2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:50.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.344+0000 7f45d9a6e640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f45d40c5b50 0x7f45d40c4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:50.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.344+0000 7f45d9a6e640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f45d4066d60 0x7f45d40c5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:50.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.344+0000 7f45d9a6e640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45d40cc820 con 0x7f45d40c5b50 2024-03-11T18:36:50.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.344+0000 7f45d9a6e640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45d40caca0 con 0x7f45d4151000 2024-03-11T18:36:50.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.344+0000 7f45d9a6e640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45d40c7980 con 0x7f45d4066d60 2024-03-11T18:36:50.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.344+0000 7f45d2ffd640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f45d40c5b50 0x7f45d40c4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:50.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.344+0000 7f45d2ffd640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f45d40c5b50 0x7f45d40c4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48372/0 (socket says 172.21.15.90:48372) 2024-03-11T18:36:50.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.344+0000 7f45d2ffd640 1 -- 172.21.15.90:0/594041028 learned_addr learned my addr 172.21.15.90:0/594041028 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:50.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.344+0000 7f45d3fff640 1 --2- 172.21.15.90:0/594041028 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f45d4066d60 0x7f45d40c5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:50.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.344+0000 7f45d37fe640 1 --2- 172.21.15.90:0/594041028 >> v2:172.21.15.92:3300/0 conn(0x7f45d4151000 0x7f45d40c2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:50.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.348+0000 7f45d2ffd640 1 -- 172.21.15.90:0/594041028 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f45d4066d60 msgr2=0x7f45d40c5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:50.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.348+0000 7f45d2ffd640 1 --2- 172.21.15.90:0/594041028 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f45d4066d60 0x7f45d40c5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:50.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.348+0000 7f45d2ffd640 1 -- 172.21.15.90:0/594041028 >> v2:172.21.15.92:3300/0 conn(0x7f45d4151000 msgr2=0x7f45d40c2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:50.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.348+0000 7f45d2ffd640 1 --2- 172.21.15.90:0/594041028 >> v2:172.21.15.92:3300/0 conn(0x7f45d4151000 0x7f45d40c2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:50.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.348+0000 7f45d2ffd640 1 -- 172.21.15.90:0/594041028 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d40c2ad0 con 0x7f45d40c5b50 2024-03-11T18:36:50.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.352+0000 7f45d2ffd640 1 --2- 172.21.15.90:0/594041028 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f45d40c5b50 0x7f45d40c4ad0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f45d406a3a0 tx=0x7f45c801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a942b4820fb09c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:50.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.356+0000 7f45d27fc640 1 -- 172.21.15.90:0/594041028 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c80ad640 con 0x7f45d40c5b50 2024-03-11T18:36:50.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.356+0000 7f45d27fc640 1 -- 172.21.15.90:0/594041028 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45c80ad7a0 con 0x7f45d40c5b50 2024-03-11T18:36:50.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.356+0000 7f45d27fc640 1 -- 172.21.15.90:0/594041028 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c80adb30 con 0x7f45d40c5b50 2024-03-11T18:36:50.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.356+0000 7f45d9a6e640 1 -- 172.21.15.90:0/594041028 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f45d40c5b50 msgr2=0x7f45d40c4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:50.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.356+0000 7f45d9a6e640 1 --2- 172.21.15.90:0/594041028 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f45d40c5b50 0x7f45d40c4ad0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f45d406a3a0 tx=0x7f45c801c3d0 comp rx=0 tx=0).stop 2024-03-11T18:36:50.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.356+0000 7f45d9a6e640 1 -- 172.21.15.90:0/594041028 shutdown_connections 2024-03-11T18:36:50.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.356+0000 7f45d9a6e640 1 --2- 172.21.15.90:0/594041028 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f45d4066d60 0x7f45d40c5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:50.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.356+0000 7f45d9a6e640 1 --2- 172.21.15.90:0/594041028 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f45d40c5b50 0x7f45d40c4ad0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:50.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.356+0000 7f45d9a6e640 1 --2- 172.21.15.90:0/594041028 >> v2:172.21.15.92:3300/0 conn(0x7f45d4151000 0x7f45d40c2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:50.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.356+0000 7f45d9a6e640 1 -- 172.21.15.90:0/594041028 >> 172.21.15.90:0/594041028 conn(0x7f45d40cb880 msgr2=0x7f45d40cbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:50.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.356+0000 7f45d9a6e640 1 -- 172.21.15.90:0/594041028 shutdown_connections 2024-03-11T18:36:50.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.356+0000 7f45d9a6e640 1 -- 172.21.15.90:0/594041028 wait complete. 2024-03-11T18:36:50.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.356+0000 7f45d9a6e640 1 Processor -- start 2024-03-11T18:36:50.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.356+0000 7f45d9a6e640 1 -- start start 2024-03-11T18:36:50.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.360+0000 7f45d9a6e640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f45d4066d60 0x7f45d41ef4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:50.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.360+0000 7f45d9a6e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f45d40c5b50 0x7f45d4159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:50.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.360+0000 7f45d9a6e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45d4151000 0x7f45d406ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:50.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.360+0000 7f45d9a6e640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d4151ee0 con 0x7f45d40c5b50 2024-03-11T18:36:50.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.360+0000 7f45d9a6e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d415ae90 con 0x7f45d4151000 2024-03-11T18:36:50.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.360+0000 7f45d9a6e640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45d415a5f0 con 0x7f45d4066d60 2024-03-11T18:36:50.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.360+0000 7f45d2ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f45d40c5b50 0x7f45d4159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:50.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.360+0000 7f45d2ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f45d40c5b50 0x7f45d4159fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48380/0 (socket says 172.21.15.90:48380) 2024-03-11T18:36:50.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.360+0000 7f45d2ffd640 1 -- 172.21.15.90:0/1676184536 learned_addr learned my addr 172.21.15.90:0/1676184536 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:50.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.360+0000 7f45d37fe640 1 --2- 172.21.15.90:0/1676184536 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f45d4066d60 0x7f45d41ef4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:50.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.360+0000 7f45d3fff640 1 --2- 172.21.15.90:0/1676184536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45d4151000 0x7f45d406ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:50.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.364+0000 7f45d2ffd640 1 -- 172.21.15.90:0/1676184536 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f45d4066d60 msgr2=0x7f45d41ef4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:50.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.364+0000 7f45d2ffd640 1 --2- 172.21.15.90:0/1676184536 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f45d4066d60 0x7f45d41ef4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:50.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.364+0000 7f45d2ffd640 1 -- 172.21.15.90:0/1676184536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45d4151000 msgr2=0x7f45d406ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:50.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.364+0000 7f45d2ffd640 1 --2- 172.21.15.90:0/1676184536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45d4151000 0x7f45d406ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:50.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.364+0000 7f45d2ffd640 1 -- 172.21.15.90:0/1676184536 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c80094d0 con 0x7f45d40c5b50 2024-03-11T18:36:50.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.364+0000 7f45d2ffd640 1 --2- 172.21.15.90:0/1676184536 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f45d40c5b50 0x7f45d4159fb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f45c801a000 tx=0x7f45c8019170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:50.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.368+0000 7f45bbfff640 1 -- 172.21.15.90:0/1676184536 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c80bd430 con 0x7f45d40c5b50 2024-03-11T18:36:50.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.368+0000 7f45bbfff640 1 -- 172.21.15.90:0/1676184536 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45c80b8070 con 0x7f45d40c5b50 2024-03-11T18:36:50.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.368+0000 7f45bbfff640 1 -- 172.21.15.90:0/1676184536 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c80bd7b0 con 0x7f45d40c5b50 2024-03-11T18:36:50.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.368+0000 7f45d9a6e640 1 -- 172.21.15.90:0/1676184536 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45d415a8d0 con 0x7f45d40c5b50 2024-03-11T18:36:50.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.368+0000 7f45d9a6e640 1 -- 172.21.15.90:0/1676184536 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45d4151b70 con 0x7f45d40c5b50 2024-03-11T18:36:50.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.372+0000 7f45d9a6e640 1 -- 172.21.15.90:0/1676184536 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45d404eef0 con 0x7f45d40c5b50 2024-03-11T18:36:50.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.408+0000 7f45bbfff640 1 -- 172.21.15.90:0/1676184536 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f45c80ba030 con 0x7f45d40c5b50 2024-03-11T18:36:50.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.408+0000 7f45bbfff640 1 --2- 172.21.15.90:0/1676184536 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f45c0066e40 0x7f45c0069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:50.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.408+0000 7f45d37fe640 1 --2- 172.21.15.90:0/1676184536 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f45c0066e40 0x7f45c0069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:50.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.408+0000 7f45d37fe640 1 --2- 172.21.15.90:0/1676184536 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f45c0066e40 0x7f45c0069300 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f45c4007940 tx=0x7f45c4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:50.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.712+0000 7f45bbfff640 1 -- 172.21.15.90:0/1676184536 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f45c80f18a0 con 0x7f45d40c5b50 2024-03-11T18:36:50.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.712+0000 7f45bbfff640 1 -- 172.21.15.90:0/1676184536 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f45c80edb40 con 0x7f45d40c5b50 2024-03-11T18:36:50.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.880+0000 7f45d9a6e640 1 -- 172.21.15.90:0/1676184536 --> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f45d4155da0 con 0x7f45c0066e40 2024-03-11T18:36:50.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.888+0000 7f45bbfff640 1 -- 172.21.15.90:0/1676184536 <== mgr.4101 v2:172.21.15.90:6800/1999720161 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332628 (secure 0 0 0) 0x7f45d4155da0 con 0x7f45c0066e40 2024-03-11T18:36:50.899 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:36:50.901 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-03-11T18:36:50.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.892+0000 7f45d9a6e640 1 -- 172.21.15.90:0/1676184536 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f45c0066e40 msgr2=0x7f45c0069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:50.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.892+0000 7f45d9a6e640 1 --2- 172.21.15.90:0/1676184536 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f45c0066e40 0x7f45c0069300 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f45c4007940 tx=0x7f45c4008040 comp rx=0 tx=0).stop 2024-03-11T18:36:50.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.892+0000 7f45d9a6e640 1 -- 172.21.15.90:0/1676184536 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f45d40c5b50 msgr2=0x7f45d4159fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:50.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.892+0000 7f45d9a6e640 1 --2- 172.21.15.90:0/1676184536 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f45d40c5b50 0x7f45d4159fb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f45c801a000 tx=0x7f45c8019170 comp rx=0 tx=0).stop 2024-03-11T18:36:50.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.892+0000 7f45d9a6e640 1 -- 172.21.15.90:0/1676184536 shutdown_connections 2024-03-11T18:36:50.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.892+0000 7f45d9a6e640 1 --2- 172.21.15.90:0/1676184536 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f45c0066e40 0x7f45c0069300 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:50.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.892+0000 7f45d9a6e640 1 --2- 172.21.15.90:0/1676184536 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f45d4151000 0x7f45d406ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:50.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.892+0000 7f45d9a6e640 1 --2- 172.21.15.90:0/1676184536 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f45d40c5b50 0x7f45d4159fb0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:50.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.892+0000 7f45d9a6e640 1 --2- 172.21.15.90:0/1676184536 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f45d4066d60 0x7f45d41ef4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:50.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.892+0000 7f45d9a6e640 1 -- 172.21.15.90:0/1676184536 >> 172.21.15.90:0/1676184536 conn(0x7f45d40cb880 msgr2=0x7f45d4155500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:50.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.892+0000 7f45d9a6e640 1 -- 172.21.15.90:0/1676184536 shutdown_connections 2024-03-11T18:36:50.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:50.892+0000 7f45d9a6e640 1 -- 172.21.15.90:0/1676184536 wait complete. 2024-03-11T18:36:50.918 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":72,"stamp":"2024-03-11T18:36:49.810111+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597854,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":292,"num_write_kb":2460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":233,"ondisk_log_size":233,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219724,"kb_used_data":3596,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754996,"statfs":{"total":773094113280,"available":772869115904,"internally_reserved":0,"allocated":3682304,"data_stored":2128050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":221236788},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2846,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":84,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005186"},"pg_stats":[{"pgid":"2.1f","version":"28'11","reported_seq":38,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:48.128889+0000","last_change":"2024-03-11T18:35:56.290140+0000","last_active":"2024-03-11T18:36:48.128889+0000","last_peered":"2024-03-11T18:36:48.128889+0000","last_clean":"2024-03-11T18:36:48.128889+0000","last_became_active":"2024-03-11T18:35:56.289178+0000","last_became_peered":"2024-03-11T18:35:56.289178+0000","last_unstale":"2024-03-11T18:36:48.128889+0000","last_undegraded":"2024-03-11T18:36:48.128889+0000","last_fullsized":"2024-03-11T18:36:48.128889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:59:46.211033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.405302+0000","last_change":"2024-03-11T18:35:57.676047+0000","last_active":"2024-03-11T18:36:06.405302+0000","last_peered":"2024-03-11T18:36:06.405302+0000","last_clean":"2024-03-11T18:36:06.405302+0000","last_became_active":"2024-03-11T18:35:57.675105+0000","last_became_peered":"2024-03-11T18:35:57.675105+0000","last_unstale":"2024-03-11T18:36:06.405302+0000","last_undegraded":"2024-03-11T18:36:06.405302+0000","last_fullsized":"2024-03-11T18:36:06.405302+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:45:39.057074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.401561+0000","last_change":"2024-03-11T18:36:04.607351+0000","last_active":"2024-03-11T18:36:06.401561+0000","last_peered":"2024-03-11T18:36:06.401561+0000","last_clean":"2024-03-11T18:36:06.401561+0000","last_became_active":"2024-03-11T18:36:04.599562+0000","last_became_peered":"2024-03-11T18:36:04.599562+0000","last_unstale":"2024-03-11T18:36:06.401561+0000","last_undegraded":"2024-03-11T18:36:06.401561+0000","last_fullsized":"2024-03-11T18:36:06.401561+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T18:57:51.134410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"28'2","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.405696+0000","last_change":"2024-03-11T18:35:56.291253+0000","last_active":"2024-03-11T18:36:06.405696+0000","last_peered":"2024-03-11T18:36:06.405696+0000","last_clean":"2024-03-11T18:36:06.405696+0000","last_became_active":"2024-03-11T18:35:56.290862+0000","last_became_peered":"2024-03-11T18:35:56.290862+0000","last_unstale":"2024-03-11T18:36:06.405696+0000","last_undegraded":"2024-03-11T18:36:06.405696+0000","last_fullsized":"2024-03-11T18:36:06.405696+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:36:17.809744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.405636+0000","last_change":"2024-03-11T18:35:57.636861+0000","last_active":"2024-03-11T18:36:06.405636+0000","last_peered":"2024-03-11T18:36:06.405636+0000","last_clean":"2024-03-11T18:36:06.405636+0000","last_became_active":"2024-03-11T18:35:57.636111+0000","last_became_peered":"2024-03-11T18:35:57.636111+0000","last_unstale":"2024-03-11T18:36:06.405636+0000","last_undegraded":"2024-03-11T18:36:06.405636+0000","last_fullsized":"2024-03-11T18:36:06.405636+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:23:11.238170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.408420+0000","last_change":"2024-03-11T18:36:04.643426+0000","last_active":"2024-03-11T18:36:06.408420+0000","last_peered":"2024-03-11T18:36:06.408420+0000","last_clean":"2024-03-11T18:36:06.408420+0000","last_became_active":"2024-03-11T18:36:04.642597+0000","last_became_peered":"2024-03-11T18:36:04.642597+0000","last_unstale":"2024-03-11T18:36:06.408420+0000","last_undegraded":"2024-03-11T18:36:06.408420+0000","last_fullsized":"2024-03-11T18:36:06.408420+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:31:27.049273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"28'4","reported_seq":31,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:44.536276+0000","last_change":"2024-03-11T18:35:56.355147+0000","last_active":"2024-03-11T18:36:44.536276+0000","last_peered":"2024-03-11T18:36:44.536276+0000","last_clean":"2024-03-11T18:36:44.536276+0000","last_became_active":"2024-03-11T18:35:56.354388+0000","last_became_peered":"2024-03-11T18:35:56.354388+0000","last_unstale":"2024-03-11T18:36:44.536276+0000","last_undegraded":"2024-03-11T18:36:44.536276+0000","last_fullsized":"2024-03-11T18:36:44.536276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:03:55.046045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.410658+0000","last_change":"2024-03-11T18:35:57.641486+0000","last_active":"2024-03-11T18:36:06.410658+0000","last_peered":"2024-03-11T18:36:06.410658+0000","last_clean":"2024-03-11T18:36:06.410658+0000","last_became_active":"2024-03-11T18:35:57.641052+0000","last_became_peered":"2024-03-11T18:35:57.641052+0000","last_unstale":"2024-03-11T18:36:06.410658+0000","last_undegraded":"2024-03-11T18:36:06.410658+0000","last_fullsized":"2024-03-11T18:36:06.410658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:29:45.662029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.411193+0000","last_change":"2024-03-11T18:36:04.629291+0000","last_active":"2024-03-11T18:36:06.411193+0000","last_peered":"2024-03-11T18:36:06.411193+0000","last_clean":"2024-03-11T18:36:06.411193+0000","last_became_active":"2024-03-11T18:36:04.624110+0000","last_became_peered":"2024-03-11T18:36:04.624110+0000","last_unstale":"2024-03-11T18:36:06.411193+0000","last_undegraded":"2024-03-11T18:36:06.411193+0000","last_fullsized":"2024-03-11T18:36:06.411193+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:04:03.368322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"23'2","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.353273+0000","last_change":"2024-03-11T18:35:56.298903+0000","last_active":"2024-03-11T18:36:06.353273+0000","last_peered":"2024-03-11T18:36:06.353273+0000","last_clean":"2024-03-11T18:36:06.353273+0000","last_became_active":"2024-03-11T18:35:56.297582+0000","last_became_peered":"2024-03-11T18:35:56.297582+0000","last_unstale":"2024-03-11T18:36:06.353273+0000","last_undegraded":"2024-03-11T18:36:06.353273+0000","last_fullsized":"2024-03-11T18:36:06.353273+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:53:25.278916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.413016+0000","last_change":"2024-03-11T18:35:57.640122+0000","last_active":"2024-03-11T18:36:06.413016+0000","last_peered":"2024-03-11T18:36:06.413016+0000","last_clean":"2024-03-11T18:36:06.413016+0000","last_became_active":"2024-03-11T18:35:57.639770+0000","last_became_peered":"2024-03-11T18:35:57.639770+0000","last_unstale":"2024-03-11T18:36:06.413016+0000","last_undegraded":"2024-03-11T18:36:06.413016+0000","last_fullsized":"2024-03-11T18:36:06.413016+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:39:25.767184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.410463+0000","last_change":"2024-03-11T18:36:04.627149+0000","last_active":"2024-03-11T18:36:06.410463+0000","last_peered":"2024-03-11T18:36:06.410463+0000","last_clean":"2024-03-11T18:36:06.410463+0000","last_became_active":"2024-03-11T18:36:04.626108+0000","last_became_peered":"2024-03-11T18:36:04.626108+0000","last_unstale":"2024-03-11T18:36:06.410463+0000","last_undegraded":"2024-03-11T18:36:06.410463+0000","last_fullsized":"2024-03-11T18:36:06.410463+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:03:40.619618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.410013+0000","last_change":"2024-03-11T18:35:57.616739+0000","last_active":"2024-03-11T18:36:06.410013+0000","last_peered":"2024-03-11T18:36:06.410013+0000","last_clean":"2024-03-11T18:36:06.410013+0000","last_became_active":"2024-03-11T18:35:57.616398+0000","last_became_peered":"2024-03-11T18:35:57.616398+0000","last_unstale":"2024-03-11T18:36:06.410013+0000","last_undegraded":"2024-03-11T18:36:06.410013+0000","last_fullsized":"2024-03-11T18:36:06.410013+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:53:30.324898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"28'4","reported_seq":31,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:37.714831+0000","last_change":"2024-03-11T18:35:56.304762+0000","last_active":"2024-03-11T18:36:37.714831+0000","last_peered":"2024-03-11T18:36:37.714831+0000","last_clean":"2024-03-11T18:36:37.714831+0000","last_became_active":"2024-03-11T18:35:56.302803+0000","last_became_peered":"2024-03-11T18:35:56.302803+0000","last_unstale":"2024-03-11T18:36:37.714831+0000","last_undegraded":"2024-03-11T18:36:37.714831+0000","last_fullsized":"2024-03-11T18:36:37.714831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:31:18.726996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.412227+0000","last_change":"2024-03-11T18:36:04.626968+0000","last_active":"2024-03-11T18:36:06.412227+0000","last_peered":"2024-03-11T18:36:06.412227+0000","last_clean":"2024-03-11T18:36:06.412227+0000","last_became_active":"2024-03-11T18:36:04.626337+0000","last_became_peered":"2024-03-11T18:36:04.626337+0000","last_unstale":"2024-03-11T18:36:06.412227+0000","last_undegraded":"2024-03-11T18:36:06.412227+0000","last_fullsized":"2024-03-11T18:36:06.412227+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:53:37.432826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.402625+0000","last_change":"2024-03-11T18:35:57.689835+0000","last_active":"2024-03-11T18:36:06.402625+0000","last_peered":"2024-03-11T18:36:06.402625+0000","last_clean":"2024-03-11T18:36:06.402625+0000","last_became_active":"2024-03-11T18:35:57.689039+0000","last_became_peered":"2024-03-11T18:35:57.689039+0000","last_unstale":"2024-03-11T18:36:06.402625+0000","last_undegraded":"2024-03-11T18:36:06.402625+0000","last_fullsized":"2024-03-11T18:36:06.402625+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:38:27.733669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"28'1","reported_seq":28,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.358940+0000","last_change":"2024-03-11T18:35:56.291070+0000","last_active":"2024-03-11T18:36:06.358940+0000","last_peered":"2024-03-11T18:36:06.358940+0000","last_clean":"2024-03-11T18:36:06.358940+0000","last_became_active":"2024-03-11T18:35:56.290504+0000","last_became_peered":"2024-03-11T18:35:56.290504+0000","last_unstale":"2024-03-11T18:36:06.358940+0000","last_undegraded":"2024-03-11T18:36:06.358940+0000","last_fullsized":"2024-03-11T18:36:06.358940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:29:31.660243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.411988+0000","last_change":"2024-03-11T18:36:04.626150+0000","last_active":"2024-03-11T18:36:06.411988+0000","last_peered":"2024-03-11T18:36:06.411988+0000","last_clean":"2024-03-11T18:36:06.411988+0000","last_became_active":"2024-03-11T18:36:04.625084+0000","last_became_peered":"2024-03-11T18:36:04.625084+0000","last_unstale":"2024-03-11T18:36:06.411988+0000","last_undegraded":"2024-03-11T18:36:06.411988+0000","last_fullsized":"2024-03-11T18:36:06.411988+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:28:58.373219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.400936+0000","last_change":"2024-03-11T18:35:57.684535+0000","last_active":"2024-03-11T18:36:06.400936+0000","last_peered":"2024-03-11T18:36:06.400936+0000","last_clean":"2024-03-11T18:36:06.400936+0000","last_became_active":"2024-03-11T18:35:57.681098+0000","last_became_peered":"2024-03-11T18:35:57.681098+0000","last_unstale":"2024-03-11T18:36:06.400936+0000","last_undegraded":"2024-03-11T18:36:06.400936+0000","last_fullsized":"2024-03-11T18:36:06.400936+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:25:49.707568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"28'3","reported_seq":30,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:37.719417+0000","last_change":"2024-03-11T18:35:56.290745+0000","last_active":"2024-03-11T18:36:37.719417+0000","last_peered":"2024-03-11T18:36:37.719417+0000","last_clean":"2024-03-11T18:36:37.719417+0000","last_became_active":"2024-03-11T18:35:56.290006+0000","last_became_peered":"2024-03-11T18:35:56.290006+0000","last_unstale":"2024-03-11T18:36:37.719417+0000","last_undegraded":"2024-03-11T18:36:37.719417+0000","last_fullsized":"2024-03-11T18:36:37.719417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:06:02.844765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.359859+0000","last_change":"2024-03-11T18:36:04.620689+0000","last_active":"2024-03-11T18:36:06.359859+0000","last_peered":"2024-03-11T18:36:06.359859+0000","last_clean":"2024-03-11T18:36:06.359859+0000","last_became_active":"2024-03-11T18:36:04.620282+0000","last_became_peered":"2024-03-11T18:36:04.620282+0000","last_unstale":"2024-03-11T18:36:06.359859+0000","last_undegraded":"2024-03-11T18:36:06.359859+0000","last_fullsized":"2024-03-11T18:36:06.359859+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:54:16.807299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.413230+0000","last_change":"2024-03-11T18:35:57.615234+0000","last_active":"2024-03-11T18:36:06.413230+0000","last_peered":"2024-03-11T18:36:06.413230+0000","last_clean":"2024-03-11T18:36:06.413230+0000","last_became_active":"2024-03-11T18:35:57.614885+0000","last_became_peered":"2024-03-11T18:35:57.614885+0000","last_unstale":"2024-03-11T18:36:06.413230+0000","last_undegraded":"2024-03-11T18:36:06.413230+0000","last_fullsized":"2024-03-11T18:36:06.413230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:22:13.730630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"28'2","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:37.714251+0000","last_change":"2024-03-11T18:35:56.304617+0000","last_active":"2024-03-11T18:36:37.714251+0000","last_peered":"2024-03-11T18:36:37.714251+0000","last_clean":"2024-03-11T18:36:37.714251+0000","last_became_active":"2024-03-11T18:35:56.301698+0000","last_became_peered":"2024-03-11T18:35:56.301698+0000","last_unstale":"2024-03-11T18:36:37.714251+0000","last_undegraded":"2024-03-11T18:36:37.714251+0000","last_fullsized":"2024-03-11T18:36:37.714251+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:29:31.660243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.412949+0000","last_change":"2024-03-11T18:36:04.643279+0000","last_active":"2024-03-11T18:36:06.412949+0000","last_peered":"2024-03-11T18:36:06.412949+0000","last_clean":"2024-03-11T18:36:06.412949+0000","last_became_active":"2024-03-11T18:36:04.641983+0000","last_became_peered":"2024-03-11T18:36:04.641983+0000","last_unstale":"2024-03-11T18:36:06.412949+0000","last_undegraded":"2024-03-11T18:36:06.412949+0000","last_fullsized":"2024-03-11T18:36:06.412949+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:13:47.499965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.406548+0000","last_change":"2024-03-11T18:35:57.506963+0000","last_active":"2024-03-11T18:36:06.406548+0000","last_peered":"2024-03-11T18:36:06.406548+0000","last_clean":"2024-03-11T18:36:06.406548+0000","last_became_active":"2024-03-11T18:35:57.506038+0000","last_became_peered":"2024-03-11T18:35:57.506038+0000","last_unstale":"2024-03-11T18:36:06.406548+0000","last_undegraded":"2024-03-11T18:36:06.406548+0000","last_fullsized":"2024-03-11T18:36:06.406548+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:52:06.024245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"28'4","reported_seq":31,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:37.955368+0000","last_change":"2024-03-11T18:35:56.364512+0000","last_active":"2024-03-11T18:36:37.955368+0000","last_peered":"2024-03-11T18:36:37.955368+0000","last_clean":"2024-03-11T18:36:37.955368+0000","last_became_active":"2024-03-11T18:35:56.363731+0000","last_became_peered":"2024-03-11T18:35:56.363731+0000","last_unstale":"2024-03-11T18:36:37.955368+0000","last_undegraded":"2024-03-11T18:36:37.955368+0000","last_fullsized":"2024-03-11T18:36:37.955368+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:36:17.809744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.408707+0000","last_change":"2024-03-11T18:36:04.643782+0000","last_active":"2024-03-11T18:36:06.408707+0000","last_peered":"2024-03-11T18:36:06.408707+0000","last_clean":"2024-03-11T18:36:06.408707+0000","last_became_active":"2024-03-11T18:36:04.642133+0000","last_became_peered":"2024-03-11T18:36:04.642133+0000","last_unstale":"2024-03-11T18:36:06.408707+0000","last_undegraded":"2024-03-11T18:36:06.408707+0000","last_fullsized":"2024-03-11T18:36:06.408707+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:53:33.601193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.405392+0000","last_change":"2024-03-11T18:35:57.675207+0000","last_active":"2024-03-11T18:36:06.405392+0000","last_peered":"2024-03-11T18:36:06.405392+0000","last_clean":"2024-03-11T18:36:06.405392+0000","last_became_active":"2024-03-11T18:35:57.674472+0000","last_became_peered":"2024-03-11T18:35:57.674472+0000","last_unstale":"2024-03-11T18:36:06.405392+0000","last_undegraded":"2024-03-11T18:36:06.405392+0000","last_fullsized":"2024-03-11T18:36:06.405392+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:29:45.662029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"28'3","reported_seq":30,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:44.760547+0000","last_change":"2024-03-11T18:35:56.348479+0000","last_active":"2024-03-11T18:36:44.760547+0000","last_peered":"2024-03-11T18:36:44.760547+0000","last_clean":"2024-03-11T18:36:44.760547+0000","last_became_active":"2024-03-11T18:35:56.348040+0000","last_became_peered":"2024-03-11T18:35:56.348040+0000","last_unstale":"2024-03-11T18:36:44.760547+0000","last_undegraded":"2024-03-11T18:36:44.760547+0000","last_fullsized":"2024-03-11T18:36:44.760547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:27:26.471550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.401155+0000","last_change":"2024-03-11T18:36:04.599805+0000","last_active":"2024-03-11T18:36:06.401155+0000","last_peered":"2024-03-11T18:36:06.401155+0000","last_clean":"2024-03-11T18:36:06.401155+0000","last_became_active":"2024-03-11T18:36:04.599065+0000","last_became_peered":"2024-03-11T18:36:04.599065+0000","last_unstale":"2024-03-11T18:36:06.401155+0000","last_undegraded":"2024-03-11T18:36:06.401155+0000","last_fullsized":"2024-03-11T18:36:06.401155+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:36:07.187003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.359096+0000","last_change":"2024-03-11T18:35:57.538551+0000","last_active":"2024-03-11T18:36:06.359096+0000","last_peered":"2024-03-11T18:36:06.359096+0000","last_clean":"2024-03-11T18:36:06.359096+0000","last_became_active":"2024-03-11T18:35:57.537706+0000","last_became_peered":"2024-03-11T18:35:57.537706+0000","last_unstale":"2024-03-11T18:36:06.359096+0000","last_undegraded":"2024-03-11T18:36:06.359096+0000","last_fullsized":"2024-03-11T18:36:06.359096+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:03:56.260394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"28'1","reported_seq":28,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.355497+0000","last_change":"2024-03-11T18:35:56.302575+0000","last_active":"2024-03-11T18:36:06.355497+0000","last_peered":"2024-03-11T18:36:06.355497+0000","last_clean":"2024-03-11T18:36:06.355497+0000","last_became_active":"2024-03-11T18:35:56.301819+0000","last_became_peered":"2024-03-11T18:35:56.301819+0000","last_unstale":"2024-03-11T18:36:06.355497+0000","last_undegraded":"2024-03-11T18:36:06.355497+0000","last_fullsized":"2024-03-11T18:36:06.355497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:38:26.519320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.404840+0000","last_change":"2024-03-11T18:36:04.609282+0000","last_active":"2024-03-11T18:36:06.404840+0000","last_peered":"2024-03-11T18:36:06.404840+0000","last_clean":"2024-03-11T18:36:06.404840+0000","last_became_active":"2024-03-11T18:36:04.608260+0000","last_became_peered":"2024-03-11T18:36:04.608260+0000","last_unstale":"2024-03-11T18:36:06.404840+0000","last_undegraded":"2024-03-11T18:36:06.404840+0000","last_fullsized":"2024-03-11T18:36:06.404840+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:49:28.143755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.413195+0000","last_change":"2024-03-11T18:35:57.688727+0000","last_active":"2024-03-11T18:36:06.413195+0000","last_peered":"2024-03-11T18:36:06.413195+0000","last_clean":"2024-03-11T18:36:06.413195+0000","last_became_active":"2024-03-11T18:35:57.688209+0000","last_became_peered":"2024-03-11T18:35:57.688209+0000","last_unstale":"2024-03-11T18:36:06.413195+0000","last_undegraded":"2024-03-11T18:36:06.413195+0000","last_fullsized":"2024-03-11T18:36:06.413195+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:58:13.389498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.359326+0000","last_change":"2024-03-11T18:35:56.293842+0000","last_active":"2024-03-11T18:36:06.359326+0000","last_peered":"2024-03-11T18:36:06.359326+0000","last_clean":"2024-03-11T18:36:06.359326+0000","last_became_active":"2024-03-11T18:35:56.293276+0000","last_became_peered":"2024-03-11T18:35:56.293276+0000","last_unstale":"2024-03-11T18:36:06.359326+0000","last_undegraded":"2024-03-11T18:36:06.359326+0000","last_fullsized":"2024-03-11T18:36:06.359326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:04:13.101160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.358354+0000","last_change":"2024-03-11T18:36:04.626320+0000","last_active":"2024-03-11T18:36:06.358354+0000","last_peered":"2024-03-11T18:36:06.358354+0000","last_clean":"2024-03-11T18:36:06.358354+0000","last_became_active":"2024-03-11T18:36:04.625663+0000","last_became_peered":"2024-03-11T18:36:04.625663+0000","last_unstale":"2024-03-11T18:36:06.358354+0000","last_undegraded":"2024-03-11T18:36:06.358354+0000","last_fullsized":"2024-03-11T18:36:06.358354+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:39:32.875112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.411275+0000","last_change":"2024-03-11T18:35:57.612242+0000","last_active":"2024-03-11T18:36:06.411275+0000","last_peered":"2024-03-11T18:36:06.411275+0000","last_clean":"2024-03-11T18:36:06.411275+0000","last_became_active":"2024-03-11T18:35:57.611509+0000","last_became_peered":"2024-03-11T18:35:57.611509+0000","last_unstale":"2024-03-11T18:36:06.411275+0000","last_undegraded":"2024-03-11T18:36:06.411275+0000","last_fullsized":"2024-03-11T18:36:06.411275+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:45:28.543623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"28'1","reported_seq":28,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.410230+0000","last_change":"2024-03-11T18:35:56.298766+0000","last_active":"2024-03-11T18:36:06.410230+0000","last_peered":"2024-03-11T18:36:06.410230+0000","last_clean":"2024-03-11T18:36:06.410230+0000","last_became_active":"2024-03-11T18:35:56.297747+0000","last_became_peered":"2024-03-11T18:35:56.297747+0000","last_unstale":"2024-03-11T18:36:06.410230+0000","last_undegraded":"2024-03-11T18:36:06.410230+0000","last_fullsized":"2024-03-11T18:36:06.410230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:13:39.177689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.358743+0000","last_change":"2024-03-11T18:36:04.635773+0000","last_active":"2024-03-11T18:36:06.358743+0000","last_peered":"2024-03-11T18:36:06.358743+0000","last_clean":"2024-03-11T18:36:06.358743+0000","last_became_active":"2024-03-11T18:36:04.635261+0000","last_became_peered":"2024-03-11T18:36:04.635261+0000","last_unstale":"2024-03-11T18:36:06.358743+0000","last_undegraded":"2024-03-11T18:36:06.358743+0000","last_fullsized":"2024-03-11T18:36:06.358743+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:44:18.985469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.358192+0000","last_change":"2024-03-11T18:35:57.538552+0000","last_active":"2024-03-11T18:36:06.358192+0000","last_peered":"2024-03-11T18:36:06.358192+0000","last_clean":"2024-03-11T18:36:06.358192+0000","last_became_active":"2024-03-11T18:35:57.537717+0000","last_became_peered":"2024-03-11T18:35:57.537717+0000","last_unstale":"2024-03-11T18:36:06.358192+0000","last_undegraded":"2024-03-11T18:36:06.358192+0000","last_fullsized":"2024-03-11T18:36:06.358192+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:53:26.493265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"28'4","reported_seq":31,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:44.730260+0000","last_change":"2024-03-11T18:35:56.294572+0000","last_active":"2024-03-11T18:36:44.730260+0000","last_peered":"2024-03-11T18:36:44.730260+0000","last_clean":"2024-03-11T18:36:44.730260+0000","last_became_active":"2024-03-11T18:35:56.294096+0000","last_became_peered":"2024-03-11T18:35:56.294096+0000","last_unstale":"2024-03-11T18:36:44.730260+0000","last_undegraded":"2024-03-11T18:36:44.730260+0000","last_fullsized":"2024-03-11T18:36:44.730260+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:22:12.516281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.359857+0000","last_change":"2024-03-11T18:36:04.620762+0000","last_active":"2024-03-11T18:36:06.359857+0000","last_peered":"2024-03-11T18:36:06.359857+0000","last_clean":"2024-03-11T18:36:06.359857+0000","last_became_active":"2024-03-11T18:36:04.620356+0000","last_became_peered":"2024-03-11T18:36:04.620356+0000","last_unstale":"2024-03-11T18:36:06.359857+0000","last_undegraded":"2024-03-11T18:36:06.359857+0000","last_fullsized":"2024-03-11T18:36:06.359857+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:58:20.497426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.404612+0000","last_change":"2024-03-11T18:35:57.602773+0000","last_active":"2024-03-11T18:36:06.404612+0000","last_peered":"2024-03-11T18:36:06.404612+0000","last_clean":"2024-03-11T18:36:06.404612+0000","last_became_active":"2024-03-11T18:35:57.602060+0000","last_became_peered":"2024-03-11T18:35:57.602060+0000","last_unstale":"2024-03-11T18:36:06.404612+0000","last_undegraded":"2024-03-11T18:36:06.404612+0000","last_fullsized":"2024-03-11T18:36:06.404612+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:36:22.201707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"28'3","reported_seq":30,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:44.536118+0000","last_change":"2024-03-11T18:35:56.305283+0000","last_active":"2024-03-11T18:36:44.536118+0000","last_peered":"2024-03-11T18:36:44.536118+0000","last_clean":"2024-03-11T18:36:44.536118+0000","last_became_active":"2024-03-11T18:35:56.303063+0000","last_became_peered":"2024-03-11T18:35:56.303063+0000","last_unstale":"2024-03-11T18:36:44.536118+0000","last_undegraded":"2024-03-11T18:36:44.536118+0000","last_fullsized":"2024-03-11T18:36:44.536118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T18:51:00.795706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.406282+0000","last_change":"2024-03-11T18:36:04.617247+0000","last_active":"2024-03-11T18:36:06.406282+0000","last_peered":"2024-03-11T18:36:06.406282+0000","last_clean":"2024-03-11T18:36:06.406282+0000","last_became_active":"2024-03-11T18:36:04.610765+0000","last_became_peered":"2024-03-11T18:36:04.610765+0000","last_unstale":"2024-03-11T18:36:06.406282+0000","last_undegraded":"2024-03-11T18:36:06.406282+0000","last_fullsized":"2024-03-11T18:36:06.406282+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:22:19.187685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.401101+0000","last_change":"2024-03-11T18:35:57.684207+0000","last_active":"2024-03-11T18:36:06.401101+0000","last_peered":"2024-03-11T18:36:06.401101+0000","last_clean":"2024-03-11T18:36:06.401101+0000","last_became_active":"2024-03-11T18:35:57.682653+0000","last_became_peered":"2024-03-11T18:35:57.682653+0000","last_unstale":"2024-03-11T18:36:06.401101+0000","last_undegraded":"2024-03-11T18:36:06.401101+0000","last_fullsized":"2024-03-11T18:36:06.401101+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:28:51.265291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"28'3","reported_seq":30,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:37.947870+0000","last_change":"2024-03-11T18:35:56.297078+0000","last_active":"2024-03-11T18:36:37.947870+0000","last_peered":"2024-03-11T18:36:37.947870+0000","last_clean":"2024-03-11T18:36:37.947870+0000","last_became_active":"2024-03-11T18:35:56.296705+0000","last_became_peered":"2024-03-11T18:35:56.296705+0000","last_unstale":"2024-03-11T18:36:37.947870+0000","last_undegraded":"2024-03-11T18:36:37.947870+0000","last_fullsized":"2024-03-11T18:36:37.947870+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:53:25.278916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":182,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.352930+0000","last_change":"2024-03-11T18:35:18.199528+0000","last_active":"2024-03-11T18:36:06.352930+0000","last_peered":"2024-03-11T18:36:06.352930+0000","last_clean":"2024-03-11T18:36:06.352930+0000","last_became_active":"2024-03-11T18:35:18.197797+0000","last_became_peered":"2024-03-11T18:35:18.197797+0000","last_unstale":"2024-03-11T18:36:06.352930+0000","last_undegraded":"2024-03-11T18:36:06.352930+0000","last_fullsized":"2024-03-11T18:36:06.352930+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:16.207500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:16.207500+0000","last_clean_scrub_stamp":"2024-03-11T18:35:16.207500+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:29:00.357496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.411807+0000","last_change":"2024-03-11T18:36:04.599815+0000","last_active":"2024-03-11T18:36:06.411807+0000","last_peered":"2024-03-11T18:36:06.411807+0000","last_clean":"2024-03-11T18:36:06.411807+0000","last_became_active":"2024-03-11T18:36:04.598777+0000","last_became_peered":"2024-03-11T18:36:04.598777+0000","last_unstale":"2024-03-11T18:36:06.411807+0000","last_undegraded":"2024-03-11T18:36:06.411807+0000","last_fullsized":"2024-03-11T18:36:06.411807+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:39:32.875112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.354816+0000","last_change":"2024-03-11T18:35:57.687766+0000","last_active":"2024-03-11T18:36:06.354816+0000","last_peered":"2024-03-11T18:36:06.354816+0000","last_clean":"2024-03-11T18:36:06.354816+0000","last_became_active":"2024-03-11T18:35:57.687059+0000","last_became_peered":"2024-03-11T18:35:57.687059+0000","last_unstale":"2024-03-11T18:36:06.354816+0000","last_undegraded":"2024-03-11T18:36:06.354816+0000","last_fullsized":"2024-03-11T18:36:06.354816+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:39:25.767184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"28'3","reported_seq":30,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:44.730609+0000","last_change":"2024-03-11T18:35:56.294626+0000","last_active":"2024-03-11T18:36:44.730609+0000","last_peered":"2024-03-11T18:36:44.730609+0000","last_clean":"2024-03-11T18:36:44.730609+0000","last_became_active":"2024-03-11T18:35:56.293961+0000","last_became_peered":"2024-03-11T18:35:56.293961+0000","last_unstale":"2024-03-11T18:36:44.730609+0000","last_undegraded":"2024-03-11T18:36:44.730609+0000","last_fullsized":"2024-03-11T18:36:44.730609+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:50:38.709532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.409872+0000","last_change":"2024-03-11T18:36:04.619862+0000","last_active":"2024-03-11T18:36:06.409872+0000","last_peered":"2024-03-11T18:36:06.409872+0000","last_clean":"2024-03-11T18:36:06.409872+0000","last_became_active":"2024-03-11T18:36:04.618908+0000","last_became_peered":"2024-03-11T18:36:04.618908+0000","last_unstale":"2024-03-11T18:36:06.409872+0000","last_undegraded":"2024-03-11T18:36:06.409872+0000","last_fullsized":"2024-03-11T18:36:06.409872+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:34:46.433500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.410912+0000","last_change":"2024-03-11T18:35:57.686416+0000","last_active":"2024-03-11T18:36:06.410912+0000","last_peered":"2024-03-11T18:36:06.410912+0000","last_clean":"2024-03-11T18:36:06.410912+0000","last_became_active":"2024-03-11T18:35:57.685828+0000","last_became_peered":"2024-03-11T18:35:57.685828+0000","last_unstale":"2024-03-11T18:36:06.410912+0000","last_undegraded":"2024-03-11T18:36:06.410912+0000","last_fullsized":"2024-03-11T18:36:06.410912+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:41:42.263652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"23'1","reported_seq":28,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.359093+0000","last_change":"2024-03-11T18:35:56.295539+0000","last_active":"2024-03-11T18:36:06.359093+0000","last_peered":"2024-03-11T18:36:06.359093+0000","last_clean":"2024-03-11T18:36:06.359093+0000","last_became_active":"2024-03-11T18:35:56.295123+0000","last_became_peered":"2024-03-11T18:35:56.295123+0000","last_unstale":"2024-03-11T18:36:06.359093+0000","last_undegraded":"2024-03-11T18:36:06.359093+0000","last_fullsized":"2024-03-11T18:36:06.359093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:03:06.585728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.406058+0000","last_change":"2024-03-11T18:36:04.616750+0000","last_active":"2024-03-11T18:36:06.406058+0000","last_peered":"2024-03-11T18:36:06.406058+0000","last_clean":"2024-03-11T18:36:06.406058+0000","last_became_active":"2024-03-11T18:36:04.616338+0000","last_became_peered":"2024-03-11T18:36:04.616338+0000","last_unstale":"2024-03-11T18:36:06.406058+0000","last_undegraded":"2024-03-11T18:36:06.406058+0000","last_fullsized":"2024-03-11T18:36:06.406058+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:14:42.558491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.354272+0000","last_change":"2024-03-11T18:35:57.689888+0000","last_active":"2024-03-11T18:36:06.354272+0000","last_peered":"2024-03-11T18:36:06.354272+0000","last_clean":"2024-03-11T18:36:06.354272+0000","last_became_active":"2024-03-11T18:35:57.689424+0000","last_became_peered":"2024-03-11T18:35:57.689424+0000","last_unstale":"2024-03-11T18:36:06.354272+0000","last_undegraded":"2024-03-11T18:36:06.354272+0000","last_fullsized":"2024-03-11T18:36:06.354272+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:25:16.564261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"23'2","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.402166+0000","last_change":"2024-03-11T18:35:56.293862+0000","last_active":"2024-03-11T18:36:06.402166+0000","last_peered":"2024-03-11T18:36:06.402166+0000","last_clean":"2024-03-11T18:36:06.402166+0000","last_became_active":"2024-03-11T18:35:56.293585+0000","last_became_peered":"2024-03-11T18:35:56.293585+0000","last_unstale":"2024-03-11T18:36:06.402166+0000","last_undegraded":"2024-03-11T18:36:06.402166+0000","last_fullsized":"2024-03-11T18:36:06.402166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:45:27.329274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.408514+0000","last_change":"2024-03-11T18:36:04.627729+0000","last_active":"2024-03-11T18:36:06.408514+0000","last_peered":"2024-03-11T18:36:06.408514+0000","last_clean":"2024-03-11T18:36:06.408514+0000","last_became_active":"2024-03-11T18:36:04.626596+0000","last_became_peered":"2024-03-11T18:36:04.626596+0000","last_unstale":"2024-03-11T18:36:06.408514+0000","last_undegraded":"2024-03-11T18:36:06.408514+0000","last_fullsized":"2024-03-11T18:36:06.408514+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:51:28.017349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.353860+0000","last_change":"2024-03-11T18:35:57.690506+0000","last_active":"2024-03-11T18:36:06.353860+0000","last_peered":"2024-03-11T18:36:06.353860+0000","last_clean":"2024-03-11T18:36:06.353860+0000","last_became_active":"2024-03-11T18:35:57.690223+0000","last_became_peered":"2024-03-11T18:35:57.690223+0000","last_unstale":"2024-03-11T18:36:06.353860+0000","last_undegraded":"2024-03-11T18:36:06.353860+0000","last_fullsized":"2024-03-11T18:36:06.353860+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:58:13.389498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"28'2","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.414795+0000","last_change":"2024-03-11T18:35:56.257209+0000","last_active":"2024-03-11T18:36:06.414795+0000","last_peered":"2024-03-11T18:36:06.414795+0000","last_clean":"2024-03-11T18:36:06.414795+0000","last_became_active":"2024-03-11T18:35:56.256454+0000","last_became_peered":"2024-03-11T18:35:56.256454+0000","last_unstale":"2024-03-11T18:36:06.414795+0000","last_undegraded":"2024-03-11T18:36:06.414795+0000","last_fullsized":"2024-03-11T18:36:06.414795+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:32:06.369248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.408104+0000","last_change":"2024-03-11T18:36:04.644184+0000","last_active":"2024-03-11T18:36:06.408104+0000","last_peered":"2024-03-11T18:36:06.408104+0000","last_clean":"2024-03-11T18:36:06.408104+0000","last_became_active":"2024-03-11T18:36:04.642079+0000","last_became_peered":"2024-03-11T18:36:04.642079+0000","last_unstale":"2024-03-11T18:36:06.408104+0000","last_undegraded":"2024-03-11T18:36:06.408104+0000","last_fullsized":"2024-03-11T18:36:06.408104+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:36:26.132020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.353155+0000","last_change":"2024-03-11T18:35:57.687311+0000","last_active":"2024-03-11T18:36:06.353155+0000","last_peered":"2024-03-11T18:36:06.353155+0000","last_clean":"2024-03-11T18:36:06.353155+0000","last_became_active":"2024-03-11T18:35:57.686850+0000","last_became_peered":"2024-03-11T18:35:57.686850+0000","last_unstale":"2024-03-11T18:36:06.353155+0000","last_undegraded":"2024-03-11T18:36:06.353155+0000","last_fullsized":"2024-03-11T18:36:06.353155+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:29:45.662029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"28'15","reported_seq":42,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:44.758260+0000","last_change":"2024-03-11T18:35:56.291232+0000","last_active":"2024-03-11T18:36:44.758260+0000","last_peered":"2024-03-11T18:36:44.758260+0000","last_clean":"2024-03-11T18:36:44.758260+0000","last_became_active":"2024-03-11T18:35:56.290944+0000","last_became_peered":"2024-03-11T18:35:56.290944+0000","last_unstale":"2024-03-11T18:36:44.758260+0000","last_undegraded":"2024-03-11T18:36:44.758260+0000","last_fullsized":"2024-03-11T18:36:44.758260+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:49:27.827765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.405992+0000","last_change":"2024-03-11T18:36:04.602142+0000","last_active":"2024-03-11T18:36:06.405992+0000","last_peered":"2024-03-11T18:36:06.405992+0000","last_clean":"2024-03-11T18:36:06.405992+0000","last_became_active":"2024-03-11T18:36:04.601224+0000","last_became_peered":"2024-03-11T18:36:04.601224+0000","last_unstale":"2024-03-11T18:36:06.405992+0000","last_undegraded":"2024-03-11T18:36:06.405992+0000","last_fullsized":"2024-03-11T18:36:06.405992+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:17:42.075355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.409133+0000","last_change":"2024-03-11T18:35:57.611305+0000","last_active":"2024-03-11T18:36:06.409133+0000","last_peered":"2024-03-11T18:36:06.409133+0000","last_clean":"2024-03-11T18:36:06.409133+0000","last_became_active":"2024-03-11T18:35:57.610955+0000","last_became_peered":"2024-03-11T18:35:57.610955+0000","last_unstale":"2024-03-11T18:36:06.409133+0000","last_undegraded":"2024-03-11T18:36:06.409133+0000","last_fullsized":"2024-03-11T18:36:06.409133+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:31:19.941345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"28'3","reported_seq":30,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:44.715902+0000","last_change":"2024-03-11T18:35:56.361952+0000","last_active":"2024-03-11T18:36:44.715902+0000","last_peered":"2024-03-11T18:36:44.715902+0000","last_clean":"2024-03-11T18:36:44.715902+0000","last_became_active":"2024-03-11T18:35:56.361141+0000","last_became_peered":"2024-03-11T18:35:56.361141+0000","last_unstale":"2024-03-11T18:36:44.715902+0000","last_undegraded":"2024-03-11T18:36:44.715902+0000","last_fullsized":"2024-03-11T18:36:44.715902+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:04:13.101160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.405270+0000","last_change":"2024-03-11T18:36:04.593604+0000","last_active":"2024-03-11T18:36:06.405270+0000","last_peered":"2024-03-11T18:36:06.405270+0000","last_clean":"2024-03-11T18:36:06.405270+0000","last_became_active":"2024-03-11T18:36:04.592304+0000","last_became_peered":"2024-03-11T18:36:04.592304+0000","last_unstale":"2024-03-11T18:36:06.405270+0000","last_undegraded":"2024-03-11T18:36:06.405270+0000","last_fullsized":"2024-03-11T18:36:06.405270+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:31:04.738940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.353525+0000","last_change":"2024-03-11T18:35:57.689404+0000","last_active":"2024-03-11T18:36:06.353525+0000","last_peered":"2024-03-11T18:36:06.353525+0000","last_clean":"2024-03-11T18:36:06.353525+0000","last_became_active":"2024-03-11T18:35:57.689139+0000","last_became_peered":"2024-03-11T18:35:57.689139+0000","last_unstale":"2024-03-11T18:36:06.353525+0000","last_undegraded":"2024-03-11T18:36:06.353525+0000","last_fullsized":"2024-03-11T18:36:06.353525+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:54:09.699371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"28'3","reported_seq":30,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:37.947614+0000","last_change":"2024-03-11T18:35:56.297682+0000","last_active":"2024-03-11T18:36:37.947614+0000","last_peered":"2024-03-11T18:36:37.947614+0000","last_clean":"2024-03-11T18:36:37.947614+0000","last_became_active":"2024-03-11T18:35:56.296426+0000","last_became_peered":"2024-03-11T18:35:56.296426+0000","last_unstale":"2024-03-11T18:36:37.947614+0000","last_undegraded":"2024-03-11T18:36:37.947614+0000","last_fullsized":"2024-03-11T18:36:37.947614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:28:50.050942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.403363+0000","last_change":"2024-03-11T18:36:04.590780+0000","last_active":"2024-03-11T18:36:06.403363+0000","last_peered":"2024-03-11T18:36:06.403363+0000","last_clean":"2024-03-11T18:36:06.403363+0000","last_became_active":"2024-03-11T18:36:04.590014+0000","last_became_peered":"2024-03-11T18:36:04.590014+0000","last_unstale":"2024-03-11T18:36:06.403363+0000","last_undegraded":"2024-03-11T18:36:06.403363+0000","last_fullsized":"2024-03-11T18:36:06.403363+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:51:13.205384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.407150+0000","last_change":"2024-03-11T18:35:57.671638+0000","last_active":"2024-03-11T18:36:06.407150+0000","last_peered":"2024-03-11T18:36:06.407150+0000","last_clean":"2024-03-11T18:36:06.407150+0000","last_became_active":"2024-03-11T18:35:57.670893+0000","last_became_peered":"2024-03-11T18:35:57.670893+0000","last_unstale":"2024-03-11T18:36:06.407150+0000","last_undegraded":"2024-03-11T18:36:06.407150+0000","last_fullsized":"2024-03-11T18:36:06.407150+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:31:21.395133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"28'7","reported_seq":34,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:44.701166+0000","last_change":"2024-03-11T18:35:56.251580+0000","last_active":"2024-03-11T18:36:44.701166+0000","last_peered":"2024-03-11T18:36:44.701166+0000","last_clean":"2024-03-11T18:36:44.701166+0000","last_became_active":"2024-03-11T18:35:56.251287+0000","last_became_peered":"2024-03-11T18:35:56.251287+0000","last_unstale":"2024-03-11T18:36:44.701166+0000","last_undegraded":"2024-03-11T18:36:44.701166+0000","last_fullsized":"2024-03-11T18:36:44.701166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:30:39.907360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.412899+0000","last_change":"2024-03-11T18:36:04.625563+0000","last_active":"2024-03-11T18:36:06.412899+0000","last_peered":"2024-03-11T18:36:06.412899+0000","last_clean":"2024-03-11T18:36:06.412899+0000","last_became_active":"2024-03-11T18:36:04.618911+0000","last_became_peered":"2024-03-11T18:36:04.618911+0000","last_unstale":"2024-03-11T18:36:06.412899+0000","last_undegraded":"2024-03-11T18:36:06.412899+0000","last_fullsized":"2024-03-11T18:36:06.412899+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:31:04.738940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.354510+0000","last_change":"2024-03-11T18:35:57.697641+0000","last_active":"2024-03-11T18:36:06.354510+0000","last_peered":"2024-03-11T18:36:06.354510+0000","last_clean":"2024-03-11T18:36:06.354510+0000","last_became_active":"2024-03-11T18:35:57.697263+0000","last_became_peered":"2024-03-11T18:35:57.697263+0000","last_unstale":"2024-03-11T18:36:06.354510+0000","last_undegraded":"2024-03-11T18:36:06.354510+0000","last_fullsized":"2024-03-11T18:36:06.354510+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:34:39.325572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"28'12","reported_seq":39,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:37.952962+0000","last_change":"2024-03-11T18:35:56.355612+0000","last_active":"2024-03-11T18:36:37.952962+0000","last_peered":"2024-03-11T18:36:37.952962+0000","last_clean":"2024-03-11T18:36:37.952962+0000","last_became_active":"2024-03-11T18:35:56.354827+0000","last_became_peered":"2024-03-11T18:35:56.354827+0000","last_unstale":"2024-03-11T18:36:37.952962+0000","last_undegraded":"2024-03-11T18:36:37.952962+0000","last_fullsized":"2024-03-11T18:36:37.952962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:06:02.844765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.358459+0000","last_change":"2024-03-11T18:36:04.617561+0000","last_active":"2024-03-11T18:36:06.358459+0000","last_peered":"2024-03-11T18:36:06.358459+0000","last_clean":"2024-03-11T18:36:06.358459+0000","last_became_active":"2024-03-11T18:36:04.616938+0000","last_became_peered":"2024-03-11T18:36:04.616938+0000","last_unstale":"2024-03-11T18:36:06.358459+0000","last_undegraded":"2024-03-11T18:36:06.358459+0000","last_fullsized":"2024-03-11T18:36:06.358459+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:16:10.508913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.412500+0000","last_change":"2024-03-11T18:35:57.679035+0000","last_active":"2024-03-11T18:36:06.412500+0000","last_peered":"2024-03-11T18:36:06.412500+0000","last_clean":"2024-03-11T18:36:06.412500+0000","last_became_active":"2024-03-11T18:35:57.678565+0000","last_became_peered":"2024-03-11T18:35:57.678565+0000","last_unstale":"2024-03-11T18:36:06.412500+0000","last_undegraded":"2024-03-11T18:36:06.412500+0000","last_fullsized":"2024-03-11T18:36:06.412500+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:18:05.980207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"28'4","reported_seq":31,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.358900+0000","last_change":"2024-03-11T18:35:56.288132+0000","last_active":"2024-03-11T18:36:06.358900+0000","last_peered":"2024-03-11T18:36:06.358900+0000","last_clean":"2024-03-11T18:36:06.358900+0000","last_became_active":"2024-03-11T18:35:56.287846+0000","last_became_peered":"2024-03-11T18:35:56.287846+0000","last_unstale":"2024-03-11T18:36:06.358900+0000","last_undegraded":"2024-03-11T18:36:06.358900+0000","last_fullsized":"2024-03-11T18:36:06.358900+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T18:51:00.795706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.411589+0000","last_change":"2024-03-11T18:36:04.643164+0000","last_active":"2024-03-11T18:36:06.411589+0000","last_peered":"2024-03-11T18:36:06.411589+0000","last_clean":"2024-03-11T18:36:06.411589+0000","last_became_active":"2024-03-11T18:36:04.642374+0000","last_became_peered":"2024-03-11T18:36:04.642374+0000","last_unstale":"2024-03-11T18:36:06.411589+0000","last_undegraded":"2024-03-11T18:36:06.411589+0000","last_fullsized":"2024-03-11T18:36:06.411589+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:25:56.815496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.405796+0000","last_change":"2024-03-11T18:35:57.636550+0000","last_active":"2024-03-11T18:36:06.405796+0000","last_peered":"2024-03-11T18:36:06.405796+0000","last_clean":"2024-03-11T18:36:06.405796+0000","last_became_active":"2024-03-11T18:35:57.636093+0000","last_became_peered":"2024-03-11T18:35:57.636093+0000","last_unstale":"2024-03-11T18:36:06.405796+0000","last_undegraded":"2024-03-11T18:36:06.405796+0000","last_fullsized":"2024-03-11T18:36:06.405796+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:33:31.344465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"28'2","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:37.946730+0000","last_change":"2024-03-11T18:35:56.355480+0000","last_active":"2024-03-11T18:36:37.946730+0000","last_peered":"2024-03-11T18:36:37.946730+0000","last_clean":"2024-03-11T18:36:37.946730+0000","last_became_active":"2024-03-11T18:35:56.354759+0000","last_became_peered":"2024-03-11T18:35:56.354759+0000","last_unstale":"2024-03-11T18:36:37.946730+0000","last_undegraded":"2024-03-11T18:36:37.946730+0000","last_fullsized":"2024-03-11T18:36:37.946730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:45:37.842725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.400682+0000","last_change":"2024-03-11T18:36:04.617364+0000","last_active":"2024-03-11T18:36:06.400682+0000","last_peered":"2024-03-11T18:36:06.400682+0000","last_clean":"2024-03-11T18:36:06.400682+0000","last_became_active":"2024-03-11T18:36:04.613159+0000","last_became_peered":"2024-03-11T18:36:04.613159+0000","last_unstale":"2024-03-11T18:36:06.400682+0000","last_undegraded":"2024-03-11T18:36:06.400682+0000","last_fullsized":"2024-03-11T18:36:06.400682+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:25:23.672189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.412882+0000","last_change":"2024-03-11T18:35:57.683426+0000","last_active":"2024-03-11T18:36:06.412882+0000","last_peered":"2024-03-11T18:36:06.412882+0000","last_clean":"2024-03-11T18:36:06.412882+0000","last_became_active":"2024-03-11T18:35:57.683150+0000","last_became_peered":"2024-03-11T18:35:57.683150+0000","last_unstale":"2024-03-11T18:36:06.412882+0000","last_undegraded":"2024-03-11T18:36:06.412882+0000","last_fullsized":"2024-03-11T18:36:06.412882+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:36:00.079075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"28'3","reported_seq":30,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:37.955896+0000","last_change":"2024-03-11T18:35:56.361613+0000","last_active":"2024-03-11T18:36:37.955896+0000","last_peered":"2024-03-11T18:36:37.955896+0000","last_clean":"2024-03-11T18:36:37.955896+0000","last_became_active":"2024-03-11T18:35:56.359629+0000","last_became_peered":"2024-03-11T18:35:56.359629+0000","last_unstale":"2024-03-11T18:36:37.955896+0000","last_undegraded":"2024-03-11T18:36:37.955896+0000","last_fullsized":"2024-03-11T18:36:37.955896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:03:06.585728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.401756+0000","last_change":"2024-03-11T18:36:04.608168+0000","last_active":"2024-03-11T18:36:06.401756+0000","last_peered":"2024-03-11T18:36:06.401756+0000","last_clean":"2024-03-11T18:36:06.401756+0000","last_became_active":"2024-03-11T18:36:04.607063+0000","last_became_peered":"2024-03-11T18:36:04.607063+0000","last_unstale":"2024-03-11T18:36:06.401756+0000","last_undegraded":"2024-03-11T18:36:06.401756+0000","last_fullsized":"2024-03-11T18:36:06.401756+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:53:41.996024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.405114+0000","last_change":"2024-03-11T18:36:04.607992+0000","last_active":"2024-03-11T18:36:06.405114+0000","last_peered":"2024-03-11T18:36:06.405114+0000","last_clean":"2024-03-11T18:36:06.405114+0000","last_became_active":"2024-03-11T18:36:04.607109+0000","last_became_peered":"2024-03-11T18:36:04.607109+0000","last_unstale":"2024-03-11T18:36:06.405114+0000","last_undegraded":"2024-03-11T18:36:06.405114+0000","last_fullsized":"2024-03-11T18:36:06.405114+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:14:37.678854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.357424+0000","last_change":"2024-03-11T18:35:57.680534+0000","last_active":"2024-03-11T18:36:06.357424+0000","last_peered":"2024-03-11T18:36:06.357424+0000","last_clean":"2024-03-11T18:36:06.357424+0000","last_became_active":"2024-03-11T18:35:57.679642+0000","last_became_peered":"2024-03-11T18:35:57.679642+0000","last_unstale":"2024-03-11T18:36:06.357424+0000","last_undegraded":"2024-03-11T18:36:06.357424+0000","last_fullsized":"2024-03-11T18:36:06.357424+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:25:49.707568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"28'1","reported_seq":28,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.409415+0000","last_change":"2024-03-11T18:35:56.303510+0000","last_active":"2024-03-11T18:36:06.409415+0000","last_peered":"2024-03-11T18:36:06.409415+0000","last_clean":"2024-03-11T18:36:06.409415+0000","last_became_active":"2024-03-11T18:35:56.302758+0000","last_became_peered":"2024-03-11T18:35:56.302758+0000","last_unstale":"2024-03-11T18:36:06.409415+0000","last_undegraded":"2024-03-11T18:36:06.409415+0000","last_fullsized":"2024-03-11T18:36:06.409415+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:03:32.297342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.412847+0000","last_change":"2024-03-11T18:36:04.618818+0000","last_active":"2024-03-11T18:36:06.412847+0000","last_peered":"2024-03-11T18:36:06.412847+0000","last_clean":"2024-03-11T18:36:06.412847+0000","last_became_active":"2024-03-11T18:36:04.617868+0000","last_became_peered":"2024-03-11T18:36:04.617868+0000","last_unstale":"2024-03-11T18:36:06.412847+0000","last_undegraded":"2024-03-11T18:36:06.412847+0000","last_fullsized":"2024-03-11T18:36:06.412847+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:16:10.508913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.400811+0000","last_change":"2024-03-11T18:35:57.690347+0000","last_active":"2024-03-11T18:36:06.400811+0000","last_peered":"2024-03-11T18:36:06.400811+0000","last_clean":"2024-03-11T18:36:06.400811+0000","last_became_active":"2024-03-11T18:35:57.689146+0000","last_became_peered":"2024-03-11T18:35:57.689146+0000","last_unstale":"2024-03-11T18:36:06.400811+0000","last_undegraded":"2024-03-11T18:36:06.400811+0000","last_fullsized":"2024-03-11T18:36:06.400811+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:03:56.260394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"28'2","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:44.532285+0000","last_change":"2024-03-11T18:35:56.348355+0000","last_active":"2024-03-11T18:36:44.532285+0000","last_peered":"2024-03-11T18:36:44.532285+0000","last_clean":"2024-03-11T18:36:44.532285+0000","last_became_active":"2024-03-11T18:35:56.347631+0000","last_became_peered":"2024-03-11T18:35:56.347631+0000","last_unstale":"2024-03-11T18:36:44.532285+0000","last_undegraded":"2024-03-11T18:36:44.532285+0000","last_fullsized":"2024-03-11T18:36:44.532285+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:06:02.844765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"28'3","reported_seq":30,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:44.716081+0000","last_change":"2024-03-11T18:35:56.361254+0000","last_active":"2024-03-11T18:36:44.716081+0000","last_peered":"2024-03-11T18:36:44.716081+0000","last_clean":"2024-03-11T18:36:44.716081+0000","last_became_active":"2024-03-11T18:35:56.360580+0000","last_became_peered":"2024-03-11T18:35:56.360580+0000","last_unstale":"2024-03-11T18:36:44.716081+0000","last_undegraded":"2024-03-11T18:36:44.716081+0000","last_fullsized":"2024-03-11T18:36:44.716081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:27:26.471550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.406163+0000","last_change":"2024-03-11T18:35:57.602778+0000","last_active":"2024-03-11T18:36:06.406163+0000","last_peered":"2024-03-11T18:36:06.406163+0000","last_clean":"2024-03-11T18:36:06.406163+0000","last_became_active":"2024-03-11T18:35:57.601990+0000","last_became_peered":"2024-03-11T18:35:57.601990+0000","last_unstale":"2024-03-11T18:36:06.406163+0000","last_undegraded":"2024-03-11T18:36:06.406163+0000","last_fullsized":"2024-03-11T18:36:06.406163+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:56:59.014871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.401528+0000","last_change":"2024-03-11T18:36:04.606098+0000","last_active":"2024-03-11T18:36:06.401528+0000","last_peered":"2024-03-11T18:36:06.401528+0000","last_clean":"2024-03-11T18:36:06.401528+0000","last_became_active":"2024-03-11T18:36:04.604823+0000","last_became_peered":"2024-03-11T18:36:04.604823+0000","last_unstale":"2024-03-11T18:36:06.401528+0000","last_undegraded":"2024-03-11T18:36:06.401528+0000","last_fullsized":"2024-03-11T18:36:06.401528+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:52:04.862452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"28'5","reported_seq":32,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:37.941879+0000","last_change":"2024-03-11T18:35:56.296312+0000","last_active":"2024-03-11T18:36:37.941879+0000","last_peered":"2024-03-11T18:36:37.941879+0000","last_clean":"2024-03-11T18:36:37.941879+0000","last_became_active":"2024-03-11T18:35:56.296090+0000","last_became_peered":"2024-03-11T18:35:56.296090+0000","last_unstale":"2024-03-11T18:36:37.941879+0000","last_undegraded":"2024-03-11T18:36:37.941879+0000","last_fullsized":"2024-03-11T18:36:37.941879+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:03:06.585728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.410741+0000","last_change":"2024-03-11T18:35:57.611809+0000","last_active":"2024-03-11T18:36:06.410741+0000","last_peered":"2024-03-11T18:36:06.410741+0000","last_clean":"2024-03-11T18:36:06.410741+0000","last_became_active":"2024-03-11T18:35:57.611097+0000","last_became_peered":"2024-03-11T18:35:57.611097+0000","last_unstale":"2024-03-11T18:36:06.410741+0000","last_undegraded":"2024-03-11T18:36:06.410741+0000","last_fullsized":"2024-03-11T18:36:06.410741+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:49:29.042114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.403540+0000","last_change":"2024-03-11T18:36:04.590064+0000","last_active":"2024-03-11T18:36:06.403540+0000","last_peered":"2024-03-11T18:36:06.403540+0000","last_clean":"2024-03-11T18:36:06.403540+0000","last_became_active":"2024-03-11T18:36:04.589037+0000","last_became_peered":"2024-03-11T18:36:04.589037+0000","last_unstale":"2024-03-11T18:36:06.403540+0000","last_undegraded":"2024-03-11T18:36:06.403540+0000","last_fullsized":"2024-03-11T18:36:06.403540+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:51:28.017349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.407866+0000","last_change":"2024-03-11T18:35:57.614944+0000","last_active":"2024-03-11T18:36:06.407866+0000","last_peered":"2024-03-11T18:36:06.407866+0000","last_clean":"2024-03-11T18:36:06.407866+0000","last_became_active":"2024-03-11T18:35:57.614144+0000","last_became_peered":"2024-03-11T18:35:57.614144+0000","last_unstale":"2024-03-11T18:36:06.407866+0000","last_undegraded":"2024-03-11T18:36:06.407866+0000","last_fullsized":"2024-03-11T18:36:06.407866+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:53:26.493265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.400811+0000","last_change":"2024-03-11T18:35:57.630211+0000","last_active":"2024-03-11T18:36:06.400811+0000","last_peered":"2024-03-11T18:36:06.400811+0000","last_clean":"2024-03-11T18:36:06.400811+0000","last_became_active":"2024-03-11T18:35:57.629599+0000","last_became_peered":"2024-03-11T18:35:57.629599+0000","last_unstale":"2024-03-11T18:36:06.400811+0000","last_undegraded":"2024-03-11T18:36:06.400811+0000","last_fullsized":"2024-03-11T18:36:06.400811+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:18:05.980207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.409629+0000","last_change":"2024-03-11T18:35:57.688700+0000","last_active":"2024-03-11T18:36:06.409629+0000","last_peered":"2024-03-11T18:36:06.409629+0000","last_clean":"2024-03-11T18:36:06.409629+0000","last_became_active":"2024-03-11T18:35:57.687968+0000","last_became_peered":"2024-03-11T18:35:57.687968+0000","last_unstale":"2024-03-11T18:36:06.409629+0000","last_undegraded":"2024-03-11T18:36:06.409629+0000","last_fullsized":"2024-03-11T18:36:06.409629+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:25:16.564261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.405998+0000","last_change":"2024-03-11T18:35:57.680730+0000","last_active":"2024-03-11T18:36:06.405998+0000","last_peered":"2024-03-11T18:36:06.405998+0000","last_clean":"2024-03-11T18:36:06.405998+0000","last_became_active":"2024-03-11T18:35:57.679935+0000","last_became_peered":"2024-03-11T18:35:57.679935+0000","last_unstale":"2024-03-11T18:36:06.405998+0000","last_undegraded":"2024-03-11T18:36:06.405998+0000","last_fullsized":"2024-03-11T18:36:06.405998+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:21:21.894805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.403523+0000","last_change":"2024-03-11T18:35:57.690799+0000","last_active":"2024-03-11T18:36:06.403523+0000","last_peered":"2024-03-11T18:36:06.403523+0000","last_clean":"2024-03-11T18:36:06.403523+0000","last_became_active":"2024-03-11T18:35:57.686623+0000","last_became_peered":"2024-03-11T18:35:57.686623+0000","last_unstale":"2024-03-11T18:36:06.403523+0000","last_undegraded":"2024-03-11T18:36:06.403523+0000","last_fullsized":"2024-03-11T18:36:06.403523+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:53:30.324898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.358254+0000","last_change":"2024-03-11T18:35:57.691955+0000","last_active":"2024-03-11T18:36:06.358254+0000","last_peered":"2024-03-11T18:36:06.358254+0000","last_clean":"2024-03-11T18:36:06.358254+0000","last_became_active":"2024-03-11T18:35:57.691413+0000","last_became_peered":"2024-03-11T18:35:57.691413+0000","last_unstale":"2024-03-11T18:36:06.358254+0000","last_undegraded":"2024-03-11T18:36:06.358254+0000","last_fullsized":"2024-03-11T18:36:06.358254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:31:19.941345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.410285+0000","last_change":"2024-03-11T18:35:57.684695+0000","last_active":"2024-03-11T18:36:06.410285+0000","last_peered":"2024-03-11T18:36:06.410285+0000","last_clean":"2024-03-11T18:36:06.410285+0000","last_became_active":"2024-03-11T18:35:57.684227+0000","last_became_peered":"2024-03-11T18:35:57.684227+0000","last_unstale":"2024-03-11T18:36:06.410285+0000","last_undegraded":"2024-03-11T18:36:06.410285+0000","last_fullsized":"2024-03-11T18:36:06.410285+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:28:51.265291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.405217+0000","last_change":"2024-03-11T18:35:57.683549+0000","last_active":"2024-03-11T18:36:06.405217+0000","last_peered":"2024-03-11T18:36:06.405217+0000","last_clean":"2024-03-11T18:36:06.405217+0000","last_became_active":"2024-03-11T18:35:57.680610+0000","last_became_peered":"2024-03-11T18:35:57.680610+0000","last_unstale":"2024-03-11T18:36:06.405217+0000","last_undegraded":"2024-03-11T18:36:06.405217+0000","last_fullsized":"2024-03-11T18:36:06.405217+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:25:52.716744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.409054+0000","last_change":"2024-03-11T18:35:57.686833+0000","last_active":"2024-03-11T18:36:06.409054+0000","last_peered":"2024-03-11T18:36:06.409054+0000","last_clean":"2024-03-11T18:36:06.409054+0000","last_became_active":"2024-03-11T18:35:57.686144+0000","last_became_peered":"2024-03-11T18:35:57.686144+0000","last_unstale":"2024-03-11T18:36:06.409054+0000","last_undegraded":"2024-03-11T18:36:06.409054+0000","last_fullsized":"2024-03-11T18:36:06.409054+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:16:03.400985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.401952+0000","last_change":"2024-03-11T18:35:57.681630+0000","last_active":"2024-03-11T18:36:06.401952+0000","last_peered":"2024-03-11T18:36:06.401952+0000","last_clean":"2024-03-11T18:36:06.401952+0000","last_became_active":"2024-03-11T18:35:57.678876+0000","last_became_peered":"2024-03-11T18:35:57.678876+0000","last_unstale":"2024-03-11T18:36:06.401952+0000","last_undegraded":"2024-03-11T18:36:06.401952+0000","last_fullsized":"2024-03-11T18:36:06.401952+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:03:33.511691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.409370+0000","last_change":"2024-03-11T18:35:57.679589+0000","last_active":"2024-03-11T18:36:06.409370+0000","last_peered":"2024-03-11T18:36:06.409370+0000","last_clean":"2024-03-11T18:36:06.409370+0000","last_became_active":"2024-03-11T18:35:57.678838+0000","last_became_peered":"2024-03-11T18:35:57.678838+0000","last_unstale":"2024-03-11T18:36:06.409370+0000","last_undegraded":"2024-03-11T18:36:06.409370+0000","last_fullsized":"2024-03-11T18:36:06.409370+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:54:09.699371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.404736+0000","last_change":"2024-03-11T18:35:57.674571+0000","last_active":"2024-03-11T18:36:06.404736+0000","last_peered":"2024-03-11T18:36:06.404736+0000","last_clean":"2024-03-11T18:36:06.404736+0000","last_became_active":"2024-03-11T18:35:57.671573+0000","last_became_peered":"2024-03-11T18:35:57.671573+0000","last_unstale":"2024-03-11T18:36:06.404736+0000","last_undegraded":"2024-03-11T18:36:06.404736+0000","last_fullsized":"2024-03-11T18:36:06.404736+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:35:10.403666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.410712+0000","last_change":"2024-03-11T18:35:57.615939+0000","last_active":"2024-03-11T18:36:06.410712+0000","last_peered":"2024-03-11T18:36:06.410712+0000","last_clean":"2024-03-11T18:36:06.410712+0000","last_became_active":"2024-03-11T18:35:57.615579+0000","last_became_peered":"2024-03-11T18:35:57.615579+0000","last_unstale":"2024-03-11T18:36:06.410712+0000","last_undegraded":"2024-03-11T18:36:06.410712+0000","last_fullsized":"2024-03-11T18:36:06.410712+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:03:33.511691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.355121+0000","last_change":"2024-03-11T18:35:57.688955+0000","last_active":"2024-03-11T18:36:06.355121+0000","last_peered":"2024-03-11T18:36:06.355121+0000","last_clean":"2024-03-11T18:36:06.355121+0000","last_became_active":"2024-03-11T18:35:57.688365+0000","last_became_peered":"2024-03-11T18:35:57.688365+0000","last_unstale":"2024-03-11T18:36:06.355121+0000","last_undegraded":"2024-03-11T18:36:06.355121+0000","last_fullsized":"2024-03-11T18:36:06.355121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:44:11.877541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.401903+0000","last_change":"2024-03-11T18:35:57.602044+0000","last_active":"2024-03-11T18:36:06.401903+0000","last_peered":"2024-03-11T18:36:06.401903+0000","last_clean":"2024-03-11T18:36:06.401903+0000","last_became_active":"2024-03-11T18:35:57.601334+0000","last_became_peered":"2024-03-11T18:35:57.601334+0000","last_unstale":"2024-03-11T18:36:06.401903+0000","last_undegraded":"2024-03-11T18:36:06.401903+0000","last_fullsized":"2024-03-11T18:36:06.401903+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:39:12.693619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.413873+0000","last_change":"2024-03-11T18:35:57.677114+0000","last_active":"2024-03-11T18:36:06.413873+0000","last_peered":"2024-03-11T18:36:06.413873+0000","last_clean":"2024-03-11T18:36:06.413873+0000","last_became_active":"2024-03-11T18:35:57.676075+0000","last_became_peered":"2024-03-11T18:35:57.676075+0000","last_unstale":"2024-03-11T18:36:06.413873+0000","last_undegraded":"2024-03-11T18:36:06.413873+0000","last_fullsized":"2024-03-11T18:36:06.413873+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:44:11.877541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.408799+0000","last_change":"2024-03-11T18:35:57.641392+0000","last_active":"2024-03-11T18:36:06.408799+0000","last_peered":"2024-03-11T18:36:06.408799+0000","last_clean":"2024-03-11T18:36:06.408799+0000","last_became_active":"2024-03-11T18:35:57.640934+0000","last_became_peered":"2024-03-11T18:35:57.640934+0000","last_unstale":"2024-03-11T18:36:06.408799+0000","last_undegraded":"2024-03-11T18:36:06.408799+0000","last_fullsized":"2024-03-11T18:36:06.408799+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:25:52.716744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.357659+0000","last_change":"2024-03-11T18:35:57.605291+0000","last_active":"2024-03-11T18:36:06.357659+0000","last_peered":"2024-03-11T18:36:06.357659+0000","last_clean":"2024-03-11T18:36:06.357659+0000","last_became_active":"2024-03-11T18:35:57.604645+0000","last_became_peered":"2024-03-11T18:35:57.604645+0000","last_unstale":"2024-03-11T18:36:06.357659+0000","last_undegraded":"2024-03-11T18:36:06.357659+0000","last_fullsized":"2024-03-11T18:36:06.357659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:03:33.511691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.404894+0000","last_change":"2024-03-11T18:35:57.503994+0000","last_active":"2024-03-11T18:36:06.404894+0000","last_peered":"2024-03-11T18:36:06.404894+0000","last_clean":"2024-03-11T18:36:06.404894+0000","last_became_active":"2024-03-11T18:35:57.503362+0000","last_became_peered":"2024-03-11T18:35:57.503362+0000","last_unstale":"2024-03-11T18:36:06.404894+0000","last_undegraded":"2024-03-11T18:36:06.404894+0000","last_fullsized":"2024-03-11T18:36:06.404894+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:03:58.875164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.402309+0000","last_change":"2024-03-11T18:35:57.679571+0000","last_active":"2024-03-11T18:36:06.402309+0000","last_peered":"2024-03-11T18:36:06.402309+0000","last_clean":"2024-03-11T18:36:06.402309+0000","last_became_active":"2024-03-11T18:35:57.678787+0000","last_became_peered":"2024-03-11T18:35:57.678787+0000","last_unstale":"2024-03-11T18:36:06.402309+0000","last_undegraded":"2024-03-11T18:36:06.402309+0000","last_fullsized":"2024-03-11T18:36:06.402309+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:13:40.392038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.412355+0000","last_change":"2024-03-11T18:35:57.613374+0000","last_active":"2024-03-11T18:36:06.412355+0000","last_peered":"2024-03-11T18:36:06.412355+0000","last_clean":"2024-03-11T18:36:06.412355+0000","last_became_active":"2024-03-11T18:35:57.613050+0000","last_became_peered":"2024-03-11T18:35:57.613050+0000","last_unstale":"2024-03-11T18:36:06.412355+0000","last_undegraded":"2024-03-11T18:36:06.412355+0000","last_fullsized":"2024-03-11T18:36:06.412355+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:50:39.923881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.411942+0000","last_change":"2024-03-11T18:35:57.636565+0000","last_active":"2024-03-11T18:36:06.411942+0000","last_peered":"2024-03-11T18:36:06.411942+0000","last_clean":"2024-03-11T18:36:06.411942+0000","last_became_active":"2024-03-11T18:35:57.635978+0000","last_became_peered":"2024-03-11T18:35:57.635978+0000","last_unstale":"2024-03-11T18:36:06.411942+0000","last_undegraded":"2024-03-11T18:36:06.411942+0000","last_fullsized":"2024-03-11T18:36:06.411942+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:03:56.260394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.410011+0000","last_change":"2024-03-11T18:35:57.641714+0000","last_active":"2024-03-11T18:36:06.410011+0000","last_peered":"2024-03-11T18:36:06.410011+0000","last_clean":"2024-03-11T18:36:06.410011+0000","last_became_active":"2024-03-11T18:35:57.641417+0000","last_became_peered":"2024-03-11T18:35:57.641417+0000","last_unstale":"2024-03-11T18:36:06.410011+0000","last_undegraded":"2024-03-11T18:36:06.410011+0000","last_fullsized":"2024-03-11T18:36:06.410011+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:25:49.707568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.354140+0000","last_change":"2024-03-11T18:35:57.689703+0000","last_active":"2024-03-11T18:36:06.354140+0000","last_peered":"2024-03-11T18:36:06.354140+0000","last_clean":"2024-03-11T18:36:06.354140+0000","last_became_active":"2024-03-11T18:35:57.689056+0000","last_became_peered":"2024-03-11T18:35:57.689056+0000","last_unstale":"2024-03-11T18:36:06.354140+0000","last_undegraded":"2024-03-11T18:36:06.354140+0000","last_fullsized":"2024-03-11T18:36:06.354140+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:41:42.263652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.400467+0000","last_change":"2024-03-11T18:35:57.684305+0000","last_active":"2024-03-11T18:36:06.400467+0000","last_peered":"2024-03-11T18:36:06.400467+0000","last_clean":"2024-03-11T18:36:06.400467+0000","last_became_active":"2024-03-11T18:35:57.682947+0000","last_became_peered":"2024-03-11T18:35:57.682947+0000","last_unstale":"2024-03-11T18:36:06.400467+0000","last_undegraded":"2024-03-11T18:36:06.400467+0000","last_fullsized":"2024-03-11T18:36:06.400467+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:53:26.493265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.352832+0000","last_change":"2024-03-11T18:35:57.692717+0000","last_active":"2024-03-11T18:36:06.352832+0000","last_peered":"2024-03-11T18:36:06.352832+0000","last_clean":"2024-03-11T18:36:06.352832+0000","last_became_active":"2024-03-11T18:35:57.692504+0000","last_became_peered":"2024-03-11T18:35:57.692504+0000","last_unstale":"2024-03-11T18:36:06.352832+0000","last_undegraded":"2024-03-11T18:36:06.352832+0000","last_fullsized":"2024-03-11T18:36:06.352832+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:25:52.716744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.409110+0000","last_change":"2024-03-11T18:35:57.640483+0000","last_active":"2024-03-11T18:36:06.409110+0000","last_peered":"2024-03-11T18:36:06.409110+0000","last_clean":"2024-03-11T18:36:06.409110+0000","last_became_active":"2024-03-11T18:35:57.639886+0000","last_became_peered":"2024-03-11T18:35:57.639886+0000","last_unstale":"2024-03-11T18:36:06.409110+0000","last_undegraded":"2024-03-11T18:36:06.409110+0000","last_fullsized":"2024-03-11T18:36:06.409110+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:34:39.325572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.352837+0000","last_change":"2024-03-11T18:35:57.687096+0000","last_active":"2024-03-11T18:36:06.352837+0000","last_peered":"2024-03-11T18:36:06.352837+0000","last_clean":"2024-03-11T18:36:06.352837+0000","last_became_active":"2024-03-11T18:35:57.686771+0000","last_became_peered":"2024-03-11T18:35:57.686771+0000","last_unstale":"2024-03-11T18:36:06.352837+0000","last_undegraded":"2024-03-11T18:36:06.352837+0000","last_fullsized":"2024-03-11T18:36:06.352837+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:16:03.400985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.409284+0000","last_change":"2024-03-11T18:35:57.694707+0000","last_active":"2024-03-11T18:36:06.409284+0000","last_peered":"2024-03-11T18:36:06.409284+0000","last_clean":"2024-03-11T18:36:06.409284+0000","last_became_active":"2024-03-11T18:35:57.694104+0000","last_became_peered":"2024-03-11T18:35:57.694104+0000","last_unstale":"2024-03-11T18:36:06.409284+0000","last_undegraded":"2024-03-11T18:36:06.409284+0000","last_fullsized":"2024-03-11T18:36:06.409284+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:47:28.579467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.405456+0000","last_change":"2024-03-11T18:35:57.538548+0000","last_active":"2024-03-11T18:36:06.405456+0000","last_peered":"2024-03-11T18:36:06.405456+0000","last_clean":"2024-03-11T18:36:06.405456+0000","last_became_active":"2024-03-11T18:35:57.537704+0000","last_became_peered":"2024-03-11T18:35:57.537704+0000","last_unstale":"2024-03-11T18:36:06.405456+0000","last_undegraded":"2024-03-11T18:36:06.405456+0000","last_fullsized":"2024-03-11T18:36:06.405456+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:40:52.251981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.359086+0000","last_change":"2024-03-11T18:35:57.507126+0000","last_active":"2024-03-11T18:36:06.359086+0000","last_peered":"2024-03-11T18:36:06.359086+0000","last_clean":"2024-03-11T18:36:06.359086+0000","last_became_active":"2024-03-11T18:35:57.506429+0000","last_became_peered":"2024-03-11T18:35:57.506429+0000","last_unstale":"2024-03-11T18:36:06.359086+0000","last_undegraded":"2024-03-11T18:36:06.359086+0000","last_fullsized":"2024-03-11T18:36:06.359086+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:28:51.265291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.353858+0000","last_change":"2024-03-11T18:35:57.687817+0000","last_active":"2024-03-11T18:36:06.353858+0000","last_peered":"2024-03-11T18:36:06.353858+0000","last_clean":"2024-03-11T18:36:06.353858+0000","last_became_active":"2024-03-11T18:35:57.687532+0000","last_became_peered":"2024-03-11T18:35:57.687532+0000","last_unstale":"2024-03-11T18:36:06.353858+0000","last_undegraded":"2024-03-11T18:36:06.353858+0000","last_fullsized":"2024-03-11T18:36:06.353858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:47:28.579467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7486,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":133,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":167936,"data_stored":14938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":116,"ondisk_log_size":116,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476761,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343940,"statfs":{"total":96636764160,"available":96608194560,"internally_reserved":0,"allocated":905216,"data_stored":708231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476761,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":311296,"data_stored":121067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476759,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":311296,"data_stored":117925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476760,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":307200,"data_stored":117261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509463,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":315392,"data_stored":121101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509463,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":311296,"data_stored":117365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509464,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":315392,"data_stored":117491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542166,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343940,"statfs":{"total":96636764160,"available":96608194560,"internally_reserved":0,"allocated":905216,"data_stored":707609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-11T18:36:50.927 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-11T18:36:51.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.072+0000 7f98ec5ba640 1 Processor -- start 2024-03-11T18:36:51.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.072+0000 7f98ec5ba640 1 -- start start 2024-03-11T18:36:51.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.072+0000 7f98ec5ba640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f98e40c65b0 0x7f98e40c37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:51.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.072+0000 7f98ec5ba640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f98e40c3d30 0x7f98e40cc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:51.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.072+0000 7f98ec5ba640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98e40c5e60 con 0x7f98e4150a90 2024-03-11T18:36:51.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.072+0000 7f98ec5ba640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98e4156a60 con 0x7f98e40c3d30 2024-03-11T18:36:51.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.072+0000 7f98ec5ba640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e40c55c0 con 0x7f98e40c65b0 2024-03-11T18:36:51.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.072+0000 7f98e9b55640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f98e40c65b0 0x7f98e40c37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:51.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.072+0000 7f98ea356640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f98e4150a90 0x7f98e4157220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.90:47708/0 (socket says 172.21.15.90:47708) 2024-03-11T18:36:51.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.072+0000 7f98ea356640 1 -- 172.21.15.90:0/3301380163 learned_addr learned my addr 172.21.15.90:0/3301380163 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T18:36:51.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.080+0000 7f98e9b55640 1 -- 172.21.15.90:0/3301380163 >> v2:172.21.15.90:3300/0 conn(0x7f98e40c3d30 msgr2=0x7f98e40cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:51.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.080+0000 7f98e9b55640 1 --2- 172.21.15.90:0/3301380163 >> v2:172.21.15.90:3300/0 conn(0x7f98e40c3d30 0x7f98e40cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:51.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.080+0000 7f98e9b55640 1 -- 172.21.15.90:0/3301380163 >> v1:172.21.15.92:6789/0 conn(0x7f98e4150a90 legacy=0x7f98e4157220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:51.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.080+0000 7f98e9b55640 1 -- 172.21.15.90:0/3301380163 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e40c58a0 con 0x7f98e40c65b0 2024-03-11T18:36:51.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.088+0000 7f98e9b55640 1 --2- 172.21.15.90:0/3301380163 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f98e40c65b0 0x7f98e40c37f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f98d80097f0 tx=0x7f98d801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6dbe1b2b00dc943e server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:51.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.200+0000 7f98e9354640 1 -- 172.21.15.90:0/3301380163 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d80ab600 con 0x7f98e40c65b0 2024-03-11T18:36:51.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.200+0000 7f98e9354640 1 -- 172.21.15.90:0/3301380163 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d80a7050 con 0x7f98e40c65b0 2024-03-11T18:36:51.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.200+0000 7f98e9354640 1 -- 172.21.15.90:0/3301380163 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d80ab980 con 0x7f98e40c65b0 2024-03-11T18:36:51.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.200+0000 7f98ec5ba640 1 -- 172.21.15.90:0/3301380163 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f98e40c65b0 msgr2=0x7f98e40c37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:51.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.200+0000 7f98ec5ba640 1 --2- 172.21.15.90:0/3301380163 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f98e40c65b0 0x7f98e40c37f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f98d80097f0 tx=0x7f98d801c4a0 comp rx=0 tx=0).stop 2024-03-11T18:36:51.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.200+0000 7f98ec5ba640 1 -- 172.21.15.90:0/3301380163 shutdown_connections 2024-03-11T18:36:51.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.200+0000 7f98ec5ba640 1 --2- 172.21.15.90:0/3301380163 >> v2:172.21.15.90:3300/0 conn(0x7f98e40c3d30 0x7f98e40cc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:51.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.200+0000 7f98ec5ba640 1 --2- 172.21.15.90:0/3301380163 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f98e40c65b0 0x7f98e40c37f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:51.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.200+0000 7f98ec5ba640 1 -- 172.21.15.90:0/3301380163 >> 172.21.15.90:0/3301380163 conn(0x7f98e4071aa0 msgr2=0x7f98e4071eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:51.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.200+0000 7f98ec5ba640 1 -- 172.21.15.90:0/3301380163 shutdown_connections 2024-03-11T18:36:51.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.200+0000 7f98ec5ba640 1 -- 172.21.15.90:0/3301380163 wait complete. 2024-03-11T18:36:51.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.204+0000 7f98ec5ba640 1 Processor -- start 2024-03-11T18:36:51.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.204+0000 7f98ec5ba640 1 -- start start 2024-03-11T18:36:51.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.204+0000 7f98ec5ba640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f98e40c3d30 0x7f98e41455f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:51.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.204+0000 7f98ec5ba640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f98e40c65b0 0x7f98e40cb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:51.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.204+0000 7f98ec5ba640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98e4150a90 0x7f98e40ca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:51.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.204+0000 7f98ec5ba640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e41f1a80 con 0x7f98e40c65b0 2024-03-11T18:36:51.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.204+0000 7f98ec5ba640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e40cc5a0 con 0x7f98e4150a90 2024-03-11T18:36:51.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.204+0000 7f98ec5ba640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e41f1f10 con 0x7f98e40c3d30 2024-03-11T18:36:51.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.204+0000 7f98e9b55640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f98e40c65b0 0x7f98e40cb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:51.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.204+0000 7f98e9b55640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f98e40c65b0 0x7f98e40cb9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48402/0 (socket says 172.21.15.90:48402) 2024-03-11T18:36:51.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.204+0000 7f98e9b55640 1 -- 172.21.15.90:0/3044759192 learned_addr learned my addr 172.21.15.90:0/3044759192 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:51.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.208+0000 7f98e9b55640 1 -- 172.21.15.90:0/3044759192 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f98e40c3d30 msgr2=0x7f98e41455f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:51.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.208+0000 7f98e9b55640 1 --2- 172.21.15.90:0/3044759192 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f98e40c3d30 0x7f98e41455f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:51.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.208+0000 7f98e9b55640 1 -- 172.21.15.90:0/3044759192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98e4150a90 msgr2=0x7f98e40ca0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:51.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.208+0000 7f98e9b55640 1 --2- 172.21.15.90:0/3044759192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98e4150a90 0x7f98e40ca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:51.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.208+0000 7f98e9b55640 1 -- 172.21.15.90:0/3044759192 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d8009420 con 0x7f98e40c65b0 2024-03-11T18:36:51.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.212+0000 7f98e9b55640 1 --2- 172.21.15.90:0/3044759192 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f98e40c65b0 0x7f98e40cb9c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f98d80ae040 tx=0x7f98d8019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:51.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.216+0000 7f98e2ffd640 1 -- 172.21.15.90:0/3044759192 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d80b4610 con 0x7f98e40c65b0 2024-03-11T18:36:51.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.216+0000 7f98ec5ba640 1 -- 172.21.15.90:0/3044759192 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98e40cc2c0 con 0x7f98e40c65b0 2024-03-11T18:36:51.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.216+0000 7f98ec5ba640 1 -- 172.21.15.90:0/3044759192 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98e41f1430 con 0x7f98e40c65b0 2024-03-11T18:36:51.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.216+0000 7f98ec5ba640 1 -- 172.21.15.90:0/3044759192 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98e404eef0 con 0x7f98e40c65b0 2024-03-11T18:36:51.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.216+0000 7f98e2ffd640 1 -- 172.21.15.90:0/3044759192 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d80abe90 con 0x7f98e40c65b0 2024-03-11T18:36:51.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.216+0000 7f98e2ffd640 1 -- 172.21.15.90:0/3044759192 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d80b4970 con 0x7f98e40c65b0 2024-03-11T18:36:51.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.536+0000 7f98e2ffd640 1 -- 172.21.15.90:0/3044759192 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f98d80b2040 con 0x7f98e40c65b0 2024-03-11T18:36:51.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.536+0000 7f98e2ffd640 1 --2- 172.21.15.90:0/3044759192 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f98d0066e70 0x7f98d0069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:51.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.536+0000 7f98ea356640 1 --2- 172.21.15.90:0/3044759192 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f98d0066e70 0x7f98d0069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:51.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.536+0000 7f98ea356640 1 --2- 172.21.15.90:0/3044759192 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f98d0066e70 0x7f98d0069330 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f98d4000f70 tx=0x7f98d4007a90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:51.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.564+0000 7f98e2ffd640 1 -- 172.21.15.90:0/3044759192 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f98d80b3e70 con 0x7f98e40c65b0 2024-03-11T18:36:51.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.564+0000 7f98e2ffd640 1 -- 172.21.15.90:0/3044759192 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f98d80ec3c0 con 0x7f98e40c65b0 2024-03-11T18:36:51.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.736+0000 7f98ec5ba640 1 -- 172.21.15.90:0/3044759192 --> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f98e40c61e0 con 0x7f98d0066e70 2024-03-11T18:36:51.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.744+0000 7f98e2ffd640 1 -- 172.21.15.90:0/3044759192 <== mgr.4101 v2:172.21.15.90:6800/1999720161 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332628 (secure 0 0 0) 0x7f98e40c61e0 con 0x7f98d0066e70 2024-03-11T18:36:51.754 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:36:51.756 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-03-11T18:36:51.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.748+0000 7f98ec5ba640 1 -- 172.21.15.90:0/3044759192 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f98d0066e70 msgr2=0x7f98d0069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:51.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.748+0000 7f98ec5ba640 1 --2- 172.21.15.90:0/3044759192 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f98d0066e70 0x7f98d0069330 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f98d4000f70 tx=0x7f98d4007a90 comp rx=0 tx=0).stop 2024-03-11T18:36:51.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.748+0000 7f98ec5ba640 1 -- 172.21.15.90:0/3044759192 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f98e40c65b0 msgr2=0x7f98e40cb9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:51.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.748+0000 7f98ec5ba640 1 --2- 172.21.15.90:0/3044759192 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f98e40c65b0 0x7f98e40cb9c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f98d80ae040 tx=0x7f98d8019240 comp rx=0 tx=0).stop 2024-03-11T18:36:51.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.748+0000 7f98ec5ba640 1 -- 172.21.15.90:0/3044759192 shutdown_connections 2024-03-11T18:36:51.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.748+0000 7f98ec5ba640 1 --2- 172.21.15.90:0/3044759192 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f98d0066e70 0x7f98d0069330 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:51.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.748+0000 7f98ec5ba640 1 --2- 172.21.15.90:0/3044759192 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f98e4150a90 0x7f98e40ca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:51.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.748+0000 7f98ec5ba640 1 --2- 172.21.15.90:0/3044759192 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f98e40c65b0 0x7f98e40cb9c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:51.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.748+0000 7f98ec5ba640 1 --2- 172.21.15.90:0/3044759192 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f98e40c3d30 0x7f98e41455f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:51.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.748+0000 7f98ec5ba640 1 -- 172.21.15.90:0/3044759192 >> 172.21.15.90:0/3044759192 conn(0x7f98e4071aa0 msgr2=0x7f98e4143ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:51.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.748+0000 7f98ec5ba640 1 -- 172.21.15.90:0/3044759192 shutdown_connections 2024-03-11T18:36:51.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.748+0000 7f98ec5ba640 1 -- 172.21.15.90:0/3044759192 wait complete. 2024-03-11T18:36:51.772 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":72,"stamp":"2024-03-11T18:36:49.810111+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597854,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":292,"num_write_kb":2460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":233,"ondisk_log_size":233,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219724,"kb_used_data":3596,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754996,"statfs":{"total":773094113280,"available":772869115904,"internally_reserved":0,"allocated":3682304,"data_stored":2128050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":221236788},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2846,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":84,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005186"},"pg_stats":[{"pgid":"2.1f","version":"28'11","reported_seq":38,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:48.128889+0000","last_change":"2024-03-11T18:35:56.290140+0000","last_active":"2024-03-11T18:36:48.128889+0000","last_peered":"2024-03-11T18:36:48.128889+0000","last_clean":"2024-03-11T18:36:48.128889+0000","last_became_active":"2024-03-11T18:35:56.289178+0000","last_became_peered":"2024-03-11T18:35:56.289178+0000","last_unstale":"2024-03-11T18:36:48.128889+0000","last_undegraded":"2024-03-11T18:36:48.128889+0000","last_fullsized":"2024-03-11T18:36:48.128889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:59:46.211033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.405302+0000","last_change":"2024-03-11T18:35:57.676047+0000","last_active":"2024-03-11T18:36:06.405302+0000","last_peered":"2024-03-11T18:36:06.405302+0000","last_clean":"2024-03-11T18:36:06.405302+0000","last_became_active":"2024-03-11T18:35:57.675105+0000","last_became_peered":"2024-03-11T18:35:57.675105+0000","last_unstale":"2024-03-11T18:36:06.405302+0000","last_undegraded":"2024-03-11T18:36:06.405302+0000","last_fullsized":"2024-03-11T18:36:06.405302+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:45:39.057074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.401561+0000","last_change":"2024-03-11T18:36:04.607351+0000","last_active":"2024-03-11T18:36:06.401561+0000","last_peered":"2024-03-11T18:36:06.401561+0000","last_clean":"2024-03-11T18:36:06.401561+0000","last_became_active":"2024-03-11T18:36:04.599562+0000","last_became_peered":"2024-03-11T18:36:04.599562+0000","last_unstale":"2024-03-11T18:36:06.401561+0000","last_undegraded":"2024-03-11T18:36:06.401561+0000","last_fullsized":"2024-03-11T18:36:06.401561+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T18:57:51.134410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"28'2","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.405696+0000","last_change":"2024-03-11T18:35:56.291253+0000","last_active":"2024-03-11T18:36:06.405696+0000","last_peered":"2024-03-11T18:36:06.405696+0000","last_clean":"2024-03-11T18:36:06.405696+0000","last_became_active":"2024-03-11T18:35:56.290862+0000","last_became_peered":"2024-03-11T18:35:56.290862+0000","last_unstale":"2024-03-11T18:36:06.405696+0000","last_undegraded":"2024-03-11T18:36:06.405696+0000","last_fullsized":"2024-03-11T18:36:06.405696+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:36:17.809744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.405636+0000","last_change":"2024-03-11T18:35:57.636861+0000","last_active":"2024-03-11T18:36:06.405636+0000","last_peered":"2024-03-11T18:36:06.405636+0000","last_clean":"2024-03-11T18:36:06.405636+0000","last_became_active":"2024-03-11T18:35:57.636111+0000","last_became_peered":"2024-03-11T18:35:57.636111+0000","last_unstale":"2024-03-11T18:36:06.405636+0000","last_undegraded":"2024-03-11T18:36:06.405636+0000","last_fullsized":"2024-03-11T18:36:06.405636+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:23:11.238170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.408420+0000","last_change":"2024-03-11T18:36:04.643426+0000","last_active":"2024-03-11T18:36:06.408420+0000","last_peered":"2024-03-11T18:36:06.408420+0000","last_clean":"2024-03-11T18:36:06.408420+0000","last_became_active":"2024-03-11T18:36:04.642597+0000","last_became_peered":"2024-03-11T18:36:04.642597+0000","last_unstale":"2024-03-11T18:36:06.408420+0000","last_undegraded":"2024-03-11T18:36:06.408420+0000","last_fullsized":"2024-03-11T18:36:06.408420+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:31:27.049273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"28'4","reported_seq":31,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:44.536276+0000","last_change":"2024-03-11T18:35:56.355147+0000","last_active":"2024-03-11T18:36:44.536276+0000","last_peered":"2024-03-11T18:36:44.536276+0000","last_clean":"2024-03-11T18:36:44.536276+0000","last_became_active":"2024-03-11T18:35:56.354388+0000","last_became_peered":"2024-03-11T18:35:56.354388+0000","last_unstale":"2024-03-11T18:36:44.536276+0000","last_undegraded":"2024-03-11T18:36:44.536276+0000","last_fullsized":"2024-03-11T18:36:44.536276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:03:55.046045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.410658+0000","last_change":"2024-03-11T18:35:57.641486+0000","last_active":"2024-03-11T18:36:06.410658+0000","last_peered":"2024-03-11T18:36:06.410658+0000","last_clean":"2024-03-11T18:36:06.410658+0000","last_became_active":"2024-03-11T18:35:57.641052+0000","last_became_peered":"2024-03-11T18:35:57.641052+0000","last_unstale":"2024-03-11T18:36:06.410658+0000","last_undegraded":"2024-03-11T18:36:06.410658+0000","last_fullsized":"2024-03-11T18:36:06.410658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:29:45.662029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.411193+0000","last_change":"2024-03-11T18:36:04.629291+0000","last_active":"2024-03-11T18:36:06.411193+0000","last_peered":"2024-03-11T18:36:06.411193+0000","last_clean":"2024-03-11T18:36:06.411193+0000","last_became_active":"2024-03-11T18:36:04.624110+0000","last_became_peered":"2024-03-11T18:36:04.624110+0000","last_unstale":"2024-03-11T18:36:06.411193+0000","last_undegraded":"2024-03-11T18:36:06.411193+0000","last_fullsized":"2024-03-11T18:36:06.411193+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:04:03.368322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"23'2","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.353273+0000","last_change":"2024-03-11T18:35:56.298903+0000","last_active":"2024-03-11T18:36:06.353273+0000","last_peered":"2024-03-11T18:36:06.353273+0000","last_clean":"2024-03-11T18:36:06.353273+0000","last_became_active":"2024-03-11T18:35:56.297582+0000","last_became_peered":"2024-03-11T18:35:56.297582+0000","last_unstale":"2024-03-11T18:36:06.353273+0000","last_undegraded":"2024-03-11T18:36:06.353273+0000","last_fullsized":"2024-03-11T18:36:06.353273+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:53:25.278916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.413016+0000","last_change":"2024-03-11T18:35:57.640122+0000","last_active":"2024-03-11T18:36:06.413016+0000","last_peered":"2024-03-11T18:36:06.413016+0000","last_clean":"2024-03-11T18:36:06.413016+0000","last_became_active":"2024-03-11T18:35:57.639770+0000","last_became_peered":"2024-03-11T18:35:57.639770+0000","last_unstale":"2024-03-11T18:36:06.413016+0000","last_undegraded":"2024-03-11T18:36:06.413016+0000","last_fullsized":"2024-03-11T18:36:06.413016+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:39:25.767184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.410463+0000","last_change":"2024-03-11T18:36:04.627149+0000","last_active":"2024-03-11T18:36:06.410463+0000","last_peered":"2024-03-11T18:36:06.410463+0000","last_clean":"2024-03-11T18:36:06.410463+0000","last_became_active":"2024-03-11T18:36:04.626108+0000","last_became_peered":"2024-03-11T18:36:04.626108+0000","last_unstale":"2024-03-11T18:36:06.410463+0000","last_undegraded":"2024-03-11T18:36:06.410463+0000","last_fullsized":"2024-03-11T18:36:06.410463+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:03:40.619618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.410013+0000","last_change":"2024-03-11T18:35:57.616739+0000","last_active":"2024-03-11T18:36:06.410013+0000","last_peered":"2024-03-11T18:36:06.410013+0000","last_clean":"2024-03-11T18:36:06.410013+0000","last_became_active":"2024-03-11T18:35:57.616398+0000","last_became_peered":"2024-03-11T18:35:57.616398+0000","last_unstale":"2024-03-11T18:36:06.410013+0000","last_undegraded":"2024-03-11T18:36:06.410013+0000","last_fullsized":"2024-03-11T18:36:06.410013+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:53:30.324898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"28'4","reported_seq":31,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:37.714831+0000","last_change":"2024-03-11T18:35:56.304762+0000","last_active":"2024-03-11T18:36:37.714831+0000","last_peered":"2024-03-11T18:36:37.714831+0000","last_clean":"2024-03-11T18:36:37.714831+0000","last_became_active":"2024-03-11T18:35:56.302803+0000","last_became_peered":"2024-03-11T18:35:56.302803+0000","last_unstale":"2024-03-11T18:36:37.714831+0000","last_undegraded":"2024-03-11T18:36:37.714831+0000","last_fullsized":"2024-03-11T18:36:37.714831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:31:18.726996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.412227+0000","last_change":"2024-03-11T18:36:04.626968+0000","last_active":"2024-03-11T18:36:06.412227+0000","last_peered":"2024-03-11T18:36:06.412227+0000","last_clean":"2024-03-11T18:36:06.412227+0000","last_became_active":"2024-03-11T18:36:04.626337+0000","last_became_peered":"2024-03-11T18:36:04.626337+0000","last_unstale":"2024-03-11T18:36:06.412227+0000","last_undegraded":"2024-03-11T18:36:06.412227+0000","last_fullsized":"2024-03-11T18:36:06.412227+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:53:37.432826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.402625+0000","last_change":"2024-03-11T18:35:57.689835+0000","last_active":"2024-03-11T18:36:06.402625+0000","last_peered":"2024-03-11T18:36:06.402625+0000","last_clean":"2024-03-11T18:36:06.402625+0000","last_became_active":"2024-03-11T18:35:57.689039+0000","last_became_peered":"2024-03-11T18:35:57.689039+0000","last_unstale":"2024-03-11T18:36:06.402625+0000","last_undegraded":"2024-03-11T18:36:06.402625+0000","last_fullsized":"2024-03-11T18:36:06.402625+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:38:27.733669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"28'1","reported_seq":28,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.358940+0000","last_change":"2024-03-11T18:35:56.291070+0000","last_active":"2024-03-11T18:36:06.358940+0000","last_peered":"2024-03-11T18:36:06.358940+0000","last_clean":"2024-03-11T18:36:06.358940+0000","last_became_active":"2024-03-11T18:35:56.290504+0000","last_became_peered":"2024-03-11T18:35:56.290504+0000","last_unstale":"2024-03-11T18:36:06.358940+0000","last_undegraded":"2024-03-11T18:36:06.358940+0000","last_fullsized":"2024-03-11T18:36:06.358940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:29:31.660243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.411988+0000","last_change":"2024-03-11T18:36:04.626150+0000","last_active":"2024-03-11T18:36:06.411988+0000","last_peered":"2024-03-11T18:36:06.411988+0000","last_clean":"2024-03-11T18:36:06.411988+0000","last_became_active":"2024-03-11T18:36:04.625084+0000","last_became_peered":"2024-03-11T18:36:04.625084+0000","last_unstale":"2024-03-11T18:36:06.411988+0000","last_undegraded":"2024-03-11T18:36:06.411988+0000","last_fullsized":"2024-03-11T18:36:06.411988+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:28:58.373219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.400936+0000","last_change":"2024-03-11T18:35:57.684535+0000","last_active":"2024-03-11T18:36:06.400936+0000","last_peered":"2024-03-11T18:36:06.400936+0000","last_clean":"2024-03-11T18:36:06.400936+0000","last_became_active":"2024-03-11T18:35:57.681098+0000","last_became_peered":"2024-03-11T18:35:57.681098+0000","last_unstale":"2024-03-11T18:36:06.400936+0000","last_undegraded":"2024-03-11T18:36:06.400936+0000","last_fullsized":"2024-03-11T18:36:06.400936+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:25:49.707568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"28'3","reported_seq":30,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:37.719417+0000","last_change":"2024-03-11T18:35:56.290745+0000","last_active":"2024-03-11T18:36:37.719417+0000","last_peered":"2024-03-11T18:36:37.719417+0000","last_clean":"2024-03-11T18:36:37.719417+0000","last_became_active":"2024-03-11T18:35:56.290006+0000","last_became_peered":"2024-03-11T18:35:56.290006+0000","last_unstale":"2024-03-11T18:36:37.719417+0000","last_undegraded":"2024-03-11T18:36:37.719417+0000","last_fullsized":"2024-03-11T18:36:37.719417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:06:02.844765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.359859+0000","last_change":"2024-03-11T18:36:04.620689+0000","last_active":"2024-03-11T18:36:06.359859+0000","last_peered":"2024-03-11T18:36:06.359859+0000","last_clean":"2024-03-11T18:36:06.359859+0000","last_became_active":"2024-03-11T18:36:04.620282+0000","last_became_peered":"2024-03-11T18:36:04.620282+0000","last_unstale":"2024-03-11T18:36:06.359859+0000","last_undegraded":"2024-03-11T18:36:06.359859+0000","last_fullsized":"2024-03-11T18:36:06.359859+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:54:16.807299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.413230+0000","last_change":"2024-03-11T18:35:57.615234+0000","last_active":"2024-03-11T18:36:06.413230+0000","last_peered":"2024-03-11T18:36:06.413230+0000","last_clean":"2024-03-11T18:36:06.413230+0000","last_became_active":"2024-03-11T18:35:57.614885+0000","last_became_peered":"2024-03-11T18:35:57.614885+0000","last_unstale":"2024-03-11T18:36:06.413230+0000","last_undegraded":"2024-03-11T18:36:06.413230+0000","last_fullsized":"2024-03-11T18:36:06.413230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:22:13.730630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"28'2","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:37.714251+0000","last_change":"2024-03-11T18:35:56.304617+0000","last_active":"2024-03-11T18:36:37.714251+0000","last_peered":"2024-03-11T18:36:37.714251+0000","last_clean":"2024-03-11T18:36:37.714251+0000","last_became_active":"2024-03-11T18:35:56.301698+0000","last_became_peered":"2024-03-11T18:35:56.301698+0000","last_unstale":"2024-03-11T18:36:37.714251+0000","last_undegraded":"2024-03-11T18:36:37.714251+0000","last_fullsized":"2024-03-11T18:36:37.714251+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:29:31.660243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.412949+0000","last_change":"2024-03-11T18:36:04.643279+0000","last_active":"2024-03-11T18:36:06.412949+0000","last_peered":"2024-03-11T18:36:06.412949+0000","last_clean":"2024-03-11T18:36:06.412949+0000","last_became_active":"2024-03-11T18:36:04.641983+0000","last_became_peered":"2024-03-11T18:36:04.641983+0000","last_unstale":"2024-03-11T18:36:06.412949+0000","last_undegraded":"2024-03-11T18:36:06.412949+0000","last_fullsized":"2024-03-11T18:36:06.412949+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:13:47.499965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.406548+0000","last_change":"2024-03-11T18:35:57.506963+0000","last_active":"2024-03-11T18:36:06.406548+0000","last_peered":"2024-03-11T18:36:06.406548+0000","last_clean":"2024-03-11T18:36:06.406548+0000","last_became_active":"2024-03-11T18:35:57.506038+0000","last_became_peered":"2024-03-11T18:35:57.506038+0000","last_unstale":"2024-03-11T18:36:06.406548+0000","last_undegraded":"2024-03-11T18:36:06.406548+0000","last_fullsized":"2024-03-11T18:36:06.406548+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:52:06.024245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"28'4","reported_seq":31,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:37.955368+0000","last_change":"2024-03-11T18:35:56.364512+0000","last_active":"2024-03-11T18:36:37.955368+0000","last_peered":"2024-03-11T18:36:37.955368+0000","last_clean":"2024-03-11T18:36:37.955368+0000","last_became_active":"2024-03-11T18:35:56.363731+0000","last_became_peered":"2024-03-11T18:35:56.363731+0000","last_unstale":"2024-03-11T18:36:37.955368+0000","last_undegraded":"2024-03-11T18:36:37.955368+0000","last_fullsized":"2024-03-11T18:36:37.955368+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:36:17.809744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.408707+0000","last_change":"2024-03-11T18:36:04.643782+0000","last_active":"2024-03-11T18:36:06.408707+0000","last_peered":"2024-03-11T18:36:06.408707+0000","last_clean":"2024-03-11T18:36:06.408707+0000","last_became_active":"2024-03-11T18:36:04.642133+0000","last_became_peered":"2024-03-11T18:36:04.642133+0000","last_unstale":"2024-03-11T18:36:06.408707+0000","last_undegraded":"2024-03-11T18:36:06.408707+0000","last_fullsized":"2024-03-11T18:36:06.408707+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:53:33.601193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.405392+0000","last_change":"2024-03-11T18:35:57.675207+0000","last_active":"2024-03-11T18:36:06.405392+0000","last_peered":"2024-03-11T18:36:06.405392+0000","last_clean":"2024-03-11T18:36:06.405392+0000","last_became_active":"2024-03-11T18:35:57.674472+0000","last_became_peered":"2024-03-11T18:35:57.674472+0000","last_unstale":"2024-03-11T18:36:06.405392+0000","last_undegraded":"2024-03-11T18:36:06.405392+0000","last_fullsized":"2024-03-11T18:36:06.405392+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:29:45.662029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"28'3","reported_seq":30,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:44.760547+0000","last_change":"2024-03-11T18:35:56.348479+0000","last_active":"2024-03-11T18:36:44.760547+0000","last_peered":"2024-03-11T18:36:44.760547+0000","last_clean":"2024-03-11T18:36:44.760547+0000","last_became_active":"2024-03-11T18:35:56.348040+0000","last_became_peered":"2024-03-11T18:35:56.348040+0000","last_unstale":"2024-03-11T18:36:44.760547+0000","last_undegraded":"2024-03-11T18:36:44.760547+0000","last_fullsized":"2024-03-11T18:36:44.760547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:27:26.471550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.401155+0000","last_change":"2024-03-11T18:36:04.599805+0000","last_active":"2024-03-11T18:36:06.401155+0000","last_peered":"2024-03-11T18:36:06.401155+0000","last_clean":"2024-03-11T18:36:06.401155+0000","last_became_active":"2024-03-11T18:36:04.599065+0000","last_became_peered":"2024-03-11T18:36:04.599065+0000","last_unstale":"2024-03-11T18:36:06.401155+0000","last_undegraded":"2024-03-11T18:36:06.401155+0000","last_fullsized":"2024-03-11T18:36:06.401155+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:36:07.187003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.359096+0000","last_change":"2024-03-11T18:35:57.538551+0000","last_active":"2024-03-11T18:36:06.359096+0000","last_peered":"2024-03-11T18:36:06.359096+0000","last_clean":"2024-03-11T18:36:06.359096+0000","last_became_active":"2024-03-11T18:35:57.537706+0000","last_became_peered":"2024-03-11T18:35:57.537706+0000","last_unstale":"2024-03-11T18:36:06.359096+0000","last_undegraded":"2024-03-11T18:36:06.359096+0000","last_fullsized":"2024-03-11T18:36:06.359096+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:03:56.260394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"28'1","reported_seq":28,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.355497+0000","last_change":"2024-03-11T18:35:56.302575+0000","last_active":"2024-03-11T18:36:06.355497+0000","last_peered":"2024-03-11T18:36:06.355497+0000","last_clean":"2024-03-11T18:36:06.355497+0000","last_became_active":"2024-03-11T18:35:56.301819+0000","last_became_peered":"2024-03-11T18:35:56.301819+0000","last_unstale":"2024-03-11T18:36:06.355497+0000","last_undegraded":"2024-03-11T18:36:06.355497+0000","last_fullsized":"2024-03-11T18:36:06.355497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:38:26.519320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.404840+0000","last_change":"2024-03-11T18:36:04.609282+0000","last_active":"2024-03-11T18:36:06.404840+0000","last_peered":"2024-03-11T18:36:06.404840+0000","last_clean":"2024-03-11T18:36:06.404840+0000","last_became_active":"2024-03-11T18:36:04.608260+0000","last_became_peered":"2024-03-11T18:36:04.608260+0000","last_unstale":"2024-03-11T18:36:06.404840+0000","last_undegraded":"2024-03-11T18:36:06.404840+0000","last_fullsized":"2024-03-11T18:36:06.404840+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:49:28.143755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.413195+0000","last_change":"2024-03-11T18:35:57.688727+0000","last_active":"2024-03-11T18:36:06.413195+0000","last_peered":"2024-03-11T18:36:06.413195+0000","last_clean":"2024-03-11T18:36:06.413195+0000","last_became_active":"2024-03-11T18:35:57.688209+0000","last_became_peered":"2024-03-11T18:35:57.688209+0000","last_unstale":"2024-03-11T18:36:06.413195+0000","last_undegraded":"2024-03-11T18:36:06.413195+0000","last_fullsized":"2024-03-11T18:36:06.413195+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:58:13.389498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.359326+0000","last_change":"2024-03-11T18:35:56.293842+0000","last_active":"2024-03-11T18:36:06.359326+0000","last_peered":"2024-03-11T18:36:06.359326+0000","last_clean":"2024-03-11T18:36:06.359326+0000","last_became_active":"2024-03-11T18:35:56.293276+0000","last_became_peered":"2024-03-11T18:35:56.293276+0000","last_unstale":"2024-03-11T18:36:06.359326+0000","last_undegraded":"2024-03-11T18:36:06.359326+0000","last_fullsized":"2024-03-11T18:36:06.359326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:04:13.101160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.358354+0000","last_change":"2024-03-11T18:36:04.626320+0000","last_active":"2024-03-11T18:36:06.358354+0000","last_peered":"2024-03-11T18:36:06.358354+0000","last_clean":"2024-03-11T18:36:06.358354+0000","last_became_active":"2024-03-11T18:36:04.625663+0000","last_became_peered":"2024-03-11T18:36:04.625663+0000","last_unstale":"2024-03-11T18:36:06.358354+0000","last_undegraded":"2024-03-11T18:36:06.358354+0000","last_fullsized":"2024-03-11T18:36:06.358354+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:39:32.875112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.411275+0000","last_change":"2024-03-11T18:35:57.612242+0000","last_active":"2024-03-11T18:36:06.411275+0000","last_peered":"2024-03-11T18:36:06.411275+0000","last_clean":"2024-03-11T18:36:06.411275+0000","last_became_active":"2024-03-11T18:35:57.611509+0000","last_became_peered":"2024-03-11T18:35:57.611509+0000","last_unstale":"2024-03-11T18:36:06.411275+0000","last_undegraded":"2024-03-11T18:36:06.411275+0000","last_fullsized":"2024-03-11T18:36:06.411275+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:45:28.543623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"28'1","reported_seq":28,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.410230+0000","last_change":"2024-03-11T18:35:56.298766+0000","last_active":"2024-03-11T18:36:06.410230+0000","last_peered":"2024-03-11T18:36:06.410230+0000","last_clean":"2024-03-11T18:36:06.410230+0000","last_became_active":"2024-03-11T18:35:56.297747+0000","last_became_peered":"2024-03-11T18:35:56.297747+0000","last_unstale":"2024-03-11T18:36:06.410230+0000","last_undegraded":"2024-03-11T18:36:06.410230+0000","last_fullsized":"2024-03-11T18:36:06.410230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:13:39.177689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.358743+0000","last_change":"2024-03-11T18:36:04.635773+0000","last_active":"2024-03-11T18:36:06.358743+0000","last_peered":"2024-03-11T18:36:06.358743+0000","last_clean":"2024-03-11T18:36:06.358743+0000","last_became_active":"2024-03-11T18:36:04.635261+0000","last_became_peered":"2024-03-11T18:36:04.635261+0000","last_unstale":"2024-03-11T18:36:06.358743+0000","last_undegraded":"2024-03-11T18:36:06.358743+0000","last_fullsized":"2024-03-11T18:36:06.358743+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:44:18.985469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.358192+0000","last_change":"2024-03-11T18:35:57.538552+0000","last_active":"2024-03-11T18:36:06.358192+0000","last_peered":"2024-03-11T18:36:06.358192+0000","last_clean":"2024-03-11T18:36:06.358192+0000","last_became_active":"2024-03-11T18:35:57.537717+0000","last_became_peered":"2024-03-11T18:35:57.537717+0000","last_unstale":"2024-03-11T18:36:06.358192+0000","last_undegraded":"2024-03-11T18:36:06.358192+0000","last_fullsized":"2024-03-11T18:36:06.358192+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:53:26.493265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"28'4","reported_seq":31,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:44.730260+0000","last_change":"2024-03-11T18:35:56.294572+0000","last_active":"2024-03-11T18:36:44.730260+0000","last_peered":"2024-03-11T18:36:44.730260+0000","last_clean":"2024-03-11T18:36:44.730260+0000","last_became_active":"2024-03-11T18:35:56.294096+0000","last_became_peered":"2024-03-11T18:35:56.294096+0000","last_unstale":"2024-03-11T18:36:44.730260+0000","last_undegraded":"2024-03-11T18:36:44.730260+0000","last_fullsized":"2024-03-11T18:36:44.730260+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:22:12.516281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.359857+0000","last_change":"2024-03-11T18:36:04.620762+0000","last_active":"2024-03-11T18:36:06.359857+0000","last_peered":"2024-03-11T18:36:06.359857+0000","last_clean":"2024-03-11T18:36:06.359857+0000","last_became_active":"2024-03-11T18:36:04.620356+0000","last_became_peered":"2024-03-11T18:36:04.620356+0000","last_unstale":"2024-03-11T18:36:06.359857+0000","last_undegraded":"2024-03-11T18:36:06.359857+0000","last_fullsized":"2024-03-11T18:36:06.359857+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:58:20.497426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.404612+0000","last_change":"2024-03-11T18:35:57.602773+0000","last_active":"2024-03-11T18:36:06.404612+0000","last_peered":"2024-03-11T18:36:06.404612+0000","last_clean":"2024-03-11T18:36:06.404612+0000","last_became_active":"2024-03-11T18:35:57.602060+0000","last_became_peered":"2024-03-11T18:35:57.602060+0000","last_unstale":"2024-03-11T18:36:06.404612+0000","last_undegraded":"2024-03-11T18:36:06.404612+0000","last_fullsized":"2024-03-11T18:36:06.404612+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:36:22.201707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"28'3","reported_seq":30,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:44.536118+0000","last_change":"2024-03-11T18:35:56.305283+0000","last_active":"2024-03-11T18:36:44.536118+0000","last_peered":"2024-03-11T18:36:44.536118+0000","last_clean":"2024-03-11T18:36:44.536118+0000","last_became_active":"2024-03-11T18:35:56.303063+0000","last_became_peered":"2024-03-11T18:35:56.303063+0000","last_unstale":"2024-03-11T18:36:44.536118+0000","last_undegraded":"2024-03-11T18:36:44.536118+0000","last_fullsized":"2024-03-11T18:36:44.536118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T18:51:00.795706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.406282+0000","last_change":"2024-03-11T18:36:04.617247+0000","last_active":"2024-03-11T18:36:06.406282+0000","last_peered":"2024-03-11T18:36:06.406282+0000","last_clean":"2024-03-11T18:36:06.406282+0000","last_became_active":"2024-03-11T18:36:04.610765+0000","last_became_peered":"2024-03-11T18:36:04.610765+0000","last_unstale":"2024-03-11T18:36:06.406282+0000","last_undegraded":"2024-03-11T18:36:06.406282+0000","last_fullsized":"2024-03-11T18:36:06.406282+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:22:19.187685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.401101+0000","last_change":"2024-03-11T18:35:57.684207+0000","last_active":"2024-03-11T18:36:06.401101+0000","last_peered":"2024-03-11T18:36:06.401101+0000","last_clean":"2024-03-11T18:36:06.401101+0000","last_became_active":"2024-03-11T18:35:57.682653+0000","last_became_peered":"2024-03-11T18:35:57.682653+0000","last_unstale":"2024-03-11T18:36:06.401101+0000","last_undegraded":"2024-03-11T18:36:06.401101+0000","last_fullsized":"2024-03-11T18:36:06.401101+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:28:51.265291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"28'3","reported_seq":30,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:37.947870+0000","last_change":"2024-03-11T18:35:56.297078+0000","last_active":"2024-03-11T18:36:37.947870+0000","last_peered":"2024-03-11T18:36:37.947870+0000","last_clean":"2024-03-11T18:36:37.947870+0000","last_became_active":"2024-03-11T18:35:56.296705+0000","last_became_peered":"2024-03-11T18:35:56.296705+0000","last_unstale":"2024-03-11T18:36:37.947870+0000","last_undegraded":"2024-03-11T18:36:37.947870+0000","last_fullsized":"2024-03-11T18:36:37.947870+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:53:25.278916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":182,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.352930+0000","last_change":"2024-03-11T18:35:18.199528+0000","last_active":"2024-03-11T18:36:06.352930+0000","last_peered":"2024-03-11T18:36:06.352930+0000","last_clean":"2024-03-11T18:36:06.352930+0000","last_became_active":"2024-03-11T18:35:18.197797+0000","last_became_peered":"2024-03-11T18:35:18.197797+0000","last_unstale":"2024-03-11T18:36:06.352930+0000","last_undegraded":"2024-03-11T18:36:06.352930+0000","last_fullsized":"2024-03-11T18:36:06.352930+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:16.207500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:16.207500+0000","last_clean_scrub_stamp":"2024-03-11T18:35:16.207500+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:29:00.357496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.411807+0000","last_change":"2024-03-11T18:36:04.599815+0000","last_active":"2024-03-11T18:36:06.411807+0000","last_peered":"2024-03-11T18:36:06.411807+0000","last_clean":"2024-03-11T18:36:06.411807+0000","last_became_active":"2024-03-11T18:36:04.598777+0000","last_became_peered":"2024-03-11T18:36:04.598777+0000","last_unstale":"2024-03-11T18:36:06.411807+0000","last_undegraded":"2024-03-11T18:36:06.411807+0000","last_fullsized":"2024-03-11T18:36:06.411807+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:39:32.875112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.354816+0000","last_change":"2024-03-11T18:35:57.687766+0000","last_active":"2024-03-11T18:36:06.354816+0000","last_peered":"2024-03-11T18:36:06.354816+0000","last_clean":"2024-03-11T18:36:06.354816+0000","last_became_active":"2024-03-11T18:35:57.687059+0000","last_became_peered":"2024-03-11T18:35:57.687059+0000","last_unstale":"2024-03-11T18:36:06.354816+0000","last_undegraded":"2024-03-11T18:36:06.354816+0000","last_fullsized":"2024-03-11T18:36:06.354816+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:39:25.767184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"28'3","reported_seq":30,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:44.730609+0000","last_change":"2024-03-11T18:35:56.294626+0000","last_active":"2024-03-11T18:36:44.730609+0000","last_peered":"2024-03-11T18:36:44.730609+0000","last_clean":"2024-03-11T18:36:44.730609+0000","last_became_active":"2024-03-11T18:35:56.293961+0000","last_became_peered":"2024-03-11T18:35:56.293961+0000","last_unstale":"2024-03-11T18:36:44.730609+0000","last_undegraded":"2024-03-11T18:36:44.730609+0000","last_fullsized":"2024-03-11T18:36:44.730609+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:50:38.709532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.409872+0000","last_change":"2024-03-11T18:36:04.619862+0000","last_active":"2024-03-11T18:36:06.409872+0000","last_peered":"2024-03-11T18:36:06.409872+0000","last_clean":"2024-03-11T18:36:06.409872+0000","last_became_active":"2024-03-11T18:36:04.618908+0000","last_became_peered":"2024-03-11T18:36:04.618908+0000","last_unstale":"2024-03-11T18:36:06.409872+0000","last_undegraded":"2024-03-11T18:36:06.409872+0000","last_fullsized":"2024-03-11T18:36:06.409872+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:34:46.433500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.410912+0000","last_change":"2024-03-11T18:35:57.686416+0000","last_active":"2024-03-11T18:36:06.410912+0000","last_peered":"2024-03-11T18:36:06.410912+0000","last_clean":"2024-03-11T18:36:06.410912+0000","last_became_active":"2024-03-11T18:35:57.685828+0000","last_became_peered":"2024-03-11T18:35:57.685828+0000","last_unstale":"2024-03-11T18:36:06.410912+0000","last_undegraded":"2024-03-11T18:36:06.410912+0000","last_fullsized":"2024-03-11T18:36:06.410912+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:41:42.263652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"23'1","reported_seq":28,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.359093+0000","last_change":"2024-03-11T18:35:56.295539+0000","last_active":"2024-03-11T18:36:06.359093+0000","last_peered":"2024-03-11T18:36:06.359093+0000","last_clean":"2024-03-11T18:36:06.359093+0000","last_became_active":"2024-03-11T18:35:56.295123+0000","last_became_peered":"2024-03-11T18:35:56.295123+0000","last_unstale":"2024-03-11T18:36:06.359093+0000","last_undegraded":"2024-03-11T18:36:06.359093+0000","last_fullsized":"2024-03-11T18:36:06.359093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:03:06.585728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.406058+0000","last_change":"2024-03-11T18:36:04.616750+0000","last_active":"2024-03-11T18:36:06.406058+0000","last_peered":"2024-03-11T18:36:06.406058+0000","last_clean":"2024-03-11T18:36:06.406058+0000","last_became_active":"2024-03-11T18:36:04.616338+0000","last_became_peered":"2024-03-11T18:36:04.616338+0000","last_unstale":"2024-03-11T18:36:06.406058+0000","last_undegraded":"2024-03-11T18:36:06.406058+0000","last_fullsized":"2024-03-11T18:36:06.406058+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:14:42.558491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.354272+0000","last_change":"2024-03-11T18:35:57.689888+0000","last_active":"2024-03-11T18:36:06.354272+0000","last_peered":"2024-03-11T18:36:06.354272+0000","last_clean":"2024-03-11T18:36:06.354272+0000","last_became_active":"2024-03-11T18:35:57.689424+0000","last_became_peered":"2024-03-11T18:35:57.689424+0000","last_unstale":"2024-03-11T18:36:06.354272+0000","last_undegraded":"2024-03-11T18:36:06.354272+0000","last_fullsized":"2024-03-11T18:36:06.354272+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:25:16.564261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"23'2","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.402166+0000","last_change":"2024-03-11T18:35:56.293862+0000","last_active":"2024-03-11T18:36:06.402166+0000","last_peered":"2024-03-11T18:36:06.402166+0000","last_clean":"2024-03-11T18:36:06.402166+0000","last_became_active":"2024-03-11T18:35:56.293585+0000","last_became_peered":"2024-03-11T18:35:56.293585+0000","last_unstale":"2024-03-11T18:36:06.402166+0000","last_undegraded":"2024-03-11T18:36:06.402166+0000","last_fullsized":"2024-03-11T18:36:06.402166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:45:27.329274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.408514+0000","last_change":"2024-03-11T18:36:04.627729+0000","last_active":"2024-03-11T18:36:06.408514+0000","last_peered":"2024-03-11T18:36:06.408514+0000","last_clean":"2024-03-11T18:36:06.408514+0000","last_became_active":"2024-03-11T18:36:04.626596+0000","last_became_peered":"2024-03-11T18:36:04.626596+0000","last_unstale":"2024-03-11T18:36:06.408514+0000","last_undegraded":"2024-03-11T18:36:06.408514+0000","last_fullsized":"2024-03-11T18:36:06.408514+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:51:28.017349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.353860+0000","last_change":"2024-03-11T18:35:57.690506+0000","last_active":"2024-03-11T18:36:06.353860+0000","last_peered":"2024-03-11T18:36:06.353860+0000","last_clean":"2024-03-11T18:36:06.353860+0000","last_became_active":"2024-03-11T18:35:57.690223+0000","last_became_peered":"2024-03-11T18:35:57.690223+0000","last_unstale":"2024-03-11T18:36:06.353860+0000","last_undegraded":"2024-03-11T18:36:06.353860+0000","last_fullsized":"2024-03-11T18:36:06.353860+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:58:13.389498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"28'2","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.414795+0000","last_change":"2024-03-11T18:35:56.257209+0000","last_active":"2024-03-11T18:36:06.414795+0000","last_peered":"2024-03-11T18:36:06.414795+0000","last_clean":"2024-03-11T18:36:06.414795+0000","last_became_active":"2024-03-11T18:35:56.256454+0000","last_became_peered":"2024-03-11T18:35:56.256454+0000","last_unstale":"2024-03-11T18:36:06.414795+0000","last_undegraded":"2024-03-11T18:36:06.414795+0000","last_fullsized":"2024-03-11T18:36:06.414795+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:32:06.369248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.408104+0000","last_change":"2024-03-11T18:36:04.644184+0000","last_active":"2024-03-11T18:36:06.408104+0000","last_peered":"2024-03-11T18:36:06.408104+0000","last_clean":"2024-03-11T18:36:06.408104+0000","last_became_active":"2024-03-11T18:36:04.642079+0000","last_became_peered":"2024-03-11T18:36:04.642079+0000","last_unstale":"2024-03-11T18:36:06.408104+0000","last_undegraded":"2024-03-11T18:36:06.408104+0000","last_fullsized":"2024-03-11T18:36:06.408104+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:36:26.132020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.353155+0000","last_change":"2024-03-11T18:35:57.687311+0000","last_active":"2024-03-11T18:36:06.353155+0000","last_peered":"2024-03-11T18:36:06.353155+0000","last_clean":"2024-03-11T18:36:06.353155+0000","last_became_active":"2024-03-11T18:35:57.686850+0000","last_became_peered":"2024-03-11T18:35:57.686850+0000","last_unstale":"2024-03-11T18:36:06.353155+0000","last_undegraded":"2024-03-11T18:36:06.353155+0000","last_fullsized":"2024-03-11T18:36:06.353155+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:29:45.662029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"28'15","reported_seq":42,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:44.758260+0000","last_change":"2024-03-11T18:35:56.291232+0000","last_active":"2024-03-11T18:36:44.758260+0000","last_peered":"2024-03-11T18:36:44.758260+0000","last_clean":"2024-03-11T18:36:44.758260+0000","last_became_active":"2024-03-11T18:35:56.290944+0000","last_became_peered":"2024-03-11T18:35:56.290944+0000","last_unstale":"2024-03-11T18:36:44.758260+0000","last_undegraded":"2024-03-11T18:36:44.758260+0000","last_fullsized":"2024-03-11T18:36:44.758260+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:49:27.827765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.405992+0000","last_change":"2024-03-11T18:36:04.602142+0000","last_active":"2024-03-11T18:36:06.405992+0000","last_peered":"2024-03-11T18:36:06.405992+0000","last_clean":"2024-03-11T18:36:06.405992+0000","last_became_active":"2024-03-11T18:36:04.601224+0000","last_became_peered":"2024-03-11T18:36:04.601224+0000","last_unstale":"2024-03-11T18:36:06.405992+0000","last_undegraded":"2024-03-11T18:36:06.405992+0000","last_fullsized":"2024-03-11T18:36:06.405992+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:17:42.075355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.409133+0000","last_change":"2024-03-11T18:35:57.611305+0000","last_active":"2024-03-11T18:36:06.409133+0000","last_peered":"2024-03-11T18:36:06.409133+0000","last_clean":"2024-03-11T18:36:06.409133+0000","last_became_active":"2024-03-11T18:35:57.610955+0000","last_became_peered":"2024-03-11T18:35:57.610955+0000","last_unstale":"2024-03-11T18:36:06.409133+0000","last_undegraded":"2024-03-11T18:36:06.409133+0000","last_fullsized":"2024-03-11T18:36:06.409133+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:31:19.941345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"28'3","reported_seq":30,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:44.715902+0000","last_change":"2024-03-11T18:35:56.361952+0000","last_active":"2024-03-11T18:36:44.715902+0000","last_peered":"2024-03-11T18:36:44.715902+0000","last_clean":"2024-03-11T18:36:44.715902+0000","last_became_active":"2024-03-11T18:35:56.361141+0000","last_became_peered":"2024-03-11T18:35:56.361141+0000","last_unstale":"2024-03-11T18:36:44.715902+0000","last_undegraded":"2024-03-11T18:36:44.715902+0000","last_fullsized":"2024-03-11T18:36:44.715902+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:04:13.101160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.405270+0000","last_change":"2024-03-11T18:36:04.593604+0000","last_active":"2024-03-11T18:36:06.405270+0000","last_peered":"2024-03-11T18:36:06.405270+0000","last_clean":"2024-03-11T18:36:06.405270+0000","last_became_active":"2024-03-11T18:36:04.592304+0000","last_became_peered":"2024-03-11T18:36:04.592304+0000","last_unstale":"2024-03-11T18:36:06.405270+0000","last_undegraded":"2024-03-11T18:36:06.405270+0000","last_fullsized":"2024-03-11T18:36:06.405270+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:31:04.738940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.353525+0000","last_change":"2024-03-11T18:35:57.689404+0000","last_active":"2024-03-11T18:36:06.353525+0000","last_peered":"2024-03-11T18:36:06.353525+0000","last_clean":"2024-03-11T18:36:06.353525+0000","last_became_active":"2024-03-11T18:35:57.689139+0000","last_became_peered":"2024-03-11T18:35:57.689139+0000","last_unstale":"2024-03-11T18:36:06.353525+0000","last_undegraded":"2024-03-11T18:36:06.353525+0000","last_fullsized":"2024-03-11T18:36:06.353525+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:54:09.699371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"28'3","reported_seq":30,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:37.947614+0000","last_change":"2024-03-11T18:35:56.297682+0000","last_active":"2024-03-11T18:36:37.947614+0000","last_peered":"2024-03-11T18:36:37.947614+0000","last_clean":"2024-03-11T18:36:37.947614+0000","last_became_active":"2024-03-11T18:35:56.296426+0000","last_became_peered":"2024-03-11T18:35:56.296426+0000","last_unstale":"2024-03-11T18:36:37.947614+0000","last_undegraded":"2024-03-11T18:36:37.947614+0000","last_fullsized":"2024-03-11T18:36:37.947614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:28:50.050942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.403363+0000","last_change":"2024-03-11T18:36:04.590780+0000","last_active":"2024-03-11T18:36:06.403363+0000","last_peered":"2024-03-11T18:36:06.403363+0000","last_clean":"2024-03-11T18:36:06.403363+0000","last_became_active":"2024-03-11T18:36:04.590014+0000","last_became_peered":"2024-03-11T18:36:04.590014+0000","last_unstale":"2024-03-11T18:36:06.403363+0000","last_undegraded":"2024-03-11T18:36:06.403363+0000","last_fullsized":"2024-03-11T18:36:06.403363+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:51:13.205384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.407150+0000","last_change":"2024-03-11T18:35:57.671638+0000","last_active":"2024-03-11T18:36:06.407150+0000","last_peered":"2024-03-11T18:36:06.407150+0000","last_clean":"2024-03-11T18:36:06.407150+0000","last_became_active":"2024-03-11T18:35:57.670893+0000","last_became_peered":"2024-03-11T18:35:57.670893+0000","last_unstale":"2024-03-11T18:36:06.407150+0000","last_undegraded":"2024-03-11T18:36:06.407150+0000","last_fullsized":"2024-03-11T18:36:06.407150+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:31:21.395133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"28'7","reported_seq":34,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:44.701166+0000","last_change":"2024-03-11T18:35:56.251580+0000","last_active":"2024-03-11T18:36:44.701166+0000","last_peered":"2024-03-11T18:36:44.701166+0000","last_clean":"2024-03-11T18:36:44.701166+0000","last_became_active":"2024-03-11T18:35:56.251287+0000","last_became_peered":"2024-03-11T18:35:56.251287+0000","last_unstale":"2024-03-11T18:36:44.701166+0000","last_undegraded":"2024-03-11T18:36:44.701166+0000","last_fullsized":"2024-03-11T18:36:44.701166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:30:39.907360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.412899+0000","last_change":"2024-03-11T18:36:04.625563+0000","last_active":"2024-03-11T18:36:06.412899+0000","last_peered":"2024-03-11T18:36:06.412899+0000","last_clean":"2024-03-11T18:36:06.412899+0000","last_became_active":"2024-03-11T18:36:04.618911+0000","last_became_peered":"2024-03-11T18:36:04.618911+0000","last_unstale":"2024-03-11T18:36:06.412899+0000","last_undegraded":"2024-03-11T18:36:06.412899+0000","last_fullsized":"2024-03-11T18:36:06.412899+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:31:04.738940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.354510+0000","last_change":"2024-03-11T18:35:57.697641+0000","last_active":"2024-03-11T18:36:06.354510+0000","last_peered":"2024-03-11T18:36:06.354510+0000","last_clean":"2024-03-11T18:36:06.354510+0000","last_became_active":"2024-03-11T18:35:57.697263+0000","last_became_peered":"2024-03-11T18:35:57.697263+0000","last_unstale":"2024-03-11T18:36:06.354510+0000","last_undegraded":"2024-03-11T18:36:06.354510+0000","last_fullsized":"2024-03-11T18:36:06.354510+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:34:39.325572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"28'12","reported_seq":39,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:37.952962+0000","last_change":"2024-03-11T18:35:56.355612+0000","last_active":"2024-03-11T18:36:37.952962+0000","last_peered":"2024-03-11T18:36:37.952962+0000","last_clean":"2024-03-11T18:36:37.952962+0000","last_became_active":"2024-03-11T18:35:56.354827+0000","last_became_peered":"2024-03-11T18:35:56.354827+0000","last_unstale":"2024-03-11T18:36:37.952962+0000","last_undegraded":"2024-03-11T18:36:37.952962+0000","last_fullsized":"2024-03-11T18:36:37.952962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:06:02.844765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.358459+0000","last_change":"2024-03-11T18:36:04.617561+0000","last_active":"2024-03-11T18:36:06.358459+0000","last_peered":"2024-03-11T18:36:06.358459+0000","last_clean":"2024-03-11T18:36:06.358459+0000","last_became_active":"2024-03-11T18:36:04.616938+0000","last_became_peered":"2024-03-11T18:36:04.616938+0000","last_unstale":"2024-03-11T18:36:06.358459+0000","last_undegraded":"2024-03-11T18:36:06.358459+0000","last_fullsized":"2024-03-11T18:36:06.358459+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:16:10.508913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.412500+0000","last_change":"2024-03-11T18:35:57.679035+0000","last_active":"2024-03-11T18:36:06.412500+0000","last_peered":"2024-03-11T18:36:06.412500+0000","last_clean":"2024-03-11T18:36:06.412500+0000","last_became_active":"2024-03-11T18:35:57.678565+0000","last_became_peered":"2024-03-11T18:35:57.678565+0000","last_unstale":"2024-03-11T18:36:06.412500+0000","last_undegraded":"2024-03-11T18:36:06.412500+0000","last_fullsized":"2024-03-11T18:36:06.412500+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:18:05.980207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"28'4","reported_seq":31,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.358900+0000","last_change":"2024-03-11T18:35:56.288132+0000","last_active":"2024-03-11T18:36:06.358900+0000","last_peered":"2024-03-11T18:36:06.358900+0000","last_clean":"2024-03-11T18:36:06.358900+0000","last_became_active":"2024-03-11T18:35:56.287846+0000","last_became_peered":"2024-03-11T18:35:56.287846+0000","last_unstale":"2024-03-11T18:36:06.358900+0000","last_undegraded":"2024-03-11T18:36:06.358900+0000","last_fullsized":"2024-03-11T18:36:06.358900+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T18:51:00.795706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.411589+0000","last_change":"2024-03-11T18:36:04.643164+0000","last_active":"2024-03-11T18:36:06.411589+0000","last_peered":"2024-03-11T18:36:06.411589+0000","last_clean":"2024-03-11T18:36:06.411589+0000","last_became_active":"2024-03-11T18:36:04.642374+0000","last_became_peered":"2024-03-11T18:36:04.642374+0000","last_unstale":"2024-03-11T18:36:06.411589+0000","last_undegraded":"2024-03-11T18:36:06.411589+0000","last_fullsized":"2024-03-11T18:36:06.411589+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:25:56.815496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.405796+0000","last_change":"2024-03-11T18:35:57.636550+0000","last_active":"2024-03-11T18:36:06.405796+0000","last_peered":"2024-03-11T18:36:06.405796+0000","last_clean":"2024-03-11T18:36:06.405796+0000","last_became_active":"2024-03-11T18:35:57.636093+0000","last_became_peered":"2024-03-11T18:35:57.636093+0000","last_unstale":"2024-03-11T18:36:06.405796+0000","last_undegraded":"2024-03-11T18:36:06.405796+0000","last_fullsized":"2024-03-11T18:36:06.405796+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:33:31.344465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"28'2","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:37.946730+0000","last_change":"2024-03-11T18:35:56.355480+0000","last_active":"2024-03-11T18:36:37.946730+0000","last_peered":"2024-03-11T18:36:37.946730+0000","last_clean":"2024-03-11T18:36:37.946730+0000","last_became_active":"2024-03-11T18:35:56.354759+0000","last_became_peered":"2024-03-11T18:35:56.354759+0000","last_unstale":"2024-03-11T18:36:37.946730+0000","last_undegraded":"2024-03-11T18:36:37.946730+0000","last_fullsized":"2024-03-11T18:36:37.946730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:45:37.842725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.400682+0000","last_change":"2024-03-11T18:36:04.617364+0000","last_active":"2024-03-11T18:36:06.400682+0000","last_peered":"2024-03-11T18:36:06.400682+0000","last_clean":"2024-03-11T18:36:06.400682+0000","last_became_active":"2024-03-11T18:36:04.613159+0000","last_became_peered":"2024-03-11T18:36:04.613159+0000","last_unstale":"2024-03-11T18:36:06.400682+0000","last_undegraded":"2024-03-11T18:36:06.400682+0000","last_fullsized":"2024-03-11T18:36:06.400682+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:25:23.672189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.412882+0000","last_change":"2024-03-11T18:35:57.683426+0000","last_active":"2024-03-11T18:36:06.412882+0000","last_peered":"2024-03-11T18:36:06.412882+0000","last_clean":"2024-03-11T18:36:06.412882+0000","last_became_active":"2024-03-11T18:35:57.683150+0000","last_became_peered":"2024-03-11T18:35:57.683150+0000","last_unstale":"2024-03-11T18:36:06.412882+0000","last_undegraded":"2024-03-11T18:36:06.412882+0000","last_fullsized":"2024-03-11T18:36:06.412882+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:36:00.079075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"28'3","reported_seq":30,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:37.955896+0000","last_change":"2024-03-11T18:35:56.361613+0000","last_active":"2024-03-11T18:36:37.955896+0000","last_peered":"2024-03-11T18:36:37.955896+0000","last_clean":"2024-03-11T18:36:37.955896+0000","last_became_active":"2024-03-11T18:35:56.359629+0000","last_became_peered":"2024-03-11T18:35:56.359629+0000","last_unstale":"2024-03-11T18:36:37.955896+0000","last_undegraded":"2024-03-11T18:36:37.955896+0000","last_fullsized":"2024-03-11T18:36:37.955896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:03:06.585728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.401756+0000","last_change":"2024-03-11T18:36:04.608168+0000","last_active":"2024-03-11T18:36:06.401756+0000","last_peered":"2024-03-11T18:36:06.401756+0000","last_clean":"2024-03-11T18:36:06.401756+0000","last_became_active":"2024-03-11T18:36:04.607063+0000","last_became_peered":"2024-03-11T18:36:04.607063+0000","last_unstale":"2024-03-11T18:36:06.401756+0000","last_undegraded":"2024-03-11T18:36:06.401756+0000","last_fullsized":"2024-03-11T18:36:06.401756+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:53:41.996024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.405114+0000","last_change":"2024-03-11T18:36:04.607992+0000","last_active":"2024-03-11T18:36:06.405114+0000","last_peered":"2024-03-11T18:36:06.405114+0000","last_clean":"2024-03-11T18:36:06.405114+0000","last_became_active":"2024-03-11T18:36:04.607109+0000","last_became_peered":"2024-03-11T18:36:04.607109+0000","last_unstale":"2024-03-11T18:36:06.405114+0000","last_undegraded":"2024-03-11T18:36:06.405114+0000","last_fullsized":"2024-03-11T18:36:06.405114+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:14:37.678854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.357424+0000","last_change":"2024-03-11T18:35:57.680534+0000","last_active":"2024-03-11T18:36:06.357424+0000","last_peered":"2024-03-11T18:36:06.357424+0000","last_clean":"2024-03-11T18:36:06.357424+0000","last_became_active":"2024-03-11T18:35:57.679642+0000","last_became_peered":"2024-03-11T18:35:57.679642+0000","last_unstale":"2024-03-11T18:36:06.357424+0000","last_undegraded":"2024-03-11T18:36:06.357424+0000","last_fullsized":"2024-03-11T18:36:06.357424+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:25:49.707568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"28'1","reported_seq":28,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.409415+0000","last_change":"2024-03-11T18:35:56.303510+0000","last_active":"2024-03-11T18:36:06.409415+0000","last_peered":"2024-03-11T18:36:06.409415+0000","last_clean":"2024-03-11T18:36:06.409415+0000","last_became_active":"2024-03-11T18:35:56.302758+0000","last_became_peered":"2024-03-11T18:35:56.302758+0000","last_unstale":"2024-03-11T18:36:06.409415+0000","last_undegraded":"2024-03-11T18:36:06.409415+0000","last_fullsized":"2024-03-11T18:36:06.409415+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:03:32.297342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.412847+0000","last_change":"2024-03-11T18:36:04.618818+0000","last_active":"2024-03-11T18:36:06.412847+0000","last_peered":"2024-03-11T18:36:06.412847+0000","last_clean":"2024-03-11T18:36:06.412847+0000","last_became_active":"2024-03-11T18:36:04.617868+0000","last_became_peered":"2024-03-11T18:36:04.617868+0000","last_unstale":"2024-03-11T18:36:06.412847+0000","last_undegraded":"2024-03-11T18:36:06.412847+0000","last_fullsized":"2024-03-11T18:36:06.412847+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:16:10.508913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.400811+0000","last_change":"2024-03-11T18:35:57.690347+0000","last_active":"2024-03-11T18:36:06.400811+0000","last_peered":"2024-03-11T18:36:06.400811+0000","last_clean":"2024-03-11T18:36:06.400811+0000","last_became_active":"2024-03-11T18:35:57.689146+0000","last_became_peered":"2024-03-11T18:35:57.689146+0000","last_unstale":"2024-03-11T18:36:06.400811+0000","last_undegraded":"2024-03-11T18:36:06.400811+0000","last_fullsized":"2024-03-11T18:36:06.400811+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:03:56.260394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"28'2","reported_seq":29,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:44.532285+0000","last_change":"2024-03-11T18:35:56.348355+0000","last_active":"2024-03-11T18:36:44.532285+0000","last_peered":"2024-03-11T18:36:44.532285+0000","last_clean":"2024-03-11T18:36:44.532285+0000","last_became_active":"2024-03-11T18:35:56.347631+0000","last_became_peered":"2024-03-11T18:35:56.347631+0000","last_unstale":"2024-03-11T18:36:44.532285+0000","last_undegraded":"2024-03-11T18:36:44.532285+0000","last_fullsized":"2024-03-11T18:36:44.532285+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:06:02.844765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"28'3","reported_seq":30,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:44.716081+0000","last_change":"2024-03-11T18:35:56.361254+0000","last_active":"2024-03-11T18:36:44.716081+0000","last_peered":"2024-03-11T18:36:44.716081+0000","last_clean":"2024-03-11T18:36:44.716081+0000","last_became_active":"2024-03-11T18:35:56.360580+0000","last_became_peered":"2024-03-11T18:35:56.360580+0000","last_unstale":"2024-03-11T18:36:44.716081+0000","last_undegraded":"2024-03-11T18:36:44.716081+0000","last_fullsized":"2024-03-11T18:36:44.716081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:27:26.471550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.406163+0000","last_change":"2024-03-11T18:35:57.602778+0000","last_active":"2024-03-11T18:36:06.406163+0000","last_peered":"2024-03-11T18:36:06.406163+0000","last_clean":"2024-03-11T18:36:06.406163+0000","last_became_active":"2024-03-11T18:35:57.601990+0000","last_became_peered":"2024-03-11T18:35:57.601990+0000","last_unstale":"2024-03-11T18:36:06.406163+0000","last_undegraded":"2024-03-11T18:36:06.406163+0000","last_fullsized":"2024-03-11T18:36:06.406163+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:56:59.014871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.401528+0000","last_change":"2024-03-11T18:36:04.606098+0000","last_active":"2024-03-11T18:36:06.401528+0000","last_peered":"2024-03-11T18:36:06.401528+0000","last_clean":"2024-03-11T18:36:06.401528+0000","last_became_active":"2024-03-11T18:36:04.604823+0000","last_became_peered":"2024-03-11T18:36:04.604823+0000","last_unstale":"2024-03-11T18:36:06.401528+0000","last_undegraded":"2024-03-11T18:36:06.401528+0000","last_fullsized":"2024-03-11T18:36:06.401528+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:52:04.862452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"28'5","reported_seq":32,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:37.941879+0000","last_change":"2024-03-11T18:35:56.296312+0000","last_active":"2024-03-11T18:36:37.941879+0000","last_peered":"2024-03-11T18:36:37.941879+0000","last_clean":"2024-03-11T18:36:37.941879+0000","last_became_active":"2024-03-11T18:35:56.296090+0000","last_became_peered":"2024-03-11T18:35:56.296090+0000","last_unstale":"2024-03-11T18:36:37.941879+0000","last_undegraded":"2024-03-11T18:36:37.941879+0000","last_fullsized":"2024-03-11T18:36:37.941879+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:03:06.585728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.410741+0000","last_change":"2024-03-11T18:35:57.611809+0000","last_active":"2024-03-11T18:36:06.410741+0000","last_peered":"2024-03-11T18:36:06.410741+0000","last_clean":"2024-03-11T18:36:06.410741+0000","last_became_active":"2024-03-11T18:35:57.611097+0000","last_became_peered":"2024-03-11T18:35:57.611097+0000","last_unstale":"2024-03-11T18:36:06.410741+0000","last_undegraded":"2024-03-11T18:36:06.410741+0000","last_fullsized":"2024-03-11T18:36:06.410741+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:49:29.042114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.403540+0000","last_change":"2024-03-11T18:36:04.590064+0000","last_active":"2024-03-11T18:36:06.403540+0000","last_peered":"2024-03-11T18:36:06.403540+0000","last_clean":"2024-03-11T18:36:06.403540+0000","last_became_active":"2024-03-11T18:36:04.589037+0000","last_became_peered":"2024-03-11T18:36:04.589037+0000","last_unstale":"2024-03-11T18:36:06.403540+0000","last_undegraded":"2024-03-11T18:36:06.403540+0000","last_fullsized":"2024-03-11T18:36:06.403540+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:51:28.017349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.407866+0000","last_change":"2024-03-11T18:35:57.614944+0000","last_active":"2024-03-11T18:36:06.407866+0000","last_peered":"2024-03-11T18:36:06.407866+0000","last_clean":"2024-03-11T18:36:06.407866+0000","last_became_active":"2024-03-11T18:35:57.614144+0000","last_became_peered":"2024-03-11T18:35:57.614144+0000","last_unstale":"2024-03-11T18:36:06.407866+0000","last_undegraded":"2024-03-11T18:36:06.407866+0000","last_fullsized":"2024-03-11T18:36:06.407866+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:53:26.493265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.400811+0000","last_change":"2024-03-11T18:35:57.630211+0000","last_active":"2024-03-11T18:36:06.400811+0000","last_peered":"2024-03-11T18:36:06.400811+0000","last_clean":"2024-03-11T18:36:06.400811+0000","last_became_active":"2024-03-11T18:35:57.629599+0000","last_became_peered":"2024-03-11T18:35:57.629599+0000","last_unstale":"2024-03-11T18:36:06.400811+0000","last_undegraded":"2024-03-11T18:36:06.400811+0000","last_fullsized":"2024-03-11T18:36:06.400811+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:18:05.980207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.409629+0000","last_change":"2024-03-11T18:35:57.688700+0000","last_active":"2024-03-11T18:36:06.409629+0000","last_peered":"2024-03-11T18:36:06.409629+0000","last_clean":"2024-03-11T18:36:06.409629+0000","last_became_active":"2024-03-11T18:35:57.687968+0000","last_became_peered":"2024-03-11T18:35:57.687968+0000","last_unstale":"2024-03-11T18:36:06.409629+0000","last_undegraded":"2024-03-11T18:36:06.409629+0000","last_fullsized":"2024-03-11T18:36:06.409629+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:25:16.564261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.405998+0000","last_change":"2024-03-11T18:35:57.680730+0000","last_active":"2024-03-11T18:36:06.405998+0000","last_peered":"2024-03-11T18:36:06.405998+0000","last_clean":"2024-03-11T18:36:06.405998+0000","last_became_active":"2024-03-11T18:35:57.679935+0000","last_became_peered":"2024-03-11T18:35:57.679935+0000","last_unstale":"2024-03-11T18:36:06.405998+0000","last_undegraded":"2024-03-11T18:36:06.405998+0000","last_fullsized":"2024-03-11T18:36:06.405998+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:21:21.894805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.403523+0000","last_change":"2024-03-11T18:35:57.690799+0000","last_active":"2024-03-11T18:36:06.403523+0000","last_peered":"2024-03-11T18:36:06.403523+0000","last_clean":"2024-03-11T18:36:06.403523+0000","last_became_active":"2024-03-11T18:35:57.686623+0000","last_became_peered":"2024-03-11T18:35:57.686623+0000","last_unstale":"2024-03-11T18:36:06.403523+0000","last_undegraded":"2024-03-11T18:36:06.403523+0000","last_fullsized":"2024-03-11T18:36:06.403523+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:53:30.324898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.358254+0000","last_change":"2024-03-11T18:35:57.691955+0000","last_active":"2024-03-11T18:36:06.358254+0000","last_peered":"2024-03-11T18:36:06.358254+0000","last_clean":"2024-03-11T18:36:06.358254+0000","last_became_active":"2024-03-11T18:35:57.691413+0000","last_became_peered":"2024-03-11T18:35:57.691413+0000","last_unstale":"2024-03-11T18:36:06.358254+0000","last_undegraded":"2024-03-11T18:36:06.358254+0000","last_fullsized":"2024-03-11T18:36:06.358254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:31:19.941345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.410285+0000","last_change":"2024-03-11T18:35:57.684695+0000","last_active":"2024-03-11T18:36:06.410285+0000","last_peered":"2024-03-11T18:36:06.410285+0000","last_clean":"2024-03-11T18:36:06.410285+0000","last_became_active":"2024-03-11T18:35:57.684227+0000","last_became_peered":"2024-03-11T18:35:57.684227+0000","last_unstale":"2024-03-11T18:36:06.410285+0000","last_undegraded":"2024-03-11T18:36:06.410285+0000","last_fullsized":"2024-03-11T18:36:06.410285+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:28:51.265291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.405217+0000","last_change":"2024-03-11T18:35:57.683549+0000","last_active":"2024-03-11T18:36:06.405217+0000","last_peered":"2024-03-11T18:36:06.405217+0000","last_clean":"2024-03-11T18:36:06.405217+0000","last_became_active":"2024-03-11T18:35:57.680610+0000","last_became_peered":"2024-03-11T18:35:57.680610+0000","last_unstale":"2024-03-11T18:36:06.405217+0000","last_undegraded":"2024-03-11T18:36:06.405217+0000","last_fullsized":"2024-03-11T18:36:06.405217+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:25:52.716744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.409054+0000","last_change":"2024-03-11T18:35:57.686833+0000","last_active":"2024-03-11T18:36:06.409054+0000","last_peered":"2024-03-11T18:36:06.409054+0000","last_clean":"2024-03-11T18:36:06.409054+0000","last_became_active":"2024-03-11T18:35:57.686144+0000","last_became_peered":"2024-03-11T18:35:57.686144+0000","last_unstale":"2024-03-11T18:36:06.409054+0000","last_undegraded":"2024-03-11T18:36:06.409054+0000","last_fullsized":"2024-03-11T18:36:06.409054+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:16:03.400985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.401952+0000","last_change":"2024-03-11T18:35:57.681630+0000","last_active":"2024-03-11T18:36:06.401952+0000","last_peered":"2024-03-11T18:36:06.401952+0000","last_clean":"2024-03-11T18:36:06.401952+0000","last_became_active":"2024-03-11T18:35:57.678876+0000","last_became_peered":"2024-03-11T18:35:57.678876+0000","last_unstale":"2024-03-11T18:36:06.401952+0000","last_undegraded":"2024-03-11T18:36:06.401952+0000","last_fullsized":"2024-03-11T18:36:06.401952+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:03:33.511691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.409370+0000","last_change":"2024-03-11T18:35:57.679589+0000","last_active":"2024-03-11T18:36:06.409370+0000","last_peered":"2024-03-11T18:36:06.409370+0000","last_clean":"2024-03-11T18:36:06.409370+0000","last_became_active":"2024-03-11T18:35:57.678838+0000","last_became_peered":"2024-03-11T18:35:57.678838+0000","last_unstale":"2024-03-11T18:36:06.409370+0000","last_undegraded":"2024-03-11T18:36:06.409370+0000","last_fullsized":"2024-03-11T18:36:06.409370+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:54:09.699371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.404736+0000","last_change":"2024-03-11T18:35:57.674571+0000","last_active":"2024-03-11T18:36:06.404736+0000","last_peered":"2024-03-11T18:36:06.404736+0000","last_clean":"2024-03-11T18:36:06.404736+0000","last_became_active":"2024-03-11T18:35:57.671573+0000","last_became_peered":"2024-03-11T18:35:57.671573+0000","last_unstale":"2024-03-11T18:36:06.404736+0000","last_undegraded":"2024-03-11T18:36:06.404736+0000","last_fullsized":"2024-03-11T18:36:06.404736+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:35:10.403666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.410712+0000","last_change":"2024-03-11T18:35:57.615939+0000","last_active":"2024-03-11T18:36:06.410712+0000","last_peered":"2024-03-11T18:36:06.410712+0000","last_clean":"2024-03-11T18:36:06.410712+0000","last_became_active":"2024-03-11T18:35:57.615579+0000","last_became_peered":"2024-03-11T18:35:57.615579+0000","last_unstale":"2024-03-11T18:36:06.410712+0000","last_undegraded":"2024-03-11T18:36:06.410712+0000","last_fullsized":"2024-03-11T18:36:06.410712+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:03:33.511691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.355121+0000","last_change":"2024-03-11T18:35:57.688955+0000","last_active":"2024-03-11T18:36:06.355121+0000","last_peered":"2024-03-11T18:36:06.355121+0000","last_clean":"2024-03-11T18:36:06.355121+0000","last_became_active":"2024-03-11T18:35:57.688365+0000","last_became_peered":"2024-03-11T18:35:57.688365+0000","last_unstale":"2024-03-11T18:36:06.355121+0000","last_undegraded":"2024-03-11T18:36:06.355121+0000","last_fullsized":"2024-03-11T18:36:06.355121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:44:11.877541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.401903+0000","last_change":"2024-03-11T18:35:57.602044+0000","last_active":"2024-03-11T18:36:06.401903+0000","last_peered":"2024-03-11T18:36:06.401903+0000","last_clean":"2024-03-11T18:36:06.401903+0000","last_became_active":"2024-03-11T18:35:57.601334+0000","last_became_peered":"2024-03-11T18:35:57.601334+0000","last_unstale":"2024-03-11T18:36:06.401903+0000","last_undegraded":"2024-03-11T18:36:06.401903+0000","last_fullsized":"2024-03-11T18:36:06.401903+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:39:12.693619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.413873+0000","last_change":"2024-03-11T18:35:57.677114+0000","last_active":"2024-03-11T18:36:06.413873+0000","last_peered":"2024-03-11T18:36:06.413873+0000","last_clean":"2024-03-11T18:36:06.413873+0000","last_became_active":"2024-03-11T18:35:57.676075+0000","last_became_peered":"2024-03-11T18:35:57.676075+0000","last_unstale":"2024-03-11T18:36:06.413873+0000","last_undegraded":"2024-03-11T18:36:06.413873+0000","last_fullsized":"2024-03-11T18:36:06.413873+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:44:11.877541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.408799+0000","last_change":"2024-03-11T18:35:57.641392+0000","last_active":"2024-03-11T18:36:06.408799+0000","last_peered":"2024-03-11T18:36:06.408799+0000","last_clean":"2024-03-11T18:36:06.408799+0000","last_became_active":"2024-03-11T18:35:57.640934+0000","last_became_peered":"2024-03-11T18:35:57.640934+0000","last_unstale":"2024-03-11T18:36:06.408799+0000","last_undegraded":"2024-03-11T18:36:06.408799+0000","last_fullsized":"2024-03-11T18:36:06.408799+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:25:52.716744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.357659+0000","last_change":"2024-03-11T18:35:57.605291+0000","last_active":"2024-03-11T18:36:06.357659+0000","last_peered":"2024-03-11T18:36:06.357659+0000","last_clean":"2024-03-11T18:36:06.357659+0000","last_became_active":"2024-03-11T18:35:57.604645+0000","last_became_peered":"2024-03-11T18:35:57.604645+0000","last_unstale":"2024-03-11T18:36:06.357659+0000","last_undegraded":"2024-03-11T18:36:06.357659+0000","last_fullsized":"2024-03-11T18:36:06.357659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:03:33.511691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.404894+0000","last_change":"2024-03-11T18:35:57.503994+0000","last_active":"2024-03-11T18:36:06.404894+0000","last_peered":"2024-03-11T18:36:06.404894+0000","last_clean":"2024-03-11T18:36:06.404894+0000","last_became_active":"2024-03-11T18:35:57.503362+0000","last_became_peered":"2024-03-11T18:35:57.503362+0000","last_unstale":"2024-03-11T18:36:06.404894+0000","last_undegraded":"2024-03-11T18:36:06.404894+0000","last_fullsized":"2024-03-11T18:36:06.404894+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:03:58.875164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.402309+0000","last_change":"2024-03-11T18:35:57.679571+0000","last_active":"2024-03-11T18:36:06.402309+0000","last_peered":"2024-03-11T18:36:06.402309+0000","last_clean":"2024-03-11T18:36:06.402309+0000","last_became_active":"2024-03-11T18:35:57.678787+0000","last_became_peered":"2024-03-11T18:35:57.678787+0000","last_unstale":"2024-03-11T18:36:06.402309+0000","last_undegraded":"2024-03-11T18:36:06.402309+0000","last_fullsized":"2024-03-11T18:36:06.402309+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:13:40.392038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.412355+0000","last_change":"2024-03-11T18:35:57.613374+0000","last_active":"2024-03-11T18:36:06.412355+0000","last_peered":"2024-03-11T18:36:06.412355+0000","last_clean":"2024-03-11T18:36:06.412355+0000","last_became_active":"2024-03-11T18:35:57.613050+0000","last_became_peered":"2024-03-11T18:35:57.613050+0000","last_unstale":"2024-03-11T18:36:06.412355+0000","last_undegraded":"2024-03-11T18:36:06.412355+0000","last_fullsized":"2024-03-11T18:36:06.412355+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:50:39.923881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.411942+0000","last_change":"2024-03-11T18:35:57.636565+0000","last_active":"2024-03-11T18:36:06.411942+0000","last_peered":"2024-03-11T18:36:06.411942+0000","last_clean":"2024-03-11T18:36:06.411942+0000","last_became_active":"2024-03-11T18:35:57.635978+0000","last_became_peered":"2024-03-11T18:35:57.635978+0000","last_unstale":"2024-03-11T18:36:06.411942+0000","last_undegraded":"2024-03-11T18:36:06.411942+0000","last_fullsized":"2024-03-11T18:36:06.411942+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:03:56.260394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.410011+0000","last_change":"2024-03-11T18:35:57.641714+0000","last_active":"2024-03-11T18:36:06.410011+0000","last_peered":"2024-03-11T18:36:06.410011+0000","last_clean":"2024-03-11T18:36:06.410011+0000","last_became_active":"2024-03-11T18:35:57.641417+0000","last_became_peered":"2024-03-11T18:35:57.641417+0000","last_unstale":"2024-03-11T18:36:06.410011+0000","last_undegraded":"2024-03-11T18:36:06.410011+0000","last_fullsized":"2024-03-11T18:36:06.410011+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:25:49.707568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.354140+0000","last_change":"2024-03-11T18:35:57.689703+0000","last_active":"2024-03-11T18:36:06.354140+0000","last_peered":"2024-03-11T18:36:06.354140+0000","last_clean":"2024-03-11T18:36:06.354140+0000","last_became_active":"2024-03-11T18:35:57.689056+0000","last_became_peered":"2024-03-11T18:35:57.689056+0000","last_unstale":"2024-03-11T18:36:06.354140+0000","last_undegraded":"2024-03-11T18:36:06.354140+0000","last_fullsized":"2024-03-11T18:36:06.354140+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:41:42.263652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.400467+0000","last_change":"2024-03-11T18:35:57.684305+0000","last_active":"2024-03-11T18:36:06.400467+0000","last_peered":"2024-03-11T18:36:06.400467+0000","last_clean":"2024-03-11T18:36:06.400467+0000","last_became_active":"2024-03-11T18:35:57.682947+0000","last_became_peered":"2024-03-11T18:35:57.682947+0000","last_unstale":"2024-03-11T18:36:06.400467+0000","last_undegraded":"2024-03-11T18:36:06.400467+0000","last_fullsized":"2024-03-11T18:36:06.400467+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:53:26.493265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.352832+0000","last_change":"2024-03-11T18:35:57.692717+0000","last_active":"2024-03-11T18:36:06.352832+0000","last_peered":"2024-03-11T18:36:06.352832+0000","last_clean":"2024-03-11T18:36:06.352832+0000","last_became_active":"2024-03-11T18:35:57.692504+0000","last_became_peered":"2024-03-11T18:35:57.692504+0000","last_unstale":"2024-03-11T18:36:06.352832+0000","last_undegraded":"2024-03-11T18:36:06.352832+0000","last_fullsized":"2024-03-11T18:36:06.352832+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:25:52.716744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.409110+0000","last_change":"2024-03-11T18:35:57.640483+0000","last_active":"2024-03-11T18:36:06.409110+0000","last_peered":"2024-03-11T18:36:06.409110+0000","last_clean":"2024-03-11T18:36:06.409110+0000","last_became_active":"2024-03-11T18:35:57.639886+0000","last_became_peered":"2024-03-11T18:35:57.639886+0000","last_unstale":"2024-03-11T18:36:06.409110+0000","last_undegraded":"2024-03-11T18:36:06.409110+0000","last_fullsized":"2024-03-11T18:36:06.409110+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:34:39.325572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.352837+0000","last_change":"2024-03-11T18:35:57.687096+0000","last_active":"2024-03-11T18:36:06.352837+0000","last_peered":"2024-03-11T18:36:06.352837+0000","last_clean":"2024-03-11T18:36:06.352837+0000","last_became_active":"2024-03-11T18:35:57.686771+0000","last_became_peered":"2024-03-11T18:35:57.686771+0000","last_unstale":"2024-03-11T18:36:06.352837+0000","last_undegraded":"2024-03-11T18:36:06.352837+0000","last_fullsized":"2024-03-11T18:36:06.352837+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:16:03.400985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.409284+0000","last_change":"2024-03-11T18:35:57.694707+0000","last_active":"2024-03-11T18:36:06.409284+0000","last_peered":"2024-03-11T18:36:06.409284+0000","last_clean":"2024-03-11T18:36:06.409284+0000","last_became_active":"2024-03-11T18:35:57.694104+0000","last_became_peered":"2024-03-11T18:35:57.694104+0000","last_unstale":"2024-03-11T18:36:06.409284+0000","last_undegraded":"2024-03-11T18:36:06.409284+0000","last_fullsized":"2024-03-11T18:36:06.409284+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:47:28.579467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.405456+0000","last_change":"2024-03-11T18:35:57.538548+0000","last_active":"2024-03-11T18:36:06.405456+0000","last_peered":"2024-03-11T18:36:06.405456+0000","last_clean":"2024-03-11T18:36:06.405456+0000","last_became_active":"2024-03-11T18:35:57.537704+0000","last_became_peered":"2024-03-11T18:35:57.537704+0000","last_unstale":"2024-03-11T18:36:06.405456+0000","last_undegraded":"2024-03-11T18:36:06.405456+0000","last_fullsized":"2024-03-11T18:36:06.405456+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:40:52.251981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.359086+0000","last_change":"2024-03-11T18:35:57.507126+0000","last_active":"2024-03-11T18:36:06.359086+0000","last_peered":"2024-03-11T18:36:06.359086+0000","last_clean":"2024-03-11T18:36:06.359086+0000","last_became_active":"2024-03-11T18:35:57.506429+0000","last_became_peered":"2024-03-11T18:35:57.506429+0000","last_unstale":"2024-03-11T18:36:06.359086+0000","last_undegraded":"2024-03-11T18:36:06.359086+0000","last_fullsized":"2024-03-11T18:36:06.359086+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:28:51.265291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-03-11T18:36:06.353858+0000","last_change":"2024-03-11T18:35:57.687817+0000","last_active":"2024-03-11T18:36:06.353858+0000","last_peered":"2024-03-11T18:36:06.353858+0000","last_clean":"2024-03-11T18:36:06.353858+0000","last_became_active":"2024-03-11T18:35:57.687532+0000","last_became_peered":"2024-03-11T18:35:57.687532+0000","last_unstale":"2024-03-11T18:36:06.353858+0000","last_undegraded":"2024-03-11T18:36:06.353858+0000","last_fullsized":"2024-03-11T18:36:06.353858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:47:28.579467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7486,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":133,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":167936,"data_stored":14938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":116,"ondisk_log_size":116,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476761,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343940,"statfs":{"total":96636764160,"available":96608194560,"internally_reserved":0,"allocated":905216,"data_stored":708231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":16,"seq":68719476761,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":311296,"data_stored":121067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":16,"seq":68719476759,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":311296,"data_stored":117925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476760,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":307200,"data_stored":117261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509463,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":315392,"data_stored":121101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509463,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":311296,"data_stored":117365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509464,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":315392,"data_stored":117491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542166,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27900,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343940,"statfs":{"total":96636764160,"available":96608194560,"internally_reserved":0,"allocated":905216,"data_stored":707609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-11T18:36:51.777 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-11T18:36:51.778 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-11T18:36:51.778 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-03-11T18:36:51.778 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-11T18:36:51.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.932+0000 7f81e9396640 1 Processor -- start 2024-03-11T18:36:51.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.932+0000 7f81e9396640 1 -- start start 2024-03-11T18:36:51.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.932+0000 7f81e9396640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f81e4070b00 0x7f81e40c4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:51.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.932+0000 7f81e9396640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81e40c5de0 con 0x7f81e4150dd0 2024-03-11T18:36:51.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.932+0000 7f81e9396640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81e4156910 con 0x7f81e41509f0 2024-03-11T18:36:51.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.932+0000 7f81e9396640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81e40c6030 con 0x7f81e4070b00 2024-03-11T18:36:51.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.936+0000 7f81e27fc640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f81e4150dd0 0x7f81e40c28e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:42996/0 (socket says 172.21.15.90:42996) 2024-03-11T18:36:51.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.936+0000 7f81e27fc640 1 -- 172.21.15.90:0/838276712 learned_addr learned my addr 172.21.15.90:0/838276712 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T18:36:51.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.936+0000 7f81e37fe640 1 --2- 172.21.15.90:0/838276712 >> v2:172.21.15.92:3300/0 conn(0x7f81e4070b00 0x7f81e40c4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:51.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.936+0000 7f81e1ffb640 1 -- 172.21.15.90:0/838276712 <== mon.0 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 974243880 0 0) 0x7f81e40c5de0 con 0x7f81e4150dd0 2024-03-11T18:36:51.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.936+0000 7f81e1ffb640 1 -- 172.21.15.90:0/838276712 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81d0005740 con 0x7f81e4150dd0 2024-03-11T18:36:51.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.940+0000 7f81e37fe640 1 -- 172.21.15.90:0/838276712 >> v1:172.21.15.92:6789/0 conn(0x7f81e41509f0 legacy=0x7f81e4157390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:51.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.940+0000 7f81e37fe640 1 -- 172.21.15.90:0/838276712 >> v1:172.21.15.90:6789/0 conn(0x7f81e4150dd0 legacy=0x7f81e40c28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:51.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.940+0000 7f81e37fe640 1 -- 172.21.15.90:0/838276712 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e40c6310 con 0x7f81e4070b00 2024-03-11T18:36:51.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.952+0000 7f81e37fe640 1 --2- 172.21.15.90:0/838276712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81e4070b00 0x7f81e40c4580 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f81dc009d90 tx=0x7f81dc01c930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a7f427bc7d115dd server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:51.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.960+0000 7f81e1ffb640 1 -- 172.21.15.90:0/838276712 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81dc0a58f0 con 0x7f81e4070b00 2024-03-11T18:36:51.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.960+0000 7f81e1ffb640 1 -- 172.21.15.90:0/838276712 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81dc0a5a50 con 0x7f81e4070b00 2024-03-11T18:36:51.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.960+0000 7f81e1ffb640 1 -- 172.21.15.90:0/838276712 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81dc0a5de0 con 0x7f81e4070b00 2024-03-11T18:36:51.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.960+0000 7f81e9396640 1 -- 172.21.15.90:0/838276712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81e4070b00 msgr2=0x7f81e40c4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:51.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.960+0000 7f81e9396640 1 --2- 172.21.15.90:0/838276712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81e4070b00 0x7f81e40c4580 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f81dc009d90 tx=0x7f81dc01c930 comp rx=0 tx=0).stop 2024-03-11T18:36:51.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.960+0000 7f81e9396640 1 -- 172.21.15.90:0/838276712 shutdown_connections 2024-03-11T18:36:51.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.960+0000 7f81e9396640 1 --2- 172.21.15.90:0/838276712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81e4070b00 0x7f81e40c4580 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:51.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.960+0000 7f81e9396640 1 -- 172.21.15.90:0/838276712 >> 172.21.15.90:0/838276712 conn(0x7f81e406ac00 msgr2=0x7f81e406b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:51.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.960+0000 7f81e9396640 1 -- 172.21.15.90:0/838276712 shutdown_connections 2024-03-11T18:36:51.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.960+0000 7f81e9396640 1 -- 172.21.15.90:0/838276712 wait complete. 2024-03-11T18:36:51.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.964+0000 7f81e9396640 1 Processor -- start 2024-03-11T18:36:51.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.964+0000 7f81e9396640 1 -- start start 2024-03-11T18:36:51.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.964+0000 7f81e9396640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81e4070b00 0x7f81e414b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:51.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.964+0000 7f81e9396640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81e41509f0 0x7f81e40ca490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:51.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.964+0000 7f81e9396640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f81e4150dd0 0x7f81e414c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:51.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.964+0000 7f81e9396640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e414d7c0 con 0x7f81e4070b00 2024-03-11T18:36:51.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.964+0000 7f81e9396640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e414dd70 con 0x7f81e41509f0 2024-03-11T18:36:51.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.964+0000 7f81e9396640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81e414db40 con 0x7f81e4150dd0 2024-03-11T18:36:51.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.964+0000 7f81e2ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81e4070b00 0x7f81e414b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:51.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.964+0000 7f81e2ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81e4070b00 0x7f81e414b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48408/0 (socket says 172.21.15.90:48408) 2024-03-11T18:36:51.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.964+0000 7f81e2ffd640 1 -- 172.21.15.90:0/3394001878 learned_addr learned my addr 172.21.15.90:0/3394001878 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:51.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.964+0000 7f81e37fe640 1 --2- 172.21.15.90:0/3394001878 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f81e4150dd0 0x7f81e414c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:51.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.968+0000 7f81e27fc640 1 --2- 172.21.15.90:0/3394001878 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81e41509f0 0x7f81e40ca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:51.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.968+0000 7f81e2ffd640 1 -- 172.21.15.90:0/3394001878 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f81e4150dd0 msgr2=0x7f81e414c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:51.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.968+0000 7f81e2ffd640 1 --2- 172.21.15.90:0/3394001878 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f81e4150dd0 0x7f81e414c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:51.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.968+0000 7f81e2ffd640 1 -- 172.21.15.90:0/3394001878 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81e41509f0 msgr2=0x7f81e40ca490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:51.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.968+0000 7f81e2ffd640 1 --2- 172.21.15.90:0/3394001878 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81e41509f0 0x7f81e40ca490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:51.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.968+0000 7f81e2ffd640 1 -- 172.21.15.90:0/3394001878 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81dc009a70 con 0x7f81e4070b00 2024-03-11T18:36:51.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.972+0000 7f81e2ffd640 1 --2- 172.21.15.90:0/3394001878 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81e4070b00 0x7f81e414b6b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f81d400c4c0 tx=0x7f81d4001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:51.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.972+0000 7f81db7fe640 1 -- 172.21.15.90:0/3394001878 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d4005480 con 0x7f81e4070b00 2024-03-11T18:36:51.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.972+0000 7f81db7fe640 1 -- 172.21.15.90:0/3394001878 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81d4003600 con 0x7f81e4070b00 2024-03-11T18:36:51.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.972+0000 7f81e9396640 1 -- 172.21.15.90:0/3394001878 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81e4160580 con 0x7f81e4070b00 2024-03-11T18:36:51.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.972+0000 7f81db7fe640 1 -- 172.21.15.90:0/3394001878 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d4003990 con 0x7f81e4070b00 2024-03-11T18:36:51.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.972+0000 7f81e9396640 1 -- 172.21.15.90:0/3394001878 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81e40cbda0 con 0x7f81e4070b00 2024-03-11T18:36:51.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.976+0000 7f81e9396640 1 -- 172.21.15.90:0/3394001878 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81e41609f0 con 0x7f81e4070b00 2024-03-11T18:36:52.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.996+0000 7f81db7fe640 1 -- 172.21.15.90:0/3394001878 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f81d401b3b0 con 0x7f81e4070b00 2024-03-11T18:36:52.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.996+0000 7f81db7fe640 1 --2- 172.21.15.90:0/3394001878 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f81c4066e90 0x7f81c4069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:52.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.996+0000 7f81e27fc640 1 --2- 172.21.15.90:0/3394001878 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f81c4066e90 0x7f81c4069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:52.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:51.996+0000 7f81e27fc640 1 --2- 172.21.15.90:0/3394001878 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f81c4066e90 0x7f81c4069350 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f81e4161860 tx=0x7f81cc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:52.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.000+0000 7f81db7fe640 1 -- 172.21.15.90:0/3394001878 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f81d4054ff0 con 0x7f81e4070b00 2024-03-11T18:36:52.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.272+0000 7f81db7fe640 1 -- 172.21.15.90:0/3394001878 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f81d4051290 con 0x7f81e4070b00 2024-03-11T18:36:52.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.500+0000 7f81e9396640 1 -- 172.21.15.90:0/3394001878 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f81e404eef0 con 0x7f81e4070b00 2024-03-11T18:36:52.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.620+0000 7f81db7fe640 1 -- 172.21.15.90:0/3394001878 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f81d4054950 con 0x7f81e4070b00 2024-03-11T18:36:52.626 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:36:52.626 INFO:teuthology.orchestra.run.smithi090.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-11T18:36:52.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.624+0000 7f81e9396640 1 -- 172.21.15.90:0/3394001878 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f81c4066e90 msgr2=0x7f81c4069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:52.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.624+0000 7f81e9396640 1 --2- 172.21.15.90:0/3394001878 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f81c4066e90 0x7f81c4069350 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f81e4161860 tx=0x7f81cc009040 comp rx=0 tx=0).stop 2024-03-11T18:36:52.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.624+0000 7f81e9396640 1 -- 172.21.15.90:0/3394001878 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81e4070b00 msgr2=0x7f81e414b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:52.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.624+0000 7f81e9396640 1 --2- 172.21.15.90:0/3394001878 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81e4070b00 0x7f81e414b6b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f81d400c4c0 tx=0x7f81d4001e80 comp rx=0 tx=0).stop 2024-03-11T18:36:52.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.624+0000 7f81e9396640 1 -- 172.21.15.90:0/3394001878 shutdown_connections 2024-03-11T18:36:52.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.624+0000 7f81e9396640 1 --2- 172.21.15.90:0/3394001878 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f81c4066e90 0x7f81c4069350 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:52.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.624+0000 7f81e9396640 1 --2- 172.21.15.90:0/3394001878 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f81e4150dd0 0x7f81e414c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:52.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.624+0000 7f81e9396640 1 --2- 172.21.15.90:0/3394001878 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f81e41509f0 0x7f81e40ca490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:52.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.624+0000 7f81e9396640 1 --2- 172.21.15.90:0/3394001878 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81e4070b00 0x7f81e414b6b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:52.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.624+0000 7f81e9396640 1 -- 172.21.15.90:0/3394001878 >> 172.21.15.90:0/3394001878 conn(0x7f81e406ac00 msgr2=0x7f81e4157390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:52.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.624+0000 7f81e9396640 1 -- 172.21.15.90:0/3394001878 shutdown_connections 2024-03-11T18:36:52.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.624+0000 7f81e9396640 1 -- 172.21.15.90:0/3394001878 wait complete. 2024-03-11T18:36:52.649 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-03-11T18:36:52.649 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T18:36:52.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.768+0000 7f4f8a25b640 1 Processor -- start 2024-03-11T18:36:52.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.768+0000 7f4f8a25b640 1 -- start start 2024-03-11T18:36:52.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.772+0000 7f4f8a25b640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f4f84154f50 0x7f4f840c5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:52.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.772+0000 7f4f8a25b640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4f840c9be0 0x7f4f84073bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:52.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.772+0000 7f4f8a25b640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f84157be0 con 0x7f4f840cb140 2024-03-11T18:36:52.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.772+0000 7f4f8a25b640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f84074330 con 0x7f4f84154f50 2024-03-11T18:36:52.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.772+0000 7f4f8a25b640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f84074130 con 0x7f4f840c9be0 2024-03-11T18:36:52.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.772+0000 7f4f837fe640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f4f84154f50 0x7f4f840c5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:52.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.772+0000 7f4f82ffd640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4f840c9be0 0x7f4f84073bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:52.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.772+0000 7f4f82ffd640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4f840c9be0 0x7f4f84073bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.90:59500/0 (socket says 172.21.15.90:59500) 2024-03-11T18:36:52.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.772+0000 7f4f82ffd640 1 -- 172.21.15.90:0/1167280249 learned_addr learned my addr 172.21.15.90:0/1167280249 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:52.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.780+0000 7f4f837fe640 1 -- 172.21.15.90:0/1167280249 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4f840c9be0 msgr2=0x7f4f84073bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:52.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.780+0000 7f4f837fe640 1 --2- 172.21.15.90:0/1167280249 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4f840c9be0 0x7f4f84073bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:52.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.780+0000 7f4f837fe640 1 -- 172.21.15.90:0/1167280249 >> v1:172.21.15.90:6789/0 conn(0x7f4f840cb140 legacy=0x7f4f840c6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:52.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.780+0000 7f4f837fe640 1 -- 172.21.15.90:0/1167280249 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f84157ec0 con 0x7f4f84154f50 2024-03-11T18:36:52.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.784+0000 7f4f837fe640 1 --2- 172.21.15.90:0/1167280249 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f84154f50 0x7f4f840c5e60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4f740097c0 tx=0x7f4f7401ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d507bfc10b609bcb server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:52.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.788+0000 7f4f827fc640 1 -- 172.21.15.90:0/1167280249 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f740a8990 con 0x7f4f84154f50 2024-03-11T18:36:52.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.788+0000 7f4f827fc640 1 -- 172.21.15.90:0/1167280249 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f740a8af0 con 0x7f4f84154f50 2024-03-11T18:36:52.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.788+0000 7f4f827fc640 1 -- 172.21.15.90:0/1167280249 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f740a8e80 con 0x7f4f84154f50 2024-03-11T18:36:52.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.792+0000 7f4f8a25b640 1 -- 172.21.15.90:0/1167280249 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f84154f50 msgr2=0x7f4f840c5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:52.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.792+0000 7f4f8a25b640 1 --2- 172.21.15.90:0/1167280249 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f84154f50 0x7f4f840c5e60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4f740097c0 tx=0x7f4f7401ec90 comp rx=0 tx=0).stop 2024-03-11T18:36:52.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.792+0000 7f4f8a25b640 1 -- 172.21.15.90:0/1167280249 shutdown_connections 2024-03-11T18:36:52.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.792+0000 7f4f8a25b640 1 --2- 172.21.15.90:0/1167280249 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4f840c9be0 0x7f4f84073bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:52.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.792+0000 7f4f8a25b640 1 --2- 172.21.15.90:0/1167280249 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f84154f50 0x7f4f840c5e60 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:52.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.792+0000 7f4f8a25b640 1 -- 172.21.15.90:0/1167280249 >> 172.21.15.90:0/1167280249 conn(0x7f4f8406fa80 msgr2=0x7f4f8406cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:52.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.792+0000 7f4f8a25b640 1 -- 172.21.15.90:0/1167280249 shutdown_connections 2024-03-11T18:36:52.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.792+0000 7f4f8a25b640 1 -- 172.21.15.90:0/1167280249 wait complete. 2024-03-11T18:36:52.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.792+0000 7f4f8a25b640 1 Processor -- start 2024-03-11T18:36:52.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.792+0000 7f4f8a25b640 1 -- start start 2024-03-11T18:36:52.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.792+0000 7f4f8a25b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f840c9be0 0x7f4f84147340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:52.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.792+0000 7f4f8a25b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f840cb140 0x7f4f840c1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:52.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.792+0000 7f4f8a25b640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4f84154f50 0x7f4f8414a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:52.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.792+0000 7f4f8a25b640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f841e38f0 con 0x7f4f840c9be0 2024-03-11T18:36:52.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.792+0000 7f4f8a25b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f841e2f60 con 0x7f4f840cb140 2024-03-11T18:36:52.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.792+0000 7f4f8a25b640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f841e3330 con 0x7f4f84154f50 2024-03-11T18:36:52.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.792+0000 7f4f837fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f840c9be0 0x7f4f84147340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:52.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.792+0000 7f4f837fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f840c9be0 0x7f4f84147340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48424/0 (socket says 172.21.15.90:48424) 2024-03-11T18:36:52.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.792+0000 7f4f837fe640 1 -- 172.21.15.90:0/3945133007 learned_addr learned my addr 172.21.15.90:0/3945133007 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:52.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.796+0000 7f4f82ffd640 1 --2- 172.21.15.90:0/3945133007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f840cb140 0x7f4f840c1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:52.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.796+0000 7f4f83fff640 1 --2- 172.21.15.90:0/3945133007 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4f84154f50 0x7f4f8414a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:52.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.800+0000 7f4f837fe640 1 -- 172.21.15.90:0/3945133007 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4f84154f50 msgr2=0x7f4f8414a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:52.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.800+0000 7f4f837fe640 1 --2- 172.21.15.90:0/3945133007 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4f84154f50 0x7f4f8414a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:52.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.800+0000 7f4f837fe640 1 -- 172.21.15.90:0/3945133007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f840cb140 msgr2=0x7f4f840c1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:52.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.800+0000 7f4f837fe640 1 --2- 172.21.15.90:0/3945133007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f840cb140 0x7f4f840c1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:52.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.800+0000 7f4f837fe640 1 -- 172.21.15.90:0/3945133007 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f74009420 con 0x7f4f840c9be0 2024-03-11T18:36:52.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.804+0000 7f4f837fe640 1 --2- 172.21.15.90:0/3945133007 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f840c9be0 0x7f4f84147340 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4f7401b8a0 tx=0x7f4f7401ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:52.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.808+0000 7f4f6bfff640 1 -- 172.21.15.90:0/3945133007 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f740b1dc0 con 0x7f4f840c9be0 2024-03-11T18:36:52.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.808+0000 7f4f6bfff640 1 -- 172.21.15.90:0/3945133007 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f740b9410 con 0x7f4f840c9be0 2024-03-11T18:36:52.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.808+0000 7f4f6bfff640 1 -- 172.21.15.90:0/3945133007 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f740b97c0 con 0x7f4f840c9be0 2024-03-11T18:36:52.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.808+0000 7f4f8a25b640 1 -- 172.21.15.90:0/3945133007 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f841e3610 con 0x7f4f840c9be0 2024-03-11T18:36:52.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.808+0000 7f4f8a25b640 1 -- 172.21.15.90:0/3945133007 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f841e2c80 con 0x7f4f840c9be0 2024-03-11T18:36:52.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:52.808+0000 7f4f8a25b640 1 -- 172.21.15.90:0/3945133007 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f8404eef0 con 0x7f4f840c9be0 2024-03-11T18:36:53.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.092+0000 7f4f6bfff640 1 -- 172.21.15.90:0/3945133007 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f4f740b7040 con 0x7f4f840c9be0 2024-03-11T18:36:53.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.092+0000 7f4f6bfff640 1 --2- 172.21.15.90:0/3945133007 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f4f70066e90 0x7f4f70069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:53.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.092+0000 7f4f82ffd640 1 --2- 172.21.15.90:0/3945133007 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f4f70066e90 0x7f4f70069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:53.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.096+0000 7f4f82ffd640 1 --2- 172.21.15.90:0/3945133007 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f4f70066e90 0x7f4f70069350 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4f841f41f0 tx=0x7f4f78009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:53.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.132+0000 7f4f6bfff640 1 -- 172.21.15.90:0/3945133007 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4f740f3ff0 con 0x7f4f840c9be0 2024-03-11T18:36:53.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.132+0000 7f4f6bfff640 1 -- 172.21.15.90:0/3945133007 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4f740f0290 con 0x7f4f840c9be0 2024-03-11T18:36:53.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.304+0000 7f4f8a25b640 1 -- 172.21.15.90:0/3945133007 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4f840c7bc0 con 0x7f4f840c9be0 2024-03-11T18:36:53.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.412+0000 7f4f6bfff640 1 -- 172.21.15.90:0/3945133007 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18430 (secure 0 0 0) 0x7f4f740b5070 con 0x7f4f840c9be0 2024-03-11T18:36:53.418 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:36:53.418 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":28,"fsid":"76d56b10-6c29-476a-804b-8482ea90f9e2","created":"2024-03-11T18:34:54.521741+0000","modified":"2024-03-11T18:36:05.526044+0000","last_up_change":"2024-03-11T18:35:16.207500+0000","last_in_change":"2024-03-11T18:35:05.213634+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T18:35:16.193842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T18:35:54.001469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T18:35:55.917321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T18:36:02.210943+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"34decff9-7280-4d84-9665-d6af8ea6de84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6819","nonce":1755667921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6821","nonce":1755667921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6825","nonce":1755667921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6823","nonce":1755667921}]},"public_addr":"172.21.15.90:6819/1755667921","cluster_addr":"172.21.15.90:6821/1755667921","heartbeat_back_addr":"172.21.15.90:6825/1755667921","heartbeat_front_addr":"172.21.15.90:6823/1755667921","state":["exists","up"]},{"osd":1,"uuid":"8ecdce9e-5486-415a-a609-dd16e945440e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6807","nonce":438682081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6809","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6811","nonce":438682081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6817","nonce":438682081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6815","nonce":438682081}]},"public_addr":"172.21.15.90:6807/438682081","cluster_addr":"172.21.15.90:6811/438682081","heartbeat_back_addr":"172.21.15.90:6817/438682081","heartbeat_front_addr":"172.21.15.90:6815/438682081","state":["exists","up"]},{"osd":2,"uuid":"737486d4-0e90-4c82-9e79-bf37c812e5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6827","nonce":2837704738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6829","nonce":2837704738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6833","nonce":2837704738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6831","nonce":2837704738}]},"public_addr":"172.21.15.90:6827/2837704738","cluster_addr":"172.21.15.90:6829/2837704738","heartbeat_back_addr":"172.21.15.90:6833/2837704738","heartbeat_front_addr":"172.21.15.90:6831/2837704738","state":["exists","up"]},{"osd":3,"uuid":"698bd661-bbef-4699-a619-f7bfe23ce12c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6803","nonce":240987971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6805","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6806","nonce":240987971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6813","nonce":240987971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6810","nonce":240987971}]},"public_addr":"172.21.15.90:6803/240987971","cluster_addr":"172.21.15.90:6806/240987971","heartbeat_back_addr":"172.21.15.90:6813/240987971","heartbeat_front_addr":"172.21.15.90:6810/240987971","state":["exists","up"]},{"osd":4,"uuid":"aba9e31f-73d9-43fe-a10b-2dc5832968b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6801","nonce":502685275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6803","nonce":502685275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6807","nonce":502685275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6805","nonce":502685275}]},"public_addr":"172.21.15.92:6801/502685275","cluster_addr":"172.21.15.92:6803/502685275","heartbeat_back_addr":"172.21.15.92:6807/502685275","heartbeat_front_addr":"172.21.15.92:6805/502685275","state":["exists","up"]},{"osd":5,"uuid":"bdd56766-4b28-4007-8de3-b5b08cf335d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6809","nonce":200142070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6811","nonce":200142070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6815","nonce":200142070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6813","nonce":200142070}]},"public_addr":"172.21.15.92:6809/200142070","cluster_addr":"172.21.15.92:6811/200142070","heartbeat_back_addr":"172.21.15.92:6815/200142070","heartbeat_front_addr":"172.21.15.92:6813/200142070","state":["exists","up"]},{"osd":6,"uuid":"6c580a59-610f-455e-a96b-033299dcc118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6817","nonce":1919010204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6819","nonce":1919010204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6823","nonce":1919010204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6821","nonce":1919010204}]},"public_addr":"172.21.15.92:6817/1919010204","cluster_addr":"172.21.15.92:6819/1919010204","heartbeat_back_addr":"172.21.15.92:6823/1919010204","heartbeat_front_addr":"172.21.15.92:6821/1919010204","state":["exists","up"]},{"osd":7,"uuid":"958184cc-aebf-4c5b-91a0-2a3e38a6b42b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6825","nonce":3475925719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6827","nonce":3475925719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6831","nonce":3475925719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6829","nonce":3475925719}]},"public_addr":"172.21.15.92:6825/3475925719","cluster_addr":"172.21.15.92:6827/3475925719","heartbeat_back_addr":"172.21.15.92:6831/3475925719","heartbeat_front_addr":"172.21.15.92:6829/3475925719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.816294+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.048084+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.137975+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.968246+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.016829+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.931676+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.041673+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.079404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T18:36:53.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.416+0000 7f4f8a25b640 1 -- 172.21.15.90:0/3945133007 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f4f70066e90 msgr2=0x7f4f70069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:53.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.416+0000 7f4f8a25b640 1 --2- 172.21.15.90:0/3945133007 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f4f70066e90 0x7f4f70069350 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4f841f41f0 tx=0x7f4f78009240 comp rx=0 tx=0).stop 2024-03-11T18:36:53.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.416+0000 7f4f8a25b640 1 -- 172.21.15.90:0/3945133007 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f840c9be0 msgr2=0x7f4f84147340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:53.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.416+0000 7f4f8a25b640 1 --2- 172.21.15.90:0/3945133007 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f840c9be0 0x7f4f84147340 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4f7401b8a0 tx=0x7f4f7401ba30 comp rx=0 tx=0).stop 2024-03-11T18:36:53.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.416+0000 7f4f8a25b640 1 -- 172.21.15.90:0/3945133007 shutdown_connections 2024-03-11T18:36:53.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.416+0000 7f4f8a25b640 1 --2- 172.21.15.90:0/3945133007 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f4f70066e90 0x7f4f70069350 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:53.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.416+0000 7f4f8a25b640 1 --2- 172.21.15.90:0/3945133007 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4f84154f50 0x7f4f8414a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:53.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.416+0000 7f4f8a25b640 1 --2- 172.21.15.90:0/3945133007 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4f840cb140 0x7f4f840c1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:53.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.416+0000 7f4f8a25b640 1 --2- 172.21.15.90:0/3945133007 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f840c9be0 0x7f4f84147340 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:53.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.416+0000 7f4f8a25b640 1 -- 172.21.15.90:0/3945133007 >> 172.21.15.90:0/3945133007 conn(0x7f4f8406fa80 msgr2=0x7f4f84071ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:53.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.416+0000 7f4f8a25b640 1 -- 172.21.15.90:0/3945133007 shutdown_connections 2024-03-11T18:36:53.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.416+0000 7f4f8a25b640 1 -- 172.21.15.90:0/3945133007 wait complete. 2024-03-11T18:36:53.449 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-11T18:35:16.193842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-11T18:35:54.001469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-11T18:35:55.917321+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-11T18:36:02.210943+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-11T18:36:53.450 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-11T18:36:53.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.564+0000 7f8beabdc640 1 Processor -- start 2024-03-11T18:36:53.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.564+0000 7f8beabdc640 1 -- start start 2024-03-11T18:36:53.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.568+0000 7f8beabdc640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f8be4151b30 0x7f8be406e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:53.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.568+0000 7f8beabdc640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8be40c4af0 0x7f8be40c4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:53.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.568+0000 7f8beabdc640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8be406f430 con 0x7f8be4070b00 2024-03-11T18:36:53.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.568+0000 7f8beabdc640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8be40c65b0 con 0x7f8be4151b30 2024-03-11T18:36:53.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.568+0000 7f8beabdc640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8be406e820 con 0x7f8be40c4af0 2024-03-11T18:36:53.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.568+0000 7f8be8978640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f8be4151b30 0x7f8be406e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:53.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.568+0000 7f8be3fff640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f8be4070b00 0x7f8be40c5de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:52784/0 (socket says 172.21.15.90:52784) 2024-03-11T18:36:53.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.568+0000 7f8be3fff640 1 -- 172.21.15.90:0/1911824127 learned_addr learned my addr 172.21.15.90:0/1911824127 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T18:36:53.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.572+0000 7f8be9179640 1 --2- 172.21.15.90:0/1911824127 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8be40c4af0 0x7f8be40c4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:53.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.572+0000 7f8be8978640 1 -- 172.21.15.90:0/1911824127 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8be40c4af0 msgr2=0x7f8be40c4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:53.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.572+0000 7f8be8978640 1 --2- 172.21.15.90:0/1911824127 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8be40c4af0 0x7f8be40c4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:53.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.572+0000 7f8be8978640 1 -- 172.21.15.90:0/1911824127 >> v1:172.21.15.90:6789/0 conn(0x7f8be4070b00 legacy=0x7f8be40c5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:53.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.572+0000 7f8be8978640 1 -- 172.21.15.90:0/1911824127 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be406eb00 con 0x7f8be4151b30 2024-03-11T18:36:53.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.584+0000 7f8be8978640 1 --2- 172.21.15.90:0/1911824127 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8be4151b30 0x7f8be406e210 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8bd40097c0 tx=0x7f8bd401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a605ed57862d1948 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:53.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.584+0000 7f8be37fe640 1 -- 172.21.15.90:0/1911824127 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd40ab670 con 0x7f8be4151b30 2024-03-11T18:36:53.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.588+0000 7f8be37fe640 1 -- 172.21.15.90:0/1911824127 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bd40ab7d0 con 0x7f8be4151b30 2024-03-11T18:36:53.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.588+0000 7f8be37fe640 1 -- 172.21.15.90:0/1911824127 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd40abb60 con 0x7f8be4151b30 2024-03-11T18:36:53.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.588+0000 7f8beabdc640 1 -- 172.21.15.90:0/1911824127 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8be4151b30 msgr2=0x7f8be406e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:53.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.588+0000 7f8beabdc640 1 --2- 172.21.15.90:0/1911824127 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8be4151b30 0x7f8be406e210 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8bd40097c0 tx=0x7f8bd401c470 comp rx=0 tx=0).stop 2024-03-11T18:36:53.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.588+0000 7f8beabdc640 1 -- 172.21.15.90:0/1911824127 shutdown_connections 2024-03-11T18:36:53.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.588+0000 7f8beabdc640 1 --2- 172.21.15.90:0/1911824127 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8be40c4af0 0x7f8be40c4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:53.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.588+0000 7f8beabdc640 1 --2- 172.21.15.90:0/1911824127 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8be4151b30 0x7f8be406e210 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:53.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.588+0000 7f8beabdc640 1 -- 172.21.15.90:0/1911824127 >> 172.21.15.90:0/1911824127 conn(0x7f8be406ac00 msgr2=0x7f8be406b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:53.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.588+0000 7f8beabdc640 1 -- 172.21.15.90:0/1911824127 shutdown_connections 2024-03-11T18:36:53.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.588+0000 7f8beabdc640 1 -- 172.21.15.90:0/1911824127 wait complete. 2024-03-11T18:36:53.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.592+0000 7f8beabdc640 1 Processor -- start 2024-03-11T18:36:53.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.592+0000 7f8beabdc640 1 -- start start 2024-03-11T18:36:53.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.592+0000 7f8beabdc640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8be4070b00 0x7f8be40c9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:53.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.592+0000 7f8beabdc640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8be40c4af0 0x7f8be414a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:53.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.592+0000 7f8beabdc640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8be414aa00 0x7f8be40c9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:53.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.592+0000 7f8beabdc640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be40caf20 con 0x7f8be40c4af0 2024-03-11T18:36:53.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.592+0000 7f8beabdc640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be414aeb0 con 0x7f8be414aa00 2024-03-11T18:36:53.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.592+0000 7f8beabdc640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8be4157300 con 0x7f8be4070b00 2024-03-11T18:36:53.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.592+0000 7f8be3fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8be40c4af0 0x7f8be414a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:53.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.592+0000 7f8be3fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8be40c4af0 0x7f8be414a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40862/0 (socket says 172.21.15.90:40862) 2024-03-11T18:36:53.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.592+0000 7f8be3fff640 1 -- 172.21.15.90:0/4189445273 learned_addr learned my addr 172.21.15.90:0/4189445273 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:53.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.592+0000 7f8be8978640 1 --2- 172.21.15.90:0/4189445273 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8be4070b00 0x7f8be40c9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:53.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.592+0000 7f8be9179640 1 --2- 172.21.15.90:0/4189445273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8be414aa00 0x7f8be40c9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:53.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.596+0000 7f8be3fff640 1 -- 172.21.15.90:0/4189445273 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8be4070b00 msgr2=0x7f8be40c9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:53.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.596+0000 7f8be3fff640 1 --2- 172.21.15.90:0/4189445273 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8be4070b00 0x7f8be40c9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:53.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.596+0000 7f8be3fff640 1 -- 172.21.15.90:0/4189445273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8be414aa00 msgr2=0x7f8be40c9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:53.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.596+0000 7f8be3fff640 1 --2- 172.21.15.90:0/4189445273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8be414aa00 0x7f8be40c9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:53.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.596+0000 7f8be3fff640 1 -- 172.21.15.90:0/4189445273 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd4009420 con 0x7f8be40c4af0 2024-03-11T18:36:53.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.600+0000 7f8be3fff640 1 --2- 172.21.15.90:0/4189445273 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8be40c4af0 0x7f8be414a4c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8bd800c430 tx=0x7f8bd800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:53.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.604+0000 7f8be17fa640 1 -- 172.21.15.90:0/4189445273 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd8005cf0 con 0x7f8be40c4af0 2024-03-11T18:36:53.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.604+0000 7f8be17fa640 1 -- 172.21.15.90:0/4189445273 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bd8005e50 con 0x7f8be40c4af0 2024-03-11T18:36:53.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.604+0000 7f8be17fa640 1 -- 172.21.15.90:0/4189445273 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd80072d0 con 0x7f8be40c4af0 2024-03-11T18:36:53.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.604+0000 7f8beabdc640 1 -- 172.21.15.90:0/4189445273 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8be413d060 con 0x7f8be40c4af0 2024-03-11T18:36:53.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.604+0000 7f8beabdc640 1 -- 172.21.15.90:0/4189445273 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8be4156a60 con 0x7f8be40c4af0 2024-03-11T18:36:53.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.604+0000 7f8beabdc640 1 -- 172.21.15.90:0/4189445273 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8be404eef0 con 0x7f8be40c4af0 2024-03-11T18:36:53.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.624+0000 7f8be17fa640 1 -- 172.21.15.90:0/4189445273 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f8bd8007430 con 0x7f8be40c4af0 2024-03-11T18:36:53.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.624+0000 7f8be17fa640 1 --2- 172.21.15.90:0/4189445273 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f8bd006b250 0x7f8bd006d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:53.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.628+0000 7f8be8978640 1 --2- 172.21.15.90:0/4189445273 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f8bd006b250 0x7f8bd006d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:53.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.628+0000 7f8be8978640 1 --2- 172.21.15.90:0/4189445273 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f8bd006b250 0x7f8bd006d710 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8bd40af000 tx=0x7f8bd40a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:53.898 INFO:tasks.ceph.mds.e.smithi090.stderr:2024-03-11T18:36:53.892+0000 f9e1640 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-03-11T18:36:53.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.920+0000 7f8be17fa640 1 -- 172.21.15.90:0/4189445273 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f8bd8055050 con 0x7f8be40c4af0 2024-03-11T18:36:53.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:53.920+0000 7f8be17fa640 1 -- 172.21.15.90:0/4189445273 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8bd80512f0 con 0x7f8be40c4af0 2024-03-11T18:36:54.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.084+0000 7f8beabdc640 1 -- 172.21.15.90:0/4189445273 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8be40c8610 con 0x7f8be40c4af0 2024-03-11T18:36:54.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.092+0000 7f8be17fa640 1 -- 172.21.15.90:0/4189445273 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8bd8017020 con 0x7f8be40c4af0 2024-03-11T18:36:54.101 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 1 2024-03-11T18:36:54.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.096+0000 7f8beabdc640 1 -- 172.21.15.90:0/4189445273 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f8bd006b250 msgr2=0x7f8bd006d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:54.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.096+0000 7f8beabdc640 1 --2- 172.21.15.90:0/4189445273 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f8bd006b250 0x7f8bd006d710 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8bd40af000 tx=0x7f8bd40a7000 comp rx=0 tx=0).stop 2024-03-11T18:36:54.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.096+0000 7f8beabdc640 1 -- 172.21.15.90:0/4189445273 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8be40c4af0 msgr2=0x7f8be414a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:54.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.096+0000 7f8beabdc640 1 --2- 172.21.15.90:0/4189445273 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8be40c4af0 0x7f8be414a4c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8bd800c430 tx=0x7f8bd800c8f0 comp rx=0 tx=0).stop 2024-03-11T18:36:54.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.096+0000 7f8beabdc640 1 -- 172.21.15.90:0/4189445273 shutdown_connections 2024-03-11T18:36:54.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.096+0000 7f8beabdc640 1 --2- 172.21.15.90:0/4189445273 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f8bd006b250 0x7f8bd006d710 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:54.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.096+0000 7f8beabdc640 1 --2- 172.21.15.90:0/4189445273 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8be414aa00 0x7f8be40c9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:54.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.096+0000 7f8beabdc640 1 --2- 172.21.15.90:0/4189445273 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8be40c4af0 0x7f8be414a4c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:54.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.096+0000 7f8beabdc640 1 --2- 172.21.15.90:0/4189445273 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8be4070b00 0x7f8be40c9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:54.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.096+0000 7f8beabdc640 1 -- 172.21.15.90:0/4189445273 >> 172.21.15.90:0/4189445273 conn(0x7f8be406ac00 msgr2=0x7f8be414fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:54.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.096+0000 7f8beabdc640 1 -- 172.21.15.90:0/4189445273 shutdown_connections 2024-03-11T18:36:54.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.096+0000 7f8beabdc640 1 -- 172.21.15.90:0/4189445273 wait complete. 2024-03-11T18:36:54.121 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-11T18:36:54.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.236+0000 7f0a4d4e0640 1 Processor -- start 2024-03-11T18:36:54.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.236+0000 7f0a4d4e0640 1 -- start start 2024-03-11T18:36:54.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.236+0000 7f0a4d4e0640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f0a481541c0 0x7f0a480caf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:54.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.236+0000 7f0a4d4e0640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a480cb5b0 con 0x7f0a480cfda0 2024-03-11T18:36:54.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.236+0000 7f0a4d4e0640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a48159e10 con 0x7f0a480c6b20 2024-03-11T18:36:54.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.236+0000 7f0a4d4e0640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a480b5410 con 0x7f0a481541c0 2024-03-11T18:36:54.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.236+0000 7f0a467fc640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f0a480cfda0 0x7f0a480ca180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:52794/0 (socket says 172.21.15.90:52794) 2024-03-11T18:36:54.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.236+0000 7f0a467fc640 1 -- 172.21.15.90:0/539052533 learned_addr learned my addr 172.21.15.90:0/539052533 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T18:36:54.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.236+0000 7f0a46ffd640 1 --2- 172.21.15.90:0/539052533 >> v2:172.21.15.92:3300/0 conn(0x7f0a481541c0 0x7f0a480caf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:54.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.240+0000 7f0a45ffb640 1 -- 172.21.15.90:0/539052533 <== mon.0 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1682133033 0 0) 0x7f0a480cb5b0 con 0x7f0a480cfda0 2024-03-11T18:36:54.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.240+0000 7f0a45ffb640 1 -- 172.21.15.90:0/539052533 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a34003530 con 0x7f0a480cfda0 2024-03-11T18:36:54.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.244+0000 7f0a45ffb640 1 -- 172.21.15.90:0/539052533 <== mon.0 v1:172.21.15.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3430421611 0 0) 0x7f0a34003530 con 0x7f0a480cfda0 2024-03-11T18:36:54.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.244+0000 7f0a45ffb640 1 -- 172.21.15.90:0/539052533 >> v2:172.21.15.92:3300/0 conn(0x7f0a481541c0 msgr2=0x7f0a480caf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:54.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.244+0000 7f0a45ffb640 1 --2- 172.21.15.90:0/539052533 >> v2:172.21.15.92:3300/0 conn(0x7f0a481541c0 0x7f0a480caf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:54.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.244+0000 7f0a45ffb640 1 -- 172.21.15.90:0/539052533 >> v1:172.21.15.92:6789/0 conn(0x7f0a480c6b20 legacy=0x7f0a4815d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:54.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.244+0000 7f0a45ffb640 1 -- 172.21.15.90:0/539052533 --> v1:172.21.15.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a4815a0f0 con 0x7f0a480cfda0 2024-03-11T18:36:54.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.244+0000 7f0a45ffb640 1 -- 172.21.15.90:0/539052533 <== mon.0 v1:172.21.15.90:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 691735435 0 0) 0x7f0a30001680 con 0x7f0a480cfda0 2024-03-11T18:36:54.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.244+0000 7f0a45ffb640 1 -- 172.21.15.90:0/539052533 >> v1:172.21.15.90:6789/0 conn(0x7f0a480cfda0 legacy=0x7f0a480ca180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:54.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.244+0000 7f0a45ffb640 1 --2- 172.21.15.90:0/539052533 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a340039d0 0x7f0a34003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:54.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.244+0000 7f0a45ffb640 1 --2- 172.21.15.90:0/539052533 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0a340047d0 0x7f0a34004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:54.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.244+0000 7f0a45ffb640 1 --2- 172.21.15.90:0/539052533 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a34005190 0x7f0a3400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:54.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.244+0000 7f0a45ffb640 1 -- 172.21.15.90:0/539052533 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3400c040 con 0x7f0a340039d0 2024-03-11T18:36:54.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.244+0000 7f0a45ffb640 1 -- 172.21.15.90:0/539052533 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3400c1b0 con 0x7f0a34005190 2024-03-11T18:36:54.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.244+0000 7f0a45ffb640 1 -- 172.21.15.90:0/539052533 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3400c370 con 0x7f0a340047d0 2024-03-11T18:36:54.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.244+0000 7f0a467fc640 1 --2- 172.21.15.90:0/539052533 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0a340047d0 0x7f0a34004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:54.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.248+0000 7f0a46ffd640 1 --2- 172.21.15.90:0/539052533 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a340039d0 0x7f0a34003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:54.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.252+0000 7f0a46ffd640 1 -- 172.21.15.90:0/539052533 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0a340047d0 msgr2=0x7f0a34004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:54.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.252+0000 7f0a46ffd640 1 --2- 172.21.15.90:0/539052533 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0a340047d0 0x7f0a34004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:54.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.252+0000 7f0a46ffd640 1 -- 172.21.15.90:0/539052533 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a34005190 msgr2=0x7f0a3400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:54.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.252+0000 7f0a46ffd640 1 --2- 172.21.15.90:0/539052533 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a34005190 0x7f0a3400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:54.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.252+0000 7f0a46ffd640 1 -- 172.21.15.90:0/539052533 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a34003530 con 0x7f0a340039d0 2024-03-11T18:36:54.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.252+0000 7f0a477fe640 1 -- 172.21.15.90:0/539052533 reap_dead start 2024-03-11T18:36:54.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.252+0000 7f0a46ffd640 1 --2- 172.21.15.90:0/539052533 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a340039d0 0x7f0a34003df0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0a40009fc0 tx=0x7f0a4001ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe23268572e5da7 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:54.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.260+0000 7f0a45ffb640 1 -- 172.21.15.90:0/539052533 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a400a5bc0 con 0x7f0a340039d0 2024-03-11T18:36:54.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.260+0000 7f0a45ffb640 1 -- 172.21.15.90:0/539052533 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a4000f040 con 0x7f0a340039d0 2024-03-11T18:36:54.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.260+0000 7f0a45ffb640 1 -- 172.21.15.90:0/539052533 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a400ad440 con 0x7f0a340039d0 2024-03-11T18:36:54.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.260+0000 7f0a4d4e0640 1 -- 172.21.15.90:0/539052533 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a340039d0 msgr2=0x7f0a34003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:54.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.260+0000 7f0a4d4e0640 1 --2- 172.21.15.90:0/539052533 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a340039d0 0x7f0a34003df0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0a40009fc0 tx=0x7f0a4001ccc0 comp rx=0 tx=0).stop 2024-03-11T18:36:54.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.260+0000 7f0a4d4e0640 1 -- 172.21.15.90:0/539052533 shutdown_connections 2024-03-11T18:36:54.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.260+0000 7f0a4d4e0640 1 --2- 172.21.15.90:0/539052533 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a340039d0 0x7f0a34003df0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:54.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.260+0000 7f0a4d4e0640 1 -- 172.21.15.90:0/539052533 >> 172.21.15.90:0/539052533 conn(0x7f0a4806b620 msgr2=0x7f0a4806ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:54.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.260+0000 7f0a4d4e0640 1 -- 172.21.15.90:0/539052533 shutdown_connections 2024-03-11T18:36:54.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.260+0000 7f0a4d4e0640 1 -- 172.21.15.90:0/539052533 wait complete. 2024-03-11T18:36:54.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.260+0000 7f0a4d4e0640 1 Processor -- start 2024-03-11T18:36:54.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.260+0000 7f0a4d4e0640 1 -- start start 2024-03-11T18:36:54.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.264+0000 7f0a4d4e0640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0a340039d0 0x7f0a481f0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:54.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.264+0000 7f0a4d4e0640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a48156e30 0x7f0a48157a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:54.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.264+0000 7f0a4d4e0640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a4814cfd0 0x7f0a4815d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:54.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.264+0000 7f0a4d4e0640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a481f0b60 con 0x7f0a48156e30 2024-03-11T18:36:54.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.264+0000 7f0a4d4e0640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a481569a0 con 0x7f0a4814cfd0 2024-03-11T18:36:54.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.264+0000 7f0a4d4e0640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a48159670 con 0x7f0a340039d0 2024-03-11T18:36:54.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.264+0000 7f0a467fc640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a48156e30 0x7f0a48157a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:54.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.264+0000 7f0a467fc640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a48156e30 0x7f0a48157a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40880/0 (socket says 172.21.15.90:40880) 2024-03-11T18:36:54.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.264+0000 7f0a467fc640 1 -- 172.21.15.90:0/789631597 learned_addr learned my addr 172.21.15.90:0/789631597 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:54.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.264+0000 7f0a46ffd640 1 --2- 172.21.15.90:0/789631597 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0a340039d0 0x7f0a481f0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:54.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.268+0000 7f0a477fe640 1 --2- 172.21.15.90:0/789631597 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a4814cfd0 0x7f0a4815d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:54.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.268+0000 7f0a467fc640 1 -- 172.21.15.90:0/789631597 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0a340039d0 msgr2=0x7f0a481f0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:54.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.268+0000 7f0a467fc640 1 --2- 172.21.15.90:0/789631597 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0a340039d0 0x7f0a481f0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:54.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.268+0000 7f0a467fc640 1 -- 172.21.15.90:0/789631597 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a4814cfd0 msgr2=0x7f0a4815d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:54.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.268+0000 7f0a467fc640 1 --2- 172.21.15.90:0/789631597 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a4814cfd0 0x7f0a4815d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:54.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.268+0000 7f0a467fc640 1 -- 172.21.15.90:0/789631597 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a48159950 con 0x7f0a48156e30 2024-03-11T18:36:54.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.268+0000 7f0a467fc640 1 --2- 172.21.15.90:0/789631597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a48156e30 0x7f0a48157a30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0a30012490 tx=0x7f0a30012950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:54.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.272+0000 7f0a3f7fe640 1 -- 172.21.15.90:0/789631597 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a30014810 con 0x7f0a48156e30 2024-03-11T18:36:54.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.272+0000 7f0a3f7fe640 1 -- 172.21.15.90:0/789631597 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a30014990 con 0x7f0a48156e30 2024-03-11T18:36:54.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.272+0000 7f0a3f7fe640 1 -- 172.21.15.90:0/789631597 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a30014d40 con 0x7f0a48156e30 2024-03-11T18:36:54.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.272+0000 7f0a4d4e0640 1 -- 172.21.15.90:0/789631597 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a48156070 con 0x7f0a48156e30 2024-03-11T18:36:54.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.272+0000 7f0a4d4e0640 1 -- 172.21.15.90:0/789631597 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a48156630 con 0x7f0a48156e30 2024-03-11T18:36:54.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.276+0000 7f0a4d4e0640 1 -- 172.21.15.90:0/789631597 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a481567f0 con 0x7f0a48156e30 2024-03-11T18:36:54.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.292+0000 7f0a3f7fe640 1 -- 172.21.15.90:0/789631597 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f0a300153e0 con 0x7f0a48156e30 2024-03-11T18:36:54.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.296+0000 7f0a3f7fe640 1 --2- 172.21.15.90:0/789631597 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0a28066ec0 0x7f0a28069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:54.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.296+0000 7f0a46ffd640 1 --2- 172.21.15.90:0/789631597 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0a28066ec0 0x7f0a28069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:54.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.296+0000 7f0a46ffd640 1 --2- 172.21.15.90:0/789631597 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0a28066ec0 0x7f0a28069380 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0a400af000 tx=0x7f0a40004ac0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:54.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.568+0000 7f0a3f7fe640 1 -- 172.21.15.90:0/789631597 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0a30014ea0 con 0x7f0a48156e30 2024-03-11T18:36:54.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.568+0000 7f0a3f7fe640 1 -- 172.21.15.90:0/789631597 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0a30054b30 con 0x7f0a48156e30 2024-03-11T18:36:54.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.744+0000 7f0a4d4e0640 1 -- 172.21.15.90:0/789631597 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0a4804eef0 con 0x7f0a48156e30 2024-03-11T18:36:54.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.780+0000 7f0a3f7fe640 1 -- 172.21.15.90:0/789631597 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0a3001c070 con 0x7f0a48156e30 2024-03-11T18:36:54.785 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 32 2024-03-11T18:36:54.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.780+0000 7f0a4d4e0640 1 -- 172.21.15.90:0/789631597 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0a28066ec0 msgr2=0x7f0a28069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:54.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.780+0000 7f0a4d4e0640 1 --2- 172.21.15.90:0/789631597 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0a28066ec0 0x7f0a28069380 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0a400af000 tx=0x7f0a40004ac0 comp rx=0 tx=0).stop 2024-03-11T18:36:54.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.780+0000 7f0a4d4e0640 1 -- 172.21.15.90:0/789631597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a48156e30 msgr2=0x7f0a48157a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:54.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.780+0000 7f0a4d4e0640 1 --2- 172.21.15.90:0/789631597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a48156e30 0x7f0a48157a30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0a30012490 tx=0x7f0a30012950 comp rx=0 tx=0).stop 2024-03-11T18:36:54.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.780+0000 7f0a4d4e0640 1 -- 172.21.15.90:0/789631597 shutdown_connections 2024-03-11T18:36:54.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.780+0000 7f0a4d4e0640 1 --2- 172.21.15.90:0/789631597 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0a28066ec0 0x7f0a28069380 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:54.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.780+0000 7f0a4d4e0640 1 --2- 172.21.15.90:0/789631597 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0a4814cfd0 0x7f0a4815d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:54.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.780+0000 7f0a4d4e0640 1 --2- 172.21.15.90:0/789631597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a48156e30 0x7f0a48157a30 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:54.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.780+0000 7f0a4d4e0640 1 --2- 172.21.15.90:0/789631597 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0a340039d0 0x7f0a481f0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:54.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.780+0000 7f0a4d4e0640 1 -- 172.21.15.90:0/789631597 >> 172.21.15.90:0/789631597 conn(0x7f0a4806b620 msgr2=0x7f0a480c6fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:54.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.780+0000 7f0a4d4e0640 1 -- 172.21.15.90:0/789631597 shutdown_connections 2024-03-11T18:36:54.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.780+0000 7f0a4d4e0640 1 -- 172.21.15.90:0/789631597 wait complete. 2024-03-11T18:36:54.805 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-11T18:36:54.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.920+0000 7f999b364640 1 Processor -- start 2024-03-11T18:36:54.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.920+0000 7f999b364640 1 -- start start 2024-03-11T18:36:54.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.920+0000 7f999b364640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9994150a90 0x7f9994157220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:54.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.920+0000 7f999b364640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f99940c65b0 0x7f99940c37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:54.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.920+0000 7f999b364640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99940c5e60 con 0x7f99940c3d30 2024-03-11T18:36:54.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.920+0000 7f999b364640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9994156a60 con 0x7f99940c65b0 2024-03-11T18:36:54.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.920+0000 7f999b364640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99940c55c0 con 0x7f9994150a90 2024-03-11T18:36:54.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.920+0000 7f99988ff640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f99940c65b0 0x7f99940c37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:54.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.920+0000 7f99988ff640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f99940c65b0 0x7f99940c37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40892/0 (socket says 172.21.15.90:40892) 2024-03-11T18:36:54.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.920+0000 7f99988ff640 1 -- 172.21.15.90:0/1388093495 learned_addr learned my addr 172.21.15.90:0/1388093495 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:54.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.924+0000 7f99988ff640 1 -- 172.21.15.90:0/1388093495 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9994150a90 msgr2=0x7f9994157220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:54.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.924+0000 7f99988ff640 1 --2- 172.21.15.90:0/1388093495 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9994150a90 0x7f9994157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:54.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.924+0000 7f99988ff640 1 -- 172.21.15.90:0/1388093495 >> v1:172.21.15.92:6789/0 conn(0x7f99940c3d30 legacy=0x7f99940cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:54.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.924+0000 7f99988ff640 1 -- 172.21.15.90:0/1388093495 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99940c58a0 con 0x7f99940c65b0 2024-03-11T18:36:54.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.928+0000 7f99988ff640 1 --2- 172.21.15.90:0/1388093495 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99940c65b0 0x7f99940c37f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f998400bb40 tx=0x7f998401ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cbcb58cbf74b56c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:54.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.932+0000 7f998bfff640 1 -- 172.21.15.90:0/1388093495 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99840a7940 con 0x7f99940c65b0 2024-03-11T18:36:54.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.932+0000 7f998bfff640 1 -- 172.21.15.90:0/1388093495 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99840a7aa0 con 0x7f99940c65b0 2024-03-11T18:36:54.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.932+0000 7f998bfff640 1 -- 172.21.15.90:0/1388093495 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99840a7e30 con 0x7f99940c65b0 2024-03-11T18:36:54.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.932+0000 7f999b364640 1 -- 172.21.15.90:0/1388093495 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99940c65b0 msgr2=0x7f99940c37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:54.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.932+0000 7f999b364640 1 --2- 172.21.15.90:0/1388093495 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99940c65b0 0x7f99940c37f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f998400bb40 tx=0x7f998401ebd0 comp rx=0 tx=0).stop 2024-03-11T18:36:54.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.932+0000 7f999b364640 1 -- 172.21.15.90:0/1388093495 shutdown_connections 2024-03-11T18:36:54.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.932+0000 7f999b364640 1 --2- 172.21.15.90:0/1388093495 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99940c65b0 0x7f99940c37f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:54.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.932+0000 7f999b364640 1 --2- 172.21.15.90:0/1388093495 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9994150a90 0x7f9994157220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:54.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.932+0000 7f999b364640 1 -- 172.21.15.90:0/1388093495 >> 172.21.15.90:0/1388093495 conn(0x7f9994071aa0 msgr2=0x7f9994071eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:54.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.932+0000 7f999b364640 1 -- 172.21.15.90:0/1388093495 shutdown_connections 2024-03-11T18:36:54.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.932+0000 7f999b364640 1 -- 172.21.15.90:0/1388093495 wait complete. 2024-03-11T18:36:54.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.936+0000 7f999b364640 1 Processor -- start 2024-03-11T18:36:54.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.936+0000 7f999b364640 1 -- start start 2024-03-11T18:36:54.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.936+0000 7f999b364640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f99940c3d30 0x7f99941eefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:54.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.936+0000 7f999b364640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99940c65b0 0x7f9994141980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:54.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.936+0000 7f999b364640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9994150a90 0x7f99941436a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:54.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.936+0000 7f999b364640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99941ee600 con 0x7f99940c65b0 2024-03-11T18:36:54.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.936+0000 7f999b364640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9994142b20 con 0x7f9994150a90 2024-03-11T18:36:54.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.936+0000 7f999b364640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f999413ea50 con 0x7f99940c3d30 2024-03-11T18:36:54.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.936+0000 7f99988ff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99940c65b0 0x7f9994141980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:54.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.936+0000 7f99988ff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99940c65b0 0x7f9994141980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40896/0 (socket says 172.21.15.90:40896) 2024-03-11T18:36:54.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.936+0000 7f99988ff640 1 -- 172.21.15.90:0/4279232283 learned_addr learned my addr 172.21.15.90:0/4279232283 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:54.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.940+0000 7f99988ff640 1 -- 172.21.15.90:0/4279232283 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f99940c3d30 msgr2=0x7f99941eefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:54.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.940+0000 7f99988ff640 1 --2- 172.21.15.90:0/4279232283 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f99940c3d30 0x7f99941eefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:54.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.940+0000 7f99988ff640 1 -- 172.21.15.90:0/4279232283 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9994150a90 msgr2=0x7f99941436a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:54.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.940+0000 7f99988ff640 1 --2- 172.21.15.90:0/4279232283 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9994150a90 0x7f99941436a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:54.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.940+0000 7f99988ff640 1 -- 172.21.15.90:0/4279232283 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f998400b770 con 0x7f99940c65b0 2024-03-11T18:36:54.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.940+0000 7f99988ff640 1 --2- 172.21.15.90:0/4279232283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99940c65b0 0x7f9994141980 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f998400b2c0 tx=0x7f998401b970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:54.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.944+0000 7f9989ffb640 1 -- 172.21.15.90:0/4279232283 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99840afe60 con 0x7f99940c65b0 2024-03-11T18:36:54.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.944+0000 7f999b364640 1 -- 172.21.15.90:0/4279232283 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99941f2a70 con 0x7f99940c65b0 2024-03-11T18:36:54.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.944+0000 7f999b364640 1 -- 172.21.15.90:0/4279232283 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99941ee810 con 0x7f99940c65b0 2024-03-11T18:36:54.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.944+0000 7f999b364640 1 -- 172.21.15.90:0/4279232283 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f999404eef0 con 0x7f99940c65b0 2024-03-11T18:36:54.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.948+0000 7f9989ffb640 1 -- 172.21.15.90:0/4279232283 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99840d0430 con 0x7f99940c65b0 2024-03-11T18:36:54.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:54.948+0000 7f9989ffb640 1 -- 172.21.15.90:0/4279232283 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99840d07e0 con 0x7f99940c65b0 2024-03-11T18:36:55.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.232+0000 7f9989ffb640 1 -- 172.21.15.90:0/4279232283 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f99840b6050 con 0x7f99940c65b0 2024-03-11T18:36:55.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.236+0000 7f9989ffb640 1 --2- 172.21.15.90:0/4279232283 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f9978066e40 0x7f9978069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:55.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.236+0000 7f9999100640 1 --2- 172.21.15.90:0/4279232283 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f9978066e40 0x7f9978069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:55.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.236+0000 7f9999100640 1 --2- 172.21.15.90:0/4279232283 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f9978066e40 0x7f9978069300 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9990009690 tx=0x7f9990006c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:55.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.272+0000 7f9989ffb640 1 -- 172.21.15.90:0/4279232283 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f99840f3880 con 0x7f99940c65b0 2024-03-11T18:36:55.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.272+0000 7f9989ffb640 1 -- 172.21.15.90:0/4279232283 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9984120e60 con 0x7f99940c65b0 2024-03-11T18:36:55.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.440+0000 7f999b364640 1 -- 172.21.15.90:0/4279232283 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f999404eef0 con 0x7f99940c65b0 2024-03-11T18:36:55.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.456+0000 7f9989ffb640 1 -- 172.21.15.90:0/4279232283 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f99840b4070 con 0x7f99940c65b0 2024-03-11T18:36:55.462 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 64 2024-03-11T18:36:55.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.456+0000 7f999b364640 1 -- 172.21.15.90:0/4279232283 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f9978066e40 msgr2=0x7f9978069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:55.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.456+0000 7f999b364640 1 --2- 172.21.15.90:0/4279232283 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f9978066e40 0x7f9978069300 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9990009690 tx=0x7f9990006c00 comp rx=0 tx=0).stop 2024-03-11T18:36:55.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.456+0000 7f999b364640 1 -- 172.21.15.90:0/4279232283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99940c65b0 msgr2=0x7f9994141980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:55.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.456+0000 7f999b364640 1 --2- 172.21.15.90:0/4279232283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99940c65b0 0x7f9994141980 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f998400b2c0 tx=0x7f998401b970 comp rx=0 tx=0).stop 2024-03-11T18:36:55.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.456+0000 7f999b364640 1 -- 172.21.15.90:0/4279232283 shutdown_connections 2024-03-11T18:36:55.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.456+0000 7f999b364640 1 --2- 172.21.15.90:0/4279232283 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f9978066e40 0x7f9978069300 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:55.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.456+0000 7f999b364640 1 --2- 172.21.15.90:0/4279232283 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9994150a90 0x7f99941436a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:55.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.456+0000 7f999b364640 1 --2- 172.21.15.90:0/4279232283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99940c65b0 0x7f9994141980 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:55.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.456+0000 7f999b364640 1 --2- 172.21.15.90:0/4279232283 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f99940c3d30 0x7f99941eefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:55.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.456+0000 7f999b364640 1 -- 172.21.15.90:0/4279232283 >> 172.21.15.90:0/4279232283 conn(0x7f9994071aa0 msgr2=0x7f99940c6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:55.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.456+0000 7f999b364640 1 -- 172.21.15.90:0/4279232283 shutdown_connections 2024-03-11T18:36:55.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.456+0000 7f999b364640 1 -- 172.21.15.90:0/4279232283 wait complete. 2024-03-11T18:36:55.481 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-11T18:36:55.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.596+0000 7f355a059640 1 Processor -- start 2024-03-11T18:36:55.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.600+0000 7f355a059640 1 -- start start 2024-03-11T18:36:55.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.600+0000 7f355a059640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3554150c90 0x7f3554159f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:55.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.600+0000 7f355a059640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f35540c7f20 0x7f35540c8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:55.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.600+0000 7f355a059640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35540c5be0 con 0x7f35540c6070 2024-03-11T18:36:55.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.600+0000 7f355a059640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f355406cad0 con 0x7f35540c7f20 2024-03-11T18:36:55.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.600+0000 7f355a059640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f355406c230 con 0x7f3554150c90 2024-03-11T18:36:55.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.600+0000 7f3553fff640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f35540c7f20 0x7f35540c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:55.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.600+0000 7f3552ffd640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f35540c6070 0x7f35540c3960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:52796/0 (socket says 172.21.15.90:52796) 2024-03-11T18:36:55.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.600+0000 7f3552ffd640 1 -- 172.21.15.90:0/3388464389 learned_addr learned my addr 172.21.15.90:0/3388464389 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T18:36:55.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.600+0000 7f35537fe640 1 --2- 172.21.15.90:0/3388464389 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3554150c90 0x7f3554159f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:55.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.604+0000 7f3553fff640 1 -- 172.21.15.90:0/3388464389 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3554150c90 msgr2=0x7f3554159f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:55.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.604+0000 7f3553fff640 1 --2- 172.21.15.90:0/3388464389 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3554150c90 0x7f3554159f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:55.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.604+0000 7f3553fff640 1 -- 172.21.15.90:0/3388464389 >> v1:172.21.15.90:6789/0 conn(0x7f35540c6070 legacy=0x7f35540c3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:55.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.604+0000 7f3553fff640 1 -- 172.21.15.90:0/3388464389 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f355406c510 con 0x7f35540c7f20 2024-03-11T18:36:55.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.612+0000 7f3553fff640 1 --2- 172.21.15.90:0/3388464389 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35540c7f20 0x7f35540c8450 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f354c009d90 tx=0x7f354c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d876b6cad1de2635 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:55.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.616+0000 7f35527fc640 1 -- 172.21.15.90:0/3388464389 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354c0a5940 con 0x7f35540c7f20 2024-03-11T18:36:55.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.616+0000 7f35527fc640 1 -- 172.21.15.90:0/3388464389 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f354c0a5aa0 con 0x7f35540c7f20 2024-03-11T18:36:55.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.616+0000 7f35527fc640 1 -- 172.21.15.90:0/3388464389 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354c0a5e30 con 0x7f35540c7f20 2024-03-11T18:36:55.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.616+0000 7f355a059640 1 -- 172.21.15.90:0/3388464389 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35540c7f20 msgr2=0x7f35540c8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:55.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.616+0000 7f355a059640 1 --2- 172.21.15.90:0/3388464389 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35540c7f20 0x7f35540c8450 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f354c009d90 tx=0x7f354c01c970 comp rx=0 tx=0).stop 2024-03-11T18:36:55.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.616+0000 7f355a059640 1 -- 172.21.15.90:0/3388464389 shutdown_connections 2024-03-11T18:36:55.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.616+0000 7f355a059640 1 --2- 172.21.15.90:0/3388464389 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35540c7f20 0x7f35540c8450 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:55.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.616+0000 7f355a059640 1 --2- 172.21.15.90:0/3388464389 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3554150c90 0x7f3554159f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:55.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.616+0000 7f355a059640 1 -- 172.21.15.90:0/3388464389 >> 172.21.15.90:0/3388464389 conn(0x7f35540cc630 msgr2=0x7f35540cca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:55.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.616+0000 7f355a059640 1 -- 172.21.15.90:0/3388464389 shutdown_connections 2024-03-11T18:36:55.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.616+0000 7f355a059640 1 -- 172.21.15.90:0/3388464389 wait complete. 2024-03-11T18:36:55.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.616+0000 7f355a059640 1 Processor -- start 2024-03-11T18:36:55.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.616+0000 7f355a059640 1 -- start start 2024-03-11T18:36:55.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.620+0000 7f355a059640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f35540c6070 0x7f35541457b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:55.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.620+0000 7f355a059640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35540c7f20 0x7f3554145cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:55.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.620+0000 7f355a059640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3554150c90 0x7f3554144500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:55.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.620+0000 7f355a059640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35541f69a0 con 0x7f3554150c90 2024-03-11T18:36:55.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.620+0000 7f355a059640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3554146560 con 0x7f35540c7f20 2024-03-11T18:36:55.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.620+0000 7f355a059640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3554146840 con 0x7f35540c6070 2024-03-11T18:36:55.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.620+0000 7f3553fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3554150c90 0x7f3554144500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:55.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.620+0000 7f3553fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3554150c90 0x7f3554144500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40908/0 (socket says 172.21.15.90:40908) 2024-03-11T18:36:55.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.620+0000 7f3553fff640 1 -- 172.21.15.90:0/755696984 learned_addr learned my addr 172.21.15.90:0/755696984 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:55.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.620+0000 7f3552ffd640 1 --2- 172.21.15.90:0/755696984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35540c7f20 0x7f3554145cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:55.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.620+0000 7f35537fe640 1 --2- 172.21.15.90:0/755696984 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f35540c6070 0x7f35541457b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:55.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.624+0000 7f3553fff640 1 -- 172.21.15.90:0/755696984 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f35540c6070 msgr2=0x7f35541457b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:55.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.624+0000 7f3553fff640 1 --2- 172.21.15.90:0/755696984 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f35540c6070 0x7f35541457b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:55.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.624+0000 7f3553fff640 1 -- 172.21.15.90:0/755696984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35540c7f20 msgr2=0x7f3554145cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:55.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.624+0000 7f3553fff640 1 --2- 172.21.15.90:0/755696984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35540c7f20 0x7f3554145cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:55.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.624+0000 7f3553fff640 1 -- 172.21.15.90:0/755696984 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f354c009a70 con 0x7f3554150c90 2024-03-11T18:36:55.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.624+0000 7f3553fff640 1 --2- 172.21.15.90:0/755696984 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3554150c90 0x7f3554144500 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f354c01b020 tx=0x7f354c019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:55.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.628+0000 7f353bfff640 1 -- 172.21.15.90:0/755696984 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354c0add90 con 0x7f3554150c90 2024-03-11T18:36:55.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.628+0000 7f353bfff640 1 -- 172.21.15.90:0/755696984 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f354c009450 con 0x7f3554150c90 2024-03-11T18:36:55.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.628+0000 7f353bfff640 1 -- 172.21.15.90:0/755696984 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354c0b9670 con 0x7f3554150c90 2024-03-11T18:36:55.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.628+0000 7f355a059640 1 -- 172.21.15.90:0/755696984 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3554146b20 con 0x7f3554150c90 2024-03-11T18:36:55.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.628+0000 7f355a059640 1 -- 172.21.15.90:0/755696984 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3554143fe0 con 0x7f3554150c90 2024-03-11T18:36:55.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.632+0000 7f355a059640 1 -- 172.21.15.90:0/755696984 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35541441d0 con 0x7f3554150c90 2024-03-11T18:36:55.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.912+0000 7f353bfff640 1 -- 172.21.15.90:0/755696984 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f354c0b97d0 con 0x7f3554150c90 2024-03-11T18:36:55.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.912+0000 7f353bfff640 1 --2- 172.21.15.90:0/755696984 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f3540066e90 0x7f3540069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:55.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.912+0000 7f35537fe640 1 --2- 172.21.15.90:0/755696984 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f3540066e90 0x7f3540069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:55.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.916+0000 7f35537fe640 1 --2- 172.21.15.90:0/755696984 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f3540066e90 0x7f3540069350 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3544009ac0 tx=0x7f3544009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:55.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.948+0000 7f353bfff640 1 -- 172.21.15.90:0/755696984 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f354c0f1640 con 0x7f3554150c90 2024-03-11T18:36:55.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:55.948+0000 7f353bfff640 1 -- 172.21.15.90:0/755696984 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f354c0ed8e0 con 0x7f3554150c90 2024-03-11T18:36:56.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.120+0000 7f355a059640 1 -- 172.21.15.90:0/755696984 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f355404eef0 con 0x7f3554150c90 2024-03-11T18:36:56.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.132+0000 7f353bfff640 1 -- 172.21.15.90:0/755696984 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f354c0b2020 con 0x7f3554150c90 2024-03-11T18:36:56.139 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 32 2024-03-11T18:36:56.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.136+0000 7f355a059640 1 -- 172.21.15.90:0/755696984 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f3540066e90 msgr2=0x7f3540069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:56.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.136+0000 7f355a059640 1 --2- 172.21.15.90:0/755696984 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f3540066e90 0x7f3540069350 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3544009ac0 tx=0x7f3544009190 comp rx=0 tx=0).stop 2024-03-11T18:36:56.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.136+0000 7f355a059640 1 -- 172.21.15.90:0/755696984 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3554150c90 msgr2=0x7f3554144500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:56.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.136+0000 7f355a059640 1 --2- 172.21.15.90:0/755696984 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3554150c90 0x7f3554144500 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f354c01b020 tx=0x7f354c019710 comp rx=0 tx=0).stop 2024-03-11T18:36:56.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.136+0000 7f355a059640 1 -- 172.21.15.90:0/755696984 shutdown_connections 2024-03-11T18:36:56.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.136+0000 7f355a059640 1 --2- 172.21.15.90:0/755696984 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f3540066e90 0x7f3540069350 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:56.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.136+0000 7f355a059640 1 --2- 172.21.15.90:0/755696984 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3554150c90 0x7f3554144500 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:56.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.136+0000 7f355a059640 1 --2- 172.21.15.90:0/755696984 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f35540c7f20 0x7f3554145cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:56.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.136+0000 7f355a059640 1 --2- 172.21.15.90:0/755696984 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f35540c6070 0x7f35541457b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:56.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.136+0000 7f355a059640 1 -- 172.21.15.90:0/755696984 >> 172.21.15.90:0/755696984 conn(0x7f35540cc630 msgr2=0x7f35540c7560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:56.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.136+0000 7f355a059640 1 -- 172.21.15.90:0/755696984 shutdown_connections 2024-03-11T18:36:56.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.136+0000 7f355a059640 1 -- 172.21.15.90:0/755696984 wait complete. 2024-03-11T18:36:56.157 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T18:36:56.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.272+0000 7f90fabe9640 1 Processor -- start 2024-03-11T18:36:56.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.272+0000 7f90fabe9640 1 -- start start 2024-03-11T18:36:56.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.272+0000 7f90fabe9640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f90f4070b00 0x7f90f406e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:56.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.272+0000 7f90fabe9640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f90f40c4af0 0x7f90f40c4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:56.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.272+0000 7f90fabe9640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90f406f430 con 0x7f90f4151b30 2024-03-11T18:36:56.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.272+0000 7f90fabe9640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90f40c65b0 con 0x7f90f4070b00 2024-03-11T18:36:56.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.272+0000 7f90fabe9640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90f406e820 con 0x7f90f40c4af0 2024-03-11T18:36:56.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.272+0000 7f90f3fff640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f90f4070b00 0x7f90f406e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:56.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.272+0000 7f90f3fff640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f90f4070b00 0x7f90f406e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40918/0 (socket says 172.21.15.90:40918) 2024-03-11T18:36:56.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.272+0000 7f90f3fff640 1 -- 172.21.15.90:0/1841367435 learned_addr learned my addr 172.21.15.90:0/1841367435 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:56.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.276+0000 7f90f3fff640 1 -- 172.21.15.90:0/1841367435 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f90f40c4af0 msgr2=0x7f90f40c4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:56.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.276+0000 7f90f3fff640 1 --2- 172.21.15.90:0/1841367435 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f90f40c4af0 0x7f90f40c4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:56.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.276+0000 7f90f3fff640 1 -- 172.21.15.90:0/1841367435 >> v1:172.21.15.90:6789/0 conn(0x7f90f4151b30 legacy=0x7f90f40c5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:56.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.276+0000 7f90f3fff640 1 -- 172.21.15.90:0/1841367435 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f406eb00 con 0x7f90f4070b00 2024-03-11T18:36:56.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.280+0000 7f90f3fff640 1 --2- 172.21.15.90:0/1841367435 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f90f4070b00 0x7f90f406e210 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f90e8009870 tx=0x7f90e801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e97cd2425dc7217 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:56.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.288+0000 7f90f37fe640 1 -- 172.21.15.90:0/1841367435 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e80a95f0 con 0x7f90f4070b00 2024-03-11T18:36:56.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.288+0000 7f90f37fe640 1 -- 172.21.15.90:0/1841367435 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90e80a9750 con 0x7f90f4070b00 2024-03-11T18:36:56.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.288+0000 7f90f37fe640 1 -- 172.21.15.90:0/1841367435 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e80a9ae0 con 0x7f90f4070b00 2024-03-11T18:36:56.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.288+0000 7f90fabe9640 1 -- 172.21.15.90:0/1841367435 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f90f4070b00 msgr2=0x7f90f406e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:56.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.288+0000 7f90fabe9640 1 --2- 172.21.15.90:0/1841367435 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f90f4070b00 0x7f90f406e210 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f90e8009870 tx=0x7f90e801c530 comp rx=0 tx=0).stop 2024-03-11T18:36:56.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.288+0000 7f90fabe9640 1 -- 172.21.15.90:0/1841367435 shutdown_connections 2024-03-11T18:36:56.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.288+0000 7f90fabe9640 1 --2- 172.21.15.90:0/1841367435 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f90f40c4af0 0x7f90f40c4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:56.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.288+0000 7f90fabe9640 1 --2- 172.21.15.90:0/1841367435 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f90f4070b00 0x7f90f406e210 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:56.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.288+0000 7f90fabe9640 1 -- 172.21.15.90:0/1841367435 >> 172.21.15.90:0/1841367435 conn(0x7f90f406ac00 msgr2=0x7f90f406b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:56.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.288+0000 7f90fabe9640 1 -- 172.21.15.90:0/1841367435 shutdown_connections 2024-03-11T18:36:56.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.288+0000 7f90fabe9640 1 -- 172.21.15.90:0/1841367435 wait complete. 2024-03-11T18:36:56.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.288+0000 7f90fabe9640 1 Processor -- start 2024-03-11T18:36:56.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.288+0000 7f90fabe9640 1 -- start start 2024-03-11T18:36:56.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.292+0000 7f90fabe9640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f90f4070b00 0x7f90f40c3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:56.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.292+0000 7f90fabe9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90f40c4af0 0x7f90f40c9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:56.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.292+0000 7f90fabe9640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f90f4151b30 0x7f90f40ca540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:56.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.292+0000 7f90fabe9640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f41f2910 con 0x7f90f4151b30 2024-03-11T18:36:56.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.292+0000 7f90fabe9640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f41f2b20 con 0x7f90f40c4af0 2024-03-11T18:36:56.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.292+0000 7f90fabe9640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90f41f2e00 con 0x7f90f4070b00 2024-03-11T18:36:56.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.292+0000 7f90f9186640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f90f4151b30 0x7f90f40ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:56.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.292+0000 7f90f9186640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f90f4151b30 0x7f90f40ca540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40922/0 (socket says 172.21.15.90:40922) 2024-03-11T18:36:56.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.292+0000 7f90f9186640 1 -- 172.21.15.90:0/1580120493 learned_addr learned my addr 172.21.15.90:0/1580120493 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:56.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.296+0000 7f90f9186640 1 -- 172.21.15.90:0/1580120493 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f90f4070b00 msgr2=0x7f90f40c3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:56.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.296+0000 7f90f9186640 1 --2- 172.21.15.90:0/1580120493 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f90f4070b00 0x7f90f40c3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:56.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.296+0000 7f90f9186640 1 -- 172.21.15.90:0/1580120493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90f40c4af0 msgr2=0x7f90f40c9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:56.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.296+0000 7f90f9186640 1 --2- 172.21.15.90:0/1580120493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90f40c4af0 0x7f90f40c9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:56.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.296+0000 7f90f9186640 1 -- 172.21.15.90:0/1580120493 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e80094d0 con 0x7f90f4151b30 2024-03-11T18:36:56.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.300+0000 7f90f9186640 1 --2- 172.21.15.90:0/1580120493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f90f4151b30 0x7f90f40ca540 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f90ec00b970 tx=0x7f90ec00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:56.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.304+0000 7f90f17fa640 1 -- 172.21.15.90:0/1580120493 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ec00cb40 con 0x7f90f4151b30 2024-03-11T18:36:56.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.304+0000 7f90f17fa640 1 -- 172.21.15.90:0/1580120493 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90ec00ccc0 con 0x7f90f4151b30 2024-03-11T18:36:56.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.304+0000 7f90f17fa640 1 -- 172.21.15.90:0/1580120493 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ec014680 con 0x7f90f4151b30 2024-03-11T18:36:56.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.304+0000 7f90fabe9640 1 -- 172.21.15.90:0/1580120493 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90f41f30e0 con 0x7f90f4151b30 2024-03-11T18:36:56.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.304+0000 7f90fabe9640 1 -- 172.21.15.90:0/1580120493 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90f41f36a0 con 0x7f90f4151b30 2024-03-11T18:36:56.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.304+0000 7f90fabe9640 1 -- 172.21.15.90:0/1580120493 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90f404eef0 con 0x7f90f4151b30 2024-03-11T18:36:56.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.324+0000 7f90f17fa640 1 -- 172.21.15.90:0/1580120493 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f90ec01d3b0 con 0x7f90f4151b30 2024-03-11T18:36:56.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.324+0000 7f90f17fa640 1 --2- 172.21.15.90:0/1580120493 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f90e0066e90 0x7f90e0069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:56.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.324+0000 7f90f8985640 1 --2- 172.21.15.90:0/1580120493 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f90e0066e90 0x7f90e0069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:56.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.328+0000 7f90f8985640 1 --2- 172.21.15.90:0/1580120493 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f90e0066e90 0x7f90e0069350 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f90e4000f70 tx=0x7f90e40079e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:56.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.328+0000 7f90f17fa640 1 -- 172.21.15.90:0/1580120493 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f90ec00ce30 con 0x7f90f4151b30 2024-03-11T18:36:56.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.704+0000 7f90f17fa640 1 -- 172.21.15.90:0/1580120493 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f90ec051c40 con 0x7f90f4151b30 2024-03-11T18:36:56.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.912+0000 7f90fabe9640 1 -- 172.21.15.90:0/1580120493 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f90f40c8b80 con 0x7f90f4151b30 2024-03-11T18:36:56.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.952+0000 7f90f17fa640 1 -- 172.21.15.90:0/1580120493 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+6575 (secure 0 0 0) 0x7f90ec019070 con 0x7f90f4151b30 2024-03-11T18:36:56.959 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:36:56.960 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/641527992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":641527992},{"type":"v1","addr":"172.21.15.90:6839","nonce":641527992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-11T18:35:57.363570+0000","modified":"2024-03-11T18:36:51.575661+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4255,"mds_1":4253,"mds_2":4241,"mds_3":4238,"mds_4":4229},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4229":{"gid":4229,"name":"c","rank":4,"incarnation":18,"state":"up:active","state_seq":17,"addr":"172.21.15.90:6835/460417177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":460417177},{"type":"v1","addr":"172.21.15.90:6835","nonce":460417177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4238":{"gid":4238,"name":"d","rank":3,"incarnation":16,"state":"up:active","state_seq":17,"addr":"172.21.15.92:6835/3568438077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":3568438077},{"type":"v1","addr":"172.21.15.92:6835","nonce":3568438077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4241":{"gid":4241,"name":"b","rank":2,"incarnation":14,"state":"up:active","state_seq":16,"addr":"172.21.15.92:6834/2306487548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":2306487548},{"type":"v1","addr":"172.21.15.92:6834","nonce":2306487548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4253":{"gid":4253,"name":"f","rank":1,"incarnation":10,"state":"up:active","state_seq":15,"addr":"172.21.15.92:6837/3209534444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":3209534444},{"type":"v1","addr":"172.21.15.92:6837","nonce":3209534444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4255":{"gid":4255,"name":"e","rank":0,"incarnation":5,"state":"up:active","state_seq":6,"addr":"172.21.15.90:6837/3485943375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3485943375},{"type":"v1","addr":"172.21.15.90:6837","nonce":3485943375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T18:36:56.960 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 19 2024-03-11T18:36:56.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.956+0000 7f90fabe9640 1 -- 172.21.15.90:0/1580120493 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f90e0066e90 msgr2=0x7f90e0069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:56.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.956+0000 7f90fabe9640 1 --2- 172.21.15.90:0/1580120493 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f90e0066e90 0x7f90e0069350 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f90e4000f70 tx=0x7f90e40079e0 comp rx=0 tx=0).stop 2024-03-11T18:36:56.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.956+0000 7f90fabe9640 1 -- 172.21.15.90:0/1580120493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f90f4151b30 msgr2=0x7f90f40ca540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:56.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.956+0000 7f90fabe9640 1 --2- 172.21.15.90:0/1580120493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f90f4151b30 0x7f90f40ca540 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f90ec00b970 tx=0x7f90ec00be30 comp rx=0 tx=0).stop 2024-03-11T18:36:56.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.956+0000 7f90fabe9640 1 -- 172.21.15.90:0/1580120493 shutdown_connections 2024-03-11T18:36:56.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.956+0000 7f90fabe9640 1 --2- 172.21.15.90:0/1580120493 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f90e0066e90 0x7f90e0069350 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:56.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.956+0000 7f90fabe9640 1 --2- 172.21.15.90:0/1580120493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f90f4151b30 0x7f90f40ca540 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:56.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.956+0000 7f90fabe9640 1 --2- 172.21.15.90:0/1580120493 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f90f40c4af0 0x7f90f40c9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:56.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.956+0000 7f90fabe9640 1 --2- 172.21.15.90:0/1580120493 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f90f4070b00 0x7f90f40c3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:56.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.956+0000 7f90fabe9640 1 -- 172.21.15.90:0/1580120493 >> 172.21.15.90:0/1580120493 conn(0x7f90f406ac00 msgr2=0x7f90f406afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:56.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.956+0000 7f90fabe9640 1 -- 172.21.15.90:0/1580120493 shutdown_connections 2024-03-11T18:36:56.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:56.956+0000 7f90fabe9640 1 -- 172.21.15.90:0/1580120493 wait complete. 2024-03-11T18:36:56.980 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T18:36:57.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.088+0000 7f3c7cfff640 1 Processor -- start 2024-03-11T18:36:57.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.092+0000 7f3c7cfff640 1 -- start start 2024-03-11T18:36:57.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.092+0000 7f3c7cfff640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f3c781509f0 0x7f3c78157390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:57.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.092+0000 7f3c7cfff640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c780c5de0 con 0x7f3c78150dd0 2024-03-11T18:36:57.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.092+0000 7f3c7cfff640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c78156910 con 0x7f3c78070b00 2024-03-11T18:36:57.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.092+0000 7f3c7cfff640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c781578d0 con 0x7f3c781509f0 2024-03-11T18:36:57.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.092+0000 7f3c75d9b640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f3c78150dd0 0x7f3c780c28e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:52802/0 (socket says 172.21.15.90:52802) 2024-03-11T18:36:57.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.092+0000 7f3c75d9b640 1 -- 172.21.15.90:0/3844427000 learned_addr learned my addr 172.21.15.90:0/3844427000 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T18:36:57.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.092+0000 7f3c7659c640 1 --2- 172.21.15.90:0/3844427000 >> v2:172.21.15.90:3300/0 conn(0x7f3c781509f0 0x7f3c78157390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:57.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.096+0000 7f3c7659c640 1 -- 172.21.15.90:0/3844427000 >> v1:172.21.15.92:6789/0 conn(0x7f3c78070b00 legacy=0x7f3c780c4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:57.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.096+0000 7f3c7659c640 1 -- 172.21.15.90:0/3844427000 >> v1:172.21.15.90:6789/0 conn(0x7f3c78150dd0 legacy=0x7f3c780c28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:57.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.096+0000 7f3c7659c640 1 -- 172.21.15.90:0/3844427000 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c780c6030 con 0x7f3c781509f0 2024-03-11T18:36:57.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.104+0000 7f3c7659c640 1 --2- 172.21.15.90:0/3844427000 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c781509f0 0x7f3c78157390 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3c680097c0 tx=0x7f3c6801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0ef3cf2115b3d3a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:57.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.108+0000 7f3c7559a640 1 -- 172.21.15.90:0/3844427000 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c680a6040 con 0x7f3c781509f0 2024-03-11T18:36:57.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.108+0000 7f3c7559a640 1 -- 172.21.15.90:0/3844427000 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c680aa600 con 0x7f3c781509f0 2024-03-11T18:36:57.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.108+0000 7f3c7559a640 1 -- 172.21.15.90:0/3844427000 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c680aa990 con 0x7f3c781509f0 2024-03-11T18:36:57.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.108+0000 7f3c7cfff640 1 -- 172.21.15.90:0/3844427000 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c781509f0 msgr2=0x7f3c78157390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:57.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.108+0000 7f3c7cfff640 1 --2- 172.21.15.90:0/3844427000 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c781509f0 0x7f3c78157390 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3c680097c0 tx=0x7f3c6801c4a0 comp rx=0 tx=0).stop 2024-03-11T18:36:57.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.108+0000 7f3c7cfff640 1 -- 172.21.15.90:0/3844427000 shutdown_connections 2024-03-11T18:36:57.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.108+0000 7f3c7cfff640 1 --2- 172.21.15.90:0/3844427000 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c781509f0 0x7f3c78157390 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:57.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.108+0000 7f3c7cfff640 1 -- 172.21.15.90:0/3844427000 >> 172.21.15.90:0/3844427000 conn(0x7f3c7806ac00 msgr2=0x7f3c7806b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:57.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.108+0000 7f3c7cfff640 1 -- 172.21.15.90:0/3844427000 shutdown_connections 2024-03-11T18:36:57.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.108+0000 7f3c7cfff640 1 -- 172.21.15.90:0/3844427000 wait complete. 2024-03-11T18:36:57.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.112+0000 7f3c7cfff640 1 Processor -- start 2024-03-11T18:36:57.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.112+0000 7f3c7cfff640 1 -- start start 2024-03-11T18:36:57.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.112+0000 7f3c7cfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c78070b00 0x7f3c78146330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:57.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.112+0000 7f3c7cfff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c781509f0 0x7f3c781451b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:57.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.112+0000 7f3c7cfff640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3c78150dd0 0x7f3c780ca360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:57.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.112+0000 7f3c7cfff640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c781f3dd0 con 0x7f3c78070b00 2024-03-11T18:36:57.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.112+0000 7f3c7cfff640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7814add0 con 0x7f3c781509f0 2024-03-11T18:36:57.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.112+0000 7f3c7cfff640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7814b0b0 con 0x7f3c78150dd0 2024-03-11T18:36:57.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.112+0000 7f3c7659c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c78070b00 0x7f3c78146330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:57.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.112+0000 7f3c7659c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c78070b00 0x7f3c78146330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40942/0 (socket says 172.21.15.90:40942) 2024-03-11T18:36:57.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.112+0000 7f3c7659c640 1 -- 172.21.15.90:0/3207015187 learned_addr learned my addr 172.21.15.90:0/3207015187 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:57.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.112+0000 7f3c75d9b640 1 --2- 172.21.15.90:0/3207015187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c781509f0 0x7f3c781451b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:57.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.112+0000 7f3c76d9d640 1 --2- 172.21.15.90:0/3207015187 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3c78150dd0 0x7f3c780ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:57.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.116+0000 7f3c7659c640 1 -- 172.21.15.90:0/3207015187 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3c78150dd0 msgr2=0x7f3c780ca360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:57.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.116+0000 7f3c7659c640 1 --2- 172.21.15.90:0/3207015187 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3c78150dd0 0x7f3c780ca360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:57.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.116+0000 7f3c7659c640 1 -- 172.21.15.90:0/3207015187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c781509f0 msgr2=0x7f3c781451b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:57.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.116+0000 7f3c7659c640 1 --2- 172.21.15.90:0/3207015187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c781509f0 0x7f3c781451b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:57.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.116+0000 7f3c7659c640 1 -- 172.21.15.90:0/3207015187 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c68009420 con 0x7f3c78070b00 2024-03-11T18:36:57.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.120+0000 7f3c7659c640 1 --2- 172.21.15.90:0/3207015187 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c78070b00 0x7f3c78146330 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3c6801a0d0 tx=0x7f3c6801ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:57.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.120+0000 7f3c66ffd640 1 -- 172.21.15.90:0/3207015187 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c680b3800 con 0x7f3c78070b00 2024-03-11T18:36:57.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.120+0000 7f3c66ffd640 1 -- 172.21.15.90:0/3207015187 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c680b3980 con 0x7f3c78070b00 2024-03-11T18:36:57.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.120+0000 7f3c66ffd640 1 -- 172.21.15.90:0/3207015187 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c680b3d30 con 0x7f3c78070b00 2024-03-11T18:36:57.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.120+0000 7f3c7cfff640 1 -- 172.21.15.90:0/3207015187 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c7814b390 con 0x7f3c78070b00 2024-03-11T18:36:57.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.120+0000 7f3c7cfff640 1 -- 172.21.15.90:0/3207015187 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c7814b950 con 0x7f3c78070b00 2024-03-11T18:36:57.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.124+0000 7f3c7cfff640 1 -- 172.21.15.90:0/3207015187 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c7804eef0 con 0x7f3c78070b00 2024-03-11T18:36:57.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.404+0000 7f3c66ffd640 1 -- 172.21.15.90:0/3207015187 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f3c680c7410 con 0x7f3c78070b00 2024-03-11T18:36:57.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.404+0000 7f3c66ffd640 1 --2- 172.21.15.90:0/3207015187 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f3c58066e40 0x7f3c58069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:57.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.404+0000 7f3c75d9b640 1 --2- 172.21.15.90:0/3207015187 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f3c58066e40 0x7f3c58069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:57.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.404+0000 7f3c75d9b640 1 --2- 172.21.15.90:0/3207015187 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f3c58066e40 0x7f3c58069300 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3c78154300 tx=0x7f3c6c007e90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:57.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.420+0000 7f3c66ffd640 1 -- 172.21.15.90:0/3207015187 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3c680eff70 con 0x7f3c78070b00 2024-03-11T18:36:57.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.420+0000 7f3c66ffd640 1 -- 172.21.15.90:0/3207015187 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3c680ec210 con 0x7f3c78070b00 2024-03-11T18:36:57.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.604+0000 7f3c7cfff640 1 -- 172.21.15.90:0/3207015187 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3c781437c0 con 0x7f3c78070b00 2024-03-11T18:36:57.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.696+0000 7f3c66ffd640 1 -- 172.21.15.90:0/3207015187 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18430 (secure 0 0 0) 0x7f3c680ef8d0 con 0x7f3c78070b00 2024-03-11T18:36:57.705 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:36:57.705 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":28,"fsid":"76d56b10-6c29-476a-804b-8482ea90f9e2","created":"2024-03-11T18:34:54.521741+0000","modified":"2024-03-11T18:36:05.526044+0000","last_up_change":"2024-03-11T18:35:16.207500+0000","last_in_change":"2024-03-11T18:35:05.213634+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T18:35:16.193842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T18:35:54.001469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T18:35:55.917321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T18:36:02.210943+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"34decff9-7280-4d84-9665-d6af8ea6de84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6819","nonce":1755667921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6821","nonce":1755667921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6825","nonce":1755667921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6823","nonce":1755667921}]},"public_addr":"172.21.15.90:6819/1755667921","cluster_addr":"172.21.15.90:6821/1755667921","heartbeat_back_addr":"172.21.15.90:6825/1755667921","heartbeat_front_addr":"172.21.15.90:6823/1755667921","state":["exists","up"]},{"osd":1,"uuid":"8ecdce9e-5486-415a-a609-dd16e945440e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6807","nonce":438682081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6809","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6811","nonce":438682081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6817","nonce":438682081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6815","nonce":438682081}]},"public_addr":"172.21.15.90:6807/438682081","cluster_addr":"172.21.15.90:6811/438682081","heartbeat_back_addr":"172.21.15.90:6817/438682081","heartbeat_front_addr":"172.21.15.90:6815/438682081","state":["exists","up"]},{"osd":2,"uuid":"737486d4-0e90-4c82-9e79-bf37c812e5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6827","nonce":2837704738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6829","nonce":2837704738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6833","nonce":2837704738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6831","nonce":2837704738}]},"public_addr":"172.21.15.90:6827/2837704738","cluster_addr":"172.21.15.90:6829/2837704738","heartbeat_back_addr":"172.21.15.90:6833/2837704738","heartbeat_front_addr":"172.21.15.90:6831/2837704738","state":["exists","up"]},{"osd":3,"uuid":"698bd661-bbef-4699-a619-f7bfe23ce12c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6803","nonce":240987971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6805","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6806","nonce":240987971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6813","nonce":240987971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6810","nonce":240987971}]},"public_addr":"172.21.15.90:6803/240987971","cluster_addr":"172.21.15.90:6806/240987971","heartbeat_back_addr":"172.21.15.90:6813/240987971","heartbeat_front_addr":"172.21.15.90:6810/240987971","state":["exists","up"]},{"osd":4,"uuid":"aba9e31f-73d9-43fe-a10b-2dc5832968b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6801","nonce":502685275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6803","nonce":502685275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6807","nonce":502685275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6805","nonce":502685275}]},"public_addr":"172.21.15.92:6801/502685275","cluster_addr":"172.21.15.92:6803/502685275","heartbeat_back_addr":"172.21.15.92:6807/502685275","heartbeat_front_addr":"172.21.15.92:6805/502685275","state":["exists","up"]},{"osd":5,"uuid":"bdd56766-4b28-4007-8de3-b5b08cf335d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6809","nonce":200142070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6811","nonce":200142070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6815","nonce":200142070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6813","nonce":200142070}]},"public_addr":"172.21.15.92:6809/200142070","cluster_addr":"172.21.15.92:6811/200142070","heartbeat_back_addr":"172.21.15.92:6815/200142070","heartbeat_front_addr":"172.21.15.92:6813/200142070","state":["exists","up"]},{"osd":6,"uuid":"6c580a59-610f-455e-a96b-033299dcc118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6817","nonce":1919010204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6819","nonce":1919010204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6823","nonce":1919010204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6821","nonce":1919010204}]},"public_addr":"172.21.15.92:6817/1919010204","cluster_addr":"172.21.15.92:6819/1919010204","heartbeat_back_addr":"172.21.15.92:6823/1919010204","heartbeat_front_addr":"172.21.15.92:6821/1919010204","state":["exists","up"]},{"osd":7,"uuid":"958184cc-aebf-4c5b-91a0-2a3e38a6b42b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6825","nonce":3475925719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6827","nonce":3475925719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6831","nonce":3475925719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6829","nonce":3475925719}]},"public_addr":"172.21.15.92:6825/3475925719","cluster_addr":"172.21.15.92:6827/3475925719","heartbeat_back_addr":"172.21.15.92:6831/3475925719","heartbeat_front_addr":"172.21.15.92:6829/3475925719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.816294+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.048084+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.137975+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.968246+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.016829+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.931676+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.041673+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.079404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T18:36:57.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.700+0000 7f3c7cfff640 1 -- 172.21.15.90:0/3207015187 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f3c58066e40 msgr2=0x7f3c58069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:57.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.700+0000 7f3c7cfff640 1 --2- 172.21.15.90:0/3207015187 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f3c58066e40 0x7f3c58069300 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3c78154300 tx=0x7f3c6c007e90 comp rx=0 tx=0).stop 2024-03-11T18:36:57.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.700+0000 7f3c7cfff640 1 -- 172.21.15.90:0/3207015187 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c78070b00 msgr2=0x7f3c78146330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:57.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.700+0000 7f3c7cfff640 1 --2- 172.21.15.90:0/3207015187 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c78070b00 0x7f3c78146330 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3c6801a0d0 tx=0x7f3c6801ace0 comp rx=0 tx=0).stop 2024-03-11T18:36:57.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.700+0000 7f3c7cfff640 1 -- 172.21.15.90:0/3207015187 shutdown_connections 2024-03-11T18:36:57.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.700+0000 7f3c7cfff640 1 --2- 172.21.15.90:0/3207015187 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f3c58066e40 0x7f3c58069300 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:57.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.700+0000 7f3c7cfff640 1 --2- 172.21.15.90:0/3207015187 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3c78150dd0 0x7f3c780ca360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:57.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.700+0000 7f3c7cfff640 1 --2- 172.21.15.90:0/3207015187 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c781509f0 0x7f3c781451b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:57.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.700+0000 7f3c7cfff640 1 --2- 172.21.15.90:0/3207015187 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c78070b00 0x7f3c78146330 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:57.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.700+0000 7f3c7cfff640 1 -- 172.21.15.90:0/3207015187 >> 172.21.15.90:0/3207015187 conn(0x7f3c7806ac00 msgr2=0x7f3c78159ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:57.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.700+0000 7f3c7cfff640 1 -- 172.21.15.90:0/3207015187 shutdown_connections 2024-03-11T18:36:57.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.700+0000 7f3c7cfff640 1 -- 172.21.15.90:0/3207015187 wait complete. 2024-03-11T18:36:57.728 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-11T18:35:16.193842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-11T18:35:54.001469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-11T18:35:55.917321+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-11T18:36:02.210943+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-11T18:36:57.728 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-11T18:36:57.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.848+0000 7f373b304640 1 Processor -- start 2024-03-11T18:36:57.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.848+0000 7f373b304640 1 -- start start 2024-03-11T18:36:57.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.848+0000 7f373b304640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3734150540 0x7f3734156b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:57.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.848+0000 7f373b304640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f373406bc10 0x7f373406c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:57.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.848+0000 7f373b304640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37340cb6d0 con 0x7f37340cc3a0 2024-03-11T18:36:57.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.848+0000 7f373b304640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37340cb400 con 0x7f373406bc10 2024-03-11T18:36:57.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.848+0000 7f373b304640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3734150920 con 0x7f3734150540 2024-03-11T18:36:57.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.852+0000 7f37398a1640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f373406bc10 0x7f373406c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:57.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.852+0000 7f37390a0640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3734150540 0x7f3734156b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:57.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.852+0000 7f37398a1640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f373406bc10 0x7f373406c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40958/0 (socket says 172.21.15.90:40958) 2024-03-11T18:36:57.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.852+0000 7f37398a1640 1 -- 172.21.15.90:0/2107236265 learned_addr learned my addr 172.21.15.90:0/2107236265 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:57.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.856+0000 7f37398a1640 1 -- 172.21.15.90:0/2107236265 >> v2:172.21.15.92:3300/0 conn(0x7f3734150540 msgr2=0x7f3734156b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:57.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.856+0000 7f37398a1640 1 --2- 172.21.15.90:0/2107236265 >> v2:172.21.15.92:3300/0 conn(0x7f3734150540 0x7f3734156b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:57.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.856+0000 7f37398a1640 1 -- 172.21.15.90:0/2107236265 >> v1:172.21.15.92:6789/0 conn(0x7f37340cc3a0 legacy=0x7f37340ca020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:57.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.856+0000 7f37398a1640 1 -- 172.21.15.90:0/2107236265 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37340ca730 con 0x7f373406bc10 2024-03-11T18:36:57.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.860+0000 7f37398a1640 1 --2- 172.21.15.90:0/2107236265 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f373406bc10 0x7f373406c060 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f373406c880 tx=0x7f372c01ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=428720f160ec2380 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:57.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.872+0000 7f372bfff640 1 -- 172.21.15.90:0/2107236265 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c00a2d0 con 0x7f373406bc10 2024-03-11T18:36:57.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.872+0000 7f372bfff640 1 -- 172.21.15.90:0/2107236265 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f372c0a7900 con 0x7f373406bc10 2024-03-11T18:36:57.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.872+0000 7f372bfff640 1 -- 172.21.15.90:0/2107236265 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c0a7c90 con 0x7f373406bc10 2024-03-11T18:36:57.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.876+0000 7f373b304640 1 -- 172.21.15.90:0/2107236265 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f373406bc10 msgr2=0x7f373406c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:57.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.876+0000 7f373b304640 1 --2- 172.21.15.90:0/2107236265 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f373406bc10 0x7f373406c060 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f373406c880 tx=0x7f372c01ebc0 comp rx=0 tx=0).stop 2024-03-11T18:36:57.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.876+0000 7f373b304640 1 -- 172.21.15.90:0/2107236265 shutdown_connections 2024-03-11T18:36:57.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.876+0000 7f373b304640 1 --2- 172.21.15.90:0/2107236265 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f373406bc10 0x7f373406c060 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:57.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.876+0000 7f373b304640 1 --2- 172.21.15.90:0/2107236265 >> v2:172.21.15.92:3300/0 conn(0x7f3734150540 0x7f3734156b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:57.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.876+0000 7f373b304640 1 -- 172.21.15.90:0/2107236265 >> 172.21.15.90:0/2107236265 conn(0x7f37340cb880 msgr2=0x7f37340cbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:57.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.876+0000 7f373b304640 1 -- 172.21.15.90:0/2107236265 shutdown_connections 2024-03-11T18:36:57.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.876+0000 7f373b304640 1 -- 172.21.15.90:0/2107236265 wait complete. 2024-03-11T18:36:57.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.876+0000 7f373b304640 1 Processor -- start 2024-03-11T18:36:57.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.876+0000 7f373b304640 1 -- start start 2024-03-11T18:36:57.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.876+0000 7f373b304640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f373406bc10 0x7f3734071140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:57.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.876+0000 7f373b304640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37340cc3a0 0x7f37341f1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:57.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.876+0000 7f373b304640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3734150540 0x7f37341f2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:57.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.876+0000 7f373b304640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3734141630 con 0x7f37340cc3a0 2024-03-11T18:36:57.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.876+0000 7f373b304640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37341436e0 con 0x7f373406bc10 2024-03-11T18:36:57.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.876+0000 7f373b304640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3734071680 con 0x7f3734150540 2024-03-11T18:36:57.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.880+0000 7f373889f640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37340cc3a0 0x7f37341f1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:57.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.880+0000 7f373889f640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37340cc3a0 0x7f37341f1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40966/0 (socket says 172.21.15.90:40966) 2024-03-11T18:36:57.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.880+0000 7f373889f640 1 -- 172.21.15.90:0/1609360910 learned_addr learned my addr 172.21.15.90:0/1609360910 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:57.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.880+0000 7f37390a0640 1 --2- 172.21.15.90:0/1609360910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f373406bc10 0x7f3734071140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:57.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.880+0000 7f37398a1640 1 --2- 172.21.15.90:0/1609360910 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3734150540 0x7f37341f2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:57.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.884+0000 7f373889f640 1 -- 172.21.15.90:0/1609360910 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3734150540 msgr2=0x7f37341f2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:57.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.884+0000 7f373889f640 1 --2- 172.21.15.90:0/1609360910 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3734150540 0x7f37341f2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:57.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.884+0000 7f373889f640 1 -- 172.21.15.90:0/1609360910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f373406bc10 msgr2=0x7f3734071140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:57.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.884+0000 7f373889f640 1 --2- 172.21.15.90:0/1609360910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f373406bc10 0x7f3734071140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:57.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.884+0000 7f373889f640 1 -- 172.21.15.90:0/1609360910 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f372c00ac40 con 0x7f37340cc3a0 2024-03-11T18:36:57.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.884+0000 7f373889f640 1 --2- 172.21.15.90:0/1609360910 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37340cc3a0 0x7f37341f1d50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f372400b410 tx=0x7f372400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:57.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.888+0000 7f3729ffb640 1 -- 172.21.15.90:0/1609360910 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3724004020 con 0x7f37340cc3a0 2024-03-11T18:36:57.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.888+0000 7f3729ffb640 1 -- 172.21.15.90:0/1609360910 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3724004180 con 0x7f37340cc3a0 2024-03-11T18:36:57.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.888+0000 7f3729ffb640 1 -- 172.21.15.90:0/1609360910 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3724011630 con 0x7f37340cc3a0 2024-03-11T18:36:57.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.888+0000 7f373b304640 1 -- 172.21.15.90:0/1609360910 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37340c7640 con 0x7f37340cc3a0 2024-03-11T18:36:57.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.888+0000 7f373b304640 1 -- 172.21.15.90:0/1609360910 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3734145260 con 0x7f37340cc3a0 2024-03-11T18:36:57.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.888+0000 7f373b304640 1 -- 172.21.15.90:0/1609360910 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37340c5bc0 con 0x7f37340cc3a0 2024-03-11T18:36:57.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.908+0000 7f3729ffb640 1 -- 172.21.15.90:0/1609360910 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f3724011790 con 0x7f37340cc3a0 2024-03-11T18:36:57.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.908+0000 7f3729ffb640 1 --2- 172.21.15.90:0/1609360910 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f3718066e90 0x7f3718069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:57.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.908+0000 7f37390a0640 1 --2- 172.21.15.90:0/1609360910 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f3718066e90 0x7f3718069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:57.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.912+0000 7f37390a0640 1 --2- 172.21.15.90:0/1609360910 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f3718066e90 0x7f3718069350 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3730009ac0 tx=0x7f3730009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:57.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:57.912+0000 7f3729ffb640 1 -- 172.21.15.90:0/1609360910 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3724055b40 con 0x7f37340cc3a0 2024-03-11T18:36:58.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.192+0000 7f3729ffb640 1 -- 172.21.15.90:0/1609360910 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3724051de0 con 0x7f37340cc3a0 2024-03-11T18:36:58.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.364+0000 7f373b304640 1 -- 172.21.15.90:0/1609360910 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f373404eef0 con 0x7f37340cc3a0 2024-03-11T18:36:58.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.376+0000 7f3729ffb640 1 -- 172.21.15.90:0/1609360910 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3724015070 con 0x7f37340cc3a0 2024-03-11T18:36:58.383 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 1 2024-03-11T18:36:58.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.380+0000 7f373b304640 1 -- 172.21.15.90:0/1609360910 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f3718066e90 msgr2=0x7f3718069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:58.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.380+0000 7f373b304640 1 --2- 172.21.15.90:0/1609360910 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f3718066e90 0x7f3718069350 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3730009ac0 tx=0x7f3730009190 comp rx=0 tx=0).stop 2024-03-11T18:36:58.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.380+0000 7f373b304640 1 -- 172.21.15.90:0/1609360910 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37340cc3a0 msgr2=0x7f37341f1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:58.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.380+0000 7f373b304640 1 --2- 172.21.15.90:0/1609360910 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37340cc3a0 0x7f37341f1d50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f372400b410 tx=0x7f372400b8d0 comp rx=0 tx=0).stop 2024-03-11T18:36:58.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.380+0000 7f373b304640 1 -- 172.21.15.90:0/1609360910 shutdown_connections 2024-03-11T18:36:58.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.380+0000 7f373b304640 1 --2- 172.21.15.90:0/1609360910 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f3718066e90 0x7f3718069350 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:58.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.380+0000 7f373b304640 1 --2- 172.21.15.90:0/1609360910 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3734150540 0x7f37341f2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:58.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.380+0000 7f373b304640 1 --2- 172.21.15.90:0/1609360910 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37340cc3a0 0x7f37341f1d50 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:58.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.380+0000 7f373b304640 1 --2- 172.21.15.90:0/1609360910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f373406bc10 0x7f3734071140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:58.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.380+0000 7f373b304640 1 -- 172.21.15.90:0/1609360910 >> 172.21.15.90:0/1609360910 conn(0x7f37340cb880 msgr2=0x7f373414e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:58.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.380+0000 7f373b304640 1 -- 172.21.15.90:0/1609360910 shutdown_connections 2024-03-11T18:36:58.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.380+0000 7f373b304640 1 -- 172.21.15.90:0/1609360910 wait complete. 2024-03-11T18:36:58.403 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-11T18:36:58.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.520+0000 7f049efb1640 1 Processor -- start 2024-03-11T18:36:58.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.520+0000 7f049efb1640 1 -- start start 2024-03-11T18:36:58.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.524+0000 7f049efb1640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f04981554a0 0x7f04980c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:58.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.524+0000 7f049efb1640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f049806fe60 0x7f04980c2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:58.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.524+0000 7f049efb1640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04980c2ae0 con 0x7f04981550c0 2024-03-11T18:36:58.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.524+0000 7f049efb1640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04980c2560 con 0x7f04981554a0 2024-03-11T18:36:58.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.524+0000 7f049efb1640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0498157cc0 con 0x7f049806fe60 2024-03-11T18:36:58.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.524+0000 7f0497fff640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f04981554a0 0x7f04980c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:58.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.524+0000 7f0497fff640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f04981554a0 0x7f04980c8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40978/0 (socket says 172.21.15.90:40978) 2024-03-11T18:36:58.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.524+0000 7f0497fff640 1 -- 172.21.15.90:0/3863908180 learned_addr learned my addr 172.21.15.90:0/3863908180 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:58.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.524+0000 7f049d54e640 1 --2- 172.21.15.90:0/3863908180 >> v2:172.21.15.92:3300/0 conn(0x7f049806fe60 0x7f04980c2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:58.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.528+0000 7f0497fff640 1 -- 172.21.15.90:0/3863908180 >> v2:172.21.15.92:3300/0 conn(0x7f049806fe60 msgr2=0x7f04980c2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:58.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.528+0000 7f0497fff640 1 --2- 172.21.15.90:0/3863908180 >> v2:172.21.15.92:3300/0 conn(0x7f049806fe60 0x7f04980c2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:58.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.528+0000 7f0497fff640 1 -- 172.21.15.90:0/3863908180 >> v1:172.21.15.90:6789/0 conn(0x7f04981550c0 legacy=0x7f049815ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:58.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.528+0000 7f0497fff640 1 -- 172.21.15.90:0/3863908180 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04980c2800 con 0x7f04981554a0 2024-03-11T18:36:58.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.532+0000 7f0497fff640 1 --2- 172.21.15.90:0/3863908180 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04981554a0 0x7f04980c8fb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f04980c4190 tx=0x7f048001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41ace57675670012 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:58.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.564+0000 7f04977fe640 1 -- 172.21.15.90:0/3863908180 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04800ab600 con 0x7f04981554a0 2024-03-11T18:36:58.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.564+0000 7f04977fe640 1 -- 172.21.15.90:0/3863908180 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04800a7050 con 0x7f04981554a0 2024-03-11T18:36:58.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.564+0000 7f04977fe640 1 -- 172.21.15.90:0/3863908180 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04800ab980 con 0x7f04981554a0 2024-03-11T18:36:58.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.564+0000 7f049efb1640 1 -- 172.21.15.90:0/3863908180 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04981554a0 msgr2=0x7f04980c8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:58.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.564+0000 7f049efb1640 1 --2- 172.21.15.90:0/3863908180 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04981554a0 0x7f04980c8fb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f04980c4190 tx=0x7f048001c450 comp rx=0 tx=0).stop 2024-03-11T18:36:58.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.564+0000 7f049efb1640 1 -- 172.21.15.90:0/3863908180 shutdown_connections 2024-03-11T18:36:58.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.564+0000 7f049efb1640 1 --2- 172.21.15.90:0/3863908180 >> v2:172.21.15.92:3300/0 conn(0x7f049806fe60 0x7f04980c2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:58.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.564+0000 7f049efb1640 1 --2- 172.21.15.90:0/3863908180 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04981554a0 0x7f04980c8fb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:58.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.564+0000 7f049efb1640 1 -- 172.21.15.90:0/3863908180 >> 172.21.15.90:0/3863908180 conn(0x7f0498070e80 msgr2=0x7f0498066990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:58.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.564+0000 7f049efb1640 1 -- 172.21.15.90:0/3863908180 shutdown_connections 2024-03-11T18:36:58.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.564+0000 7f049efb1640 1 -- 172.21.15.90:0/3863908180 wait complete. 2024-03-11T18:36:58.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.564+0000 7f049efb1640 1 Processor -- start 2024-03-11T18:36:58.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.564+0000 7f049efb1640 1 -- start start 2024-03-11T18:36:58.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.568+0000 7f049efb1640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f049806fe60 0x7f04980b1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:58.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.568+0000 7f049efb1640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04981550c0 0x7f0498150da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:58.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.568+0000 7f049efb1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04981554a0 0x7f0498156c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:58.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.568+0000 7f049efb1640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04981512e0 con 0x7f04981550c0 2024-03-11T18:36:58.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.568+0000 7f049efb1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0498150910 con 0x7f04981554a0 2024-03-11T18:36:58.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.568+0000 7f049efb1640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f049814fa40 con 0x7f049806fe60 2024-03-11T18:36:58.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.568+0000 7f049cd4d640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f049806fe60 0x7f04980b1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:58.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.568+0000 7f049d54e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04981554a0 0x7f0498156c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:58.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.568+0000 7f049cd4d640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f049806fe60 0x7f04980b1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.90:41012/0 (socket says 172.21.15.90:41012) 2024-03-11T18:36:58.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.568+0000 7f049cd4d640 1 -- 172.21.15.90:0/3704132746 learned_addr learned my addr 172.21.15.90:0/3704132746 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:58.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.576+0000 7f0497fff640 1 --2- 172.21.15.90:0/3704132746 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04981550c0 0x7f0498150da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:58.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.580+0000 7f049cd4d640 1 -- 172.21.15.90:0/3704132746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04981554a0 msgr2=0x7f0498156c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:58.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.580+0000 7f049cd4d640 1 --2- 172.21.15.90:0/3704132746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04981554a0 0x7f0498156c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:58.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.580+0000 7f049cd4d640 1 -- 172.21.15.90:0/3704132746 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04981550c0 msgr2=0x7f0498150da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:58.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.580+0000 7f049cd4d640 1 --2- 172.21.15.90:0/3704132746 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04981550c0 0x7f0498150da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:58.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.580+0000 7f049cd4d640 1 -- 172.21.15.90:0/3704132746 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04800094d0 con 0x7f049806fe60 2024-03-11T18:36:58.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.604+0000 7f049cd4d640 1 --2- 172.21.15.90:0/3704132746 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f049806fe60 0x7f04980b1360 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f048800c7c0 tx=0x7f048800cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:58.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.728+0000 7f04957fa640 1 -- 172.21.15.90:0/3704132746 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048800aa50 con 0x7f049806fe60 2024-03-11T18:36:58.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.728+0000 7f04957fa640 1 -- 172.21.15.90:0/3704132746 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f048800abb0 con 0x7f049806fe60 2024-03-11T18:36:58.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.728+0000 7f04957fa640 1 -- 172.21.15.90:0/3704132746 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0488012420 con 0x7f049806fe60 2024-03-11T18:36:58.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.728+0000 7f049efb1640 1 -- 172.21.15.90:0/3704132746 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f049814fd00 con 0x7f049806fe60 2024-03-11T18:36:58.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.728+0000 7f049efb1640 1 -- 172.21.15.90:0/3704132746 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04981502c0 con 0x7f049806fe60 2024-03-11T18:36:58.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:58.728+0000 7f049efb1640 1 -- 172.21.15.90:0/3704132746 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04980b05f0 con 0x7f049806fe60 2024-03-11T18:36:59.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.040+0000 7f04957fa640 1 -- 172.21.15.90:0/3704132746 <== mon.2 v2:172.21.15.92:3301/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f0488006b60 con 0x7f049806fe60 2024-03-11T18:36:59.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.044+0000 7f04957fa640 1 --2- 172.21.15.90:0/3704132746 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0478066e90 0x7f0478069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:59.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.044+0000 7f0497fff640 1 --2- 172.21.15.90:0/3704132746 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0478066e90 0x7f0478069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:59.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.044+0000 7f0497fff640 1 --2- 172.21.15.90:0/3704132746 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0478066e90 0x7f0478069350 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f04980c47e0 tx=0x7f04800a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:59.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.052+0000 7f04957fa640 1 -- 172.21.15.90:0/3704132746 <== mon.2 v2:172.21.15.92:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0488055c90 con 0x7f049806fe60 2024-03-11T18:36:59.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.080+0000 7f04957fa640 1 -- 172.21.15.90:0/3704132746 <== mon.2 v2:172.21.15.92:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0488051f30 con 0x7f049806fe60 2024-03-11T18:36:59.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.256+0000 7f049efb1640 1 -- 172.21.15.90:0/3704132746 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f049804eef0 con 0x7f049806fe60 2024-03-11T18:36:59.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.308+0000 7f04957fa640 1 -- 172.21.15.90:0/3704132746 <== mon.2 v2:172.21.15.92:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0488016020 con 0x7f049806fe60 2024-03-11T18:36:59.317 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 32 2024-03-11T18:36:59.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.312+0000 7f049efb1640 1 -- 172.21.15.90:0/3704132746 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0478066e90 msgr2=0x7f0478069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:59.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.312+0000 7f049efb1640 1 --2- 172.21.15.90:0/3704132746 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0478066e90 0x7f0478069350 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f04980c47e0 tx=0x7f04800a6000 comp rx=0 tx=0).stop 2024-03-11T18:36:59.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.312+0000 7f049efb1640 1 -- 172.21.15.90:0/3704132746 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f049806fe60 msgr2=0x7f04980b1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:59.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.312+0000 7f049efb1640 1 --2- 172.21.15.90:0/3704132746 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f049806fe60 0x7f04980b1360 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f048800c7c0 tx=0x7f048800cc80 comp rx=0 tx=0).stop 2024-03-11T18:36:59.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.312+0000 7f049efb1640 1 -- 172.21.15.90:0/3704132746 shutdown_connections 2024-03-11T18:36:59.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.312+0000 7f049efb1640 1 --2- 172.21.15.90:0/3704132746 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0478066e90 0x7f0478069350 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:59.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.312+0000 7f049efb1640 1 --2- 172.21.15.90:0/3704132746 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f04981554a0 0x7f0498156c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:59.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.312+0000 7f049efb1640 1 --2- 172.21.15.90:0/3704132746 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04981550c0 0x7f0498150da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:59.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.312+0000 7f049efb1640 1 --2- 172.21.15.90:0/3704132746 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f049806fe60 0x7f04980b1360 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:59.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.312+0000 7f049efb1640 1 -- 172.21.15.90:0/3704132746 >> 172.21.15.90:0/3704132746 conn(0x7f0498070e80 msgr2=0x7f0498153d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:59.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.312+0000 7f049efb1640 1 -- 172.21.15.90:0/3704132746 shutdown_connections 2024-03-11T18:36:59.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.312+0000 7f049efb1640 1 -- 172.21.15.90:0/3704132746 wait complete. 2024-03-11T18:36:59.340 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-11T18:36:59.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.460+0000 7fbf79604640 1 Processor -- start 2024-03-11T18:36:59.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.460+0000 7fbf79604640 1 -- start start 2024-03-11T18:36:59.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.464+0000 7fbf79604640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbf74151c70 0x7fbf7406bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:59.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.464+0000 7fbf79604640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbf740c7f20 0x7fbf7406c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:59.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.464+0000 7fbf79604640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf740c6720 con 0x7fbf74066d60 2024-03-11T18:36:59.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.464+0000 7fbf79604640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf740cc7b0 con 0x7fbf74151c70 2024-03-11T18:36:59.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.464+0000 7fbf79604640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf740c5d20 con 0x7fbf740c7f20 2024-03-11T18:36:59.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.464+0000 7fbf727fc640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbf740c7f20 0x7fbf7406c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:59.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.464+0000 7fbf72ffd640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbf74151c70 0x7fbf7406bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:59.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.464+0000 7fbf727fc640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbf740c7f20 0x7fbf7406c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.90:41016/0 (socket says 172.21.15.90:41016) 2024-03-11T18:36:59.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.464+0000 7fbf737fe640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7fbf74066d60 0x7fbf74157ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.90:53862/0 (socket says 172.21.15.90:53862) 2024-03-11T18:36:59.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.464+0000 7fbf727fc640 1 -- 172.21.15.90:0/2120829929 learned_addr learned my addr 172.21.15.90:0/2120829929 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:59.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.472+0000 7fbf727fc640 1 -- 172.21.15.90:0/2120829929 >> v2:172.21.15.92:3300/0 conn(0x7fbf74151c70 msgr2=0x7fbf7406bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:36:59.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.472+0000 7fbf727fc640 1 --2- 172.21.15.90:0/2120829929 >> v2:172.21.15.92:3300/0 conn(0x7fbf74151c70 0x7fbf7406bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:59.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.472+0000 7fbf727fc640 1 -- 172.21.15.90:0/2120829929 >> v1:172.21.15.92:6789/0 conn(0x7fbf74066d60 legacy=0x7fbf74157ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:59.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.472+0000 7fbf727fc640 1 -- 172.21.15.90:0/2120829929 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf740c5a40 con 0x7fbf740c7f20 2024-03-11T18:36:59.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.476+0000 7fbf727fc640 1 --2- 172.21.15.90:0/2120829929 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbf740c7f20 0x7fbf7406c150 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbf68009870 tx=0x7fbf6801c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b3d3ae894dc685 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:59.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.488+0000 7fbf71ffb640 1 -- 172.21.15.90:0/2120829929 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf680ab610 con 0x7fbf740c7f20 2024-03-11T18:36:59.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.488+0000 7fbf71ffb640 1 -- 172.21.15.90:0/2120829929 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf680ab770 con 0x7fbf740c7f20 2024-03-11T18:36:59.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.488+0000 7fbf71ffb640 1 -- 172.21.15.90:0/2120829929 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf680abb00 con 0x7fbf740c7f20 2024-03-11T18:36:59.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.488+0000 7fbf79604640 1 -- 172.21.15.90:0/2120829929 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbf740c7f20 msgr2=0x7fbf7406c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:59.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.488+0000 7fbf79604640 1 --2- 172.21.15.90:0/2120829929 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbf740c7f20 0x7fbf7406c150 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbf68009870 tx=0x7fbf6801c410 comp rx=0 tx=0).stop 2024-03-11T18:36:59.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.488+0000 7fbf79604640 1 -- 172.21.15.90:0/2120829929 shutdown_connections 2024-03-11T18:36:59.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.488+0000 7fbf79604640 1 --2- 172.21.15.90:0/2120829929 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbf740c7f20 0x7fbf7406c150 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:59.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.488+0000 7fbf79604640 1 --2- 172.21.15.90:0/2120829929 >> v2:172.21.15.92:3300/0 conn(0x7fbf74151c70 0x7fbf7406bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:59.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.488+0000 7fbf79604640 1 -- 172.21.15.90:0/2120829929 >> 172.21.15.90:0/2120829929 conn(0x7fbf740cb880 msgr2=0x7fbf740cbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:36:59.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.488+0000 7fbf79604640 1 -- 172.21.15.90:0/2120829929 shutdown_connections 2024-03-11T18:36:59.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.488+0000 7fbf79604640 1 -- 172.21.15.90:0/2120829929 wait complete. 2024-03-11T18:36:59.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.492+0000 7fbf79604640 1 Processor -- start 2024-03-11T18:36:59.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.492+0000 7fbf79604640 1 -- start start 2024-03-11T18:36:59.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.492+0000 7fbf79604640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbf74066d60 0x7fbf741425a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:59.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.492+0000 7fbf79604640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf740c7f20 0x7fbf74146130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:59.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.492+0000 7fbf79604640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf74151c70 0x7fbf74146670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:59.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.492+0000 7fbf79604640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf741531f0 con 0x7fbf740c7f20 2024-03-11T18:36:59.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.492+0000 7fbf79604640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf74146e50 con 0x7fbf74151c70 2024-03-11T18:36:59.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.492+0000 7fbf79604640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf74147340 con 0x7fbf74066d60 2024-03-11T18:36:59.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.492+0000 7fbf727fc640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf740c7f20 0x7fbf74146130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:59.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.492+0000 7fbf727fc640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf740c7f20 0x7fbf74146130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41004/0 (socket says 172.21.15.90:41004) 2024-03-11T18:36:59.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.492+0000 7fbf727fc640 1 -- 172.21.15.90:0/2720563759 learned_addr learned my addr 172.21.15.90:0/2720563759 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:36:59.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.492+0000 7fbf737fe640 1 --2- 172.21.15.90:0/2720563759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf74151c70 0x7fbf74146670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:59.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.492+0000 7fbf72ffd640 1 --2- 172.21.15.90:0/2720563759 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbf74066d60 0x7fbf741425a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:59.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.496+0000 7fbf727fc640 1 -- 172.21.15.90:0/2720563759 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbf74066d60 msgr2=0x7fbf741425a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:59.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.496+0000 7fbf727fc640 1 --2- 172.21.15.90:0/2720563759 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbf74066d60 0x7fbf741425a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:59.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.496+0000 7fbf727fc640 1 -- 172.21.15.90:0/2720563759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf74151c70 msgr2=0x7fbf74146670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:36:59.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.496+0000 7fbf727fc640 1 --2- 172.21.15.90:0/2720563759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf74151c70 0x7fbf74146670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:36:59.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.496+0000 7fbf727fc640 1 -- 172.21.15.90:0/2720563759 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf680094d0 con 0x7fbf740c7f20 2024-03-11T18:36:59.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.500+0000 7fbf727fc640 1 --2- 172.21.15.90:0/2720563759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf740c7f20 0x7fbf74146130 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbf680ad040 tx=0x7fbf6801ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:59.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.504+0000 7fbf5b7fe640 1 -- 172.21.15.90:0/2720563759 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf680be5f0 con 0x7fbf740c7f20 2024-03-11T18:36:59.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.504+0000 7fbf5b7fe640 1 -- 172.21.15.90:0/2720563759 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf680be770 con 0x7fbf740c7f20 2024-03-11T18:36:59.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.504+0000 7fbf5b7fe640 1 -- 172.21.15.90:0/2720563759 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf680beb20 con 0x7fbf740c7f20 2024-03-11T18:36:59.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.504+0000 7fbf79604640 1 -- 172.21.15.90:0/2720563759 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf74072cc0 con 0x7fbf740c7f20 2024-03-11T18:36:59.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.504+0000 7fbf79604640 1 -- 172.21.15.90:0/2720563759 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf74147080 con 0x7fbf740c7f20 2024-03-11T18:36:59.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.508+0000 7fbf79604640 1 -- 172.21.15.90:0/2720563759 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf7404eef0 con 0x7fbf740c7f20 2024-03-11T18:36:59.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.524+0000 7fbf5b7fe640 1 -- 172.21.15.90:0/2720563759 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fbf680b4600 con 0x7fbf740c7f20 2024-03-11T18:36:59.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.524+0000 7fbf5b7fe640 1 --2- 172.21.15.90:0/2720563759 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fbf60066e90 0x7fbf60069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:36:59.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.524+0000 7fbf72ffd640 1 --2- 172.21.15.90:0/2720563759 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fbf60066e90 0x7fbf60069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:36:59.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.528+0000 7fbf72ffd640 1 --2- 172.21.15.90:0/2720563759 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fbf60066e90 0x7fbf60069350 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbf64009ac0 tx=0x7fbf64009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:36:59.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.528+0000 7fbf5b7fe640 1 -- 172.21.15.90:0/2720563759 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fbf680f0ec0 con 0x7fbf740c7f20 2024-03-11T18:36:59.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.804+0000 7fbf5b7fe640 1 -- 172.21.15.90:0/2720563759 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbf680ed160 con 0x7fbf740c7f20 2024-03-11T18:36:59.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:36:59.976+0000 7fbf79604640 1 -- 172.21.15.90:0/2720563759 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbf741620e0 con 0x7fbf740c7f20 2024-03-11T18:37:00.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.080+0000 7fbf5b7fe640 1 -- 172.21.15.90:0/2720563759 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbf680f0820 con 0x7fbf740c7f20 2024-03-11T18:37:00.087 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 64 2024-03-11T18:37:00.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.084+0000 7fbf79604640 1 -- 172.21.15.90:0/2720563759 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fbf60066e90 msgr2=0x7fbf60069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:00.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.084+0000 7fbf79604640 1 --2- 172.21.15.90:0/2720563759 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fbf60066e90 0x7fbf60069350 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbf64009ac0 tx=0x7fbf64009190 comp rx=0 tx=0).stop 2024-03-11T18:37:00.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.084+0000 7fbf79604640 1 -- 172.21.15.90:0/2720563759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf740c7f20 msgr2=0x7fbf74146130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:00.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.084+0000 7fbf79604640 1 --2- 172.21.15.90:0/2720563759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf740c7f20 0x7fbf74146130 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbf680ad040 tx=0x7fbf6801ac50 comp rx=0 tx=0).stop 2024-03-11T18:37:00.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.084+0000 7fbf79604640 1 -- 172.21.15.90:0/2720563759 shutdown_connections 2024-03-11T18:37:00.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.084+0000 7fbf79604640 1 --2- 172.21.15.90:0/2720563759 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fbf60066e90 0x7fbf60069350 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:00.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.084+0000 7fbf79604640 1 --2- 172.21.15.90:0/2720563759 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbf74151c70 0x7fbf74146670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:00.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.084+0000 7fbf79604640 1 --2- 172.21.15.90:0/2720563759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf740c7f20 0x7fbf74146130 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:00.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.084+0000 7fbf79604640 1 --2- 172.21.15.90:0/2720563759 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbf74066d60 0x7fbf741425a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:00.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.084+0000 7fbf79604640 1 -- 172.21.15.90:0/2720563759 >> 172.21.15.90:0/2720563759 conn(0x7fbf740cb880 msgr2=0x7fbf740c5090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:37:00.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.084+0000 7fbf79604640 1 -- 172.21.15.90:0/2720563759 shutdown_connections 2024-03-11T18:37:00.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.084+0000 7fbf79604640 1 -- 172.21.15.90:0/2720563759 wait complete. 2024-03-11T18:37:00.113 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-11T18:37:00.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.236+0000 7f125c84e640 1 Processor -- start 2024-03-11T18:37:00.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.236+0000 7f125c84e640 1 -- start start 2024-03-11T18:37:00.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.236+0000 7f125c84e640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f12581550c0 0x7f12580c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:00.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.236+0000 7f125c84e640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f12581554a0 0x7f125815ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:00.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.236+0000 7f125c84e640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f125806fe60 0x7f12580c2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:00.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.236+0000 7f125c84e640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f125815bfb0 con 0x7f12581550c0 2024-03-11T18:37:00.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.236+0000 7f125c84e640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12580c2ae0 con 0x7f125806fe60 2024-03-11T18:37:00.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.236+0000 7f125c84e640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12580c2560 con 0x7f12581554a0 2024-03-11T18:37:00.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.236+0000 7f125659c640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f12581550c0 0x7f12580c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:00.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.236+0000 7f125659c640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f12581550c0 0x7f12580c8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41016/0 (socket says 172.21.15.90:41016) 2024-03-11T18:37:00.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.236+0000 7f125659c640 1 -- 172.21.15.90:0/797730041 learned_addr learned my addr 172.21.15.90:0/797730041 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:37:00.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.236+0000 7f1256d9d640 1 --2- 172.21.15.90:0/797730041 >> v2:172.21.15.92:3300/0 conn(0x7f125806fe60 0x7f12580c2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:00.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.240+0000 7f125659c640 1 -- 172.21.15.90:0/797730041 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f12581554a0 msgr2=0x7f125815ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:00.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.240+0000 7f125659c640 1 --2- 172.21.15.90:0/797730041 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f12581554a0 0x7f125815ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:00.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.240+0000 7f125659c640 1 -- 172.21.15.90:0/797730041 >> v2:172.21.15.92:3300/0 conn(0x7f125806fe60 msgr2=0x7f12580c2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:00.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.240+0000 7f125659c640 1 --2- 172.21.15.90:0/797730041 >> v2:172.21.15.92:3300/0 conn(0x7f125806fe60 0x7f12580c2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:00.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.240+0000 7f125659c640 1 -- 172.21.15.90:0/797730041 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1258157cc0 con 0x7f12581550c0 2024-03-11T18:37:00.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.248+0000 7f125659c640 1 --2- 172.21.15.90:0/797730041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12581550c0 0x7f12580c8fb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f12580ca950 tx=0x7f124801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a353da7d1f22db1 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:37:00.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.252+0000 7f125559a640 1 -- 172.21.15.90:0/797730041 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12480ab600 con 0x7f12581550c0 2024-03-11T18:37:00.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.252+0000 7f125559a640 1 -- 172.21.15.90:0/797730041 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12480a7050 con 0x7f12581550c0 2024-03-11T18:37:00.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.252+0000 7f125559a640 1 -- 172.21.15.90:0/797730041 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12480ab980 con 0x7f12581550c0 2024-03-11T18:37:00.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.252+0000 7f125c84e640 1 -- 172.21.15.90:0/797730041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12581550c0 msgr2=0x7f12580c8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:00.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.252+0000 7f125c84e640 1 --2- 172.21.15.90:0/797730041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12581550c0 0x7f12580c8fb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f12580ca950 tx=0x7f124801c490 comp rx=0 tx=0).stop 2024-03-11T18:37:00.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.252+0000 7f125c84e640 1 -- 172.21.15.90:0/797730041 shutdown_connections 2024-03-11T18:37:00.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.252+0000 7f125c84e640 1 --2- 172.21.15.90:0/797730041 >> v2:172.21.15.92:3300/0 conn(0x7f125806fe60 0x7f12580c2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:00.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.252+0000 7f125c84e640 1 --2- 172.21.15.90:0/797730041 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f12581554a0 0x7f125815ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:00.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.252+0000 7f125c84e640 1 --2- 172.21.15.90:0/797730041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12581550c0 0x7f12580c8fb0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:00.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.252+0000 7f125c84e640 1 -- 172.21.15.90:0/797730041 >> 172.21.15.90:0/797730041 conn(0x7f1258070e80 msgr2=0x7f1258066990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:37:00.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.252+0000 7f125c84e640 1 -- 172.21.15.90:0/797730041 shutdown_connections 2024-03-11T18:37:00.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.252+0000 7f125c84e640 1 -- 172.21.15.90:0/797730041 wait complete. 2024-03-11T18:37:00.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.252+0000 7f125c84e640 1 Processor -- start 2024-03-11T18:37:00.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.252+0000 7f125c84e640 1 -- start start 2024-03-11T18:37:00.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.256+0000 7f125c84e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f125806fe60 0x7f12581f1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:00.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.256+0000 7f125c84e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12581550c0 0x7f12581f40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:00.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.256+0000 7f125c84e640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f12581554a0 0x7f12581487b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:00.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.256+0000 7f125c84e640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1258159640 con 0x7f12581550c0 2024-03-11T18:37:00.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.256+0000 7f125c84e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f125815a160 con 0x7f125806fe60 2024-03-11T18:37:00.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.256+0000 7f125c84e640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12581f22e0 con 0x7f12581554a0 2024-03-11T18:37:00.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.256+0000 7f1255d9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12581550c0 0x7f12581f40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:00.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.256+0000 7f1255d9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12581550c0 0x7f12581f40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41024/0 (socket says 172.21.15.90:41024) 2024-03-11T18:37:00.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.256+0000 7f1255d9b640 1 -- 172.21.15.90:0/259480749 learned_addr learned my addr 172.21.15.90:0/259480749 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:37:00.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.256+0000 7f1256d9d640 1 --2- 172.21.15.90:0/259480749 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f12581554a0 0x7f12581487b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:00.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.256+0000 7f125659c640 1 --2- 172.21.15.90:0/259480749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f125806fe60 0x7f12581f1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:00.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.260+0000 7f1255d9b640 1 -- 172.21.15.90:0/259480749 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f12581554a0 msgr2=0x7f12581487b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:00.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.260+0000 7f1255d9b640 1 --2- 172.21.15.90:0/259480749 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f12581554a0 0x7f12581487b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:00.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.260+0000 7f1255d9b640 1 -- 172.21.15.90:0/259480749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f125806fe60 msgr2=0x7f12581f1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:00.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.260+0000 7f1255d9b640 1 --2- 172.21.15.90:0/259480749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f125806fe60 0x7f12581f1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:00.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.260+0000 7f1255d9b640 1 -- 172.21.15.90:0/259480749 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12480094d0 con 0x7f12581550c0 2024-03-11T18:37:00.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.264+0000 7f1255d9b640 1 --2- 172.21.15.90:0/259480749 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12581550c0 0x7f12581f40c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f124c00b550 tx=0x7f124c00ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:37:00.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.268+0000 7f1246ffd640 1 -- 172.21.15.90:0/259480749 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124c010870 con 0x7f12581550c0 2024-03-11T18:37:00.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.268+0000 7f1246ffd640 1 -- 172.21.15.90:0/259480749 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f124c0109f0 con 0x7f12581550c0 2024-03-11T18:37:00.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.268+0000 7f1246ffd640 1 -- 172.21.15.90:0/259480749 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124c010da0 con 0x7f12581550c0 2024-03-11T18:37:00.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.268+0000 7f125c84e640 1 -- 172.21.15.90:0/259480749 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12580ce1b0 con 0x7f12581550c0 2024-03-11T18:37:00.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.268+0000 7f125c84e640 1 -- 172.21.15.90:0/259480749 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1258159830 con 0x7f12581550c0 2024-03-11T18:37:00.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.268+0000 7f125c84e640 1 -- 172.21.15.90:0/259480749 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f125815a2f0 con 0x7f12581550c0 2024-03-11T18:37:00.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.288+0000 7f1246ffd640 1 -- 172.21.15.90:0/259480749 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f124c023480 con 0x7f12581550c0 2024-03-11T18:37:00.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.288+0000 7f1246ffd640 1 --2- 172.21.15.90:0/259480749 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1238066e90 0x7f1238069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:00.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.288+0000 7f125659c640 1 --2- 172.21.15.90:0/259480749 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1238066e90 0x7f1238069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:00.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.288+0000 7f125659c640 1 --2- 172.21.15.90:0/259480749 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1238066e90 0x7f1238069350 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f12580c47e0 tx=0x7f1248008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:37:00.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.568+0000 7f1246ffd640 1 -- 172.21.15.90:0/259480749 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f124c0559f0 con 0x7f12581550c0 2024-03-11T18:37:00.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.568+0000 7f1246ffd640 1 -- 172.21.15.90:0/259480749 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f124c051c90 con 0x7f12581550c0 2024-03-11T18:37:00.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.740+0000 7f125c84e640 1 -- 172.21.15.90:0/259480749 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f125815b490 con 0x7f12581550c0 2024-03-11T18:37:00.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.756+0000 7f1246ffd640 1 -- 172.21.15.90:0/259480749 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f124c01e0a0 con 0x7f12581550c0 2024-03-11T18:37:00.761 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 32 2024-03-11T18:37:00.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.756+0000 7f125c84e640 1 -- 172.21.15.90:0/259480749 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1238066e90 msgr2=0x7f1238069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:00.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.756+0000 7f125c84e640 1 --2- 172.21.15.90:0/259480749 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1238066e90 0x7f1238069350 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f12580c47e0 tx=0x7f1248008040 comp rx=0 tx=0).stop 2024-03-11T18:37:00.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.756+0000 7f125c84e640 1 -- 172.21.15.90:0/259480749 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12581550c0 msgr2=0x7f12581f40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:00.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.756+0000 7f125c84e640 1 --2- 172.21.15.90:0/259480749 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12581550c0 0x7f12581f40c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f124c00b550 tx=0x7f124c00ba10 comp rx=0 tx=0).stop 2024-03-11T18:37:00.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.756+0000 7f125c84e640 1 -- 172.21.15.90:0/259480749 shutdown_connections 2024-03-11T18:37:00.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.756+0000 7f125c84e640 1 --2- 172.21.15.90:0/259480749 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1238066e90 0x7f1238069350 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:00.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.756+0000 7f125c84e640 1 --2- 172.21.15.90:0/259480749 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f12581554a0 0x7f12581487b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:00.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.756+0000 7f125c84e640 1 --2- 172.21.15.90:0/259480749 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12581550c0 0x7f12581f40c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:00.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.756+0000 7f125c84e640 1 --2- 172.21.15.90:0/259480749 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f125806fe60 0x7f12581f1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:00.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.756+0000 7f125c84e640 1 -- 172.21.15.90:0/259480749 >> 172.21.15.90:0/259480749 conn(0x7f1258070e80 msgr2=0x7f125806f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:37:00.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.756+0000 7f125c84e640 1 -- 172.21.15.90:0/259480749 shutdown_connections 2024-03-11T18:37:00.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.756+0000 7f125c84e640 1 -- 172.21.15.90:0/259480749 wait complete. 2024-03-11T18:37:00.782 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T18:37:00.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.896+0000 7fdc3f750640 1 Processor -- start 2024-03-11T18:37:00.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.896+0000 7fdc3f750640 1 -- start start 2024-03-11T18:37:00.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.896+0000 7fdc3f750640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fdc380cb7d0 0x7fdc380cf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:00.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.896+0000 7fdc3f750640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc3806d000 con 0x7fdc380cfd40 2024-03-11T18:37:00.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.896+0000 7fdc3f750640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc380cad00 con 0x7fdc38153fe0 2024-03-11T18:37:00.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.896+0000 7fdc3f750640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc381543c0 con 0x7fdc380cb7d0 2024-03-11T18:37:00.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.896+0000 7fdc3dced640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7fdc380cfd40 0x7fdc3806bdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:52820/0 (socket says 172.21.15.90:52820) 2024-03-11T18:37:00.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.896+0000 7fdc3dced640 1 -- 172.21.15.90:0/1980388728 learned_addr learned my addr 172.21.15.90:0/1980388728 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T18:37:00.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.896+0000 7fdc3cceb640 1 --2- 172.21.15.90:0/1980388728 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fdc380cb7d0 0x7fdc380cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:00.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.900+0000 7fdc2ffff640 1 -- 172.21.15.90:0/1980388728 <== mon.0 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3332322464 0 0) 0x7fdc3806d000 con 0x7fdc380cfd40 2024-03-11T18:37:00.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.900+0000 7fdc2ffff640 1 -- 172.21.15.90:0/1980388728 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc24003530 con 0x7fdc380cfd40 2024-03-11T18:37:00.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.904+0000 7fdc2ffff640 1 -- 172.21.15.90:0/1980388728 <== mon.0 v1:172.21.15.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1252094423 0 0) 0x7fdc24003530 con 0x7fdc380cfd40 2024-03-11T18:37:00.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.904+0000 7fdc2ffff640 1 -- 172.21.15.90:0/1980388728 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fdc380cb7d0 msgr2=0x7fdc380cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:00.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.904+0000 7fdc2ffff640 1 --2- 172.21.15.90:0/1980388728 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fdc380cb7d0 0x7fdc380cf800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:00.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.904+0000 7fdc2ffff640 1 -- 172.21.15.90:0/1980388728 >> v1:172.21.15.92:6789/0 conn(0x7fdc38153fe0 legacy=0x7fdc380ca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:00.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.904+0000 7fdc2ffff640 1 -- 172.21.15.90:0/1980388728 --> v1:172.21.15.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc38159ca0 con 0x7fdc380cfd40 2024-03-11T18:37:00.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.904+0000 7fdc2ffff640 1 -- 172.21.15.90:0/1980388728 <== mon.0 v1:172.21.15.90:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 691735435 0 0) 0x7fdc30001a00 con 0x7fdc380cfd40 2024-03-11T18:37:00.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.904+0000 7fdc2ffff640 1 -- 172.21.15.90:0/1980388728 >> v1:172.21.15.90:6789/0 conn(0x7fdc380cfd40 legacy=0x7fdc3806bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:00.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.904+0000 7fdc2ffff640 1 --2- 172.21.15.90:0/1980388728 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fdc380cb7d0 0x7fdc240039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:00.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.904+0000 7fdc2ffff640 1 --2- 172.21.15.90:0/1980388728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc24003f80 0x7fdc240047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:00.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.904+0000 7fdc2ffff640 1 --2- 172.21.15.90:0/1980388728 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc24004d10 0x7fdc24005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:00.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.904+0000 7fdc2ffff640 1 -- 172.21.15.90:0/1980388728 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc381543c0 con 0x7fdc24004d10 2024-03-11T18:37:00.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.904+0000 7fdc2ffff640 1 -- 172.21.15.90:0/1980388728 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2400bad0 con 0x7fdc24003f80 2024-03-11T18:37:00.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.904+0000 7fdc2ffff640 1 -- 172.21.15.90:0/1980388728 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2400bc90 con 0x7fdc380cb7d0 2024-03-11T18:37:00.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.908+0000 7fdc3dced640 1 --2- 172.21.15.90:0/1980388728 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc24004d10 0x7fdc24005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:00.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.908+0000 7fdc3dced640 1 -- 172.21.15.90:0/1980388728 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fdc380cb7d0 msgr2=0x7fdc240039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:00.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.908+0000 7fdc3dced640 1 --2- 172.21.15.90:0/1980388728 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fdc380cb7d0 0x7fdc240039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:00.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.908+0000 7fdc3dced640 1 -- 172.21.15.90:0/1980388728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc24003f80 msgr2=0x7fdc240047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:00.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.908+0000 7fdc3dced640 1 --2- 172.21.15.90:0/1980388728 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc24003f80 0x7fdc240047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:00.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.908+0000 7fdc3dced640 1 -- 172.21.15.90:0/1980388728 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc24003530 con 0x7fdc24004d10 2024-03-11T18:37:00.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.912+0000 7fdc3dced640 1 --2- 172.21.15.90:0/1980388728 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc24004d10 0x7fdc24005190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdc3000a760 tx=0x7fdc3001d280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a06803e1982d2fb server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:37:00.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.916+0000 7fdc2ffff640 1 -- 172.21.15.90:0/1980388728 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc300ad630 con 0x7fdc24004d10 2024-03-11T18:37:00.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.916+0000 7fdc2ffff640 1 -- 172.21.15.90:0/1980388728 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc300a8070 con 0x7fdc24004d10 2024-03-11T18:37:00.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.916+0000 7fdc2ffff640 1 -- 172.21.15.90:0/1980388728 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc300ada90 con 0x7fdc24004d10 2024-03-11T18:37:00.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.916+0000 7fdc3f750640 1 -- 172.21.15.90:0/1980388728 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc24004d10 msgr2=0x7fdc24005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:00.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.916+0000 7fdc3f750640 1 --2- 172.21.15.90:0/1980388728 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc24004d10 0x7fdc24005190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdc3000a760 tx=0x7fdc3001d280 comp rx=0 tx=0).stop 2024-03-11T18:37:00.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.916+0000 7fdc3dced640 1 -- 172.21.15.90:0/1980388728 reap_dead start 2024-03-11T18:37:00.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.916+0000 7fdc3f750640 1 -- 172.21.15.90:0/1980388728 shutdown_connections 2024-03-11T18:37:00.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.916+0000 7fdc3f750640 1 -- 172.21.15.90:0/1980388728 >> 172.21.15.90:0/1980388728 conn(0x7fdc38072500 msgr2=0x7fdc38072910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:37:00.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.916+0000 7fdc3f750640 1 -- 172.21.15.90:0/1980388728 shutdown_connections 2024-03-11T18:37:00.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.916+0000 7fdc3f750640 1 -- 172.21.15.90:0/1980388728 wait complete. 2024-03-11T18:37:00.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.920+0000 7fdc3f750640 1 Processor -- start 2024-03-11T18:37:00.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.920+0000 7fdc3f750640 1 -- start start 2024-03-11T18:37:00.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.920+0000 7fdc3f750640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc380b47d0 0x7fdc38152660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:00.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.920+0000 7fdc3f750640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc380cd3c0 0x7fdc380ced50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:00.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.920+0000 7fdc3f750640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fdc380b4bb0 0x7fdc381efd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:00.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.920+0000 7fdc3f750640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc381ef340 con 0x7fdc380b47d0 2024-03-11T18:37:00.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.920+0000 7fdc3f750640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc380b5290 con 0x7fdc380cd3c0 2024-03-11T18:37:00.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.920+0000 7fdc3f750640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc380cf290 con 0x7fdc380b4bb0 2024-03-11T18:37:00.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.920+0000 7fdc3d4ec640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc380b47d0 0x7fdc38152660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:00.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.920+0000 7fdc3d4ec640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc380b47d0 0x7fdc38152660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41050/0 (socket says 172.21.15.90:41050) 2024-03-11T18:37:00.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.920+0000 7fdc3d4ec640 1 -- 172.21.15.90:0/1245956018 learned_addr learned my addr 172.21.15.90:0/1245956018 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:37:00.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.920+0000 7fdc3dced640 1 --2- 172.21.15.90:0/1245956018 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fdc380b4bb0 0x7fdc381efd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:00.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.920+0000 7fdc3cceb640 1 --2- 172.21.15.90:0/1245956018 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc380cd3c0 0x7fdc380ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:00.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.924+0000 7fdc3d4ec640 1 -- 172.21.15.90:0/1245956018 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fdc380b4bb0 msgr2=0x7fdc381efd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:00.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.924+0000 7fdc3d4ec640 1 --2- 172.21.15.90:0/1245956018 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fdc380b4bb0 0x7fdc381efd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:00.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.924+0000 7fdc3d4ec640 1 -- 172.21.15.90:0/1245956018 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc380cd3c0 msgr2=0x7fdc380ced50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:00.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.924+0000 7fdc3d4ec640 1 --2- 172.21.15.90:0/1245956018 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc380cd3c0 0x7fdc380ced50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:00.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.924+0000 7fdc3d4ec640 1 -- 172.21.15.90:0/1245956018 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc38149570 con 0x7fdc380b47d0 2024-03-11T18:37:00.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.928+0000 7fdc3d4ec640 1 --2- 172.21.15.90:0/1245956018 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc380b47d0 0x7fdc38152660 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdc3400c560 tx=0x7fdc3400ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:37:00.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.932+0000 7fdc2dffb640 1 -- 172.21.15.90:0/1245956018 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc340052d0 con 0x7fdc380b47d0 2024-03-11T18:37:00.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.932+0000 7fdc2dffb640 1 -- 172.21.15.90:0/1245956018 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc34005470 con 0x7fdc380b47d0 2024-03-11T18:37:00.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.932+0000 7fdc3f750640 1 -- 172.21.15.90:0/1245956018 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc381f0270 con 0x7fdc380b47d0 2024-03-11T18:37:00.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.932+0000 7fdc2dffb640 1 -- 172.21.15.90:0/1245956018 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc34012870 con 0x7fdc380b47d0 2024-03-11T18:37:00.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.932+0000 7fdc3f750640 1 -- 172.21.15.90:0/1245956018 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc38149af0 con 0x7fdc380b47d0 2024-03-11T18:37:00.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.932+0000 7fdc3f750640 1 -- 172.21.15.90:0/1245956018 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc380681b0 con 0x7fdc380b47d0 2024-03-11T18:37:00.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.952+0000 7fdc2dffb640 1 -- 172.21.15.90:0/1245956018 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fdc340129d0 con 0x7fdc380b47d0 2024-03-11T18:37:00.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.952+0000 7fdc2dffb640 1 --2- 172.21.15.90:0/1245956018 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fdc1c066e90 0x7fdc1c069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:00.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.952+0000 7fdc3cceb640 1 --2- 172.21.15.90:0/1245956018 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fdc1c066e90 0x7fdc1c069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:00.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.952+0000 7fdc3cceb640 1 --2- 172.21.15.90:0/1245956018 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fdc1c066e90 0x7fdc1c069350 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdc3806cf30 tx=0x7fdc28009320 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:37:00.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:00.952+0000 7fdc2dffb640 1 -- 172.21.15.90:0/1245956018 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fdc34055ac0 con 0x7fdc380b47d0 2024-03-11T18:37:01.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.220+0000 7fdc2dffb640 1 -- 172.21.15.90:0/1245956018 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdc34051d60 con 0x7fdc380b47d0 2024-03-11T18:37:01.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.420+0000 7fdc3f750640 1 -- 172.21.15.90:0/1245956018 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc3804eef0 con 0x7fdc380b47d0 2024-03-11T18:37:01.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.436+0000 7fdc2dffb640 1 -- 172.21.15.90:0/1245956018 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+6575 (secure 0 0 0) 0x7fdc34055420 con 0x7fdc380b47d0 2024-03-11T18:37:01.441 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:37:01.441 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/641527992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":641527992},{"type":"v1","addr":"172.21.15.90:6839","nonce":641527992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-11T18:35:57.363570+0000","modified":"2024-03-11T18:36:51.575661+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4255,"mds_1":4253,"mds_2":4241,"mds_3":4238,"mds_4":4229},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4229":{"gid":4229,"name":"c","rank":4,"incarnation":18,"state":"up:active","state_seq":17,"addr":"172.21.15.90:6835/460417177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":460417177},{"type":"v1","addr":"172.21.15.90:6835","nonce":460417177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4238":{"gid":4238,"name":"d","rank":3,"incarnation":16,"state":"up:active","state_seq":17,"addr":"172.21.15.92:6835/3568438077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":3568438077},{"type":"v1","addr":"172.21.15.92:6835","nonce":3568438077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4241":{"gid":4241,"name":"b","rank":2,"incarnation":14,"state":"up:active","state_seq":16,"addr":"172.21.15.92:6834/2306487548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":2306487548},{"type":"v1","addr":"172.21.15.92:6834","nonce":2306487548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4253":{"gid":4253,"name":"f","rank":1,"incarnation":10,"state":"up:active","state_seq":15,"addr":"172.21.15.92:6837/3209534444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":3209534444},{"type":"v1","addr":"172.21.15.92:6837","nonce":3209534444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4255":{"gid":4255,"name":"e","rank":0,"incarnation":5,"state":"up:active","state_seq":6,"addr":"172.21.15.90:6837/3485943375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3485943375},{"type":"v1","addr":"172.21.15.90:6837","nonce":3485943375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T18:37:01.442 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 19 2024-03-11T18:37:01.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.436+0000 7fdc3f750640 1 -- 172.21.15.90:0/1245956018 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fdc1c066e90 msgr2=0x7fdc1c069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:01.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.436+0000 7fdc3f750640 1 --2- 172.21.15.90:0/1245956018 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fdc1c066e90 0x7fdc1c069350 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdc3806cf30 tx=0x7fdc28009320 comp rx=0 tx=0).stop 2024-03-11T18:37:01.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.436+0000 7fdc3f750640 1 -- 172.21.15.90:0/1245956018 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc380b47d0 msgr2=0x7fdc38152660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:01.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.436+0000 7fdc3f750640 1 --2- 172.21.15.90:0/1245956018 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc380b47d0 0x7fdc38152660 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdc3400c560 tx=0x7fdc3400ca20 comp rx=0 tx=0).stop 2024-03-11T18:37:01.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.436+0000 7fdc3f750640 1 -- 172.21.15.90:0/1245956018 shutdown_connections 2024-03-11T18:37:01.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.436+0000 7fdc3f750640 1 --2- 172.21.15.90:0/1245956018 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fdc1c066e90 0x7fdc1c069350 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:01.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.436+0000 7fdc3f750640 1 --2- 172.21.15.90:0/1245956018 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fdc380b4bb0 0x7fdc381efd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:01.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.436+0000 7fdc3f750640 1 --2- 172.21.15.90:0/1245956018 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fdc380cd3c0 0x7fdc380ced50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:01.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.436+0000 7fdc3f750640 1 --2- 172.21.15.90:0/1245956018 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc380b47d0 0x7fdc38152660 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:01.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.436+0000 7fdc3f750640 1 -- 172.21.15.90:0/1245956018 >> 172.21.15.90:0/1245956018 conn(0x7fdc38072500 msgr2=0x7fdc380c5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:37:01.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.436+0000 7fdc3f750640 1 -- 172.21.15.90:0/1245956018 shutdown_connections 2024-03-11T18:37:01.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.436+0000 7fdc3f750640 1 -- 172.21.15.90:0/1245956018 wait complete. 2024-03-11T18:37:01.463 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T18:37:01.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.576+0000 7f38ba516640 1 Processor -- start 2024-03-11T18:37:01.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.576+0000 7f38ba516640 1 -- start start 2024-03-11T18:37:01.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.576+0000 7f38ba516640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f38b41509f0 0x7f38b4157390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:01.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.576+0000 7f38ba516640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f38b4150dd0 0x7f38b40c28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:01.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.576+0000 7f38ba516640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38b40c5de0 con 0x7f38b4070b00 2024-03-11T18:37:01.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.576+0000 7f38ba516640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38b4156910 con 0x7f38b41509f0 2024-03-11T18:37:01.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.576+0000 7f38ba516640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38b41578d0 con 0x7f38b4150dd0 2024-03-11T18:37:01.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.576+0000 7f38b37fe640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f38b4150dd0 0x7f38b40c28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:01.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.580+0000 7f38b37fe640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f38b4150dd0 0x7f38b40c28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.90:41072/0 (socket says 172.21.15.90:41072) 2024-03-11T18:37:01.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.580+0000 7f38b37fe640 1 -- 172.21.15.90:0/2730443645 learned_addr learned my addr 172.21.15.90:0/2730443645 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:37:01.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.580+0000 7f38b3fff640 1 --2- 172.21.15.90:0/2730443645 >> v2:172.21.15.92:3300/0 conn(0x7f38b41509f0 0x7f38b4157390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:01.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.584+0000 7f38b37fe640 1 -- 172.21.15.90:0/2730443645 >> v2:172.21.15.92:3300/0 conn(0x7f38b41509f0 msgr2=0x7f38b4157390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:01.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.584+0000 7f38b37fe640 1 --2- 172.21.15.90:0/2730443645 >> v2:172.21.15.92:3300/0 conn(0x7f38b41509f0 0x7f38b4157390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:01.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.584+0000 7f38b37fe640 1 -- 172.21.15.90:0/2730443645 >> v1:172.21.15.92:6789/0 conn(0x7f38b4070b00 legacy=0x7f38b40c4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:01.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.584+0000 7f38b37fe640 1 -- 172.21.15.90:0/2730443645 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b40c6030 con 0x7f38b4150dd0 2024-03-11T18:37:01.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.588+0000 7f38b37fe640 1 --2- 172.21.15.90:0/2730443645 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f38b4150dd0 0x7f38b40c28e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f38a8009870 tx=0x7f38a801ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f31ae9eb0b89b15 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:37:01.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.604+0000 7f38b2ffd640 1 -- 172.21.15.90:0/2730443645 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a80a8a50 con 0x7f38b4150dd0 2024-03-11T18:37:01.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.604+0000 7f38b2ffd640 1 -- 172.21.15.90:0/2730443645 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38a80a8bb0 con 0x7f38b4150dd0 2024-03-11T18:37:01.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.604+0000 7f38b2ffd640 1 -- 172.21.15.90:0/2730443645 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a80b0440 con 0x7f38b4150dd0 2024-03-11T18:37:01.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.604+0000 7f38ba516640 1 -- 172.21.15.90:0/2730443645 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f38b4150dd0 msgr2=0x7f38b40c28e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:01.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.604+0000 7f38ba516640 1 --2- 172.21.15.90:0/2730443645 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f38b4150dd0 0x7f38b40c28e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f38a8009870 tx=0x7f38a801ed50 comp rx=0 tx=0).stop 2024-03-11T18:37:01.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.604+0000 7f38ba516640 1 -- 172.21.15.90:0/2730443645 shutdown_connections 2024-03-11T18:37:01.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.604+0000 7f38ba516640 1 --2- 172.21.15.90:0/2730443645 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f38b4150dd0 0x7f38b40c28e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:01.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.604+0000 7f38ba516640 1 --2- 172.21.15.90:0/2730443645 >> v2:172.21.15.92:3300/0 conn(0x7f38b41509f0 0x7f38b4157390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:01.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.604+0000 7f38ba516640 1 -- 172.21.15.90:0/2730443645 >> 172.21.15.90:0/2730443645 conn(0x7f38b406ac00 msgr2=0x7f38b406b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:37:01.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.604+0000 7f38ba516640 1 -- 172.21.15.90:0/2730443645 shutdown_connections 2024-03-11T18:37:01.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.604+0000 7f38ba516640 1 -- 172.21.15.90:0/2730443645 wait complete. 2024-03-11T18:37:01.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.608+0000 7f38ba516640 1 Processor -- start 2024-03-11T18:37:01.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.608+0000 7f38ba516640 1 -- start start 2024-03-11T18:37:01.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.608+0000 7f38ba516640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f38b4070b00 0x7f38b414d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:01.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.608+0000 7f38ba516640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f38b41509f0 0x7f38b40ca5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:01.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.608+0000 7f38ba516640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38b4150dd0 0x7f38b41f2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:01.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.608+0000 7f38ba516640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b413c650 con 0x7f38b4070b00 2024-03-11T18:37:01.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.608+0000 7f38ba516640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b415ad70 con 0x7f38b4150dd0 2024-03-11T18:37:01.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.608+0000 7f38ba516640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38b415a7f0 con 0x7f38b41509f0 2024-03-11T18:37:01.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.608+0000 7f38b3fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f38b4070b00 0x7f38b414d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:01.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.608+0000 7f38b3fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f38b4070b00 0x7f38b414d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41066/0 (socket says 172.21.15.90:41066) 2024-03-11T18:37:01.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.608+0000 7f38b3fff640 1 -- 172.21.15.90:0/3349113931 learned_addr learned my addr 172.21.15.90:0/3349113931 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:37:01.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.612+0000 7f38b37fe640 1 --2- 172.21.15.90:0/3349113931 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f38b41509f0 0x7f38b40ca5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:01.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.612+0000 7f38b8ab3640 1 --2- 172.21.15.90:0/3349113931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38b4150dd0 0x7f38b41f2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:01.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.616+0000 7f38b3fff640 1 -- 172.21.15.90:0/3349113931 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f38b41509f0 msgr2=0x7f38b40ca5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:01.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.616+0000 7f38b3fff640 1 --2- 172.21.15.90:0/3349113931 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f38b41509f0 0x7f38b40ca5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:01.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.616+0000 7f38b3fff640 1 -- 172.21.15.90:0/3349113931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38b4150dd0 msgr2=0x7f38b41f2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:01.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.616+0000 7f38b3fff640 1 --2- 172.21.15.90:0/3349113931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38b4150dd0 0x7f38b41f2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:01.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.616+0000 7f38b3fff640 1 -- 172.21.15.90:0/3349113931 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a80094d0 con 0x7f38b4070b00 2024-03-11T18:37:01.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.620+0000 7f38b3fff640 1 --2- 172.21.15.90:0/3349113931 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f38b4070b00 0x7f38b414d7a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f38a400d5f0 tx=0x7f38a400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:37:01.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.620+0000 7f38b0ff9640 1 -- 172.21.15.90:0/3349113931 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a40125f0 con 0x7f38b4070b00 2024-03-11T18:37:01.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.624+0000 7f38b0ff9640 1 -- 172.21.15.90:0/3349113931 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38a4012750 con 0x7f38b4070b00 2024-03-11T18:37:01.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.624+0000 7f38b0ff9640 1 -- 172.21.15.90:0/3349113931 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a4012ae0 con 0x7f38b4070b00 2024-03-11T18:37:01.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.624+0000 7f38ba516640 1 -- 172.21.15.90:0/3349113931 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38b415aa90 con 0x7f38b4070b00 2024-03-11T18:37:01.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.624+0000 7f38ba516640 1 -- 172.21.15.90:0/3349113931 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38b414b090 con 0x7f38b4070b00 2024-03-11T18:37:01.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.624+0000 7f38ba516640 1 -- 172.21.15.90:0/3349113931 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38b404eef0 con 0x7f38b4070b00 2024-03-11T18:37:01.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.912+0000 7f38b0ff9640 1 -- 172.21.15.90:0/3349113931 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f38a4012c40 con 0x7f38b4070b00 2024-03-11T18:37:01.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.912+0000 7f38b0ff9640 1 --2- 172.21.15.90:0/3349113931 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f38a0066e90 0x7f38a0069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:01.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.912+0000 7f38b37fe640 1 --2- 172.21.15.90:0/3349113931 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f38a0066e90 0x7f38a0069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:01.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.912+0000 7f38b37fe640 1 --2- 172.21.15.90:0/3349113931 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f38a0066e90 0x7f38a0069350 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f38a80b2000 tx=0x7f38a8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:37:01.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.928+0000 7f38b0ff9640 1 -- 172.21.15.90:0/3349113931 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f38a4015e20 con 0x7f38b4070b00 2024-03-11T18:37:01.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:01.928+0000 7f38b0ff9640 1 -- 172.21.15.90:0/3349113931 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f38a4052700 con 0x7f38b4070b00 2024-03-11T18:37:02.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.096+0000 7f38ba516640 1 -- 172.21.15.90:0/3349113931 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38b40721d0 con 0x7f38b4070b00 2024-03-11T18:37:02.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.232+0000 7f38b0ff9640 1 -- 172.21.15.90:0/3349113931 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18430 (secure 0 0 0) 0x7f38a4055dc0 con 0x7f38b4070b00 2024-03-11T18:37:02.238 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:37:02.238 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":28,"fsid":"76d56b10-6c29-476a-804b-8482ea90f9e2","created":"2024-03-11T18:34:54.521741+0000","modified":"2024-03-11T18:36:05.526044+0000","last_up_change":"2024-03-11T18:35:16.207500+0000","last_in_change":"2024-03-11T18:35:05.213634+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T18:35:16.193842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T18:35:54.001469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T18:35:55.917321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T18:36:02.210943+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"34decff9-7280-4d84-9665-d6af8ea6de84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6819","nonce":1755667921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6821","nonce":1755667921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6825","nonce":1755667921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6823","nonce":1755667921}]},"public_addr":"172.21.15.90:6819/1755667921","cluster_addr":"172.21.15.90:6821/1755667921","heartbeat_back_addr":"172.21.15.90:6825/1755667921","heartbeat_front_addr":"172.21.15.90:6823/1755667921","state":["exists","up"]},{"osd":1,"uuid":"8ecdce9e-5486-415a-a609-dd16e945440e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6807","nonce":438682081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6809","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6811","nonce":438682081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6817","nonce":438682081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6815","nonce":438682081}]},"public_addr":"172.21.15.90:6807/438682081","cluster_addr":"172.21.15.90:6811/438682081","heartbeat_back_addr":"172.21.15.90:6817/438682081","heartbeat_front_addr":"172.21.15.90:6815/438682081","state":["exists","up"]},{"osd":2,"uuid":"737486d4-0e90-4c82-9e79-bf37c812e5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6827","nonce":2837704738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6829","nonce":2837704738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6833","nonce":2837704738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6831","nonce":2837704738}]},"public_addr":"172.21.15.90:6827/2837704738","cluster_addr":"172.21.15.90:6829/2837704738","heartbeat_back_addr":"172.21.15.90:6833/2837704738","heartbeat_front_addr":"172.21.15.90:6831/2837704738","state":["exists","up"]},{"osd":3,"uuid":"698bd661-bbef-4699-a619-f7bfe23ce12c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6803","nonce":240987971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6805","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6806","nonce":240987971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6813","nonce":240987971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6810","nonce":240987971}]},"public_addr":"172.21.15.90:6803/240987971","cluster_addr":"172.21.15.90:6806/240987971","heartbeat_back_addr":"172.21.15.90:6813/240987971","heartbeat_front_addr":"172.21.15.90:6810/240987971","state":["exists","up"]},{"osd":4,"uuid":"aba9e31f-73d9-43fe-a10b-2dc5832968b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6801","nonce":502685275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6803","nonce":502685275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6807","nonce":502685275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6805","nonce":502685275}]},"public_addr":"172.21.15.92:6801/502685275","cluster_addr":"172.21.15.92:6803/502685275","heartbeat_back_addr":"172.21.15.92:6807/502685275","heartbeat_front_addr":"172.21.15.92:6805/502685275","state":["exists","up"]},{"osd":5,"uuid":"bdd56766-4b28-4007-8de3-b5b08cf335d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6809","nonce":200142070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6811","nonce":200142070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6815","nonce":200142070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6813","nonce":200142070}]},"public_addr":"172.21.15.92:6809/200142070","cluster_addr":"172.21.15.92:6811/200142070","heartbeat_back_addr":"172.21.15.92:6815/200142070","heartbeat_front_addr":"172.21.15.92:6813/200142070","state":["exists","up"]},{"osd":6,"uuid":"6c580a59-610f-455e-a96b-033299dcc118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6817","nonce":1919010204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6819","nonce":1919010204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6823","nonce":1919010204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6821","nonce":1919010204}]},"public_addr":"172.21.15.92:6817/1919010204","cluster_addr":"172.21.15.92:6819/1919010204","heartbeat_back_addr":"172.21.15.92:6823/1919010204","heartbeat_front_addr":"172.21.15.92:6821/1919010204","state":["exists","up"]},{"osd":7,"uuid":"958184cc-aebf-4c5b-91a0-2a3e38a6b42b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6825","nonce":3475925719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6827","nonce":3475925719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6831","nonce":3475925719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6829","nonce":3475925719}]},"public_addr":"172.21.15.92:6825/3475925719","cluster_addr":"172.21.15.92:6827/3475925719","heartbeat_back_addr":"172.21.15.92:6831/3475925719","heartbeat_front_addr":"172.21.15.92:6829/3475925719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.816294+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.048084+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.137975+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.968246+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.016829+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.931676+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.041673+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.079404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T18:37:02.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.236+0000 7f38ba516640 1 -- 172.21.15.90:0/3349113931 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f38a0066e90 msgr2=0x7f38a0069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:02.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.236+0000 7f38ba516640 1 --2- 172.21.15.90:0/3349113931 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f38a0066e90 0x7f38a0069350 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f38a80b2000 tx=0x7f38a8009040 comp rx=0 tx=0).stop 2024-03-11T18:37:02.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.236+0000 7f38ba516640 1 -- 172.21.15.90:0/3349113931 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f38b4070b00 msgr2=0x7f38b414d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:02.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.236+0000 7f38ba516640 1 --2- 172.21.15.90:0/3349113931 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f38b4070b00 0x7f38b414d7a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f38a400d5f0 tx=0x7f38a400dab0 comp rx=0 tx=0).stop 2024-03-11T18:37:02.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.236+0000 7f38ba516640 1 -- 172.21.15.90:0/3349113931 shutdown_connections 2024-03-11T18:37:02.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.236+0000 7f38ba516640 1 --2- 172.21.15.90:0/3349113931 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f38a0066e90 0x7f38a0069350 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:02.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.236+0000 7f38ba516640 1 --2- 172.21.15.90:0/3349113931 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f38b4150dd0 0x7f38b41f2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:02.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.236+0000 7f38ba516640 1 --2- 172.21.15.90:0/3349113931 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f38b41509f0 0x7f38b40ca5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:02.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.236+0000 7f38ba516640 1 --2- 172.21.15.90:0/3349113931 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f38b4070b00 0x7f38b414d7a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:02.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.236+0000 7f38ba516640 1 -- 172.21.15.90:0/3349113931 >> 172.21.15.90:0/3349113931 conn(0x7f38b406ac00 msgr2=0x7f38b40c7580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:37:02.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.236+0000 7f38ba516640 1 -- 172.21.15.90:0/3349113931 shutdown_connections 2024-03-11T18:37:02.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.236+0000 7f38ba516640 1 -- 172.21.15.90:0/3349113931 wait complete. 2024-03-11T18:37:02.265 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T18:37:02.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.412+0000 7ff43c2de640 1 Processor -- start 2024-03-11T18:37:02.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.412+0000 7ff43c2de640 1 -- start start 2024-03-11T18:37:02.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.416+0000 7ff43c2de640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff4340c65b0 0x7ff4340c37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:02.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.416+0000 7ff43c2de640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff4340c3d30 0x7ff4340cc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:02.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.416+0000 7ff43c2de640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4340c5e60 con 0x7ff434150a90 2024-03-11T18:37:02.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.416+0000 7ff43c2de640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff434156a60 con 0x7ff4340c3d30 2024-03-11T18:37:02.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.416+0000 7ff43c2de640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4340c55c0 con 0x7ff4340c65b0 2024-03-11T18:37:02.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.420+0000 7ff43a07a640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7ff434150a90 0x7ff434157220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:52836/0 (socket says 172.21.15.90:52836) 2024-03-11T18:37:02.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.420+0000 7ff43a07a640 1 -- 172.21.15.90:0/1876739459 learned_addr learned my addr 172.21.15.90:0/1876739459 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T18:37:02.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.420+0000 7ff439879640 1 --2- 172.21.15.90:0/1876739459 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff4340c65b0 0x7ff4340c37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:02.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.424+0000 7ff439078640 1 -- 172.21.15.90:0/1876739459 <== mon.0 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2642643285 0 0) 0x7ff4340c5e60 con 0x7ff434150a90 2024-03-11T18:37:02.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.424+0000 7ff439078640 1 -- 172.21.15.90:0/1876739459 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff41c003530 con 0x7ff434150a90 2024-03-11T18:37:02.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.428+0000 7ff439078640 1 -- 172.21.15.90:0/1876739459 <== mon.0 v1:172.21.15.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2334922891 0 0) 0x7ff41c003530 con 0x7ff434150a90 2024-03-11T18:37:02.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.428+0000 7ff439078640 1 -- 172.21.15.90:0/1876739459 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff4340c65b0 msgr2=0x7ff4340c37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:02.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.428+0000 7ff439078640 1 --2- 172.21.15.90:0/1876739459 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff4340c65b0 0x7ff4340c37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:02.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.428+0000 7ff439078640 1 -- 172.21.15.90:0/1876739459 >> v2:172.21.15.92:3300/0 conn(0x7ff4340c3d30 msgr2=0x7ff4340cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:02.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.428+0000 7ff439078640 1 --2- 172.21.15.90:0/1876739459 >> v2:172.21.15.92:3300/0 conn(0x7ff4340c3d30 0x7ff4340cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:02.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.428+0000 7ff439078640 1 -- 172.21.15.90:0/1876739459 --> v1:172.21.15.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4340c58a0 con 0x7ff434150a90 2024-03-11T18:37:02.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.428+0000 7ff439078640 1 -- 172.21.15.90:0/1876739459 <== mon.0 v1:172.21.15.90:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 691735435 0 0) 0x7ff42c001a00 con 0x7ff434150a90 2024-03-11T18:37:02.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.428+0000 7ff439078640 1 -- 172.21.15.90:0/1876739459 >> v1:172.21.15.90:6789/0 conn(0x7ff434150a90 legacy=0x7ff434157220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:02.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.428+0000 7ff439078640 1 --2- 172.21.15.90:0/1876739459 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff41c003980 0x7ff41c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:02.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.428+0000 7ff439078640 1 --2- 172.21.15.90:0/1876739459 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff41c0047d0 0x7ff41c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:02.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.428+0000 7ff439078640 1 --2- 172.21.15.90:0/1876739459 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff4340c65b0 0x7ff41c0051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:02.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.428+0000 7ff439078640 1 -- 172.21.15.90:0/1876739459 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff434156a60 con 0x7ff41c0047d0 2024-03-11T18:37:02.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.428+0000 7ff439078640 1 -- 172.21.15.90:0/1876739459 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4340c55c0 con 0x7ff41c003980 2024-03-11T18:37:02.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.428+0000 7ff439078640 1 -- 172.21.15.90:0/1876739459 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff41c00bb70 con 0x7ff4340c65b0 2024-03-11T18:37:02.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.428+0000 7ff439879640 1 --2- 172.21.15.90:0/1876739459 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff41c0047d0 0x7ff41c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:02.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.432+0000 7ff439879640 1 -- 172.21.15.90:0/1876739459 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff4340c65b0 msgr2=0x7ff41c0051f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:02.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.432+0000 7ff439879640 1 --2- 172.21.15.90:0/1876739459 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff4340c65b0 0x7ff41c0051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:02.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.432+0000 7ff439879640 1 -- 172.21.15.90:0/1876739459 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff41c003980 msgr2=0x7ff41c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:02.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.432+0000 7ff439879640 1 --2- 172.21.15.90:0/1876739459 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff41c003980 0x7ff41c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:02.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.432+0000 7ff439879640 1 -- 172.21.15.90:0/1876739459 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff41c003530 con 0x7ff41c0047d0 2024-03-11T18:37:02.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.436+0000 7ff439879640 1 --2- 172.21.15.90:0/1876739459 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff41c0047d0 0x7ff41c004c50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff42800ba30 tx=0x7ff42801c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3fb2fcd9015c64d server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:37:02.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.440+0000 7ff439078640 1 -- 172.21.15.90:0/1876739459 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4280a9680 con 0x7ff41c0047d0 2024-03-11T18:37:02.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.440+0000 7ff439078640 1 -- 172.21.15.90:0/1876739459 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4280a97e0 con 0x7ff41c0047d0 2024-03-11T18:37:02.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.440+0000 7ff439078640 1 -- 172.21.15.90:0/1876739459 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4280a9b90 con 0x7ff41c0047d0 2024-03-11T18:37:02.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.440+0000 7ff43c2de640 1 -- 172.21.15.90:0/1876739459 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff41c0047d0 msgr2=0x7ff41c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:02.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.440+0000 7ff43c2de640 1 --2- 172.21.15.90:0/1876739459 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff41c0047d0 0x7ff41c004c50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff42800ba30 tx=0x7ff42801c6a0 comp rx=0 tx=0).stop 2024-03-11T18:37:02.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.440+0000 7ff43a87b640 1 -- 172.21.15.90:0/1876739459 reap_dead start 2024-03-11T18:37:02.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.440+0000 7ff43c2de640 1 -- 172.21.15.90:0/1876739459 shutdown_connections 2024-03-11T18:37:02.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.440+0000 7ff43c2de640 1 -- 172.21.15.90:0/1876739459 >> 172.21.15.90:0/1876739459 conn(0x7ff434071aa0 msgr2=0x7ff434071eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:37:02.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.440+0000 7ff43c2de640 1 -- 172.21.15.90:0/1876739459 shutdown_connections 2024-03-11T18:37:02.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.440+0000 7ff43c2de640 1 -- 172.21.15.90:0/1876739459 wait complete. 2024-03-11T18:37:02.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.440+0000 7ff43c2de640 1 Processor -- start 2024-03-11T18:37:02.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.440+0000 7ff43c2de640 1 -- start start 2024-03-11T18:37:02.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.444+0000 7ff43c2de640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff41c0047d0 0x7ff4340ca350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:02.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.444+0000 7ff43c2de640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff43406b4d0 0x7ff4340cb540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:02.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.444+0000 7ff43c2de640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff4340cba80 0x7ff434154670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:02.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.444+0000 7ff43c2de640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff43406c660 con 0x7ff43406b4d0 2024-03-11T18:37:02.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.444+0000 7ff43c2de640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff43413f800 con 0x7ff41c0047d0 2024-03-11T18:37:02.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.444+0000 7ff43c2de640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff43406c890 con 0x7ff4340cba80 2024-03-11T18:37:02.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.444+0000 7ff439879640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff43406b4d0 0x7ff4340cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:02.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.444+0000 7ff439879640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff43406b4d0 0x7ff4340cb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41084/0 (socket says 172.21.15.90:41084) 2024-03-11T18:37:02.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.444+0000 7ff439879640 1 -- 172.21.15.90:0/2801585912 learned_addr learned my addr 172.21.15.90:0/2801585912 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:37:02.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.448+0000 7ff439879640 1 -- 172.21.15.90:0/2801585912 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff4340cba80 msgr2=0x7ff434154670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:02.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.448+0000 7ff439879640 1 --2- 172.21.15.90:0/2801585912 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff4340cba80 0x7ff434154670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:02.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.448+0000 7ff439879640 1 -- 172.21.15.90:0/2801585912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff41c0047d0 msgr2=0x7ff4340ca350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:02.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.448+0000 7ff439879640 1 --2- 172.21.15.90:0/2801585912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff41c0047d0 0x7ff4340ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:02.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.448+0000 7ff439879640 1 -- 172.21.15.90:0/2801585912 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff43413ec10 con 0x7ff43406b4d0 2024-03-11T18:37:02.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.448+0000 7ff439879640 1 --2- 172.21.15.90:0/2801585912 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff43406b4d0 0x7ff4340cb540 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff42801c4d0 tx=0x7ff41c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:37:02.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.452+0000 7ff432ffd640 1 -- 172.21.15.90:0/2801585912 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4280a9ea0 con 0x7ff43406b4d0 2024-03-11T18:37:02.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.452+0000 7ff43c2de640 1 -- 172.21.15.90:0/2801585912 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff43406c660 con 0x7ff43406b4d0 2024-03-11T18:37:02.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.452+0000 7ff43c2de640 1 -- 172.21.15.90:0/2801585912 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff43413f1b0 con 0x7ff43406b4d0 2024-03-11T18:37:02.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.452+0000 7ff43c2de640 1 -- 172.21.15.90:0/2801585912 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff43404eef0 con 0x7ff43406b4d0 2024-03-11T18:37:02.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.452+0000 7ff432ffd640 1 -- 172.21.15.90:0/2801585912 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4280059a0 con 0x7ff43406b4d0 2024-03-11T18:37:02.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.452+0000 7ff432ffd640 1 -- 172.21.15.90:0/2801585912 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff428005d10 con 0x7ff43406b4d0 2024-03-11T18:37:02.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.780+0000 7ff432ffd640 1 -- 172.21.15.90:0/2801585912 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7ff4280b3450 con 0x7ff43406b4d0 2024-03-11T18:37:02.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.780+0000 7ff432ffd640 1 --2- 172.21.15.90:0/2801585912 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7ff420066e70 0x7ff420069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:02.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.780+0000 7ff43a07a640 1 --2- 172.21.15.90:0/2801585912 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7ff420066e70 0x7ff420069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:02.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.780+0000 7ff43a07a640 1 --2- 172.21.15.90:0/2801585912 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7ff420066e70 0x7ff420069330 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff42c005aa0 tx=0x7ff42c0018e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:37:02.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.816+0000 7ff432ffd640 1 -- 172.21.15.90:0/2801585912 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff4280a83e0 con 0x7ff43406b4d0 2024-03-11T18:37:02.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:02.816+0000 7ff432ffd640 1 -- 172.21.15.90:0/2801585912 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff4280ec750 con 0x7ff43406b4d0 2024-03-11T18:37:03.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:03.076+0000 7ff43c2de640 1 -- 172.21.15.90:0/2801585912 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff434154390 con 0x7ff43406b4d0 2024-03-11T18:37:03.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:03.192+0000 7ff432ffd640 1 -- 172.21.15.90:0/2801585912 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+6575 (secure 0 0 0) 0x7ff434154390 con 0x7ff43406b4d0 2024-03-11T18:37:03.199 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T18:37:03.199 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/641527992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":641527992},{"type":"v1","addr":"172.21.15.90:6839","nonce":641527992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-11T18:35:57.363570+0000","modified":"2024-03-11T18:36:51.575661+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4255,"mds_1":4253,"mds_2":4241,"mds_3":4238,"mds_4":4229},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4229":{"gid":4229,"name":"c","rank":4,"incarnation":18,"state":"up:active","state_seq":17,"addr":"172.21.15.90:6835/460417177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":460417177},{"type":"v1","addr":"172.21.15.90:6835","nonce":460417177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4238":{"gid":4238,"name":"d","rank":3,"incarnation":16,"state":"up:active","state_seq":17,"addr":"172.21.15.92:6835/3568438077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":3568438077},{"type":"v1","addr":"172.21.15.92:6835","nonce":3568438077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4241":{"gid":4241,"name":"b","rank":2,"incarnation":14,"state":"up:active","state_seq":16,"addr":"172.21.15.92:6834/2306487548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":2306487548},{"type":"v1","addr":"172.21.15.92:6834","nonce":2306487548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4253":{"gid":4253,"name":"f","rank":1,"incarnation":10,"state":"up:active","state_seq":15,"addr":"172.21.15.92:6837/3209534444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":3209534444},{"type":"v1","addr":"172.21.15.92:6837","nonce":3209534444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4255":{"gid":4255,"name":"e","rank":0,"incarnation":5,"state":"up:active","state_seq":6,"addr":"172.21.15.90:6837/3485943375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3485943375},{"type":"v1","addr":"172.21.15.90:6837","nonce":3485943375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T18:37:03.199 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 19 2024-03-11T18:37:03.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:03.196+0000 7ff43c2de640 1 -- 172.21.15.90:0/2801585912 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7ff420066e70 msgr2=0x7ff420069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:03.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:03.196+0000 7ff43c2de640 1 --2- 172.21.15.90:0/2801585912 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7ff420066e70 0x7ff420069330 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff42c005aa0 tx=0x7ff42c0018e0 comp rx=0 tx=0).stop 2024-03-11T18:37:03.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:03.196+0000 7ff43c2de640 1 -- 172.21.15.90:0/2801585912 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff43406b4d0 msgr2=0x7ff4340cb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:03.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:03.196+0000 7ff43c2de640 1 --2- 172.21.15.90:0/2801585912 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff43406b4d0 0x7ff4340cb540 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff42801c4d0 tx=0x7ff41c005800 comp rx=0 tx=0).stop 2024-03-11T18:37:03.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:03.196+0000 7ff43c2de640 1 -- 172.21.15.90:0/2801585912 shutdown_connections 2024-03-11T18:37:03.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:03.196+0000 7ff43c2de640 1 --2- 172.21.15.90:0/2801585912 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7ff420066e70 0x7ff420069330 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:03.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:03.196+0000 7ff43c2de640 1 --2- 172.21.15.90:0/2801585912 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff4340cba80 0x7ff434154670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:03.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:03.196+0000 7ff43c2de640 1 --2- 172.21.15.90:0/2801585912 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff43406b4d0 0x7ff4340cb540 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:03.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:03.196+0000 7ff43c2de640 1 --2- 172.21.15.90:0/2801585912 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff41c0047d0 0x7ff4340ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:03.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:03.196+0000 7ff43c2de640 1 -- 172.21.15.90:0/2801585912 >> 172.21.15.90:0/2801585912 conn(0x7ff434071aa0 msgr2=0x7ff43414e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:37:03.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:03.196+0000 7ff43c2de640 1 -- 172.21.15.90:0/2801585912 shutdown_connections 2024-03-11T18:37:03.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:03.196+0000 7ff43c2de640 1 -- 172.21.15.90:0/2801585912 wait complete. 2024-03-11T18:37:03.827 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 19, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-11T18:35:57.363570+0000', 'modified': '2024-03-11T18:36:51.575661+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 4255, 'mds_1': 4253, 'mds_2': 4241, 'mds_3': 4238, 'mds_4': 4229}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4229': {'gid': 4229, 'name': 'c', 'rank': 4, 'incarnation': 18, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.90:6835/460417177', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.90:6834', 'nonce': 460417177}, {'type': 'v1', 'addr': '172.21.15.90:6835', 'nonce': 460417177}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_4238': {'gid': 4238, 'name': 'd', 'rank': 3, 'incarnation': 16, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.92:6835/3568438077', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6833', 'nonce': 3568438077}, {'type': 'v1', 'addr': '172.21.15.92:6835', 'nonce': 3568438077}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_4241': {'gid': 4241, 'name': 'b', 'rank': 2, 'incarnation': 14, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.92:6834/2306487548', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6832', 'nonce': 2306487548}, {'type': 'v1', 'addr': '172.21.15.92:6834', 'nonce': 2306487548}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_4253': {'gid': 4253, 'name': 'f', 'rank': 1, 'incarnation': 10, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.92:6837/3209534444', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.92:6836', 'nonce': 3209534444}, {'type': 'v1', 'addr': '172.21.15.92:6837', 'nonce': 3209534444}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_4255': {'gid': 4255, 'name': 'e', 'rank': 0, 'incarnation': 5, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.90:6837/3485943375', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.90:6836', 'nonce': 3485943375}, {'type': 'v1', 'addr': '172.21.15.90:6837', 'nonce': 3485943375}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-11T18:37:03.827 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-11T18:37:03.827 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-11T18:37:03.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:03.956+0000 7f52cf3e8640 1 Processor -- start 2024-03-11T18:37:03.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:03.956+0000 7f52cf3e8640 1 -- start start 2024-03-11T18:37:03.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:03.956+0000 7f52cf3e8640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f52c8151d50 0x7f52c80c3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:03.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:03.956+0000 7f52cf3e8640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f52c80c95f0 0x7f52c80c5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:03.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:03.956+0000 7f52cf3e8640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52c80c7430 con 0x7f52c806b860 2024-03-11T18:37:03.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:03.956+0000 7f52cf3e8640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52c8152130 con 0x7f52c8151d50 2024-03-11T18:37:03.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:03.956+0000 7f52cf3e8640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52c80c7a40 con 0x7f52c80c95f0 2024-03-11T18:37:04.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.016+0000 7f52cd184640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f52c8151d50 0x7f52c80c3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:04.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.032+0000 7f52cd184640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f52c8151d50 0x7f52c80c3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44352/0 (socket says 172.21.15.90:44352) 2024-03-11T18:37:04.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.032+0000 7f52cd184640 1 -- 172.21.15.90:0/205589647 learned_addr learned my addr 172.21.15.90:0/205589647 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:37:04.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.040+0000 7f52cd184640 1 -- 172.21.15.90:0/205589647 >> v2:172.21.15.92:3300/0 conn(0x7f52c80c95f0 msgr2=0x7f52c80c5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:04.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.040+0000 7f52cd184640 1 --2- 172.21.15.90:0/205589647 >> v2:172.21.15.92:3300/0 conn(0x7f52c80c95f0 0x7f52c80c5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:04.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.040+0000 7f52cd184640 1 -- 172.21.15.90:0/205589647 >> v1:172.21.15.90:6789/0 conn(0x7f52c806b860 legacy=0x7f52c8157970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:04.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.040+0000 7f52cd184640 1 -- 172.21.15.90:0/205589647 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c80c6e70 con 0x7f52c8151d50 2024-03-11T18:37:04.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.052+0000 7f52cd184640 1 --2- 172.21.15.90:0/205589647 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52c8151d50 0x7f52c80c3130 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f52c8073140 tx=0x7f52b801ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aee21e47c2de17be server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:37:04.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.068+0000 7f52bffff640 1 -- 172.21.15.90:0/205589647 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b80a89d0 con 0x7f52c8151d50 2024-03-11T18:37:04.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.068+0000 7f52bffff640 1 -- 172.21.15.90:0/205589647 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b80a8b30 con 0x7f52c8151d50 2024-03-11T18:37:04.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.068+0000 7f52bffff640 1 -- 172.21.15.90:0/205589647 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b80b0440 con 0x7f52c8151d50 2024-03-11T18:37:04.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.072+0000 7f52cf3e8640 1 -- 172.21.15.90:0/205589647 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52c8151d50 msgr2=0x7f52c80c3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:04.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.072+0000 7f52cf3e8640 1 --2- 172.21.15.90:0/205589647 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52c8151d50 0x7f52c80c3130 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f52c8073140 tx=0x7f52b801ecd0 comp rx=0 tx=0).stop 2024-03-11T18:37:04.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.072+0000 7f52cf3e8640 1 -- 172.21.15.90:0/205589647 shutdown_connections 2024-03-11T18:37:04.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.072+0000 7f52cf3e8640 1 --2- 172.21.15.90:0/205589647 >> v2:172.21.15.92:3300/0 conn(0x7f52c80c95f0 0x7f52c80c5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:04.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.072+0000 7f52cf3e8640 1 --2- 172.21.15.90:0/205589647 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52c8151d50 0x7f52c80c3130 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:04.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.072+0000 7f52cf3e8640 1 -- 172.21.15.90:0/205589647 >> 172.21.15.90:0/205589647 conn(0x7f52c8071aa0 msgr2=0x7f52c8071eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:37:04.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.072+0000 7f52cf3e8640 1 -- 172.21.15.90:0/205589647 shutdown_connections 2024-03-11T18:37:04.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.072+0000 7f52cf3e8640 1 -- 172.21.15.90:0/205589647 wait complete. 2024-03-11T18:37:04.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.072+0000 7f52cf3e8640 1 Processor -- start 2024-03-11T18:37:04.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.072+0000 7f52cf3e8640 1 -- start start 2024-03-11T18:37:04.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.076+0000 7f52cf3e8640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52c806b860 0x7f52c81e2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:04.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.076+0000 7f52cf3e8640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52c80c95f0 0x7f52c81456b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:04.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.076+0000 7f52cf3e8640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f52c8151d50 0x7f52c814a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:04.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.076+0000 7f52cf3e8640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c80c5070 con 0x7f52c806b860 2024-03-11T18:37:04.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.076+0000 7f52cf3e8640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c80c5310 con 0x7f52c80c95f0 2024-03-11T18:37:04.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.076+0000 7f52cf3e8640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c81e3300 con 0x7f52c8151d50 2024-03-11T18:37:04.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.076+0000 7f52cc983640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52c80c95f0 0x7f52c81456b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:04.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.076+0000 7f52cd985640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f52c8151d50 0x7f52c814a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:04.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.076+0000 7f52cc983640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52c80c95f0 0x7f52c81456b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.90:49564/0 (socket says 172.21.15.90:49564) 2024-03-11T18:37:04.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.076+0000 7f52cc983640 1 -- 172.21.15.90:0/4091697083 learned_addr learned my addr 172.21.15.90:0/4091697083 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:37:04.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.088+0000 7f52cc983640 1 -- 172.21.15.90:0/4091697083 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f52c8151d50 msgr2=0x7f52c814a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:04.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.088+0000 7f52cc983640 1 --2- 172.21.15.90:0/4091697083 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f52c8151d50 0x7f52c814a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:04.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.088+0000 7f52cc983640 1 -- 172.21.15.90:0/4091697083 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52c806b860 msgr2=0x7f52c81e2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:04.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.088+0000 7f52cc983640 1 --2- 172.21.15.90:0/4091697083 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52c806b860 0x7f52c81e2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:04.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.088+0000 7f52cc983640 1 -- 172.21.15.90:0/4091697083 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b80094d0 con 0x7f52c80c95f0 2024-03-11T18:37:04.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.096+0000 7f52cc983640 1 --2- 172.21.15.90:0/4091697083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52c80c95f0 0x7f52c81456b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f52c400c450 tx=0x7f52c400c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:37:04.111 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:37:04.113 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:37:04.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.108+0000 7f52bdffb640 1 -- 172.21.15.90:0/4091697083 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c4012830 con 0x7f52c80c95f0 2024-03-11T18:37:04.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.108+0000 7f52cf3e8640 1 -- 172.21.15.90:0/4091697083 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52c81e35e0 con 0x7f52c80c95f0 2024-03-11T18:37:04.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.108+0000 7f52cf3e8640 1 -- 172.21.15.90:0/4091697083 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52c81e2d40 con 0x7f52c80c95f0 2024-03-11T18:37:04.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.112+0000 7f52d2551000 1 Processor -- start 2024-03-11T18:37:04.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.112+0000 7f52d2551000 1 -- start start 2024-03-11T18:37:04.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.112+0000 7f52d2551000 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x562b82357f70 0x562b8231e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:04.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.112+0000 7f52d2551000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b8231e890 0x562b82359680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:04.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.112+0000 7f52d2551000 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b82359bc0 0x562b82359fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:04.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.112+0000 7f52d2551000 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b821e0d00 con 0x562b82359bc0 2024-03-11T18:37:04.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.112+0000 7f52d2551000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b82172430 con 0x562b8231e890 2024-03-11T18:37:04.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.112+0000 7f52d2551000 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x562b8216f960 con 0x562b82357f70 2024-03-11T18:37:04.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.128+0000 7f52cd184640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b8231e890 0x562b82359680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:04.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.132+0000 7f52cd184640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b8231e890 0x562b82359680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.90:49578/0 (socket says 172.21.15.90:49578) 2024-03-11T18:37:04.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.132+0000 7f52cd184640 1 -- 172.21.15.90:0/3893312466 learned_addr learned my addr 172.21.15.90:0/3893312466 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:37:04.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.160+0000 7f52cc983640 1 --2- 172.21.15.90:0/3893312466 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b82359bc0 0x562b82359fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:04.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.164+0000 7f52cc983640 1 -- 172.21.15.90:0/3893312466 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x562b82357f70 msgr2=0x562b8231e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:04.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.164+0000 7f52cc983640 1 --2- 172.21.15.90:0/3893312466 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x562b82357f70 0x562b8231e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:04.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.164+0000 7f52cc983640 1 -- 172.21.15.90:0/3893312466 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b8231e890 msgr2=0x562b82359680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:04.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.164+0000 7f52cc983640 1 --2- 172.21.15.90:0/3893312466 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b8231e890 0x562b82359680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:04.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.164+0000 7f52cc983640 1 -- 172.21.15.90:0/3893312466 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b81e61890 con 0x562b82359bc0 2024-03-11T18:37:04.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.168+0000 7f52cc983640 1 --2- 172.21.15.90:0/3893312466 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b82359bc0 0x562b82359fa0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f52c4016000 tx=0x7f52c4007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c1357cb43864ad7 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:37:04.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.172+0000 7f52b2ffd640 1 -- 172.21.15.90:0/3893312466 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c4012ea0 con 0x562b82359bc0 2024-03-11T18:37:04.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.172+0000 7f52b2ffd640 1 -- 172.21.15.90:0/3893312466 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c4006c70 con 0x562b82359bc0 2024-03-11T18:37:04.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.172+0000 7f52b2ffd640 1 -- 172.21.15.90:0/3893312466 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c401c660 con 0x562b82359bc0 2024-03-11T18:37:04.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.172+0000 7f52d2551000 1 -- 172.21.15.90:0/3893312466 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b82359bc0 msgr2=0x562b82359fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:04.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.172+0000 7f52d2551000 1 --2- 172.21.15.90:0/3893312466 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b82359bc0 0x562b82359fa0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f52c4016000 tx=0x7f52c4007f90 comp rx=0 tx=0).stop 2024-03-11T18:37:04.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.172+0000 7f52d2551000 1 -- 172.21.15.90:0/3893312466 shutdown_connections 2024-03-11T18:37:04.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.172+0000 7f52d2551000 1 --2- 172.21.15.90:0/3893312466 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b82359bc0 0x562b82359fa0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:04.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.172+0000 7f52d2551000 1 --2- 172.21.15.90:0/3893312466 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b8231e890 0x562b82359680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:04.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.172+0000 7f52d2551000 1 --2- 172.21.15.90:0/3893312466 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x562b82357f70 0x562b8231e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:04.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.172+0000 7f52d2551000 1 -- 172.21.15.90:0/3893312466 >> 172.21.15.90:0/3893312466 conn(0x562b8231eff0 msgr2=0x562b82357440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:37:04.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.172+0000 7f52d2551000 1 -- 172.21.15.90:0/3893312466 shutdown_connections 2024-03-11T18:37:04.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.172+0000 7f52d2551000 1 -- 172.21.15.90:0/3893312466 wait complete. 2024-03-11T18:37:04.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.176+0000 7f52d2551000 1 -- start start 2024-03-11T18:37:04.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.176+0000 7f52d2551000 1 Processor -- start 2024-03-11T18:37:04.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.176+0000 7f52d2551000 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x562b8231e890 0x562b82364f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:04.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.176+0000 7f52d2551000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b82357f70 0x562b82362a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:04.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.176+0000 7f52d2551000 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b82359bc0 0x562b82362f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:04.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.176+0000 7f52d2551000 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b8216b140 con 0x562b82359bc0 2024-03-11T18:37:04.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.176+0000 7f52d2551000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b82105630 con 0x562b82357f70 2024-03-11T18:37:04.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.176+0000 7f52d2551000 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x562b821052d0 con 0x562b8231e890 2024-03-11T18:37:04.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.176+0000 7f52d2551000 10 client.0 _ll_register_callbacks cb 0x562b8221b1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T18:37:04.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.176+0000 7f52cc983640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b82359bc0 0x562b82362f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:04.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.176+0000 7f52cc983640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b82359bc0 0x562b82362f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44382/0 (socket says 172.21.15.90:44382) 2024-03-11T18:37:04.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.176+0000 7f52cc983640 1 -- 172.21.15.90:0/4222405560 learned_addr learned my addr 172.21.15.90:0/4222405560 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:37:04.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.180+0000 7f52cc983640 1 -- 172.21.15.90:0/4222405560 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x562b8231e890 msgr2=0x562b82364f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:04.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.180+0000 7f52cc983640 1 --2- 172.21.15.90:0/4222405560 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x562b8231e890 0x562b82364f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:04.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.180+0000 7f52cc983640 1 -- 172.21.15.90:0/4222405560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b82357f70 msgr2=0x562b82362a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:04.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.180+0000 7f52cc983640 1 --2- 172.21.15.90:0/4222405560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b82357f70 0x562b82362a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:04.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.180+0000 7f52cc983640 1 -- 172.21.15.90:0/4222405560 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52c81e2d40 con 0x562b82359bc0 2024-03-11T18:37:04.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.184+0000 7f52cc983640 1 --2- 172.21.15.90:0/4222405560 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b82359bc0 0x562b82362f60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f52c4013000 tx=0x7f52c4003770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99e0e51b9041b5d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:37:04.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.184+0000 7f52a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-03-11T18:37:04.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.188+0000 7f52a7fff640 1 -- 172.21.15.90:0/4222405560 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c401c660 con 0x562b82359bc0 2024-03-11T18:37:04.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.188+0000 7f52a7fff640 1 -- 172.21.15.90:0/4222405560 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f52c401c7c0 con 0x562b82359bc0 2024-03-11T18:37:04.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.188+0000 7f52d2551000 1 -- 172.21.15.90:0/4222405560 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562b8211e850 con 0x562b82359bc0 2024-03-11T18:37:04.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.188+0000 7f52a7fff640 1 -- 172.21.15.90:0/4222405560 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52c401c980 con 0x562b82359bc0 2024-03-11T18:37:04.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.188+0000 7f52d2551000 10 client.4662 fetch_fsmap learned FSMap version 19 2024-03-11T18:37:04.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.188+0000 7f52d2551000 1 -- 172.21.15.90:0/4222405560 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) v3 -- 0x562b8212ae70 con 0x562b82359bc0 2024-03-11T18:37:04.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.192+0000 7f52a7fff640 1 -- 172.21.15.90:0/4222405560 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562b8211e850 con 0x562b82359bc0 2024-03-11T18:37:04.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.192+0000 7f52a7fff640 1 -- 172.21.15.90:0/4222405560 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 19) v1 ==== 4500+0+0 (secure 0 0 0) 0x7f52c402d020 con 0x562b82359bc0 2024-03-11T18:37:04.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.192+0000 7f52d2551000 10 client.4662 fetch_fsmap finished waiting for FSMap version 19 2024-03-11T18:37:04.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.192+0000 7f52d2551000 10 client.4662 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-11T18:37:04.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.192+0000 7f52d2551000 20 client.4662 populate_metadata read hostname 'smithi090' 2024-03-11T18:37:04.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.192+0000 7f52d2551000 1 --2- 172.21.15.90:0/4222405560 >> [v2:172.21.15.90:6834/460417177,v1:172.21.15.90:6835/460417177] conn(0x562b823539d0 0x562b82353db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:04.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.192+0000 7f52d2551000 4 client.4662 mds_command: new command op to 4229 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T18:37:04.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.192+0000 7f52d2551000 1 -- 172.21.15.90:0/4222405560 --> [v2:172.21.15.90:6834/460417177,v1:172.21.15.90:6835/460417177] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562b82167f20 con 0x562b823539d0 2024-03-11T18:37:04.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.192+0000 7f52cd985640 1 --2- 172.21.15.90:0/4222405560 >> [v2:172.21.15.90:6834/460417177,v1:172.21.15.90:6835/460417177] conn(0x562b823539d0 0x562b82353db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:04.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.204+0000 7f52bdffb640 1 -- 172.21.15.90:0/4091697083 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c402fdd0 con 0x7f52c80c95f0 2024-03-11T18:37:04.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.204+0000 7f52bdffb640 1 -- 172.21.15.90:0/4091697083 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c4011580 con 0x7f52c80c95f0 2024-03-11T18:37:04.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.208+0000 7f52bdffb640 1 -- 172.21.15.90:0/4091697083 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f52c4027c50 con 0x7f52c80c95f0 2024-03-11T18:37:04.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.208+0000 7f52bdffb640 1 --2- 172.21.15.90:0/4091697083 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f52ac066cc0 0x7f52ac069180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:04.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.208+0000 7f52cd184640 1 --2- 172.21.15.90:0/4091697083 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f52ac066cc0 0x7f52ac069180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:04.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.208+0000 7f52cd184640 1 --2- 172.21.15.90:0/4091697083 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f52ac066cc0 0x7f52ac069180 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f52b8000c10 tx=0x7f52b80b2040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:37:04.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.208+0000 7f52bdffb640 1 -- 172.21.15.90:0/4091697083 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f52c401e030 con 0x7f52c80c95f0 2024-03-11T18:37:04.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.236+0000 7f52cd985640 1 --2- 172.21.15.90:0/4222405560 >> [v2:172.21.15.90:6834/460417177,v1:172.21.15.90:6835/460417177] conn(0x562b823539d0 0x562b82353db0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=af4478cb339d8ca6 server_cookie=faf5f5e967f3658e in_seq=0 out_seq=0 2024-03-11T18:37:04.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.236+0000 7f52a7fff640 10 client.4662 ms_handle_connect on v2:172.21.15.90:6834/460417177 2024-03-11T18:37:04.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.384+0000 7f52a7fff640 1 -- 172.21.15.90:0/4222405560 <== mds.4 v2:172.21.15.90:6834/460417177 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562b82167f20 con 0x562b823539d0 2024-03-11T18:37:04.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.384+0000 7f52a7fff640 10 client.4662 handle_command_reply: tid=0 2024-03-11T18:37:04.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.384+0000 7f52d2551000 1 client.4662 shutdown 2024-03-11T18:37:04.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.384+0000 7f52d2551000 1 -- 172.21.15.90:0/4222405560 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b82359bc0 msgr2=0x562b82362f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:04.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.384+0000 7f52d2551000 1 --2- 172.21.15.90:0/4222405560 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b82359bc0 0x562b82362f60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f52c4013000 tx=0x7f52c4003770 comp rx=0 tx=0).stop 2024-03-11T18:37:04.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.384+0000 7f52d2551000 1 -- 172.21.15.90:0/4222405560 shutdown_connections 2024-03-11T18:37:04.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.384+0000 7f52d2551000 1 --2- 172.21.15.90:0/4222405560 >> [v2:172.21.15.90:6834/460417177,v1:172.21.15.90:6835/460417177] conn(0x562b823539d0 0x562b82353db0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:04.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.384+0000 7f52d2551000 1 --2- 172.21.15.90:0/4222405560 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b82359bc0 0x562b82362f60 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:04.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.384+0000 7f52a7fff640 0 client.4662 ms_handle_reset on v2:172.21.15.90:6834/460417177 2024-03-11T18:37:04.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.384+0000 7f52d2551000 1 --2- 172.21.15.90:0/4222405560 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b82357f70 0x562b82362a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:04.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.384+0000 7f52d2551000 1 --2- 172.21.15.90:0/4222405560 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x562b8231e890 0x562b82364f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:04.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.384+0000 7f52d2551000 1 -- 172.21.15.90:0/4222405560 >> 172.21.15.90:0/4222405560 conn(0x562b8231eff0 msgr2=0x562b82357660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:37:04.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.384+0000 7f52d2551000 1 -- 172.21.15.90:0/4222405560 shutdown_connections 2024-03-11T18:37:04.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.384+0000 7f52d2551000 1 -- 172.21.15.90:0/4222405560 wait complete. 2024-03-11T18:37:04.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.384+0000 7f52d2551000 20 client.4662 trim_cache size 0 max 16384 2024-03-11T18:37:04.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.428+0000 7f52d2551000 1 Processor -- start 2024-03-11T18:37:04.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.432+0000 7f52d2551000 1 -- start start 2024-03-11T18:37:04.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.432+0000 7f52d2551000 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x562b8231e890 0x562b82350da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:04.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.432+0000 7f52d2551000 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b823539d0 0x562b82358670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:04.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.432+0000 7f52d2551000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b82357f70 0x562b82358bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:04.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.432+0000 7f52d2551000 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b81e22940 con 0x562b823539d0 2024-03-11T18:37:04.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.432+0000 7f52d2551000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b82173c80 con 0x562b82357f70 2024-03-11T18:37:04.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.432+0000 7f52d2551000 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x562b81e621b0 con 0x562b8231e890 2024-03-11T18:37:04.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.432+0000 7f52cc983640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b823539d0 0x562b82358670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:04.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.432+0000 7f52cd184640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x562b8231e890 0x562b82350da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:04.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.432+0000 7f52cd985640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b82357f70 0x562b82358bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:04.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.432+0000 7f52cc983640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b823539d0 0x562b82358670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44390/0 (socket says 172.21.15.90:44390) 2024-03-11T18:37:04.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.432+0000 7f52cc983640 1 -- 172.21.15.90:0/2722622448 learned_addr learned my addr 172.21.15.90:0/2722622448 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:37:04.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.436+0000 7f52cc983640 1 -- 172.21.15.90:0/2722622448 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x562b8231e890 msgr2=0x562b82350da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:04.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.436+0000 7f52cc983640 1 --2- 172.21.15.90:0/2722622448 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x562b8231e890 0x562b82350da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:04.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.436+0000 7f52cc983640 1 -- 172.21.15.90:0/2722622448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b82357f70 msgr2=0x562b82358bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:04.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.436+0000 7f52cc983640 1 --2- 172.21.15.90:0/2722622448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b82357f70 0x562b82358bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:04.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.436+0000 7f52cc983640 1 -- 172.21.15.90:0/2722622448 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b82110e10 con 0x562b823539d0 2024-03-11T18:37:04.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.440+0000 7f52cc983640 1 --2- 172.21.15.90:0/2722622448 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b823539d0 0x562b82358670 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f52c4034f60 tx=0x7f52c4089db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f3ad5e222762341 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:37:04.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.448+0000 7f52b27fc640 1 -- 172.21.15.90:0/2722622448 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c4087070 con 0x562b823539d0 2024-03-11T18:37:04.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.448+0000 7f52b27fc640 1 -- 172.21.15.90:0/2722622448 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c402f400 con 0x562b823539d0 2024-03-11T18:37:04.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.448+0000 7f52b27fc640 1 -- 172.21.15.90:0/2722622448 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c400ce40 con 0x562b823539d0 2024-03-11T18:37:04.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.452+0000 7f52d2551000 1 -- 172.21.15.90:0/2722622448 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b823539d0 msgr2=0x562b82358670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:04.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.452+0000 7f52d2551000 1 --2- 172.21.15.90:0/2722622448 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b823539d0 0x562b82358670 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f52c4034f60 tx=0x7f52c4089db0 comp rx=0 tx=0).stop 2024-03-11T18:37:04.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.452+0000 7f52d2551000 1 -- 172.21.15.90:0/2722622448 shutdown_connections 2024-03-11T18:37:04.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.452+0000 7f52d2551000 1 --2- 172.21.15.90:0/2722622448 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b82357f70 0x562b82358bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:04.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.452+0000 7f52d2551000 1 --2- 172.21.15.90:0/2722622448 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b823539d0 0x562b82358670 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:04.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.452+0000 7f52d2551000 1 --2- 172.21.15.90:0/2722622448 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x562b8231e890 0x562b82350da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:04.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.452+0000 7f52d2551000 1 -- 172.21.15.90:0/2722622448 >> 172.21.15.90:0/2722622448 conn(0x562b8231eff0 msgr2=0x562b823745d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:37:04.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.452+0000 7f52d2551000 1 -- 172.21.15.90:0/2722622448 shutdown_connections 2024-03-11T18:37:04.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.452+0000 7f52d2551000 1 -- 172.21.15.90:0/2722622448 wait complete. 2024-03-11T18:37:04.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.452+0000 7f52d2551000 1 -- start start 2024-03-11T18:37:04.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.452+0000 7f52d2551000 1 Processor -- start 2024-03-11T18:37:04.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.456+0000 7f52d2551000 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x562b8231e890 0x562b82362b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:04.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.456+0000 7f52d2551000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b823539d0 0x562b82370370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:04.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.456+0000 7f52d2551000 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b82357f70 0x562b823708b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:04.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.456+0000 7f52d2551000 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b821ab1a0 con 0x562b82357f70 2024-03-11T18:37:04.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.456+0000 7f52d2551000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b8216b140 con 0x562b823539d0 2024-03-11T18:37:04.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.456+0000 7f52d2551000 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x562b8212a8c0 con 0x562b8231e890 2024-03-11T18:37:04.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.456+0000 7f52d2551000 10 client.0 _ll_register_callbacks cb 0x562b8216d2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T18:37:04.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.456+0000 7f52cd985640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b82357f70 0x562b823708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:04.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.456+0000 7f52cd985640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b82357f70 0x562b823708b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44402/0 (socket says 172.21.15.90:44402) 2024-03-11T18:37:04.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.456+0000 7f52cd985640 1 -- 172.21.15.90:0/3179171036 learned_addr learned my addr 172.21.15.90:0/3179171036 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:37:04.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.456+0000 7f52cd184640 1 --2- 172.21.15.90:0/3179171036 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x562b8231e890 0x562b82362b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:04.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.456+0000 7f52cc983640 1 --2- 172.21.15.90:0/3179171036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b823539d0 0x562b82370370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:04.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.460+0000 7f52cd985640 1 -- 172.21.15.90:0/3179171036 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x562b8231e890 msgr2=0x562b82362b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:04.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.460+0000 7f52cd985640 1 --2- 172.21.15.90:0/3179171036 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x562b8231e890 0x562b82362b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:04.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.460+0000 7f52cd985640 1 -- 172.21.15.90:0/3179171036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b823539d0 msgr2=0x562b82370370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:04.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.460+0000 7f52cd985640 1 --2- 172.21.15.90:0/3179171036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b823539d0 0x562b82370370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:04.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.460+0000 7f52cd985640 1 -- 172.21.15.90:0/3179171036 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562b8212ae70 con 0x562b82357f70 2024-03-11T18:37:04.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.464+0000 7f52cd985640 1 --2- 172.21.15.90:0/3179171036 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b82357f70 0x562b823708b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x562b821b80c0 tx=0x7f52c000a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ea59d081b3d2c3c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:37:04.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.464+0000 7f52a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-03-11T18:37:04.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.468+0000 7f52a7fff640 1 -- 172.21.15.90:0/3179171036 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c0010040 con 0x562b82357f70 2024-03-11T18:37:04.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.468+0000 7f52a7fff640 1 -- 172.21.15.90:0/3179171036 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f52c0021670 con 0x562b82357f70 2024-03-11T18:37:04.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.468+0000 7f52d2551000 1 -- 172.21.15.90:0/3179171036 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562b820f0e80 con 0x562b82357f70 2024-03-11T18:37:04.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.472+0000 7f52a7fff640 1 -- 172.21.15.90:0/3179171036 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52c0021830 con 0x562b82357f70 2024-03-11T18:37:04.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.472+0000 7f52d2551000 10 client.4668 fetch_fsmap learned FSMap version 19 2024-03-11T18:37:04.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.472+0000 7f52d2551000 1 -- 172.21.15.90:0/3179171036 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) v3 -- 0x562b81e61cc0 con 0x562b82357f70 2024-03-11T18:37:04.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.476+0000 7f52a7fff640 1 -- 172.21.15.90:0/3179171036 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562b820f0e80 con 0x562b82357f70 2024-03-11T18:37:04.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.476+0000 7f52a7fff640 1 -- 172.21.15.90:0/3179171036 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 19) v1 ==== 4500+0+0 (secure 0 0 0) 0x7f52c0008070 con 0x562b82357f70 2024-03-11T18:37:04.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.476+0000 7f52d2551000 10 client.4668 fetch_fsmap finished waiting for FSMap version 19 2024-03-11T18:37:04.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.476+0000 7f52d2551000 10 client.4668 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-11T18:37:04.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.476+0000 7f52d2551000 20 client.4668 populate_metadata read hostname 'smithi090' 2024-03-11T18:37:04.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.476+0000 7f52d2551000 1 --2- 172.21.15.90:0/3179171036 >> [v2:172.21.15.90:6834/460417177,v1:172.21.15.90:6835/460417177] conn(0x562b82359bc0 0x562b823c6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:04.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.476+0000 7f52d2551000 4 client.4668 mds_command: new command op to 4229 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-11T18:37:04.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.476+0000 7f52d2551000 1 -- 172.21.15.90:0/3179171036 --> [v2:172.21.15.90:6834/460417177,v1:172.21.15.90:6835/460417177] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562b82133fa0 con 0x562b82359bc0 2024-03-11T18:37:04.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.476+0000 7f52cd184640 1 --2- 172.21.15.90:0/3179171036 >> [v2:172.21.15.90:6834/460417177,v1:172.21.15.90:6835/460417177] conn(0x562b82359bc0 0x562b823c6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:04.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.480+0000 7f52a7fff640 10 client.4668 ms_handle_connect on v2:172.21.15.90:6834/460417177 2024-03-11T18:37:04.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.480+0000 7f52cd184640 1 --2- 172.21.15.90:0/3179171036 >> [v2:172.21.15.90:6834/460417177,v1:172.21.15.90:6835/460417177] conn(0x562b82359bc0 0x562b823c6cf0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=294d3fc577f72d41 server_cookie=a733a290411f6652 in_seq=0 out_seq=0 2024-03-11T18:37:04.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.500+0000 7f52a7fff640 1 -- 172.21.15.90:0/3179171036 <== mds.4 v2:172.21.15.90:6834/460417177 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x562b82133fa0 con 0x562b82359bc0 2024-03-11T18:37:04.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.500+0000 7f52a7fff640 10 client.4668 handle_command_reply: tid=0 2024-03-11T18:37:04.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.500+0000 7f52d2551000 1 client.4668 shutdown 2024-03-11T18:37:04.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.500+0000 7f52d2551000 1 -- 172.21.15.90:0/3179171036 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b82357f70 msgr2=0x562b823708b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:04.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.500+0000 7f52d2551000 1 --2- 172.21.15.90:0/3179171036 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b82357f70 0x562b823708b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x562b821b80c0 tx=0x7f52c000a7d0 comp rx=0 tx=0).stop 2024-03-11T18:37:04.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.500+0000 7f52d2551000 1 -- 172.21.15.90:0/3179171036 shutdown_connections 2024-03-11T18:37:04.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.500+0000 7f52d2551000 1 --2- 172.21.15.90:0/3179171036 >> [v2:172.21.15.90:6834/460417177,v1:172.21.15.90:6835/460417177] conn(0x562b82359bc0 0x562b823c6cf0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:04.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.500+0000 7f52d2551000 1 --2- 172.21.15.90:0/3179171036 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b82357f70 0x562b823708b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:04.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.500+0000 7f52d2551000 1 --2- 172.21.15.90:0/3179171036 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562b823539d0 0x562b82370370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:04.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.500+0000 7f52d2551000 1 --2- 172.21.15.90:0/3179171036 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x562b8231e890 0x562b82362b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:04.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.500+0000 7f52d2551000 1 -- 172.21.15.90:0/3179171036 >> 172.21.15.90:0/3179171036 conn(0x562b8231eff0 msgr2=0x562b8235a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:37:04.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.500+0000 7f52a7fff640 0 client.4668 ms_handle_reset on v2:172.21.15.90:6834/460417177 2024-03-11T18:37:04.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.500+0000 7f52d2551000 1 -- 172.21.15.90:0/3179171036 shutdown_connections 2024-03-11T18:37:04.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.500+0000 7f52d2551000 1 -- 172.21.15.90:0/3179171036 wait complete. 2024-03-11T18:37:04.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.500+0000 7f52d2551000 20 client.4668 trim_cache size 0 max 16384 2024-03-11T18:37:04.511 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-03-11T18:37:04.511 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "76d56b10-6c29-476a-804b-8482ea90f9e2", 2024-03-11T18:37:04.511 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 4, 2024-03-11T18:37:04.511 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 4229, 2024-03-11T18:37:04.511 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-03-11T18:37:04.511 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-03-11T18:37:04.511 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-03-11T18:37:04.511 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 14.291533662999999, 2024-03-11T18:37:04.511 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 19, 2024-03-11T18:37:04.511 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 30, 2024-03-11T18:37:04.512 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 28, 2024-03-11T18:37:04.512 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 84.317867331000002 2024-03-11T18:37:04.512 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-03-11T18:37:04.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.500+0000 7f52cf3e8640 1 -- 172.21.15.90:0/4091697083 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f52ac066cc0 msgr2=0x7f52ac069180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:04.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.500+0000 7f52cf3e8640 1 --2- 172.21.15.90:0/4091697083 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f52ac066cc0 0x7f52ac069180 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f52b8000c10 tx=0x7f52b80b2040 comp rx=0 tx=0).stop 2024-03-11T18:37:04.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.500+0000 7f52cf3e8640 1 -- 172.21.15.90:0/4091697083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52c80c95f0 msgr2=0x7f52c81456b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:04.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.500+0000 7f52cf3e8640 1 --2- 172.21.15.90:0/4091697083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52c80c95f0 0x7f52c81456b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f52c400c450 tx=0x7f52c400c910 comp rx=0 tx=0).stop 2024-03-11T18:37:04.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.500+0000 7f52cf3e8640 1 -- 172.21.15.90:0/4091697083 shutdown_connections 2024-03-11T18:37:04.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.500+0000 7f52cf3e8640 1 --2- 172.21.15.90:0/4091697083 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f52ac066cc0 0x7f52ac069180 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:04.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.500+0000 7f52cf3e8640 1 --2- 172.21.15.90:0/4091697083 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f52c8151d50 0x7f52c814a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:04.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.500+0000 7f52cf3e8640 1 --2- 172.21.15.90:0/4091697083 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f52c80c95f0 0x7f52c81456b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:04.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.500+0000 7f52cf3e8640 1 --2- 172.21.15.90:0/4091697083 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52c806b860 0x7f52c81e2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:04.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.500+0000 7f52cf3e8640 1 -- 172.21.15.90:0/4091697083 >> 172.21.15.90:0/4091697083 conn(0x7f52c8071aa0 msgr2=0x7f52c8147d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:37:04.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.504+0000 7f52cf3e8640 1 -- 172.21.15.90:0/4091697083 shutdown_connections 2024-03-11T18:37:04.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.504+0000 7f52cf3e8640 1 -- 172.21.15.90:0/4091697083 wait complete. 2024-03-11T18:37:04.529 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-03-11T18:37:04.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.696+0000 7fb49bc79640 1 Processor -- start 2024-03-11T18:37:04.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.696+0000 7fb49bc79640 1 -- start start 2024-03-11T18:37:04.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.696+0000 7fb49bc79640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fb494151b30 0x7fb49406e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:04.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.696+0000 7fb49bc79640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fb494070b00 0x7fb4940c5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:04.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.696+0000 7fb49bc79640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb49406f430 con 0x7fb4940c4af0 2024-03-11T18:37:04.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.696+0000 7fb49bc79640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4940c65b0 con 0x7fb494151b30 2024-03-11T18:37:04.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.696+0000 7fb49bc79640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4940c6320 con 0x7fb494070b00 2024-03-11T18:37:04.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.696+0000 7fb499a15640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fb494151b30 0x7fb49406e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:04.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.696+0000 7fb499a15640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fb494151b30 0x7fb49406e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44406/0 (socket says 172.21.15.90:44406) 2024-03-11T18:37:04.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.696+0000 7fb499a15640 1 -- 172.21.15.90:0/1773470619 learned_addr learned my addr 172.21.15.90:0/1773470619 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:37:04.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.696+0000 7fb499214640 1 --2- 172.21.15.90:0/1773470619 >> v2:172.21.15.92:3300/0 conn(0x7fb494070b00 0x7fb4940c5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:04.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.700+0000 7fb499a15640 1 -- 172.21.15.90:0/1773470619 >> v2:172.21.15.92:3300/0 conn(0x7fb494070b00 msgr2=0x7fb4940c5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:04.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.700+0000 7fb499a15640 1 --2- 172.21.15.90:0/1773470619 >> v2:172.21.15.92:3300/0 conn(0x7fb494070b00 0x7fb4940c5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:04.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.700+0000 7fb499a15640 1 -- 172.21.15.90:0/1773470619 >> v1:172.21.15.90:6789/0 conn(0x7fb4940c4af0 legacy=0x7fb494157c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:04.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.700+0000 7fb499a15640 1 -- 172.21.15.90:0/1773470619 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb49406e820 con 0x7fb494151b30 2024-03-11T18:37:04.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.700+0000 7fb499214640 1 --2- 172.21.15.90:0/1773470619 >> v2:172.21.15.92:3300/0 conn(0x7fb494070b00 0x7fb4940c5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T18:37:04.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.704+0000 7fb499a15640 1 --2- 172.21.15.90:0/1773470619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb494151b30 0x7fb49406e210 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb49406ca40 tx=0x7fb49001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=225119496b65647 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:37:04.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.708+0000 7fb498a13640 1 -- 172.21.15.90:0/1773470619 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4900aa650 con 0x7fb494151b30 2024-03-11T18:37:04.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.708+0000 7fb498a13640 1 -- 172.21.15.90:0/1773470619 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4900aa7b0 con 0x7fb494151b30 2024-03-11T18:37:04.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.708+0000 7fb498a13640 1 -- 172.21.15.90:0/1773470619 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4900aab40 con 0x7fb494151b30 2024-03-11T18:37:04.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.708+0000 7fb49bc79640 1 -- 172.21.15.90:0/1773470619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb494151b30 msgr2=0x7fb49406e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:04.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.708+0000 7fb49bc79640 1 --2- 172.21.15.90:0/1773470619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb494151b30 0x7fb49406e210 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb49406ca40 tx=0x7fb49001c4f0 comp rx=0 tx=0).stop 2024-03-11T18:37:04.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.708+0000 7fb49bc79640 1 -- 172.21.15.90:0/1773470619 shutdown_connections 2024-03-11T18:37:04.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.708+0000 7fb49bc79640 1 --2- 172.21.15.90:0/1773470619 >> v2:172.21.15.92:3300/0 conn(0x7fb494070b00 0x7fb4940c5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:04.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.708+0000 7fb49bc79640 1 --2- 172.21.15.90:0/1773470619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb494151b30 0x7fb49406e210 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:04.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.708+0000 7fb49bc79640 1 -- 172.21.15.90:0/1773470619 >> 172.21.15.90:0/1773470619 conn(0x7fb49406ac00 msgr2=0x7fb49406b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:37:04.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.708+0000 7fb49bc79640 1 -- 172.21.15.90:0/1773470619 shutdown_connections 2024-03-11T18:37:04.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.708+0000 7fb49bc79640 1 -- 172.21.15.90:0/1773470619 wait complete. 2024-03-11T18:37:04.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.712+0000 7fb49bc79640 1 Processor -- start 2024-03-11T18:37:04.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.712+0000 7fb49bc79640 1 -- start start 2024-03-11T18:37:04.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.712+0000 7fb49bc79640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb494070b00 0x7fb4940c9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:04.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.712+0000 7fb49bc79640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4940c4af0 0x7fb4940c3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:04.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.712+0000 7fb49bc79640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb494151b30 0x7fb4940c44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:04.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.712+0000 7fb49bc79640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4940c21a0 con 0x7fb494070b00 2024-03-11T18:37:04.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.712+0000 7fb49bc79640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4940c3390 con 0x7fb4940c4af0 2024-03-11T18:37:04.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.712+0000 7fb49bc79640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4940cabc0 con 0x7fb494151b30 2024-03-11T18:37:04.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.712+0000 7fb499a15640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb494070b00 0x7fb4940c9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:04.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.712+0000 7fb499a15640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb494070b00 0x7fb4940c9ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44416/0 (socket says 172.21.15.90:44416) 2024-03-11T18:37:04.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.712+0000 7fb499a15640 1 -- 172.21.15.90:0/2104313910 learned_addr learned my addr 172.21.15.90:0/2104313910 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:37:04.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.712+0000 7fb499214640 1 --2- 172.21.15.90:0/2104313910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4940c4af0 0x7fb4940c3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:04.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.712+0000 7fb49a216640 1 --2- 172.21.15.90:0/2104313910 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb494151b30 0x7fb4940c44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:04.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.716+0000 7fb499a15640 1 -- 172.21.15.90:0/2104313910 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb494151b30 msgr2=0x7fb4940c44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:04.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.716+0000 7fb499a15640 1 --2- 172.21.15.90:0/2104313910 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb494151b30 0x7fb4940c44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:04.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.716+0000 7fb499a15640 1 -- 172.21.15.90:0/2104313910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4940c4af0 msgr2=0x7fb4940c3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:04.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.716+0000 7fb499a15640 1 --2- 172.21.15.90:0/2104313910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4940c4af0 0x7fb4940c3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:04.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.716+0000 7fb499a15640 1 -- 172.21.15.90:0/2104313910 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4900094d0 con 0x7fb494070b00 2024-03-11T18:37:04.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.720+0000 7fb499a15640 1 --2- 172.21.15.90:0/2104313910 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb494070b00 0x7fb4940c9ff0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb490019100 tx=0x7fb49001ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:37:04.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.724+0000 7fb48a7fc640 1 -- 172.21.15.90:0/2104313910 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4900b3800 con 0x7fb494070b00 2024-03-11T18:37:04.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.724+0000 7fb48a7fc640 1 -- 172.21.15.90:0/2104313910 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4900b3980 con 0x7fb494070b00 2024-03-11T18:37:04.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.724+0000 7fb49bc79640 1 -- 172.21.15.90:0/2104313910 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4941f78d0 con 0x7fb494070b00 2024-03-11T18:37:04.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.724+0000 7fb48a7fc640 1 -- 172.21.15.90:0/2104313910 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4900b3d30 con 0x7fb494070b00 2024-03-11T18:37:04.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.724+0000 7fb49bc79640 1 -- 172.21.15.90:0/2104313910 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4940ca8e0 con 0x7fb494070b00 2024-03-11T18:37:04.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.724+0000 7fb49ede2000 1 Processor -- start 2024-03-11T18:37:04.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.724+0000 7fb49ede2000 1 -- start start 2024-03-11T18:37:04.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.728+0000 7fb49ede2000 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55829ad27f70 0x55829acee350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:04.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.728+0000 7fb49ede2000 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55829acee890 0x55829ad29680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:04.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.728+0000 7fb49ede2000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55829ad29bc0 0x55829ad29fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:04.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.728+0000 7fb49ede2000 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55829abb0d00 con 0x55829ad27f70 2024-03-11T18:37:04.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.728+0000 7fb49ede2000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55829ab42430 con 0x55829ad29bc0 2024-03-11T18:37:04.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.728+0000 7fb49ede2000 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x55829ab3f960 con 0x55829acee890 2024-03-11T18:37:04.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.732+0000 7fb499a15640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55829acee890 0x55829ad29680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:04.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.732+0000 7fb499214640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55829ad29bc0 0x55829ad29fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:04.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.732+0000 7fb499214640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55829ad29bc0 0x55829ad29fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.90:49624/0 (socket says 172.21.15.90:49624) 2024-03-11T18:37:04.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.732+0000 7fb499214640 1 -- 172.21.15.90:0/473613431 learned_addr learned my addr 172.21.15.90:0/473613431 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:37:04.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.740+0000 7fb499214640 1 -- 172.21.15.90:0/473613431 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55829acee890 msgr2=0x55829ad29680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:04.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.740+0000 7fb499214640 1 --2- 172.21.15.90:0/473613431 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55829acee890 0x55829ad29680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:04.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.740+0000 7fb499214640 1 -- 172.21.15.90:0/473613431 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55829ad27f70 msgr2=0x55829acee350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:04.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.740+0000 7fb499214640 1 --2- 172.21.15.90:0/473613431 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55829ad27f70 0x55829acee350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:04.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.740+0000 7fb499214640 1 -- 172.21.15.90:0/473613431 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55829a831890 con 0x55829ad29bc0 2024-03-11T18:37:04.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.744+0000 7fb48a7fc640 1 -- 172.21.15.90:0/2104313910 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fb4900bd3f0 con 0x7fb494070b00 2024-03-11T18:37:04.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.744+0000 7fb48a7fc640 1 --2- 172.21.15.90:0/2104313910 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fb478066ec0 0x7fb478069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:04.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.744+0000 7fb49a216640 1 --2- 172.21.15.90:0/2104313910 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fb478066ec0 0x7fb478069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:04.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.744+0000 7fb49a216640 1 --2- 172.21.15.90:0/2104313910 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fb478066ec0 0x7fb478069380 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb4940c9d50 tx=0x7fb48c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:37:04.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.744+0000 7fb499214640 1 --2- 172.21.15.90:0/473613431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55829ad29bc0 0x55829ad29fa0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb4840044b0 tx=0x7fb4840039f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fada950c5d82908 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:37:04.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.744+0000 7fb48a7fc640 1 -- 172.21.15.90:0/2104313910 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fb4900f02e0 con 0x7fb494070b00 2024-03-11T18:37:04.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.756+0000 7fb4777fe640 1 -- 172.21.15.90:0/473613431 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48400b910 con 0x55829ad29bc0 2024-03-11T18:37:04.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.756+0000 7fb4777fe640 1 -- 172.21.15.90:0/473613431 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb48400ba70 con 0x55829ad29bc0 2024-03-11T18:37:04.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.756+0000 7fb4777fe640 1 -- 172.21.15.90:0/473613431 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48400bde0 con 0x55829ad29bc0 2024-03-11T18:37:04.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.756+0000 7fb49ede2000 1 -- 172.21.15.90:0/473613431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55829ad29bc0 msgr2=0x55829ad29fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:04.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.756+0000 7fb49ede2000 1 --2- 172.21.15.90:0/473613431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55829ad29bc0 0x55829ad29fa0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb4840044b0 tx=0x7fb4840039f0 comp rx=0 tx=0).stop 2024-03-11T18:37:04.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.756+0000 7fb49ede2000 1 -- 172.21.15.90:0/473613431 shutdown_connections 2024-03-11T18:37:04.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.756+0000 7fb49ede2000 1 --2- 172.21.15.90:0/473613431 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55829ad29bc0 0x55829ad29fa0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:04.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.756+0000 7fb49ede2000 1 --2- 172.21.15.90:0/473613431 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55829acee890 0x55829ad29680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:04.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.756+0000 7fb49ede2000 1 --2- 172.21.15.90:0/473613431 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55829ad27f70 0x55829acee350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:04.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.756+0000 7fb49ede2000 1 -- 172.21.15.90:0/473613431 >> 172.21.15.90:0/473613431 conn(0x55829aceeff0 msgr2=0x55829ad27440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:37:04.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.756+0000 7fb49ede2000 1 -- 172.21.15.90:0/473613431 shutdown_connections 2024-03-11T18:37:04.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.756+0000 7fb49ede2000 1 -- 172.21.15.90:0/473613431 wait complete. 2024-03-11T18:37:04.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.760+0000 7fb49ede2000 1 -- start start 2024-03-11T18:37:04.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.760+0000 7fb49ede2000 1 Processor -- start 2024-03-11T18:37:04.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.760+0000 7fb49ede2000 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55829acee890 0x55829ad34b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:04.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.760+0000 7fb49ede2000 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55829ad27f70 0x55829ad35060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:04.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.760+0000 7fb49ede2000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55829ad29bc0 0x55829aced720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:04.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.760+0000 7fb49ede2000 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55829ab3b140 con 0x55829acee890 2024-03-11T18:37:04.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.760+0000 7fb49ede2000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55829aad5630 con 0x55829ad29bc0 2024-03-11T18:37:04.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.760+0000 7fb49ede2000 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x55829aad52d0 con 0x55829ad27f70 2024-03-11T18:37:04.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.760+0000 7fb49ede2000 10 client.0 _ll_register_callbacks cb 0x55829abeb1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T18:37:04.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.760+0000 7fb499a15640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55829acee890 0x55829ad34b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:04.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.760+0000 7fb499a15640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55829acee890 0x55829ad34b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44438/0 (socket says 172.21.15.90:44438) 2024-03-11T18:37:04.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.760+0000 7fb499a15640 1 -- 172.21.15.90:0/2896146981 learned_addr learned my addr 172.21.15.90:0/2896146981 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:37:04.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.760+0000 7fb499214640 1 --2- 172.21.15.90:0/2896146981 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55829ad27f70 0x55829ad35060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:04.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.764+0000 7fb49a216640 1 --2- 172.21.15.90:0/2896146981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55829ad29bc0 0x55829aced720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:04.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.764+0000 7fb499a15640 1 -- 172.21.15.90:0/2896146981 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55829ad27f70 msgr2=0x55829ad35060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:04.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.764+0000 7fb499a15640 1 --2- 172.21.15.90:0/2896146981 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55829ad27f70 0x55829ad35060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:04.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.764+0000 7fb499a15640 1 -- 172.21.15.90:0/2896146981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55829ad29bc0 msgr2=0x55829aced720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:04.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.764+0000 7fb499a15640 1 --2- 172.21.15.90:0/2896146981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55829ad29bc0 0x55829aced720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:04.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.764+0000 7fb499a15640 1 -- 172.21.15.90:0/2896146981 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb48400e040 con 0x55829acee890 2024-03-11T18:37:04.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.764+0000 7fb499a15640 1 --2- 172.21.15.90:0/2896146981 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55829acee890 0x55829ad34b20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb4900b6040 tx=0x7fb4900f8a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99b2f80802aa3383 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:37:04.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.764+0000 7fb474ff9640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-03-11T18:37:04.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.772+0000 7fb474ff9640 1 -- 172.21.15.90:0/2896146981 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4900f4020 con 0x55829acee890 2024-03-11T18:37:04.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.772+0000 7fb474ff9640 1 -- 172.21.15.90:0/2896146981 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fb4900f6310 con 0x55829acee890 2024-03-11T18:37:04.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.772+0000 7fb49ede2000 1 -- 172.21.15.90:0/2896146981 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55829aaee850 con 0x55829acee890 2024-03-11T18:37:04.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.772+0000 7fb474ff9640 1 -- 172.21.15.90:0/2896146981 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4900f4180 con 0x55829acee890 2024-03-11T18:37:04.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.772+0000 7fb49ede2000 10 client.4680 fetch_fsmap learned FSMap version 19 2024-03-11T18:37:04.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.772+0000 7fb49ede2000 1 -- 172.21.15.90:0/2896146981 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) v3 -- 0x55829aafae70 con 0x55829acee890 2024-03-11T18:37:04.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.776+0000 7fb474ff9640 1 -- 172.21.15.90:0/2896146981 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55829aaee850 con 0x55829acee890 2024-03-11T18:37:04.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.776+0000 7fb474ff9640 1 -- 172.21.15.90:0/2896146981 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 19) v1 ==== 4500+0+0 (secure 0 0 0) 0x7fb4900f64f0 con 0x55829acee890 2024-03-11T18:37:04.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.776+0000 7fb49ede2000 10 client.4680 fetch_fsmap finished waiting for FSMap version 19 2024-03-11T18:37:04.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.776+0000 7fb49ede2000 10 client.4680 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-11T18:37:04.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.776+0000 7fb49ede2000 20 client.4680 populate_metadata read hostname 'smithi090' 2024-03-11T18:37:04.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.776+0000 7fb49ede2000 1 --2- 172.21.15.90:0/2896146981 >> [v2:172.21.15.92:6833/3568438077,v1:172.21.15.92:6835/3568438077] conn(0x55829ad32a20 0x55829ad32e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:04.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.776+0000 7fb49ede2000 4 client.4680 mds_command: new command op to 4238 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T18:37:04.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.776+0000 7fb49ede2000 1 -- 172.21.15.90:0/2896146981 --> [v2:172.21.15.92:6833/3568438077,v1:172.21.15.92:6835/3568438077] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55829ab37f20 con 0x55829ad32a20 2024-03-11T18:37:04.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.804+0000 7fb499214640 1 --2- 172.21.15.90:0/2896146981 >> [v2:172.21.15.92:6833/3568438077,v1:172.21.15.92:6835/3568438077] conn(0x55829ad32a20 0x55829ad32e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:04.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.944+0000 7fb499214640 1 --2- 172.21.15.90:0/2896146981 >> [v2:172.21.15.92:6833/3568438077,v1:172.21.15.92:6835/3568438077] conn(0x55829ad32a20 0x55829ad32e00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2123c9c76499cdd0 server_cookie=c391f269373f2672 in_seq=0 out_seq=0 2024-03-11T18:37:04.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:04.944+0000 7fb474ff9640 10 client.4680 ms_handle_connect on v2:172.21.15.92:6833/3568438077 2024-03-11T18:37:05.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.136+0000 7fb474ff9640 1 -- 172.21.15.90:0/2896146981 <== mds.3 v2:172.21.15.92:6833/3568438077 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55829ab37f20 con 0x55829ad32a20 2024-03-11T18:37:05.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.136+0000 7fb474ff9640 10 client.4680 handle_command_reply: tid=0 2024-03-11T18:37:05.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.136+0000 7fb49ede2000 1 client.4680 shutdown 2024-03-11T18:37:05.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.136+0000 7fb49ede2000 1 -- 172.21.15.90:0/2896146981 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55829acee890 msgr2=0x55829ad34b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:05.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.136+0000 7fb49ede2000 1 --2- 172.21.15.90:0/2896146981 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55829acee890 0x55829ad34b20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb4900b6040 tx=0x7fb4900f8a80 comp rx=0 tx=0).stop 2024-03-11T18:37:05.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.136+0000 7fb49ede2000 1 -- 172.21.15.90:0/2896146981 shutdown_connections 2024-03-11T18:37:05.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.136+0000 7fb49ede2000 1 --2- 172.21.15.90:0/2896146981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55829ad29bc0 0x55829aced720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:05.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.136+0000 7fb49ede2000 1 --2- 172.21.15.90:0/2896146981 >> [v2:172.21.15.92:6833/3568438077,v1:172.21.15.92:6835/3568438077] conn(0x55829ad32a20 0x55829ad32e00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:05.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.136+0000 7fb49ede2000 1 --2- 172.21.15.90:0/2896146981 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55829ad27f70 0x55829ad35060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:05.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.136+0000 7fb474ff9640 0 client.4680 ms_handle_reset on v2:172.21.15.92:6833/3568438077 2024-03-11T18:37:05.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.136+0000 7fb49ede2000 1 --2- 172.21.15.90:0/2896146981 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55829acee890 0x55829ad34b20 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:05.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.136+0000 7fb49ede2000 1 -- 172.21.15.90:0/2896146981 >> 172.21.15.90:0/2896146981 conn(0x55829aceeff0 msgr2=0x55829ad27020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:37:05.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.136+0000 7fb49ede2000 1 -- 172.21.15.90:0/2896146981 shutdown_connections 2024-03-11T18:37:05.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.136+0000 7fb49ede2000 1 -- 172.21.15.90:0/2896146981 wait complete. 2024-03-11T18:37:05.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.136+0000 7fb49ede2000 20 client.4680 trim_cache size 0 max 16384 2024-03-11T18:37:05.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.188+0000 7fb49ede2000 1 Processor -- start 2024-03-11T18:37:05.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.188+0000 7fb49ede2000 1 -- start start 2024-03-11T18:37:05.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.192+0000 7fb49ede2000 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55829aceeff0 0x55829ad3b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:05.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.192+0000 7fb49ede2000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55829acee890 0x55829ad28a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:05.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.192+0000 7fb49ede2000 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55829ad27f70 0x55829ad37b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:05.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.192+0000 7fb49ede2000 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55829a7f2940 con 0x55829aceeff0 2024-03-11T18:37:05.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.192+0000 7fb49ede2000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55829aafa8c0 con 0x55829acee890 2024-03-11T18:37:05.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.192+0000 7fb49ede2000 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x55829ad27580 con 0x55829ad27f70 2024-03-11T18:37:05.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.216+0000 7fb499214640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55829acee890 0x55829ad28a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:05.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.216+0000 7fb499214640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55829acee890 0x55829ad28a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.90:49644/0 (socket says 172.21.15.90:49644) 2024-03-11T18:37:05.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.216+0000 7fb499214640 1 -- 172.21.15.90:0/2987191063 learned_addr learned my addr 172.21.15.90:0/2987191063 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:37:05.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.220+0000 7fb499214640 1 -- 172.21.15.90:0/2987191063 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55829ad27f70 msgr2=0x55829ad37b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:05.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.220+0000 7fb499214640 1 --2- 172.21.15.90:0/2987191063 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55829ad27f70 0x55829ad37b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:05.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.220+0000 7fb499214640 1 -- 172.21.15.90:0/2987191063 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55829aceeff0 msgr2=0x55829ad3b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:05.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.220+0000 7fb499214640 1 --2- 172.21.15.90:0/2987191063 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55829aceeff0 0x55829ad3b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:05.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.220+0000 7fb499214640 1 -- 172.21.15.90:0/2987191063 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55829aae0e10 con 0x55829acee890 2024-03-11T18:37:05.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.228+0000 7fb499214640 1 --2- 172.21.15.90:0/2987191063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55829acee890 0x55829ad28a60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb484004240 tx=0x7fb484004620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea0194987382bfc0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:37:05.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.244+0000 7fb476ffd640 1 -- 172.21.15.90:0/2987191063 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb484023070 con 0x55829acee890 2024-03-11T18:37:05.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.244+0000 7fb476ffd640 1 -- 172.21.15.90:0/2987191063 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb48400b910 con 0x55829acee890 2024-03-11T18:37:05.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.244+0000 7fb476ffd640 1 -- 172.21.15.90:0/2987191063 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb484027a90 con 0x55829acee890 2024-03-11T18:37:05.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.244+0000 7fb49ede2000 1 -- 172.21.15.90:0/2987191063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55829acee890 msgr2=0x55829ad28a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:05.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.244+0000 7fb49ede2000 1 --2- 172.21.15.90:0/2987191063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55829acee890 0x55829ad28a60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb484004240 tx=0x7fb484004620 comp rx=0 tx=0).stop 2024-03-11T18:37:05.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.244+0000 7fb49ede2000 1 -- 172.21.15.90:0/2987191063 shutdown_connections 2024-03-11T18:37:05.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.244+0000 7fb49ede2000 1 --2- 172.21.15.90:0/2987191063 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55829ad27f70 0x55829ad37b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:05.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.244+0000 7fb49ede2000 1 --2- 172.21.15.90:0/2987191063 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55829acee890 0x55829ad28a60 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:05.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.244+0000 7fb49ede2000 1 --2- 172.21.15.90:0/2987191063 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55829aceeff0 0x55829ad3b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:05.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.244+0000 7fb49ede2000 1 -- 172.21.15.90:0/2987191063 >> 172.21.15.90:0/2987191063 conn(0x55829ad2ae10 msgr2=0x55829ad28350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:37:05.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.244+0000 7fb49ede2000 1 -- 172.21.15.90:0/2987191063 shutdown_connections 2024-03-11T18:37:05.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.244+0000 7fb49ede2000 1 -- 172.21.15.90:0/2987191063 wait complete. 2024-03-11T18:37:05.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.248+0000 7fb49ede2000 1 -- start start 2024-03-11T18:37:05.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.248+0000 7fb49ede2000 1 Processor -- start 2024-03-11T18:37:05.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.248+0000 7fb49ede2000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55829acee890 0x55829ad3acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:05.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.248+0000 7fb49ede2000 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55829aceeff0 0x55829ad8f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:05.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.248+0000 7fb49ede2000 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55829ad27f70 0x55829ad8f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:05.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.248+0000 7fb49ede2000 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55829ab05c80 con 0x55829ad27f70 2024-03-11T18:37:05.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.248+0000 7fb49ede2000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55829ab38950 con 0x55829acee890 2024-03-11T18:37:05.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.248+0000 7fb49ede2000 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x55829aafda60 con 0x55829aceeff0 2024-03-11T18:37:05.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.248+0000 7fb49ede2000 10 client.0 _ll_register_callbacks cb 0x55829ab3d2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T18:37:05.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.248+0000 7fb49a216640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55829ad27f70 0x55829ad8f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:05.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.248+0000 7fb499214640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55829aceeff0 0x55829ad8f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:05.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.252+0000 7fb49a216640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55829ad27f70 0x55829ad8f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44456/0 (socket says 172.21.15.90:44456) 2024-03-11T18:37:05.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.252+0000 7fb49a216640 1 -- 172.21.15.90:0/3861368486 learned_addr learned my addr 172.21.15.90:0/3861368486 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:37:05.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.252+0000 7fb499a15640 1 --2- 172.21.15.90:0/3861368486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55829acee890 0x55829ad3acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:05.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.256+0000 7fb49a216640 1 -- 172.21.15.90:0/3861368486 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55829aceeff0 msgr2=0x55829ad8f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:05.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.256+0000 7fb49a216640 1 --2- 172.21.15.90:0/3861368486 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55829aceeff0 0x55829ad8f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:05.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.256+0000 7fb49a216640 1 -- 172.21.15.90:0/3861368486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55829acee890 msgr2=0x55829ad3acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:05.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.256+0000 7fb49a216640 1 --2- 172.21.15.90:0/3861368486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55829acee890 0x55829ad3acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:05.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.256+0000 7fb49a216640 1 -- 172.21.15.90:0/3861368486 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55829a831890 con 0x55829ad27f70 2024-03-11T18:37:05.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.260+0000 7fb49a216640 1 --2- 172.21.15.90:0/3861368486 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55829ad27f70 0x55829ad8f750 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb48c0014f0 tx=0x7fb48c014aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce353fe3db1987ea server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:37:05.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.260+0000 7fb474ff9640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-03-11T18:37:05.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.264+0000 7fb474ff9640 1 -- 172.21.15.90:0/3861368486 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48c0126a0 con 0x55829ad27f70 2024-03-11T18:37:05.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.264+0000 7fb474ff9640 1 -- 172.21.15.90:0/3861368486 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fb48c012800 con 0x55829ad27f70 2024-03-11T18:37:05.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.264+0000 7fb49ede2000 1 -- 172.21.15.90:0/3861368486 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55829ab03fa0 con 0x55829ad27f70 2024-03-11T18:37:05.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.268+0000 7fb474ff9640 1 -- 172.21.15.90:0/3861368486 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb48c0129c0 con 0x55829ad27f70 2024-03-11T18:37:05.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.268+0000 7fb49ede2000 10 client.4683 fetch_fsmap learned FSMap version 19 2024-03-11T18:37:05.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.268+0000 7fb49ede2000 1 -- 172.21.15.90:0/3861368486 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) v3 -- 0x55829a831cc0 con 0x55829ad27f70 2024-03-11T18:37:05.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.272+0000 7fb474ff9640 1 -- 172.21.15.90:0/3861368486 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55829ab03fa0 con 0x55829ad27f70 2024-03-11T18:37:05.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.272+0000 7fb474ff9640 1 -- 172.21.15.90:0/3861368486 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 19) v1 ==== 4500+0+0 (secure 0 0 0) 0x7fb48c012070 con 0x55829ad27f70 2024-03-11T18:37:05.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.272+0000 7fb49ede2000 10 client.4683 fetch_fsmap finished waiting for FSMap version 19 2024-03-11T18:37:05.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.272+0000 7fb49ede2000 10 client.4683 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-11T18:37:05.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.272+0000 7fb49ede2000 20 client.4683 populate_metadata read hostname 'smithi090' 2024-03-11T18:37:05.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.272+0000 7fb49ede2000 1 --2- 172.21.15.90:0/3861368486 >> [v2:172.21.15.92:6833/3568438077,v1:172.21.15.92:6835/3568438077] conn(0x55829ad29bc0 0x55829ad8fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:05.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.272+0000 7fb49ede2000 4 client.4683 mds_command: new command op to 4238 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-11T18:37:05.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.272+0000 7fb49ede2000 1 -- 172.21.15.90:0/3861368486 --> [v2:172.21.15.92:6833/3568438077,v1:172.21.15.92:6835/3568438077] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55829ab33b60 con 0x55829ad29bc0 2024-03-11T18:37:05.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.276+0000 7fb499a15640 1 --2- 172.21.15.90:0/3861368486 >> [v2:172.21.15.92:6833/3568438077,v1:172.21.15.92:6835/3568438077] conn(0x55829ad29bc0 0x55829ad8fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:05.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.284+0000 7fb499a15640 1 --2- 172.21.15.90:0/3861368486 >> [v2:172.21.15.92:6833/3568438077,v1:172.21.15.92:6835/3568438077] conn(0x55829ad29bc0 0x55829ad8fc90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6ef7f553f638a05a server_cookie=76007fce5d8d5e7a in_seq=0 out_seq=0 2024-03-11T18:37:05.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.284+0000 7fb474ff9640 10 client.4683 ms_handle_connect on v2:172.21.15.92:6833/3568438077 2024-03-11T18:37:05.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.312+0000 7fb474ff9640 1 -- 172.21.15.90:0/3861368486 <== mds.3 v2:172.21.15.92:6833/3568438077 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55829ab33b60 con 0x55829ad29bc0 2024-03-11T18:37:05.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.312+0000 7fb474ff9640 10 client.4683 handle_command_reply: tid=0 2024-03-11T18:37:05.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.312+0000 7fb49ede2000 1 client.4683 shutdown 2024-03-11T18:37:05.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.312+0000 7fb49ede2000 1 -- 172.21.15.90:0/3861368486 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55829ad27f70 msgr2=0x55829ad8f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:05.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.312+0000 7fb49ede2000 1 --2- 172.21.15.90:0/3861368486 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55829ad27f70 0x55829ad8f750 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb48c0014f0 tx=0x7fb48c014aa0 comp rx=0 tx=0).stop 2024-03-11T18:37:05.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.312+0000 7fb49ede2000 1 -- 172.21.15.90:0/3861368486 shutdown_connections 2024-03-11T18:37:05.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.312+0000 7fb49ede2000 1 --2- 172.21.15.90:0/3861368486 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55829ad27f70 0x55829ad8f750 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:05.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.312+0000 7fb49ede2000 1 --2- 172.21.15.90:0/3861368486 >> [v2:172.21.15.92:6833/3568438077,v1:172.21.15.92:6835/3568438077] conn(0x55829ad29bc0 0x55829ad8fc90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:05.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.312+0000 7fb49ede2000 1 --2- 172.21.15.90:0/3861368486 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55829aceeff0 0x55829ad8f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:05.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.312+0000 7fb49ede2000 1 --2- 172.21.15.90:0/3861368486 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55829acee890 0x55829ad3acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:05.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.312+0000 7fb474ff9640 0 client.4683 ms_handle_reset on v2:172.21.15.92:6833/3568438077 2024-03-11T18:37:05.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.312+0000 7fb49ede2000 1 -- 172.21.15.90:0/3861368486 >> 172.21.15.90:0/3861368486 conn(0x55829ad2ae10 msgr2=0x55829ad28be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:37:05.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.312+0000 7fb49ede2000 1 -- 172.21.15.90:0/3861368486 shutdown_connections 2024-03-11T18:37:05.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.312+0000 7fb49ede2000 1 -- 172.21.15.90:0/3861368486 wait complete. 2024-03-11T18:37:05.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.312+0000 7fb49ede2000 20 client.4683 trim_cache size 0 max 16384 2024-03-11T18:37:05.319 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-03-11T18:37:05.320 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "76d56b10-6c29-476a-804b-8482ea90f9e2", 2024-03-11T18:37:05.320 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 3, 2024-03-11T18:37:05.320 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 4238, 2024-03-11T18:37:05.320 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-03-11T18:37:05.320 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-03-11T18:37:05.320 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-03-11T18:37:05.320 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 18.807969343, 2024-03-11T18:37:05.320 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 19, 2024-03-11T18:37:05.320 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 30, 2024-03-11T18:37:05.320 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 28, 2024-03-11T18:37:05.320 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 83.932843797999993 2024-03-11T18:37:05.320 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-03-11T18:37:05.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.312+0000 7fb49bc79640 1 -- 172.21.15.90:0/2104313910 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fb478066ec0 msgr2=0x7fb478069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:05.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.312+0000 7fb49bc79640 1 --2- 172.21.15.90:0/2104313910 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fb478066ec0 0x7fb478069380 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb4940c9d50 tx=0x7fb48c009040 comp rx=0 tx=0).stop 2024-03-11T18:37:05.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.312+0000 7fb49bc79640 1 -- 172.21.15.90:0/2104313910 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb494070b00 msgr2=0x7fb4940c9ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:05.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.312+0000 7fb49bc79640 1 --2- 172.21.15.90:0/2104313910 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb494070b00 0x7fb4940c9ff0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb490019100 tx=0x7fb49001ad30 comp rx=0 tx=0).stop 2024-03-11T18:37:05.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.312+0000 7fb49bc79640 1 -- 172.21.15.90:0/2104313910 shutdown_connections 2024-03-11T18:37:05.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.312+0000 7fb49bc79640 1 --2- 172.21.15.90:0/2104313910 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fb478066ec0 0x7fb478069380 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:05.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.312+0000 7fb49bc79640 1 --2- 172.21.15.90:0/2104313910 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb494151b30 0x7fb4940c44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:05.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.312+0000 7fb49bc79640 1 --2- 172.21.15.90:0/2104313910 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb4940c4af0 0x7fb4940c3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:05.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.312+0000 7fb49bc79640 1 --2- 172.21.15.90:0/2104313910 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb494070b00 0x7fb4940c9ff0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:05.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.312+0000 7fb49bc79640 1 -- 172.21.15.90:0/2104313910 >> 172.21.15.90:0/2104313910 conn(0x7fb49406ac00 msgr2=0x7fb494072010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:37:05.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.312+0000 7fb49bc79640 1 -- 172.21.15.90:0/2104313910 shutdown_connections 2024-03-11T18:37:05.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.312+0000 7fb49bc79640 1 -- 172.21.15.90:0/2104313910 wait complete. 2024-03-11T18:37:05.345 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-11T18:37:05.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.468+0000 7fe116459640 1 Processor -- start 2024-03-11T18:37:05.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.468+0000 7fe116459640 1 -- start start 2024-03-11T18:37:05.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.468+0000 7fe116459640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe110153cf0 0x7fe11015d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:05.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.468+0000 7fe116459640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe11015a0b0 con 0x7fe1100c4b60 2024-03-11T18:37:05.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.468+0000 7fe116459640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe11015d880 con 0x7fe1100c42f0 2024-03-11T18:37:05.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.468+0000 7fe116459640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe11015e120 con 0x7fe110153cf0 2024-03-11T18:37:05.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.468+0000 7fe10f7fe640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7fe1100c4b60 0x7fe11015a540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:60170/0 (socket says 172.21.15.90:60170) 2024-03-11T18:37:05.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.468+0000 7fe10f7fe640 1 -- 172.21.15.90:0/4037057595 learned_addr learned my addr 172.21.15.90:0/4037057595 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T18:37:05.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.472+0000 7fe10ffff640 1 --2- 172.21.15.90:0/4037057595 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe110153cf0 0x7fe11015d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:05.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.472+0000 7fe10effd640 1 -- 172.21.15.90:0/4037057595 <== mon.0 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2709403901 0 0) 0x7fe11015a0b0 con 0x7fe1100c4b60 2024-03-11T18:37:05.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.472+0000 7fe10effd640 1 -- 172.21.15.90:0/4037057595 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0fc003530 con 0x7fe1100c4b60 2024-03-11T18:37:05.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.476+0000 7fe10ffff640 1 -- 172.21.15.90:0/4037057595 >> v1:172.21.15.92:6789/0 conn(0x7fe1100c42f0 legacy=0x7fe1100ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:05.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.476+0000 7fe10ffff640 1 -- 172.21.15.90:0/4037057595 >> v1:172.21.15.90:6789/0 conn(0x7fe1100c4b60 legacy=0x7fe11015a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:05.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.476+0000 7fe10ffff640 1 -- 172.21.15.90:0/4037057595 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe11015db60 con 0x7fe110153cf0 2024-03-11T18:37:05.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.488+0000 7fe10ffff640 1 --2- 172.21.15.90:0/4037057595 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe110153cf0 0x7fe11015d120 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe0f80097f0 tx=0x7fe0f801c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93e55081987ded2e server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:37:05.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.496+0000 7fe10effd640 1 -- 172.21.15.90:0/4037057595 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f80ab600 con 0x7fe110153cf0 2024-03-11T18:37:05.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.496+0000 7fe10effd640 1 -- 172.21.15.90:0/4037057595 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f80a7050 con 0x7fe110153cf0 2024-03-11T18:37:05.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.496+0000 7fe10effd640 1 -- 172.21.15.90:0/4037057595 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f80ab980 con 0x7fe110153cf0 2024-03-11T18:37:05.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.496+0000 7fe116459640 1 -- 172.21.15.90:0/4037057595 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe110153cf0 msgr2=0x7fe11015d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:05.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.496+0000 7fe116459640 1 --2- 172.21.15.90:0/4037057595 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe110153cf0 0x7fe11015d120 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe0f80097f0 tx=0x7fe0f801c4e0 comp rx=0 tx=0).stop 2024-03-11T18:37:05.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.496+0000 7fe116459640 1 -- 172.21.15.90:0/4037057595 shutdown_connections 2024-03-11T18:37:05.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.496+0000 7fe116459640 1 --2- 172.21.15.90:0/4037057595 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe110153cf0 0x7fe11015d120 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:05.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.496+0000 7fe116459640 1 -- 172.21.15.90:0/4037057595 >> 172.21.15.90:0/4037057595 conn(0x7fe11006b640 msgr2=0x7fe11006ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:37:05.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.496+0000 7fe116459640 1 -- 172.21.15.90:0/4037057595 shutdown_connections 2024-03-11T18:37:05.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.496+0000 7fe116459640 1 -- 172.21.15.90:0/4037057595 wait complete. 2024-03-11T18:37:05.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.500+0000 7fe116459640 1 Processor -- start 2024-03-11T18:37:05.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.500+0000 7fe116459640 1 -- start start 2024-03-11T18:37:05.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.500+0000 7fe116459640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe1100c42f0 0x7fe11015c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:05.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.500+0000 7fe116459640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe1100c4b60 0x7fe110143ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:05.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.500+0000 7fe116459640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe110153cf0 0x7fe1100c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:05.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.500+0000 7fe116459640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1100c7a50 con 0x7fe1100c4b60 2024-03-11T18:37:05.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.500+0000 7fe116459640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1100c8600 con 0x7fe110153cf0 2024-03-11T18:37:05.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.500+0000 7fe116459640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe11015ae20 con 0x7fe1100c42f0 2024-03-11T18:37:05.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.500+0000 7fe10f7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe1100c4b60 0x7fe110143ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:05.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.500+0000 7fe10f7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe1100c4b60 0x7fe110143ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44458/0 (socket says 172.21.15.90:44458) 2024-03-11T18:37:05.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.500+0000 7fe10f7fe640 1 -- 172.21.15.90:0/978825474 learned_addr learned my addr 172.21.15.90:0/978825474 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:37:05.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.500+0000 7fe1149f6640 1 --2- 172.21.15.90:0/978825474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe110153cf0 0x7fe1100c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:05.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.504+0000 7fe10ffff640 1 --2- 172.21.15.90:0/978825474 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe1100c42f0 0x7fe11015c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:05.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.504+0000 7fe10f7fe640 1 -- 172.21.15.90:0/978825474 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe1100c42f0 msgr2=0x7fe11015c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:05.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.504+0000 7fe10f7fe640 1 --2- 172.21.15.90:0/978825474 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe1100c42f0 0x7fe11015c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:05.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.504+0000 7fe10f7fe640 1 -- 172.21.15.90:0/978825474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe110153cf0 msgr2=0x7fe1100c88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:05.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.504+0000 7fe10f7fe640 1 --2- 172.21.15.90:0/978825474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe110153cf0 0x7fe1100c88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:05.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.504+0000 7fe10f7fe640 1 -- 172.21.15.90:0/978825474 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f8009420 con 0x7fe1100c4b60 2024-03-11T18:37:05.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.508+0000 7fe10f7fe640 1 --2- 172.21.15.90:0/978825474 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe1100c4b60 0x7fe110143ff0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe10000c420 tx=0x7fe10000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:37:05.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.512+0000 7fe10cff9640 1 -- 172.21.15.90:0/978825474 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe100005cf0 con 0x7fe1100c4b60 2024-03-11T18:37:05.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.512+0000 7fe10cff9640 1 -- 172.21.15.90:0/978825474 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe100005e70 con 0x7fe1100c4b60 2024-03-11T18:37:05.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.512+0000 7fe10cff9640 1 -- 172.21.15.90:0/978825474 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1000072f0 con 0x7fe1100c4b60 2024-03-11T18:37:05.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.512+0000 7fe116459640 1 -- 172.21.15.90:0/978825474 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe11015b100 con 0x7fe1100c4b60 2024-03-11T18:37:05.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.512+0000 7fe116459640 1 -- 172.21.15.90:0/978825474 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe11015b6c0 con 0x7fe1100c4b60 2024-03-11T18:37:05.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.512+0000 7fe1195c2000 1 Processor -- start 2024-03-11T18:37:05.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.512+0000 7fe1195c2000 1 -- start start 2024-03-11T18:37:05.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.516+0000 7fe1195c2000 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899a83df70 0x55899a804350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:05.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.516+0000 7fe1195c2000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55899a804890 0x55899a83f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:05.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.516+0000 7fe1195c2000 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55899a83fbc0 0x55899a83ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:05.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.516+0000 7fe1195c2000 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55899a6c6d00 con 0x55899a83df70 2024-03-11T18:37:05.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.516+0000 7fe1195c2000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55899a658430 con 0x55899a804890 2024-03-11T18:37:05.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.516+0000 7fe1195c2000 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x55899a655960 con 0x55899a83fbc0 2024-03-11T18:37:05.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.516+0000 7fe10f7fe640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55899a83fbc0 0x55899a83ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:05.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.516+0000 7fe10f7fe640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55899a83fbc0 0x55899a83ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.90:50122/0 (socket says 172.21.15.90:50122) 2024-03-11T18:37:05.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.516+0000 7fe10f7fe640 1 -- 172.21.15.90:0/1983015687 learned_addr learned my addr 172.21.15.90:0/1983015687 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:37:05.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.520+0000 7fe10ffff640 1 --2- 172.21.15.90:0/1983015687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55899a804890 0x55899a83f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:05.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.524+0000 7fe10f7fe640 1 -- 172.21.15.90:0/1983015687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55899a804890 msgr2=0x55899a83f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:05.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.524+0000 7fe10f7fe640 1 --2- 172.21.15.90:0/1983015687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55899a804890 0x55899a83f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:05.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.524+0000 7fe10f7fe640 1 -- 172.21.15.90:0/1983015687 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899a83df70 msgr2=0x55899a804350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:05.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.524+0000 7fe10f7fe640 1 --2- 172.21.15.90:0/1983015687 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899a83df70 0x55899a804350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:05.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.524+0000 7fe10f7fe640 1 -- 172.21.15.90:0/1983015687 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55899a347890 con 0x55899a83fbc0 2024-03-11T18:37:05.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.528+0000 7fe10f7fe640 1 --2- 172.21.15.90:0/1983015687 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55899a83fbc0 0x55899a83ffa0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe100005890 tx=0x7fe100009050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8bdd53f4b0291ab server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:37:05.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.532+0000 7fe10cff9640 1 -- 172.21.15.90:0/978825474 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fe1000047f0 con 0x7fe1100c4b60 2024-03-11T18:37:05.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.532+0000 7fe10cff9640 1 --2- 172.21.15.90:0/978825474 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe0f0066ec0 0x7fe0f0069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:05.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.532+0000 7fe1149f6640 1 --2- 172.21.15.90:0/978825474 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe0f0066ec0 0x7fe0f0069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:05.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.532+0000 7fe1149f6640 1 --2- 172.21.15.90:0/978825474 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe0f0066ec0 0x7fe0f0069380 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe1080012f0 tx=0x7fe108009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:37:05.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.536+0000 7fe10cff9640 1 -- 172.21.15.90:0/978825474 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fe100057080 con 0x7fe1100c4b60 2024-03-11T18:37:05.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.540+0000 7fe105ffb640 1 -- 172.21.15.90:0/1983015687 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe100053320 con 0x55899a83fbc0 2024-03-11T18:37:05.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.540+0000 7fe105ffb640 1 -- 172.21.15.90:0/1983015687 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe100056e50 con 0x55899a83fbc0 2024-03-11T18:37:05.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.540+0000 7fe105ffb640 1 -- 172.21.15.90:0/1983015687 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe100056bc0 con 0x55899a83fbc0 2024-03-11T18:37:05.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.540+0000 7fe1195c2000 1 -- 172.21.15.90:0/1983015687 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55899a83fbc0 msgr2=0x55899a83ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:05.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.540+0000 7fe1195c2000 1 --2- 172.21.15.90:0/1983015687 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55899a83fbc0 0x55899a83ffa0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe100005890 tx=0x7fe100009050 comp rx=0 tx=0).stop 2024-03-11T18:37:05.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.540+0000 7fe1195c2000 1 -- 172.21.15.90:0/1983015687 shutdown_connections 2024-03-11T18:37:05.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.540+0000 7fe1195c2000 1 --2- 172.21.15.90:0/1983015687 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55899a83fbc0 0x55899a83ffa0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:05.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.540+0000 7fe1195c2000 1 --2- 172.21.15.90:0/1983015687 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55899a804890 0x55899a83f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:05.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.540+0000 7fe1195c2000 1 --2- 172.21.15.90:0/1983015687 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899a83df70 0x55899a804350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:05.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.540+0000 7fe1195c2000 1 -- 172.21.15.90:0/1983015687 >> 172.21.15.90:0/1983015687 conn(0x55899a804ff0 msgr2=0x55899a83d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:37:05.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.540+0000 7fe1195c2000 1 -- 172.21.15.90:0/1983015687 shutdown_connections 2024-03-11T18:37:05.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.540+0000 7fe1195c2000 1 -- 172.21.15.90:0/1983015687 wait complete. 2024-03-11T18:37:05.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.540+0000 7fe1195c2000 1 -- start start 2024-03-11T18:37:05.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.544+0000 7fe1195c2000 1 Processor -- start 2024-03-11T18:37:05.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.544+0000 7fe1195c2000 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55899a804890 0x55899a84ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:05.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.544+0000 7fe1195c2000 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899a83df70 0x55899a848810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:05.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.544+0000 7fe1195c2000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55899a83fbc0 0x55899a848d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:05.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.544+0000 7fe1195c2000 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55899a651140 con 0x55899a83df70 2024-03-11T18:37:05.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.544+0000 7fe1195c2000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55899a5eb630 con 0x55899a83fbc0 2024-03-11T18:37:05.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.544+0000 7fe1195c2000 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x55899a5eb2d0 con 0x55899a804890 2024-03-11T18:37:05.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.544+0000 7fe1195c2000 10 client.0 _ll_register_callbacks cb 0x55899a7011c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T18:37:05.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.544+0000 7fe10f7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899a83df70 0x55899a848810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:05.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.544+0000 7fe10f7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899a83df70 0x55899a848810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44472/0 (socket says 172.21.15.90:44472) 2024-03-11T18:37:05.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.544+0000 7fe10f7fe640 1 -- 172.21.15.90:0/3390048155 learned_addr learned my addr 172.21.15.90:0/3390048155 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:37:05.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.544+0000 7fe1149f6640 1 --2- 172.21.15.90:0/3390048155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55899a83fbc0 0x55899a848d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:05.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.544+0000 7fe10ffff640 1 --2- 172.21.15.90:0/3390048155 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55899a804890 0x55899a84ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:05.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.548+0000 7fe10f7fe640 1 -- 172.21.15.90:0/3390048155 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55899a804890 msgr2=0x55899a84ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:05.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.548+0000 7fe10f7fe640 1 --2- 172.21.15.90:0/3390048155 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55899a804890 0x55899a84ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:05.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.548+0000 7fe10f7fe640 1 -- 172.21.15.90:0/3390048155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55899a83fbc0 msgr2=0x55899a848d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:05.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.548+0000 7fe10f7fe640 1 --2- 172.21.15.90:0/3390048155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55899a83fbc0 0x55899a848d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:05.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.548+0000 7fe10f7fe640 1 -- 172.21.15.90:0/3390048155 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe11015b6c0 con 0x55899a83df70 2024-03-11T18:37:05.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.548+0000 7fe10f7fe640 1 --2- 172.21.15.90:0/3390048155 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899a83df70 0x55899a848810 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe100000f20 tx=0x7fe10005b210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa666e5e2c18bd26 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:37:05.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.548+0000 7fe0eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-03-11T18:37:05.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.556+0000 7fe0eeffd640 1 -- 172.21.15.90:0/3390048155 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10006e070 con 0x55899a83df70 2024-03-11T18:37:05.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.556+0000 7fe0eeffd640 1 -- 172.21.15.90:0/3390048155 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fe10005f250 con 0x55899a83df70 2024-03-11T18:37:05.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.556+0000 7fe1195c2000 1 -- 172.21.15.90:0/3390048155 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55899a604850 con 0x55899a83df70 2024-03-11T18:37:05.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.556+0000 7fe0eeffd640 1 -- 172.21.15.90:0/3390048155 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe10005f410 con 0x55899a83df70 2024-03-11T18:37:05.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.556+0000 7fe1195c2000 10 client.4692 fetch_fsmap learned FSMap version 19 2024-03-11T18:37:05.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.556+0000 7fe1195c2000 1 -- 172.21.15.90:0/3390048155 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) v3 -- 0x55899a610e70 con 0x55899a83df70 2024-03-11T18:37:05.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.560+0000 7fe0eeffd640 1 -- 172.21.15.90:0/3390048155 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe100072080 con 0x55899a83df70 2024-03-11T18:37:05.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.560+0000 7fe0eeffd640 1 -- 172.21.15.90:0/3390048155 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 19) v1 ==== 4500+0+0 (secure 0 0 0) 0x7fe100025040 con 0x55899a83df70 2024-03-11T18:37:05.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.560+0000 7fe1195c2000 10 client.4692 fetch_fsmap finished waiting for FSMap version 19 2024-03-11T18:37:05.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.560+0000 7fe1195c2000 10 client.4692 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-11T18:37:05.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.560+0000 7fe1195c2000 20 client.4692 populate_metadata read hostname 'smithi090' 2024-03-11T18:37:05.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.560+0000 7fe1195c2000 1 --2- 172.21.15.90:0/3390048155 >> [v2:172.21.15.92:6832/2306487548,v1:172.21.15.92:6834/2306487548] conn(0x55899a8396a0 0x55899a839a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:05.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.560+0000 7fe1195c2000 4 client.4692 mds_command: new command op to 4241 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T18:37:05.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.560+0000 7fe1195c2000 1 -- 172.21.15.90:0/3390048155 --> [v2:172.21.15.92:6832/2306487548,v1:172.21.15.92:6834/2306487548] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55899a64df20 con 0x55899a8396a0 2024-03-11T18:37:05.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.564+0000 7fe10ffff640 1 --2- 172.21.15.90:0/3390048155 >> [v2:172.21.15.92:6832/2306487548,v1:172.21.15.92:6834/2306487548] conn(0x55899a8396a0 0x55899a839a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:05.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.656+0000 7fe0eeffd640 10 client.4692 ms_handle_connect on v2:172.21.15.92:6832/2306487548 2024-03-11T18:37:05.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.656+0000 7fe10ffff640 1 --2- 172.21.15.90:0/3390048155 >> [v2:172.21.15.92:6832/2306487548,v1:172.21.15.92:6834/2306487548] conn(0x55899a8396a0 0x55899a839a80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=61383f0be286c7ca server_cookie=7a64b294e87fa3dc in_seq=0 out_seq=0 2024-03-11T18:37:05.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.884+0000 7fe0eeffd640 1 -- 172.21.15.90:0/3390048155 <== mds.2 v2:172.21.15.92:6832/2306487548 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55899a64df20 con 0x55899a8396a0 2024-03-11T18:37:05.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.884+0000 7fe0eeffd640 10 client.4692 handle_command_reply: tid=0 2024-03-11T18:37:05.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.884+0000 7fe1195c2000 1 client.4692 shutdown 2024-03-11T18:37:05.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.884+0000 7fe1195c2000 1 -- 172.21.15.90:0/3390048155 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899a83df70 msgr2=0x55899a848810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:05.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.884+0000 7fe1195c2000 1 --2- 172.21.15.90:0/3390048155 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899a83df70 0x55899a848810 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe100000f20 tx=0x7fe10005b210 comp rx=0 tx=0).stop 2024-03-11T18:37:05.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.884+0000 7fe1195c2000 1 -- 172.21.15.90:0/3390048155 shutdown_connections 2024-03-11T18:37:05.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.884+0000 7fe1195c2000 1 --2- 172.21.15.90:0/3390048155 >> [v2:172.21.15.92:6832/2306487548,v1:172.21.15.92:6834/2306487548] conn(0x55899a8396a0 0x55899a839a80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:05.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.884+0000 7fe1195c2000 1 --2- 172.21.15.90:0/3390048155 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55899a83fbc0 0x55899a848d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:05.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.884+0000 7fe0eeffd640 0 client.4692 ms_handle_reset on v2:172.21.15.92:6832/2306487548 2024-03-11T18:37:05.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.884+0000 7fe1195c2000 1 --2- 172.21.15.90:0/3390048155 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899a83df70 0x55899a848810 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:05.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.884+0000 7fe1195c2000 1 --2- 172.21.15.90:0/3390048155 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55899a804890 0x55899a84ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:05.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.884+0000 7fe1195c2000 1 -- 172.21.15.90:0/3390048155 >> 172.21.15.90:0/3390048155 conn(0x55899a804ff0 msgr2=0x55899a83d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:37:05.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.884+0000 7fe1195c2000 1 -- 172.21.15.90:0/3390048155 shutdown_connections 2024-03-11T18:37:05.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.884+0000 7fe1195c2000 1 -- 172.21.15.90:0/3390048155 wait complete. 2024-03-11T18:37:05.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.884+0000 7fe1195c2000 20 client.4692 trim_cache size 0 max 16384 2024-03-11T18:37:05.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.932+0000 7fe1195c2000 1 Processor -- start 2024-03-11T18:37:05.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.932+0000 7fe1195c2000 1 -- start start 2024-03-11T18:37:05.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.932+0000 7fe1195c2000 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899a804ff0 0x55899a848d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:05.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.932+0000 7fe1195c2000 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55899a804890 0x55899a840830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:05.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.932+0000 7fe1195c2000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55899a8396a0 0x55899a840d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:05.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.932+0000 7fe1195c2000 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55899a8519c0 con 0x55899a804ff0 2024-03-11T18:37:05.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.932+0000 7fe1195c2000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55899a308940 con 0x55899a8396a0 2024-03-11T18:37:05.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.932+0000 7fe1195c2000 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x55899a61bc80 con 0x55899a804890 2024-03-11T18:37:05.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.932+0000 7fe10ffff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899a804ff0 0x55899a848d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:05.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.932+0000 7fe10ffff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899a804ff0 0x55899a848d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44488/0 (socket says 172.21.15.90:44488) 2024-03-11T18:37:05.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.932+0000 7fe10ffff640 1 -- 172.21.15.90:0/1364230926 learned_addr learned my addr 172.21.15.90:0/1364230926 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:37:05.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.932+0000 7fe10f7fe640 1 --2- 172.21.15.90:0/1364230926 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55899a804890 0x55899a840830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:05.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.936+0000 7fe1149f6640 1 --2- 172.21.15.90:0/1364230926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55899a8396a0 0x55899a840d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:05.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.936+0000 7fe10ffff640 1 -- 172.21.15.90:0/1364230926 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55899a804890 msgr2=0x55899a840830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:05.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.936+0000 7fe10ffff640 1 --2- 172.21.15.90:0/1364230926 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55899a804890 0x55899a840830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:05.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.936+0000 7fe10ffff640 1 -- 172.21.15.90:0/1364230926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55899a8396a0 msgr2=0x55899a840d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:05.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.936+0000 7fe10ffff640 1 --2- 172.21.15.90:0/1364230926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55899a8396a0 0x55899a840d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:05.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.936+0000 7fe10ffff640 1 -- 172.21.15.90:0/1364230926 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55899a5f6e10 con 0x55899a804ff0 2024-03-11T18:37:05.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.940+0000 7fe10ffff640 1 --2- 172.21.15.90:0/1364230926 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899a804ff0 0x55899a848d10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe0f801a110 tx=0x7fe0f8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7140b186578ce0cb server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:37:05.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.948+0000 7fe1057fa640 1 -- 172.21.15.90:0/1364230926 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f80bb040 con 0x55899a804ff0 2024-03-11T18:37:05.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.948+0000 7fe1057fa640 1 -- 172.21.15.90:0/1364230926 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f80b6850 con 0x55899a804ff0 2024-03-11T18:37:05.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.948+0000 7fe1057fa640 1 -- 172.21.15.90:0/1364230926 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f80b6bc0 con 0x55899a804ff0 2024-03-11T18:37:05.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.948+0000 7fe1195c2000 1 -- 172.21.15.90:0/1364230926 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899a804ff0 msgr2=0x55899a848d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:05.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.948+0000 7fe1195c2000 1 --2- 172.21.15.90:0/1364230926 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899a804ff0 0x55899a848d10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe0f801a110 tx=0x7fe0f8004740 comp rx=0 tx=0).stop 2024-03-11T18:37:05.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.948+0000 7fe1195c2000 1 -- 172.21.15.90:0/1364230926 shutdown_connections 2024-03-11T18:37:05.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.948+0000 7fe1195c2000 1 --2- 172.21.15.90:0/1364230926 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55899a8396a0 0x55899a840d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:05.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.948+0000 7fe1195c2000 1 --2- 172.21.15.90:0/1364230926 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55899a804890 0x55899a840830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:05.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.948+0000 7fe1195c2000 1 --2- 172.21.15.90:0/1364230926 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899a804ff0 0x55899a848d10 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:05.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.948+0000 7fe1195c2000 1 -- 172.21.15.90:0/1364230926 >> 172.21.15.90:0/1364230926 conn(0x55899a849990 msgr2=0x55899a85a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:37:05.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.948+0000 7fe1195c2000 1 -- 172.21.15.90:0/1364230926 shutdown_connections 2024-03-11T18:37:05.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.948+0000 7fe1195c2000 1 -- 172.21.15.90:0/1364230926 wait complete. 2024-03-11T18:37:05.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.952+0000 7fe1195c2000 1 -- start start 2024-03-11T18:37:05.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.952+0000 7fe1195c2000 1 Processor -- start 2024-03-11T18:37:05.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.952+0000 7fe1195c2000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55899a804890 0x55899a803cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:05.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.952+0000 7fe1195c2000 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55899a804ff0 0x55899a8a61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:05.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.952+0000 7fe1195c2000 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899a8396a0 0x55899a8a66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:05.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.952+0000 7fe1195c2000 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x55899a5e9660 con 0x55899a8396a0 2024-03-11T18:37:05.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.952+0000 7fe1195c2000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x55899a658e80 con 0x55899a804890 2024-03-11T18:37:05.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.952+0000 7fe1195c2000 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x55899a609800 con 0x55899a804ff0 2024-03-11T18:37:05.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.952+0000 7fe1195c2000 10 client.0 _ll_register_callbacks cb 0x55899a6532c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T18:37:05.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.952+0000 7fe1149f6640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899a8396a0 0x55899a8a66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:05.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.952+0000 7fe1149f6640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899a8396a0 0x55899a8a66e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44492/0 (socket says 172.21.15.90:44492) 2024-03-11T18:37:05.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.952+0000 7fe1149f6640 1 -- 172.21.15.90:0/3375266594 learned_addr learned my addr 172.21.15.90:0/3375266594 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:37:05.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.952+0000 7fe10ffff640 1 --2- 172.21.15.90:0/3375266594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55899a804890 0x55899a803cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:05.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.952+0000 7fe10f7fe640 1 --2- 172.21.15.90:0/3375266594 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55899a804ff0 0x55899a8a61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:05.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.956+0000 7fe1149f6640 1 -- 172.21.15.90:0/3375266594 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55899a804ff0 msgr2=0x55899a8a61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:05.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.956+0000 7fe1149f6640 1 --2- 172.21.15.90:0/3375266594 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55899a804ff0 0x55899a8a61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:05.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.956+0000 7fe1149f6640 1 -- 172.21.15.90:0/3375266594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55899a804890 msgr2=0x55899a803cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:05.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.956+0000 7fe1149f6640 1 --2- 172.21.15.90:0/3375266594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55899a804890 0x55899a803cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:05.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.956+0000 7fe1149f6640 1 -- 172.21.15.90:0/3375266594 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe11015db60 con 0x55899a8396a0 2024-03-11T18:37:05.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.960+0000 7fe1149f6640 1 --2- 172.21.15.90:0/3375266594 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899a8396a0 0x55899a8a66e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe108010010 tx=0x7fe108001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaefb09f90ad6922 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:37:05.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.960+0000 7fe0eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-03-11T18:37:05.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.960+0000 7fe0eeffd640 1 -- 172.21.15.90:0/3375266594 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe108013830 con 0x55899a8396a0 2024-03-11T18:37:05.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.960+0000 7fe1195c2000 1 -- 172.21.15.90:0/3375266594 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55899a5d6e80 con 0x55899a8396a0 2024-03-11T18:37:05.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.964+0000 7fe0eeffd640 1 -- 172.21.15.90:0/3375266594 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fe10801c040 con 0x55899a8396a0 2024-03-11T18:37:05.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.964+0000 7fe0eeffd640 1 -- 172.21.15.90:0/3375266594 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe108013990 con 0x55899a8396a0 2024-03-11T18:37:05.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.964+0000 7fe1195c2000 10 client.4698 fetch_fsmap learned FSMap version 19 2024-03-11T18:37:05.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.964+0000 7fe1195c2000 1 -- 172.21.15.90:0/3375266594 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) v3 -- 0x55899a347cc0 con 0x55899a8396a0 2024-03-11T18:37:05.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.968+0000 7fe0eeffd640 1 -- 172.21.15.90:0/3375266594 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55899a5d6e80 con 0x55899a8396a0 2024-03-11T18:37:05.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.968+0000 7fe0eeffd640 1 -- 172.21.15.90:0/3375266594 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 19) v1 ==== 4500+0+0 (secure 0 0 0) 0x7fe108012070 con 0x55899a8396a0 2024-03-11T18:37:05.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.968+0000 7fe1195c2000 10 client.4698 fetch_fsmap finished waiting for FSMap version 19 2024-03-11T18:37:05.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.968+0000 7fe1195c2000 10 client.4698 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-11T18:37:05.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.968+0000 7fe1195c2000 20 client.4698 populate_metadata read hostname 'smithi090' 2024-03-11T18:37:05.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.968+0000 7fe1195c2000 1 --2- 172.21.15.90:0/3375266594 >> [v2:172.21.15.92:6832/2306487548,v1:172.21.15.92:6834/2306487548] conn(0x55899a83df70 0x55899a8a6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:05.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.968+0000 7fe1195c2000 4 client.4698 mds_command: new command op to 4241 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-11T18:37:05.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.968+0000 7fe1195c2000 1 -- 172.21.15.90:0/3375266594 --> [v2:172.21.15.92:6832/2306487548,v1:172.21.15.92:6834/2306487548] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55899a619fa0 con 0x55899a83df70 2024-03-11T18:37:05.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.972+0000 7fe10ffff640 1 --2- 172.21.15.90:0/3375266594 >> [v2:172.21.15.92:6832/2306487548,v1:172.21.15.92:6834/2306487548] conn(0x55899a83df70 0x55899a8a6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:05.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.976+0000 7fe10ffff640 1 --2- 172.21.15.90:0/3375266594 >> [v2:172.21.15.92:6832/2306487548,v1:172.21.15.92:6834/2306487548] conn(0x55899a83df70 0x55899a8a6c20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8fbb9475093327cc server_cookie=be20d39641c5fd38 in_seq=0 out_seq=0 2024-03-11T18:37:05.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:05.980+0000 7fe0eeffd640 10 client.4698 ms_handle_connect on v2:172.21.15.92:6832/2306487548 2024-03-11T18:37:06.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.008+0000 7fe0eeffd640 1 -- 172.21.15.90:0/3375266594 <== mds.2 v2:172.21.15.92:6832/2306487548 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55899a619fa0 con 0x55899a83df70 2024-03-11T18:37:06.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.008+0000 7fe0eeffd640 10 client.4698 handle_command_reply: tid=0 2024-03-11T18:37:06.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.008+0000 7fe1195c2000 1 client.4698 shutdown 2024-03-11T18:37:06.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.008+0000 7fe1195c2000 1 -- 172.21.15.90:0/3375266594 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899a8396a0 msgr2=0x55899a8a66e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:06.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.008+0000 7fe1195c2000 1 --2- 172.21.15.90:0/3375266594 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899a8396a0 0x55899a8a66e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe108010010 tx=0x7fe108001260 comp rx=0 tx=0).stop 2024-03-11T18:37:06.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.008+0000 7fe1195c2000 1 -- 172.21.15.90:0/3375266594 shutdown_connections 2024-03-11T18:37:06.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.008+0000 7fe1195c2000 1 --2- 172.21.15.90:0/3375266594 >> [v2:172.21.15.92:6832/2306487548,v1:172.21.15.92:6834/2306487548] conn(0x55899a83df70 0x55899a8a6c20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.008+0000 7fe1195c2000 1 --2- 172.21.15.90:0/3375266594 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55899a8396a0 0x55899a8a66e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.008+0000 7fe1195c2000 1 --2- 172.21.15.90:0/3375266594 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x55899a804ff0 0x55899a8a61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.008+0000 7fe1195c2000 1 --2- 172.21.15.90:0/3375266594 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x55899a804890 0x55899a803cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.008+0000 7fe0eeffd640 0 client.4698 ms_handle_reset on v2:172.21.15.92:6832/2306487548 2024-03-11T18:37:06.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.008+0000 7fe1195c2000 1 -- 172.21.15.90:0/3375266594 >> 172.21.15.90:0/3375266594 conn(0x55899a849990 msgr2=0x55899a85a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:37:06.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.008+0000 7fe1195c2000 1 -- 172.21.15.90:0/3375266594 shutdown_connections 2024-03-11T18:37:06.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.008+0000 7fe1195c2000 1 -- 172.21.15.90:0/3375266594 wait complete. 2024-03-11T18:37:06.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.008+0000 7fe1195c2000 20 client.4698 trim_cache size 0 max 16384 2024-03-11T18:37:06.016 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-03-11T18:37:06.016 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "76d56b10-6c29-476a-804b-8482ea90f9e2", 2024-03-11T18:37:06.016 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 2, 2024-03-11T18:37:06.016 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 4241, 2024-03-11T18:37:06.016 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-03-11T18:37:06.016 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-03-11T18:37:06.016 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-03-11T18:37:06.016 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 22.605437875, 2024-03-11T18:37:06.016 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 19, 2024-03-11T18:37:06.016 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 30, 2024-03-11T18:37:06.016 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 28, 2024-03-11T18:37:06.017 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 84.631137269000007 2024-03-11T18:37:06.017 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-03-11T18:37:06.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.008+0000 7fe116459640 1 -- 172.21.15.90:0/978825474 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe0f0066ec0 msgr2=0x7fe0f0069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:06.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.008+0000 7fe116459640 1 --2- 172.21.15.90:0/978825474 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe0f0066ec0 0x7fe0f0069380 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe1080012f0 tx=0x7fe108009040 comp rx=0 tx=0).stop 2024-03-11T18:37:06.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.008+0000 7fe116459640 1 -- 172.21.15.90:0/978825474 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe1100c4b60 msgr2=0x7fe110143ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:06.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.008+0000 7fe116459640 1 --2- 172.21.15.90:0/978825474 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe1100c4b60 0x7fe110143ff0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe10000c420 tx=0x7fe10000c8e0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.008+0000 7fe116459640 1 -- 172.21.15.90:0/978825474 shutdown_connections 2024-03-11T18:37:06.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.008+0000 7fe116459640 1 --2- 172.21.15.90:0/978825474 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe0f0066ec0 0x7fe0f0069380 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.008+0000 7fe116459640 1 --2- 172.21.15.90:0/978825474 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe110153cf0 0x7fe1100c88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.008+0000 7fe116459640 1 --2- 172.21.15.90:0/978825474 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe1100c4b60 0x7fe110143ff0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.008+0000 7fe116459640 1 --2- 172.21.15.90:0/978825474 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe1100c42f0 0x7fe11015c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.008+0000 7fe116459640 1 -- 172.21.15.90:0/978825474 >> 172.21.15.90:0/978825474 conn(0x7fe11006b640 msgr2=0x7fe1101579a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:37:06.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.008+0000 7fe116459640 1 -- 172.21.15.90:0/978825474 shutdown_connections 2024-03-11T18:37:06.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.008+0000 7fe116459640 1 -- 172.21.15.90:0/978825474 wait complete. 2024-03-11T18:37:06.040 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-03-11T18:37:06.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.160+0000 7f0254879640 1 Processor -- start 2024-03-11T18:37:06.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.160+0000 7f0254879640 1 -- start start 2024-03-11T18:37:06.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.160+0000 7f0254879640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f025006f400 0x7f02500c6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:06.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.160+0000 7f0254879640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02500c7d50 con 0x7f02501540c0 2024-03-11T18:37:06.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.160+0000 7f0254879640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02500703f0 con 0x7f02500cc280 2024-03-11T18:37:06.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.160+0000 7f0254879640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f025015d900 con 0x7f025006f400 2024-03-11T18:37:06.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.164+0000 7f024dd9b640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f025006f400 0x7f02500c6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:06.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.164+0000 7f024dd9b640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f025006f400 0x7f02500c6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.90:50168/0 (socket says 172.21.15.90:50168) 2024-03-11T18:37:06.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.164+0000 7f024dd9b640 1 -- 172.21.15.90:0/2249227980 learned_addr learned my addr 172.21.15.90:0/2249227980 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:37:06.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.168+0000 7f024dd9b640 1 -- 172.21.15.90:0/2249227980 >> v1:172.21.15.92:6789/0 conn(0x7f02500cc280 legacy=0x7f0250164580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:06.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.168+0000 7f024dd9b640 1 -- 172.21.15.90:0/2249227980 >> v1:172.21.15.90:6789/0 conn(0x7f02501540c0 legacy=0x7f025015d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:06.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.168+0000 7f024dd9b640 1 -- 172.21.15.90:0/2249227980 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f025006b480 con 0x7f025006f400 2024-03-11T18:37:06.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.176+0000 7f024dd9b640 1 --2- 172.21.15.90:0/2249227980 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f025006f400 0x7f02500c6cf0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f02400098a0 tx=0x7f024001ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1932b97e3d95b591 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:37:06.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.184+0000 7f024d59a640 1 -- 172.21.15.90:0/2249227980 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02400a89b0 con 0x7f025006f400 2024-03-11T18:37:06.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.184+0000 7f024d59a640 1 -- 172.21.15.90:0/2249227980 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02400a8b10 con 0x7f025006f400 2024-03-11T18:37:06.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.184+0000 7f024d59a640 1 -- 172.21.15.90:0/2249227980 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02400a8ea0 con 0x7f025006f400 2024-03-11T18:37:06.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.184+0000 7f0254879640 1 -- 172.21.15.90:0/2249227980 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f025006f400 msgr2=0x7f02500c6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:06.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.184+0000 7f0254879640 1 --2- 172.21.15.90:0/2249227980 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f025006f400 0x7f02500c6cf0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f02400098a0 tx=0x7f024001ecf0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.184+0000 7f0254879640 1 -- 172.21.15.90:0/2249227980 shutdown_connections 2024-03-11T18:37:06.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.184+0000 7f0254879640 1 --2- 172.21.15.90:0/2249227980 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f025006f400 0x7f02500c6cf0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.184+0000 7f0254879640 1 -- 172.21.15.90:0/2249227980 >> 172.21.15.90:0/2249227980 conn(0x7f0250071aa0 msgr2=0x7f0250071eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:37:06.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.184+0000 7f0254879640 1 -- 172.21.15.90:0/2249227980 shutdown_connections 2024-03-11T18:37:06.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.184+0000 7f0254879640 1 -- 172.21.15.90:0/2249227980 wait complete. 2024-03-11T18:37:06.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.188+0000 7f0254879640 1 Processor -- start 2024-03-11T18:37:06.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.188+0000 7f0254879640 1 -- start start 2024-03-11T18:37:06.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.188+0000 7f0254879640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f025006f400 0x7f025014ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:06.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.188+0000 7f0254879640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02500cc280 0x7f025014d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:06.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.188+0000 7f0254879640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f02501540c0 0x7f02501f3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:06.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.188+0000 7f0254879640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02501e39a0 con 0x7f025006f400 2024-03-11T18:37:06.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.188+0000 7f0254879640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025014c100 con 0x7f02500cc280 2024-03-11T18:37:06.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.188+0000 7f0254879640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f025014c9a0 con 0x7f02501540c0 2024-03-11T18:37:06.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.192+0000 7f024ed9d640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f02501540c0 0x7f02501f3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:06.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.192+0000 7f024ed9d640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f02501540c0 0x7f02501f3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.90:50178/0 (socket says 172.21.15.90:50178) 2024-03-11T18:37:06.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.192+0000 7f024ed9d640 1 -- 172.21.15.90:0/394930663 learned_addr learned my addr 172.21.15.90:0/394930663 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:37:06.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.200+0000 7f024ed9d640 1 -- 172.21.15.90:0/394930663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02500cc280 msgr2=0x7f025014d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:06.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.200+0000 7f024ed9d640 1 --2- 172.21.15.90:0/394930663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02500cc280 0x7f025014d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.200+0000 7f024ed9d640 1 -- 172.21.15.90:0/394930663 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f025006f400 msgr2=0x7f025014ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:06.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.200+0000 7f024ed9d640 1 --2- 172.21.15.90:0/394930663 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f025006f400 0x7f025014ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.200+0000 7f024ed9d640 1 -- 172.21.15.90:0/394930663 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02400094d0 con 0x7f02501540c0 2024-03-11T18:37:06.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.208+0000 7f024ed9d640 1 --2- 172.21.15.90:0/394930663 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f02501540c0 0x7f02501f3e20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f024800c7b0 tx=0x7f024800cc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:37:06.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.216+0000 7f023effd640 1 -- 172.21.15.90:0/394930663 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0248003d90 con 0x7f02501540c0 2024-03-11T18:37:06.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.216+0000 7f023effd640 1 -- 172.21.15.90:0/394930663 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0248005070 con 0x7f02501540c0 2024-03-11T18:37:06.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.216+0000 7f023effd640 1 -- 172.21.15.90:0/394930663 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0248005420 con 0x7f02501540c0 2024-03-11T18:37:06.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.216+0000 7f0254879640 1 -- 172.21.15.90:0/394930663 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02501f4360 con 0x7f02501540c0 2024-03-11T18:37:06.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.216+0000 7f0254879640 1 -- 172.21.15.90:0/394930663 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f025015b200 con 0x7f02501540c0 2024-03-11T18:37:06.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.220+0000 7f02579e2000 1 Processor -- start 2024-03-11T18:37:06.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.220+0000 7f02579e2000 1 -- start start 2024-03-11T18:37:06.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.220+0000 7f02579e2000 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563556764f70 0x56355672b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:06.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.220+0000 7f02579e2000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56355672b890 0x563556766680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:06.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.220+0000 7f02579e2000 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x563556766bc0 0x563556766fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:06.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.220+0000 7f02579e2000 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635565edd00 con 0x563556764f70 2024-03-11T18:37:06.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.220+0000 7f02579e2000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56355657f430 con 0x56355672b890 2024-03-11T18:37:06.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.220+0000 7f02579e2000 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x56355657c960 con 0x563556766bc0 2024-03-11T18:37:06.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.224+0000 7f024dd9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563556764f70 0x56355672b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:06.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.224+0000 7f024dd9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563556764f70 0x56355672b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44504/0 (socket says 172.21.15.90:44504) 2024-03-11T18:37:06.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.224+0000 7f024dd9b640 1 -- 172.21.15.90:0/291330586 learned_addr learned my addr 172.21.15.90:0/291330586 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:37:06.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.228+0000 7f024dd9b640 1 -- 172.21.15.90:0/291330586 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x563556766bc0 msgr2=0x563556766fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:06.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.228+0000 7f024dd9b640 1 --2- 172.21.15.90:0/291330586 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x563556766bc0 0x563556766fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.228+0000 7f024dd9b640 1 -- 172.21.15.90:0/291330586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56355672b890 msgr2=0x563556766680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:06.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.228+0000 7f024dd9b640 1 --2- 172.21.15.90:0/291330586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56355672b890 0x563556766680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.228+0000 7f024dd9b640 1 -- 172.21.15.90:0/291330586 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56355626e890 con 0x563556764f70 2024-03-11T18:37:06.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.232+0000 7f024dd9b640 1 --2- 172.21.15.90:0/291330586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563556764f70 0x56355672b350 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f024001d3a0 tx=0x7f024001d530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ebf7cfb60787dd server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:37:06.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.236+0000 7f022ffff640 1 -- 172.21.15.90:0/291330586 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02400b0440 con 0x563556764f70 2024-03-11T18:37:06.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.236+0000 7f022ffff640 1 -- 172.21.15.90:0/291330586 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02400b05a0 con 0x563556764f70 2024-03-11T18:37:06.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.236+0000 7f022ffff640 1 -- 172.21.15.90:0/291330586 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02400b0930 con 0x563556764f70 2024-03-11T18:37:06.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.236+0000 7f02579e2000 1 -- 172.21.15.90:0/291330586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563556764f70 msgr2=0x56355672b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:06.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.236+0000 7f02579e2000 1 --2- 172.21.15.90:0/291330586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563556764f70 0x56355672b350 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f024001d3a0 tx=0x7f024001d530 comp rx=0 tx=0).stop 2024-03-11T18:37:06.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.236+0000 7f02579e2000 1 -- 172.21.15.90:0/291330586 shutdown_connections 2024-03-11T18:37:06.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.236+0000 7f02579e2000 1 --2- 172.21.15.90:0/291330586 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x563556766bc0 0x563556766fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.236+0000 7f02579e2000 1 --2- 172.21.15.90:0/291330586 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56355672b890 0x563556766680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.236+0000 7f02579e2000 1 --2- 172.21.15.90:0/291330586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563556764f70 0x56355672b350 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.236+0000 7f02579e2000 1 -- 172.21.15.90:0/291330586 >> 172.21.15.90:0/291330586 conn(0x56355672bff0 msgr2=0x563556764440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:37:06.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.236+0000 7f02579e2000 1 -- 172.21.15.90:0/291330586 shutdown_connections 2024-03-11T18:37:06.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.236+0000 7f02579e2000 1 -- 172.21.15.90:0/291330586 wait complete. 2024-03-11T18:37:06.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.236+0000 7f02579e2000 1 -- start start 2024-03-11T18:37:06.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.236+0000 7f02579e2000 1 Processor -- start 2024-03-11T18:37:06.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.240+0000 7f02579e2000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56355672b890 0x563556771b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:06.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.240+0000 7f02579e2000 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x563556764f70 0x563556772060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:06.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.240+0000 7f02579e2000 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563556766bc0 0x56355672a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:06.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.240+0000 7f02579e2000 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x563556578140 con 0x563556766bc0 2024-03-11T18:37:06.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.240+0000 7f02579e2000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x563556512630 con 0x56355672b890 2024-03-11T18:37:06.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.240+0000 7f02579e2000 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x5635565122d0 con 0x563556764f70 2024-03-11T18:37:06.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.240+0000 7f02579e2000 10 client.0 _ll_register_callbacks cb 0x5635566281c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T18:37:06.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.240+0000 7f024dd9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563556766bc0 0x56355672a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:06.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.240+0000 7f024dd9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563556766bc0 0x56355672a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44518/0 (socket says 172.21.15.90:44518) 2024-03-11T18:37:06.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.240+0000 7f024dd9b640 1 -- 172.21.15.90:0/2186454169 learned_addr learned my addr 172.21.15.90:0/2186454169 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:37:06.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.244+0000 7f024dd9b640 1 -- 172.21.15.90:0/2186454169 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x563556764f70 msgr2=0x563556772060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:06.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.244+0000 7f024dd9b640 1 --2- 172.21.15.90:0/2186454169 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x563556764f70 0x563556772060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.244+0000 7f024dd9b640 1 -- 172.21.15.90:0/2186454169 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56355672b890 msgr2=0x563556771b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:06.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.244+0000 7f024dd9b640 1 --2- 172.21.15.90:0/2186454169 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56355672b890 0x563556771b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.244+0000 7f024dd9b640 1 -- 172.21.15.90:0/2186454169 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f025006b480 con 0x563556766bc0 2024-03-11T18:37:06.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.244+0000 7f024dd9b640 1 --2- 172.21.15.90:0/2186454169 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563556766bc0 0x56355672a720 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f024001f590 tx=0x7f024001ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6ad3916524b7a84 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:37:06.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.244+0000 7f022d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-03-11T18:37:06.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.260+0000 7f022d7fa640 1 -- 172.21.15.90:0/2186454169 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0240004220 con 0x563556766bc0 2024-03-11T18:37:06.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.260+0000 7f02579e2000 1 -- 172.21.15.90:0/2186454169 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56355652b850 con 0x563556766bc0 2024-03-11T18:37:06.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.260+0000 7f022d7fa640 1 -- 172.21.15.90:0/2186454169 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f0240004380 con 0x563556766bc0 2024-03-11T18:37:06.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.264+0000 7f022d7fa640 1 -- 172.21.15.90:0/2186454169 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0240003620 con 0x563556766bc0 2024-03-11T18:37:06.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.264+0000 7f02579e2000 10 client.4704 fetch_fsmap learned FSMap version 19 2024-03-11T18:37:06.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.264+0000 7f02579e2000 1 -- 172.21.15.90:0/2186454169 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) v3 -- 0x563556537e70 con 0x563556766bc0 2024-03-11T18:37:06.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.268+0000 7f022d7fa640 1 -- 172.21.15.90:0/2186454169 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56355652b850 con 0x563556766bc0 2024-03-11T18:37:06.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.268+0000 7f022d7fa640 1 -- 172.21.15.90:0/2186454169 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 19) v1 ==== 4500+0+0 (secure 0 0 0) 0x7f02400c4930 con 0x563556766bc0 2024-03-11T18:37:06.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.268+0000 7f02579e2000 10 client.4704 fetch_fsmap finished waiting for FSMap version 19 2024-03-11T18:37:06.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.268+0000 7f02579e2000 10 client.4704 resolve_mds: resolved name 'f' to daemon mds.f 2024-03-11T18:37:06.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.268+0000 7f02579e2000 20 client.4704 populate_metadata read hostname 'smithi090' 2024-03-11T18:37:06.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.268+0000 7f02579e2000 1 --2- 172.21.15.90:0/2186454169 >> [v2:172.21.15.92:6836/3209534444,v1:172.21.15.92:6837/3209534444] conn(0x56355676fa20 0x56355676fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:06.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.268+0000 7f02579e2000 4 client.4704 mds_command: new command op to 4253 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T18:37:06.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.268+0000 7f02579e2000 1 -- 172.21.15.90:0/2186454169 --> [v2:172.21.15.92:6836/3209534444,v1:172.21.15.92:6837/3209534444] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563556574f20 con 0x56355676fa20 2024-03-11T18:37:06.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.272+0000 7f024e59c640 1 --2- 172.21.15.90:0/2186454169 >> [v2:172.21.15.92:6836/3209534444,v1:172.21.15.92:6837/3209534444] conn(0x56355676fa20 0x56355676fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:06.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.280+0000 7f023effd640 1 -- 172.21.15.90:0/394930663 <== mon.2 v2:172.21.15.92:3301/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f0248005580 con 0x7f02501540c0 2024-03-11T18:37:06.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.280+0000 7f023effd640 1 --2- 172.21.15.90:0/394930663 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0230066e00 0x7f02300692c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:06.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.280+0000 7f024dd9b640 1 --2- 172.21.15.90:0/394930663 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0230066e00 0x7f02300692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:06.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.280+0000 7f024dd9b640 1 --2- 172.21.15.90:0/394930663 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0230066e00 0x7f02300692c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f02400090b0 tx=0x7f02400cf040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:37:06.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.372+0000 7f024e59c640 1 --2- 172.21.15.90:0/2186454169 >> [v2:172.21.15.92:6836/3209534444,v1:172.21.15.92:6837/3209534444] conn(0x56355676fa20 0x56355676fe00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=39596e8657698481 server_cookie=75f28f15b7575f97 in_seq=0 out_seq=0 2024-03-11T18:37:06.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.372+0000 7f022d7fa640 10 client.4704 ms_handle_connect on v2:172.21.15.92:6836/3209534444 2024-03-11T18:37:06.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.392+0000 7f023effd640 1 -- 172.21.15.90:0/394930663 <== mon.2 v2:172.21.15.92:3301/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f0248055560 con 0x7f02501540c0 2024-03-11T18:37:06.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.564+0000 7f022d7fa640 1 -- 172.21.15.90:0/2186454169 <== mds.1 v2:172.21.15.92:6836/3209534444 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563556574f20 con 0x56355676fa20 2024-03-11T18:37:06.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.564+0000 7f022d7fa640 10 client.4704 handle_command_reply: tid=0 2024-03-11T18:37:06.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.568+0000 7f02579e2000 1 client.4704 shutdown 2024-03-11T18:37:06.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.568+0000 7f02579e2000 1 -- 172.21.15.90:0/2186454169 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563556766bc0 msgr2=0x56355672a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:06.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.568+0000 7f02579e2000 1 --2- 172.21.15.90:0/2186454169 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563556766bc0 0x56355672a720 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f024001f590 tx=0x7f024001ba90 comp rx=0 tx=0).stop 2024-03-11T18:37:06.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.568+0000 7f02579e2000 1 -- 172.21.15.90:0/2186454169 shutdown_connections 2024-03-11T18:37:06.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.568+0000 7f02579e2000 1 --2- 172.21.15.90:0/2186454169 >> [v2:172.21.15.92:6836/3209534444,v1:172.21.15.92:6837/3209534444] conn(0x56355676fa20 0x56355676fe00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.568+0000 7f02579e2000 1 --2- 172.21.15.90:0/2186454169 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563556766bc0 0x56355672a720 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.568+0000 7f02579e2000 1 --2- 172.21.15.90:0/2186454169 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x563556764f70 0x563556772060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.568+0000 7f02579e2000 1 --2- 172.21.15.90:0/2186454169 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56355672b890 0x563556771b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.568+0000 7f022d7fa640 0 client.4704 ms_handle_reset on v2:172.21.15.92:6836/3209534444 2024-03-11T18:37:06.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.568+0000 7f02579e2000 1 -- 172.21.15.90:0/2186454169 >> 172.21.15.90:0/2186454169 conn(0x56355672bff0 msgr2=0x563556764020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:37:06.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.568+0000 7f02579e2000 1 -- 172.21.15.90:0/2186454169 shutdown_connections 2024-03-11T18:37:06.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.568+0000 7f02579e2000 1 -- 172.21.15.90:0/2186454169 wait complete. 2024-03-11T18:37:06.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.568+0000 7f02579e2000 20 client.4704 trim_cache size 0 max 16384 2024-03-11T18:37:06.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.596+0000 7f02579e2000 1 Processor -- start 2024-03-11T18:37:06.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.596+0000 7f02579e2000 1 -- start start 2024-03-11T18:37:06.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.596+0000 7f02579e2000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56355672b890 0x563556774b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:06.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.596+0000 7f02579e2000 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563556764f70 0x563556775080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:06.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.596+0000 7f02579e2000 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x563556766bc0 0x5635567755c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:06.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.596+0000 7f02579e2000 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x56355622f940 con 0x563556764f70 2024-03-11T18:37:06.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.596+0000 7f02579e2000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x563556533050 con 0x56355672b890 2024-03-11T18:37:06.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.596+0000 7f02579e2000 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x5635565b81a0 con 0x563556766bc0 2024-03-11T18:37:06.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.600+0000 7f024dd9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563556764f70 0x563556775080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:06.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.600+0000 7f024dd9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563556764f70 0x563556775080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44524/0 (socket says 172.21.15.90:44524) 2024-03-11T18:37:06.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.600+0000 7f024dd9b640 1 -- 172.21.15.90:0/2739900105 learned_addr learned my addr 172.21.15.90:0/2739900105 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:37:06.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.600+0000 7f024e59c640 1 --2- 172.21.15.90:0/2739900105 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56355672b890 0x563556774b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:06.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.604+0000 7f024dd9b640 1 -- 172.21.15.90:0/2739900105 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x563556766bc0 msgr2=0x5635567755c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:06.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.604+0000 7f024dd9b640 1 --2- 172.21.15.90:0/2739900105 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x563556766bc0 0x5635567755c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.604+0000 7f024dd9b640 1 -- 172.21.15.90:0/2739900105 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56355672b890 msgr2=0x563556774b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:06.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.604+0000 7f024dd9b640 1 --2- 172.21.15.90:0/2739900105 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56355672b890 0x563556774b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.604+0000 7f024dd9b640 1 -- 172.21.15.90:0/2739900105 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56355651de10 con 0x563556764f70 2024-03-11T18:37:06.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.604+0000 7f024dd9b640 1 --2- 172.21.15.90:0/2739900105 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563556764f70 0x563556775080 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f024009b4d0 tx=0x7f024000b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd0f91c9be1da393 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:37:06.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.612+0000 7f022f7fe640 1 -- 172.21.15.90:0/2739900105 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02400b9370 con 0x563556764f70 2024-03-11T18:37:06.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.612+0000 7f022f7fe640 1 -- 172.21.15.90:0/2739900105 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02400b0440 con 0x563556764f70 2024-03-11T18:37:06.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.612+0000 7f022f7fe640 1 -- 172.21.15.90:0/2739900105 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02400cc4d0 con 0x563556764f70 2024-03-11T18:37:06.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.612+0000 7f02579e2000 1 -- 172.21.15.90:0/2739900105 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563556764f70 msgr2=0x563556775080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:06.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.612+0000 7f02579e2000 1 --2- 172.21.15.90:0/2739900105 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563556764f70 0x563556775080 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f024009b4d0 tx=0x7f024000b050 comp rx=0 tx=0).stop 2024-03-11T18:37:06.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.612+0000 7f02579e2000 1 -- 172.21.15.90:0/2739900105 shutdown_connections 2024-03-11T18:37:06.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.612+0000 7f02579e2000 1 --2- 172.21.15.90:0/2739900105 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x563556766bc0 0x5635567755c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.612+0000 7f02579e2000 1 --2- 172.21.15.90:0/2739900105 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563556764f70 0x563556775080 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.612+0000 7f02579e2000 1 --2- 172.21.15.90:0/2739900105 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56355672b890 0x563556774b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.612+0000 7f02579e2000 1 -- 172.21.15.90:0/2739900105 >> 172.21.15.90:0/2739900105 conn(0x56355672bff0 msgr2=0x56355676ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:37:06.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.612+0000 7f02579e2000 1 -- 172.21.15.90:0/2739900105 shutdown_connections 2024-03-11T18:37:06.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.612+0000 7f02579e2000 1 -- 172.21.15.90:0/2739900105 wait complete. 2024-03-11T18:37:06.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.616+0000 7f02579e2000 1 -- start start 2024-03-11T18:37:06.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.616+0000 7f02579e2000 1 Processor -- start 2024-03-11T18:37:06.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.616+0000 7f02579e2000 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56355672b890 0x56355678c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:06.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.616+0000 7f02579e2000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563556764f70 0x56355678cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:06.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.616+0000 7f02579e2000 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x563556766bc0 0x5635567cbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:06.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.616+0000 7f02579e2000 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x563556575950 con 0x56355672b890 2024-03-11T18:37:06.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.616+0000 7f02579e2000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x56355652bc60 con 0x563556764f70 2024-03-11T18:37:06.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.616+0000 7f02579e2000 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x563556575540 con 0x563556766bc0 2024-03-11T18:37:06.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.616+0000 7f02579e2000 10 client.0 _ll_register_callbacks cb 0x56355657a2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T18:37:06.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.616+0000 7f024dd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563556764f70 0x56355678cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:06.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.620+0000 7f024dd9b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563556764f70 0x56355678cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.90:49746/0 (socket says 172.21.15.90:49746) 2024-03-11T18:37:06.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.620+0000 7f024dd9b640 1 -- 172.21.15.90:0/3432342616 learned_addr learned my addr 172.21.15.90:0/3432342616 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:37:06.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.624+0000 7f024ed9d640 1 --2- 172.21.15.90:0/3432342616 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x563556766bc0 0x5635567cbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:06.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.628+0000 7f024dd9b640 1 -- 172.21.15.90:0/3432342616 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x563556766bc0 msgr2=0x5635567cbac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:06.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.628+0000 7f024dd9b640 1 --2- 172.21.15.90:0/3432342616 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x563556766bc0 0x5635567cbac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.628+0000 7f024dd9b640 1 -- 172.21.15.90:0/3432342616 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56355672b890 msgr2=0x56355678c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:06.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.628+0000 7f024dd9b640 1 --2- 172.21.15.90:0/3432342616 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56355672b890 0x56355678c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.628+0000 7f024dd9b640 1 -- 172.21.15.90:0/3432342616 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563556537e70 con 0x563556764f70 2024-03-11T18:37:06.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.636+0000 7f024dd9b640 1 --2- 172.21.15.90:0/3432342616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563556764f70 0x56355678cdf0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f024001e760 tx=0x7f02400b7000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9980d132ca4bfabe server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:37:06.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.636+0000 7f022d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.92:3300/0 2024-03-11T18:37:06.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.648+0000 7f022d7fa640 1 -- 172.21.15.90:0/3432342616 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02400e4070 con 0x563556764f70 2024-03-11T18:37:06.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.648+0000 7f022d7fa640 1 -- 172.21.15.90:0/3432342616 <== mon.1 v2:172.21.15.92:3300/0 2 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f0240003620 con 0x563556764f70 2024-03-11T18:37:06.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.648+0000 7f02579e2000 1 -- 172.21.15.90:0/3432342616 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563556540fa0 con 0x563556764f70 2024-03-11T18:37:06.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.652+0000 7f022d7fa640 1 -- 172.21.15.90:0/3432342616 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02400b9370 con 0x563556764f70 2024-03-11T18:37:06.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.652+0000 7f02579e2000 10 client.4648 fetch_fsmap learned FSMap version 19 2024-03-11T18:37:06.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.652+0000 7f02579e2000 1 -- 172.21.15.90:0/3432342616 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) v3 -- 0x56355626ecc0 con 0x563556764f70 2024-03-11T18:37:06.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.688+0000 7f022d7fa640 1 -- 172.21.15.90:0/3432342616 <== mon.1 v2:172.21.15.92:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563556540fa0 con 0x563556764f70 2024-03-11T18:37:06.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.688+0000 7f022d7fa640 1 -- 172.21.15.90:0/3432342616 <== mon.1 v2:172.21.15.92:3300/0 5 ==== fsmap(e 19) v1 ==== 4500+0+0 (secure 0 0 0) 0x7f02400cc4d0 con 0x563556764f70 2024-03-11T18:37:06.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.688+0000 7f02579e2000 10 client.4648 fetch_fsmap finished waiting for FSMap version 19 2024-03-11T18:37:06.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.688+0000 7f02579e2000 10 client.4648 resolve_mds: resolved name 'f' to daemon mds.f 2024-03-11T18:37:06.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.688+0000 7f02579e2000 20 client.4648 populate_metadata read hostname 'smithi090' 2024-03-11T18:37:06.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.688+0000 7f02579e2000 1 --2- 172.21.15.90:0/3432342616 >> [v2:172.21.15.92:6836/3209534444,v1:172.21.15.92:6837/3209534444] conn(0x5635567cc000 0x5635567cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:06.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.688+0000 7f02579e2000 4 client.4648 mds_command: new command op to 4253 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-11T18:37:06.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.688+0000 7f02579e2000 1 -- 172.21.15.90:0/3432342616 --> [v2:172.21.15.92:6836/3209534444,v1:172.21.15.92:6837/3209534444] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563556570b60 con 0x5635567cc000 2024-03-11T18:37:06.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.692+0000 7f024e59c640 1 --2- 172.21.15.90:0/3432342616 >> [v2:172.21.15.92:6836/3209534444,v1:172.21.15.92:6837/3209534444] conn(0x5635567cc000 0x5635567cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:06.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.700+0000 7f024e59c640 1 --2- 172.21.15.90:0/3432342616 >> [v2:172.21.15.92:6836/3209534444,v1:172.21.15.92:6837/3209534444] conn(0x5635567cc000 0x5635567cc3e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6bb95f3d7f53b705 server_cookie=edfe59e27388a93b in_seq=0 out_seq=0 2024-03-11T18:37:06.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.700+0000 7f022d7fa640 10 client.4648 ms_handle_connect on v2:172.21.15.92:6836/3209534444 2024-03-11T18:37:06.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.724+0000 7f022d7fa640 1 -- 172.21.15.90:0/3432342616 <== mds.1 v2:172.21.15.92:6836/3209534444 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x563556570b60 con 0x5635567cc000 2024-03-11T18:37:06.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.724+0000 7f022d7fa640 10 client.4648 handle_command_reply: tid=0 2024-03-11T18:37:06.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.724+0000 7f02579e2000 1 client.4648 shutdown 2024-03-11T18:37:06.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.724+0000 7f02579e2000 1 -- 172.21.15.90:0/3432342616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563556764f70 msgr2=0x56355678cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:06.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.724+0000 7f02579e2000 1 --2- 172.21.15.90:0/3432342616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563556764f70 0x56355678cdf0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f024001e760 tx=0x7f02400b7000 comp rx=0 tx=0).stop 2024-03-11T18:37:06.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.724+0000 7f02579e2000 1 -- 172.21.15.90:0/3432342616 shutdown_connections 2024-03-11T18:37:06.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.724+0000 7f02579e2000 1 --2- 172.21.15.90:0/3432342616 >> [v2:172.21.15.92:6836/3209534444,v1:172.21.15.92:6837/3209534444] conn(0x5635567cc000 0x5635567cc3e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.724+0000 7f02579e2000 1 --2- 172.21.15.90:0/3432342616 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x563556766bc0 0x5635567cbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.724+0000 7f022d7fa640 0 client.4648 ms_handle_reset on v2:172.21.15.92:6836/3209534444 2024-03-11T18:37:06.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.724+0000 7f02579e2000 1 --2- 172.21.15.90:0/3432342616 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x563556764f70 0x56355678cdf0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.724+0000 7f02579e2000 1 --2- 172.21.15.90:0/3432342616 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56355672b890 0x56355678c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.724+0000 7f02579e2000 1 -- 172.21.15.90:0/3432342616 >> 172.21.15.90:0/3432342616 conn(0x56355672bff0 msgr2=0x563556766fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:37:06.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.724+0000 7f02579e2000 1 -- 172.21.15.90:0/3432342616 shutdown_connections 2024-03-11T18:37:06.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.728+0000 7f02579e2000 1 -- 172.21.15.90:0/3432342616 wait complete. 2024-03-11T18:37:06.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.728+0000 7f02579e2000 20 client.4648 trim_cache size 0 max 16384 2024-03-11T18:37:06.734 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-03-11T18:37:06.734 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "76d56b10-6c29-476a-804b-8482ea90f9e2", 2024-03-11T18:37:06.734 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 1, 2024-03-11T18:37:06.734 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 4253, 2024-03-11T18:37:06.734 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-03-11T18:37:06.735 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-03-11T18:37:06.735 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-03-11T18:37:06.735 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 30.166009280000001, 2024-03-11T18:37:06.735 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 19, 2024-03-11T18:37:06.735 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 30, 2024-03-11T18:37:06.735 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 28, 2024-03-11T18:37:06.735 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 83.269705778000002 2024-03-11T18:37:06.735 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-03-11T18:37:06.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.728+0000 7f0254879640 1 -- 172.21.15.90:0/394930663 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0230066e00 msgr2=0x7f02300692c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:06.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.728+0000 7f0254879640 1 --2- 172.21.15.90:0/394930663 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0230066e00 0x7f02300692c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f02400090b0 tx=0x7f02400cf040 comp rx=0 tx=0).stop 2024-03-11T18:37:06.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.728+0000 7f0254879640 1 -- 172.21.15.90:0/394930663 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f02501540c0 msgr2=0x7f02501f3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:06.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.728+0000 7f0254879640 1 --2- 172.21.15.90:0/394930663 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f02501540c0 0x7f02501f3e20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f024800c7b0 tx=0x7f024800cc70 comp rx=0 tx=0).stop 2024-03-11T18:37:06.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.728+0000 7f0254879640 1 -- 172.21.15.90:0/394930663 shutdown_connections 2024-03-11T18:37:06.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.728+0000 7f0254879640 1 --2- 172.21.15.90:0/394930663 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0230066e00 0x7f02300692c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.728+0000 7f0254879640 1 --2- 172.21.15.90:0/394930663 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f02501540c0 0x7f02501f3e20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.728+0000 7f0254879640 1 --2- 172.21.15.90:0/394930663 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f02500cc280 0x7f025014d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.728+0000 7f0254879640 1 --2- 172.21.15.90:0/394930663 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f025006f400 0x7f025014ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.728+0000 7f0254879640 1 -- 172.21.15.90:0/394930663 >> 172.21.15.90:0/394930663 conn(0x7f0250071aa0 msgr2=0x7f02500c69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:37:06.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.728+0000 7f0254879640 1 -- 172.21.15.90:0/394930663 shutdown_connections 2024-03-11T18:37:06.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.728+0000 7f0254879640 1 -- 172.21.15.90:0/394930663 wait complete. 2024-03-11T18:37:06.759 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-03-11T18:37:06.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.876+0000 7fc07a239640 1 Processor -- start 2024-03-11T18:37:06.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.876+0000 7fc07a239640 1 -- start start 2024-03-11T18:37:06.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.880+0000 7fc07a239640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc07406e210 0x7fc074157c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:06.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.880+0000 7fc07a239640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc07406f430 con 0x7fc074070b00 2024-03-11T18:37:06.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.880+0000 7fc07a239640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc07406e820 con 0x7fc074151b30 2024-03-11T18:37:06.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.880+0000 7fc07a239640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0740c65b0 con 0x7fc07406e210 2024-03-11T18:37:06.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.880+0000 7fc072ffd640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7fc074070b00 0x7fc0740c4af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:60188/0 (socket says 172.21.15.90:60188) 2024-03-11T18:37:06.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.880+0000 7fc072ffd640 1 -- 172.21.15.90:0/2346688619 learned_addr learned my addr 172.21.15.90:0/2346688619 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T18:37:06.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.880+0000 7fc073fff640 1 --2- 172.21.15.90:0/2346688619 >> v2:172.21.15.92:3300/0 conn(0x7fc07406e210 0x7fc074157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:06.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.880+0000 7fc0727fc640 1 -- 172.21.15.90:0/2346688619 <== mon.0 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3033858616 0 0) 0x7fc07406f430 con 0x7fc074070b00 2024-03-11T18:37:06.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.880+0000 7fc0727fc640 1 -- 172.21.15.90:0/2346688619 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc05c003530 con 0x7fc074070b00 2024-03-11T18:37:06.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.884+0000 7fc0727fc640 1 -- 172.21.15.90:0/2346688619 <== mon.0 v1:172.21.15.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3760688170 0 0) 0x7fc05c003530 con 0x7fc074070b00 2024-03-11T18:37:06.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.884+0000 7fc0727fc640 1 -- 172.21.15.90:0/2346688619 >> v2:172.21.15.92:3300/0 conn(0x7fc07406e210 msgr2=0x7fc074157c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:06.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.884+0000 7fc0727fc640 1 --2- 172.21.15.90:0/2346688619 >> v2:172.21.15.92:3300/0 conn(0x7fc07406e210 0x7fc074157c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.884+0000 7fc0727fc640 1 -- 172.21.15.90:0/2346688619 >> v1:172.21.15.92:6789/0 conn(0x7fc074151b30 legacy=0x7fc0740c5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:06.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.884+0000 7fc0727fc640 1 -- 172.21.15.90:0/2346688619 --> v1:172.21.15.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc07406eb00 con 0x7fc074070b00 2024-03-11T18:37:06.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.884+0000 7fc0727fc640 1 -- 172.21.15.90:0/2346688619 <== mon.0 v1:172.21.15.90:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 691735435 0 0) 0x7fc068001a00 con 0x7fc074070b00 2024-03-11T18:37:06.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.884+0000 7fc0727fc640 1 -- 172.21.15.90:0/2346688619 >> v1:172.21.15.90:6789/0 conn(0x7fc074070b00 legacy=0x7fc0740c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:06.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.884+0000 7fc0727fc640 1 --2- 172.21.15.90:0/2346688619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc05c003980 0x7fc05c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:06.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.884+0000 7fc0727fc640 1 --2- 172.21.15.90:0/2346688619 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc05c0047d0 0x7fc05c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:06.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.884+0000 7fc0727fc640 1 --2- 172.21.15.90:0/2346688619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc05c005190 0x7fc05c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:06.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.884+0000 7fc0727fc640 1 -- 172.21.15.90:0/2346688619 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05c00c000 con 0x7fc05c005190 2024-03-11T18:37:06.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.884+0000 7fc0727fc640 1 -- 172.21.15.90:0/2346688619 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05c00c170 con 0x7fc05c003980 2024-03-11T18:37:06.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.884+0000 7fc0727fc640 1 -- 172.21.15.90:0/2346688619 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc05c00c330 con 0x7fc05c0047d0 2024-03-11T18:37:06.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.888+0000 7fc073fff640 1 --2- 172.21.15.90:0/2346688619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc05c005190 0x7fc05c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:06.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.888+0000 7fc072ffd640 1 --2- 172.21.15.90:0/2346688619 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc05c0047d0 0x7fc05c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:06.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.892+0000 7fc073fff640 1 -- 172.21.15.90:0/2346688619 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc05c0047d0 msgr2=0x7fc05c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:06.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.892+0000 7fc073fff640 1 --2- 172.21.15.90:0/2346688619 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc05c0047d0 0x7fc05c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.892+0000 7fc073fff640 1 -- 172.21.15.90:0/2346688619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc05c003980 msgr2=0x7fc05c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:06.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.892+0000 7fc073fff640 1 --2- 172.21.15.90:0/2346688619 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc05c003980 0x7fc05c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.892+0000 7fc073fff640 1 -- 172.21.15.90:0/2346688619 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc07406eb00 con 0x7fc05c005190 2024-03-11T18:37:06.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.892+0000 7fc073fff640 1 -- 172.21.15.90:0/2346688619 reap_dead start 2024-03-11T18:37:06.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.892+0000 7fc073fff640 1 --2- 172.21.15.90:0/2346688619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc05c005190 0x7fc05c00b990 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc06c009ae0 tx=0x7fc06c01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c99c61af4d373cb server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:37:06.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.896+0000 7fc0727fc640 1 -- 172.21.15.90:0/2346688619 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06c004020 con 0x7fc05c005190 2024-03-11T18:37:06.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.896+0000 7fc0727fc640 1 -- 172.21.15.90:0/2346688619 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc06c0041c0 con 0x7fc05c005190 2024-03-11T18:37:06.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.896+0000 7fc0727fc640 1 -- 172.21.15.90:0/2346688619 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06c0a56a0 con 0x7fc05c005190 2024-03-11T18:37:06.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.900+0000 7fc07a239640 1 -- 172.21.15.90:0/2346688619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc05c005190 msgr2=0x7fc05c00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:06.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.900+0000 7fc07a239640 1 --2- 172.21.15.90:0/2346688619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc05c005190 0x7fc05c00b990 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc06c009ae0 tx=0x7fc06c01cab0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.900+0000 7fc07a239640 1 -- 172.21.15.90:0/2346688619 shutdown_connections 2024-03-11T18:37:06.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.900+0000 7fc07a239640 1 --2- 172.21.15.90:0/2346688619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc05c005190 0x7fc05c00b990 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.900+0000 7fc07a239640 1 -- 172.21.15.90:0/2346688619 >> 172.21.15.90:0/2346688619 conn(0x7fc07406ac00 msgr2=0x7fc07406b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:37:06.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.900+0000 7fc07a239640 1 -- 172.21.15.90:0/2346688619 shutdown_connections 2024-03-11T18:37:06.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.900+0000 7fc07a239640 1 -- 172.21.15.90:0/2346688619 wait complete. 2024-03-11T18:37:06.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.900+0000 7fc07a239640 1 Processor -- start 2024-03-11T18:37:06.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.900+0000 7fc07a239640 1 -- start start 2024-03-11T18:37:06.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.900+0000 7fc07a239640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc05c005190 0x7fc07413fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:06.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.900+0000 7fc07a239640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0741443b0 0x7fc074145b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:06.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.900+0000 7fc07a239640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc074157310 0x7fc074148ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:06.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.900+0000 7fc07a239640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc074145500 con 0x7fc0741443b0 2024-03-11T18:37:06.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.900+0000 7fc07a239640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc074152980 con 0x7fc05c005190 2024-03-11T18:37:06.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.900+0000 7fc07a239640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0741eceb0 con 0x7fc074157310 2024-03-11T18:37:06.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.900+0000 7fc072ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0741443b0 0x7fc074145b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:06.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.900+0000 7fc072ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0741443b0 0x7fc074145b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44550/0 (socket says 172.21.15.90:44550) 2024-03-11T18:37:06.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.900+0000 7fc072ffd640 1 -- 172.21.15.90:0/3258335372 learned_addr learned my addr 172.21.15.90:0/3258335372 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:37:06.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.904+0000 7fc073fff640 1 --2- 172.21.15.90:0/3258335372 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc074157310 0x7fc074148ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:06.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.904+0000 7fc072ffd640 1 -- 172.21.15.90:0/3258335372 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc074157310 msgr2=0x7fc074148ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:06.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.904+0000 7fc072ffd640 1 --2- 172.21.15.90:0/3258335372 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc074157310 0x7fc074148ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.904+0000 7fc072ffd640 1 -- 172.21.15.90:0/3258335372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc05c005190 msgr2=0x7fc07413fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:06.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.904+0000 7fc072ffd640 1 --2- 172.21.15.90:0/3258335372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc05c005190 0x7fc07413fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.904+0000 7fc072ffd640 1 -- 172.21.15.90:0/3258335372 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0741ede80 con 0x7fc0741443b0 2024-03-11T18:37:06.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.908+0000 7fc072ffd640 1 --2- 172.21.15.90:0/3258335372 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0741443b0 0x7fc074145b50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc0680079a0 tx=0x7fc06800a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:37:06.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.912+0000 7fc05bfff640 1 -- 172.21.15.90:0/3258335372 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0680133f0 con 0x7fc0741443b0 2024-03-11T18:37:06.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.912+0000 7fc05bfff640 1 -- 172.21.15.90:0/3258335372 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc068013570 con 0x7fc0741443b0 2024-03-11T18:37:06.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.912+0000 7fc05bfff640 1 -- 172.21.15.90:0/3258335372 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc068013920 con 0x7fc0741443b0 2024-03-11T18:37:06.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.912+0000 7fc07a239640 1 -- 172.21.15.90:0/3258335372 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0741ed9f0 con 0x7fc0741443b0 2024-03-11T18:37:06.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.912+0000 7fc07a239640 1 -- 172.21.15.90:0/3258335372 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0741ed0c0 con 0x7fc0741443b0 2024-03-11T18:37:06.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.916+0000 7fc07d3a2000 1 Processor -- start 2024-03-11T18:37:06.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.916+0000 7fc07d3a2000 1 -- start start 2024-03-11T18:37:06.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.916+0000 7fc07d3a2000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562948097f70 0x56294805e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:06.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.916+0000 7fc07d3a2000 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x56294805e890 0x562948099680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:06.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.916+0000 7fc07d3a2000 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562948099bc0 0x562948099fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:06.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.916+0000 7fc07d3a2000 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x562947f20d00 con 0x562948099bc0 2024-03-11T18:37:06.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.916+0000 7fc07d3a2000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x562947eb2430 con 0x562948097f70 2024-03-11T18:37:06.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.916+0000 7fc07d3a2000 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x562947eaf960 con 0x56294805e890 2024-03-11T18:37:06.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.916+0000 7fc073fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562948097f70 0x56294805e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:06.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.916+0000 7fc073fff640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562948097f70 0x56294805e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.90:49786/0 (socket says 172.21.15.90:49786) 2024-03-11T18:37:06.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.916+0000 7fc073fff640 1 -- 172.21.15.90:0/1510975082 learned_addr learned my addr 172.21.15.90:0/1510975082 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:37:06.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.920+0000 7fc0737fe640 1 --2- 172.21.15.90:0/1510975082 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x56294805e890 0x562948099680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:06.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.924+0000 7fc073fff640 1 -- 172.21.15.90:0/1510975082 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x56294805e890 msgr2=0x562948099680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:06.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.924+0000 7fc073fff640 1 --2- 172.21.15.90:0/1510975082 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x56294805e890 0x562948099680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.924+0000 7fc073fff640 1 -- 172.21.15.90:0/1510975082 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562948099bc0 msgr2=0x562948099fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:06.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.924+0000 7fc073fff640 1 --2- 172.21.15.90:0/1510975082 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562948099bc0 0x562948099fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.924+0000 7fc073fff640 1 -- 172.21.15.90:0/1510975082 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562947ba1890 con 0x562948097f70 2024-03-11T18:37:06.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.928+0000 7fc073fff640 1 --2- 172.21.15.90:0/1510975082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562948097f70 0x56294805e350 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc06c01d350 tx=0x7fc06c004740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5dc2deae524910b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:37:06.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.932+0000 7fc05bfff640 1 -- 172.21.15.90:0/3258335372 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fc068013a80 con 0x7fc0741443b0 2024-03-11T18:37:06.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.932+0000 7fc05bfff640 1 --2- 172.21.15.90:0/3258335372 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fc060066e40 0x7fc060069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:06.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.932+0000 7fc0737fe640 1 --2- 172.21.15.90:0/3258335372 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fc060066e40 0x7fc060069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:06.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.932+0000 7fc0737fe640 1 --2- 172.21.15.90:0/3258335372 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fc060066e40 0x7fc060069300 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc064007fd0 tx=0x7fc064009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:37:06.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.936+0000 7fc05bfff640 1 -- 172.21.15.90:0/3258335372 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fc0680568b0 con 0x7fc0741443b0 2024-03-11T18:37:06.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.940+0000 7fc0597fa640 1 -- 172.21.15.90:0/1510975082 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06c004530 con 0x562948097f70 2024-03-11T18:37:06.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.940+0000 7fc0597fa640 1 -- 172.21.15.90:0/1510975082 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc06c0b2050 con 0x562948097f70 2024-03-11T18:37:06.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.940+0000 7fc0597fa640 1 -- 172.21.15.90:0/1510975082 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06c0a58a0 con 0x562948097f70 2024-03-11T18:37:06.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.940+0000 7fc07d3a2000 1 -- 172.21.15.90:0/1510975082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562948097f70 msgr2=0x56294805e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:06.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.940+0000 7fc07d3a2000 1 --2- 172.21.15.90:0/1510975082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562948097f70 0x56294805e350 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc06c01d350 tx=0x7fc06c004740 comp rx=0 tx=0).stop 2024-03-11T18:37:06.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.940+0000 7fc07d3a2000 1 -- 172.21.15.90:0/1510975082 shutdown_connections 2024-03-11T18:37:06.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.940+0000 7fc07d3a2000 1 --2- 172.21.15.90:0/1510975082 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562948099bc0 0x562948099fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.940+0000 7fc07d3a2000 1 --2- 172.21.15.90:0/1510975082 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x56294805e890 0x562948099680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.940+0000 7fc07d3a2000 1 --2- 172.21.15.90:0/1510975082 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562948097f70 0x56294805e350 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.940+0000 7fc07d3a2000 1 -- 172.21.15.90:0/1510975082 >> 172.21.15.90:0/1510975082 conn(0x56294805eff0 msgr2=0x562948097440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:37:06.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.940+0000 7fc07d3a2000 1 -- 172.21.15.90:0/1510975082 shutdown_connections 2024-03-11T18:37:06.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.940+0000 7fc07d3a2000 1 -- 172.21.15.90:0/1510975082 wait complete. 2024-03-11T18:37:06.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.940+0000 7fc07d3a2000 1 -- start start 2024-03-11T18:37:06.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.940+0000 7fc07d3a2000 1 Processor -- start 2024-03-11T18:37:06.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.940+0000 7fc07d3a2000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56294805e890 0x5629480a4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:06.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.940+0000 7fc07d3a2000 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562948097f70 0x5629480a2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:06.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.940+0000 7fc07d3a2000 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x562948099bc0 0x5629480a2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:06.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.940+0000 7fc07d3a2000 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x562947eab140 con 0x562948097f70 2024-03-11T18:37:06.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.940+0000 7fc07d3a2000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x562947e45630 con 0x56294805e890 2024-03-11T18:37:06.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.940+0000 7fc07d3a2000 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x562947e452d0 con 0x562948099bc0 2024-03-11T18:37:06.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.940+0000 7fc07d3a2000 10 client.0 _ll_register_callbacks cb 0x562947f5b1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T18:37:06.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.940+0000 7fc072ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562948097f70 0x5629480a2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:06.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.940+0000 7fc072ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562948097f70 0x5629480a2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44572/0 (socket says 172.21.15.90:44572) 2024-03-11T18:37:06.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.940+0000 7fc072ffd640 1 -- 172.21.15.90:0/1950115908 learned_addr learned my addr 172.21.15.90:0/1950115908 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:37:06.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.944+0000 7fc073fff640 1 --2- 172.21.15.90:0/1950115908 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x562948099bc0 0x5629480a2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:06.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.944+0000 7fc0737fe640 1 --2- 172.21.15.90:0/1950115908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56294805e890 0x5629480a4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:06.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.944+0000 7fc072ffd640 1 -- 172.21.15.90:0/1950115908 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x562948099bc0 msgr2=0x5629480a2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:06.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.944+0000 7fc072ffd640 1 --2- 172.21.15.90:0/1950115908 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x562948099bc0 0x5629480a2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.944+0000 7fc072ffd640 1 -- 172.21.15.90:0/1950115908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56294805e890 msgr2=0x5629480a4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:06.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.944+0000 7fc072ffd640 1 --2- 172.21.15.90:0/1950115908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56294805e890 0x5629480a4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.944+0000 7fc072ffd640 1 -- 172.21.15.90:0/1950115908 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc07406eb00 con 0x562948097f70 2024-03-11T18:37:06.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.948+0000 7fc072ffd640 1 --2- 172.21.15.90:0/1950115908 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562948097f70 0x5629480a2810 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc06805e390 tx=0x7fc06805a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aa3680b317b0756 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:37:06.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.948+0000 7fc0527fc640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-03-11T18:37:06.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.952+0000 7fc0527fc640 1 -- 172.21.15.90:0/1950115908 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc068055800 con 0x562948097f70 2024-03-11T18:37:06.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.952+0000 7fc0527fc640 1 -- 172.21.15.90:0/1950115908 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fc06805c2a0 con 0x562948097f70 2024-03-11T18:37:06.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.952+0000 7fc07d3a2000 1 -- 172.21.15.90:0/1950115908 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562947e5e850 con 0x562948097f70 2024-03-11T18:37:06.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.952+0000 7fc0527fc640 1 -- 172.21.15.90:0/1950115908 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc068055960 con 0x562948097f70 2024-03-11T18:37:06.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.952+0000 7fc07d3a2000 10 client.4716 fetch_fsmap learned FSMap version 19 2024-03-11T18:37:06.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.952+0000 7fc07d3a2000 1 -- 172.21.15.90:0/1950115908 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) v3 -- 0x562947e6ae70 con 0x562948097f70 2024-03-11T18:37:06.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.956+0000 7fc0527fc640 1 -- 172.21.15.90:0/1950115908 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562947e5e850 con 0x562948097f70 2024-03-11T18:37:06.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.956+0000 7fc0527fc640 1 -- 172.21.15.90:0/1950115908 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 19) v1 ==== 4500+0+0 (secure 0 0 0) 0x7fc06806d030 con 0x562948097f70 2024-03-11T18:37:06.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.956+0000 7fc07d3a2000 10 client.4716 fetch_fsmap finished waiting for FSMap version 19 2024-03-11T18:37:06.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.956+0000 7fc07d3a2000 10 client.4716 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-11T18:37:06.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.956+0000 7fc07d3a2000 20 client.4716 populate_metadata read hostname 'smithi090' 2024-03-11T18:37:06.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.956+0000 7fc07d3a2000 1 --2- 172.21.15.90:0/1950115908 >> [v2:172.21.15.90:6836/3485943375,v1:172.21.15.90:6837/3485943375] conn(0x5629480938d0 0x562948093cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:06.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.956+0000 7fc07d3a2000 4 client.4716 mds_command: new command op to 4255 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-11T18:37:06.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.956+0000 7fc07d3a2000 1 -- 172.21.15.90:0/1950115908 --> [v2:172.21.15.90:6836/3485943375,v1:172.21.15.90:6837/3485943375] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562947ea7f20 con 0x5629480938d0 2024-03-11T18:37:06.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.960+0000 7fc0737fe640 1 --2- 172.21.15.90:0/1950115908 >> [v2:172.21.15.90:6836/3485943375,v1:172.21.15.90:6837/3485943375] conn(0x5629480938d0 0x562948093cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:06.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.960+0000 7fc0737fe640 1 --2- 172.21.15.90:0/1950115908 >> [v2:172.21.15.90:6836/3485943375,v1:172.21.15.90:6837/3485943375] conn(0x5629480938d0 0x562948093cb0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b00366259e437913 server_cookie=c43129975e81a461 in_seq=0 out_seq=0 2024-03-11T18:37:06.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.960+0000 7fc0527fc640 10 client.4716 ms_handle_connect on v2:172.21.15.90:6836/3485943375 2024-03-11T18:37:06.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.980+0000 7fc0527fc640 1 -- 172.21.15.90:0/1950115908 <== mds.0 v2:172.21.15.90:6836/3485943375 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562947ea7f20 con 0x5629480938d0 2024-03-11T18:37:06.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.980+0000 7fc0527fc640 10 client.4716 handle_command_reply: tid=0 2024-03-11T18:37:06.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.980+0000 7fc07d3a2000 1 client.4716 shutdown 2024-03-11T18:37:06.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.980+0000 7fc07d3a2000 1 -- 172.21.15.90:0/1950115908 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562948097f70 msgr2=0x5629480a2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:06.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.980+0000 7fc07d3a2000 1 --2- 172.21.15.90:0/1950115908 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562948097f70 0x5629480a2810 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc06805e390 tx=0x7fc06805a300 comp rx=0 tx=0).stop 2024-03-11T18:37:06.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.980+0000 7fc07d3a2000 1 -- 172.21.15.90:0/1950115908 shutdown_connections 2024-03-11T18:37:06.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.980+0000 7fc07d3a2000 1 --2- 172.21.15.90:0/1950115908 >> [v2:172.21.15.90:6836/3485943375,v1:172.21.15.90:6837/3485943375] conn(0x5629480938d0 0x562948093cb0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.980+0000 7fc07d3a2000 1 --2- 172.21.15.90:0/1950115908 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x562948099bc0 0x5629480a2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.980+0000 7fc07d3a2000 1 --2- 172.21.15.90:0/1950115908 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562948097f70 0x5629480a2810 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.980+0000 7fc07d3a2000 1 --2- 172.21.15.90:0/1950115908 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56294805e890 0x5629480a4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:06.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.980+0000 7fc07d3a2000 1 -- 172.21.15.90:0/1950115908 >> 172.21.15.90:0/1950115908 conn(0x56294805eff0 msgr2=0x562948097020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:37:06.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.980+0000 7fc0527fc640 0 client.4716 ms_handle_reset on v2:172.21.15.90:6836/3485943375 2024-03-11T18:37:06.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.980+0000 7fc07d3a2000 1 -- 172.21.15.90:0/1950115908 shutdown_connections 2024-03-11T18:37:06.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.980+0000 7fc07d3a2000 1 -- 172.21.15.90:0/1950115908 wait complete. 2024-03-11T18:37:06.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.980+0000 7fc07d3a2000 20 client.4716 trim_cache size 0 max 16384 2024-03-11T18:37:07.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.996+0000 7fc07d3a2000 1 Processor -- start 2024-03-11T18:37:07.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:06.996+0000 7fc07d3a2000 1 -- start start 2024-03-11T18:37:07.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.000+0000 7fc07d3a2000 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x56294805eff0 0x56294809a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:07.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.000+0000 7fc07d3a2000 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56294805e890 0x56294809abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:07.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.000+0000 7fc07d3a2000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562948097f70 0x56294805da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:07.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.000+0000 7fc07d3a2000 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x562947b62940 con 0x56294805e890 2024-03-11T18:37:07.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.000+0000 7fc07d3a2000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x562947eadca0 con 0x562948097f70 2024-03-11T18:37:07.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.000+0000 7fc07d3a2000 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x562947e2cb10 con 0x56294805eff0 2024-03-11T18:37:07.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.000+0000 7fc072ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56294805e890 0x56294809abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:07.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.000+0000 7fc072ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56294805e890 0x56294809abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44580/0 (socket says 172.21.15.90:44580) 2024-03-11T18:37:07.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.000+0000 7fc072ffd640 1 -- 172.21.15.90:0/3110346263 learned_addr learned my addr 172.21.15.90:0/3110346263 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:37:07.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.000+0000 7fc0737fe640 1 --2- 172.21.15.90:0/3110346263 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x56294805eff0 0x56294809a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:07.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.000+0000 7fc073fff640 1 --2- 172.21.15.90:0/3110346263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562948097f70 0x56294805da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:07.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.000+0000 7fc072ffd640 1 -- 172.21.15.90:0/3110346263 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x56294805eff0 msgr2=0x56294809a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:07.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.000+0000 7fc072ffd640 1 --2- 172.21.15.90:0/3110346263 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x56294805eff0 0x56294809a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:07.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.000+0000 7fc072ffd640 1 -- 172.21.15.90:0/3110346263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562948097f70 msgr2=0x56294805da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:07.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.000+0000 7fc072ffd640 1 --2- 172.21.15.90:0/3110346263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562948097f70 0x56294805da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:07.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.000+0000 7fc072ffd640 1 -- 172.21.15.90:0/3110346263 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562948094c70 con 0x56294805e890 2024-03-11T18:37:07.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.004+0000 7fc072ffd640 1 --2- 172.21.15.90:0/3110346263 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56294805e890 0x56294809abf0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc068074210 tx=0x7fc06805a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9de0e7f5b9e3b33 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:37:07.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.008+0000 7fc058ff9640 1 -- 172.21.15.90:0/3110346263 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc068001a70 con 0x56294805e890 2024-03-11T18:37:07.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.008+0000 7fc058ff9640 1 -- 172.21.15.90:0/3110346263 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc068074290 con 0x56294805e890 2024-03-11T18:37:07.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.008+0000 7fc058ff9640 1 -- 172.21.15.90:0/3110346263 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc068055800 con 0x56294805e890 2024-03-11T18:37:07.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.008+0000 7fc07d3a2000 1 -- 172.21.15.90:0/3110346263 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56294805e890 msgr2=0x56294809abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:07.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.008+0000 7fc07d3a2000 1 --2- 172.21.15.90:0/3110346263 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56294805e890 0x56294809abf0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc068074210 tx=0x7fc06805a000 comp rx=0 tx=0).stop 2024-03-11T18:37:07.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.008+0000 7fc07d3a2000 1 -- 172.21.15.90:0/3110346263 shutdown_connections 2024-03-11T18:37:07.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.008+0000 7fc07d3a2000 1 --2- 172.21.15.90:0/3110346263 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x562948097f70 0x56294805da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:07.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.008+0000 7fc07d3a2000 1 --2- 172.21.15.90:0/3110346263 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56294805e890 0x56294809abf0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:07.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.008+0000 7fc07d3a2000 1 --2- 172.21.15.90:0/3110346263 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x56294805eff0 0x56294809a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:07.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.008+0000 7fc07d3a2000 1 -- 172.21.15.90:0/3110346263 >> 172.21.15.90:0/3110346263 conn(0x5629480a5070 msgr2=0x562948099fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:37:07.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.008+0000 7fc07d3a2000 1 -- 172.21.15.90:0/3110346263 shutdown_connections 2024-03-11T18:37:07.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.008+0000 7fc07d3a2000 1 -- 172.21.15.90:0/3110346263 wait complete. 2024-03-11T18:37:07.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.008+0000 7fc07d3a2000 1 -- start start 2024-03-11T18:37:07.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.008+0000 7fc07d3a2000 1 Processor -- start 2024-03-11T18:37:07.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.012+0000 7fc07d3a2000 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56294805e890 0x5629480ffea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:07.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.012+0000 7fc07d3a2000 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x56294805eff0 0x5629481003e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:07.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.012+0000 7fc07d3a2000 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562948097f70 0x5629480a7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:07.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.012+0000 7fc07d3a2000 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x562947ea6930 con 0x562948097f70 2024-03-11T18:37:07.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.012+0000 7fc07d3a2000 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x562947e43660 con 0x56294805e890 2024-03-11T18:37:07.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.012+0000 7fc07d3a2000 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x562947eeb1a0 con 0x56294805eff0 2024-03-11T18:37:07.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.012+0000 7fc07d3a2000 10 client.0 _ll_register_callbacks cb 0x562947ead2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-11T18:37:07.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.012+0000 7fc073fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562948097f70 0x5629480a7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:07.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.012+0000 7fc073fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562948097f70 0x5629480a7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44596/0 (socket says 172.21.15.90:44596) 2024-03-11T18:37:07.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.012+0000 7fc073fff640 1 -- 172.21.15.90:0/2464001216 learned_addr learned my addr 172.21.15.90:0/2464001216 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:37:07.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.012+0000 7fc0737fe640 1 --2- 172.21.15.90:0/2464001216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56294805e890 0x5629480ffea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:07.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.012+0000 7fc072ffd640 1 --2- 172.21.15.90:0/2464001216 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x56294805eff0 0x5629481003e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:07.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.016+0000 7fc073fff640 1 -- 172.21.15.90:0/2464001216 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x56294805eff0 msgr2=0x5629481003e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:07.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.016+0000 7fc073fff640 1 --2- 172.21.15.90:0/2464001216 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x56294805eff0 0x5629481003e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:07.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.016+0000 7fc073fff640 1 -- 172.21.15.90:0/2464001216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56294805e890 msgr2=0x5629480ffea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:07.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.016+0000 7fc073fff640 1 --2- 172.21.15.90:0/2464001216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56294805e890 0x5629480ffea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:07.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.016+0000 7fc073fff640 1 -- 172.21.15.90:0/2464001216 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562947e6ae70 con 0x562948097f70 2024-03-11T18:37:07.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.016+0000 7fc073fff640 1 --2- 172.21.15.90:0/2464001216 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562948097f70 0x5629480a7840 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc06c0bb040 tx=0x7fc06c019850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd1d3fdc22251a6a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:37:07.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.016+0000 7fc0527fc640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-03-11T18:37:07.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.020+0000 7fc0527fc640 1 -- 172.21.15.90:0/2464001216 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06c005ba0 con 0x562948097f70 2024-03-11T18:37:07.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.020+0000 7fc0527fc640 1 -- 172.21.15.90:0/2464001216 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fc06c005d00 con 0x562948097f70 2024-03-11T18:37:07.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.020+0000 7fc07d3a2000 1 -- 172.21.15.90:0/2464001216 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562947e30e80 con 0x562948097f70 2024-03-11T18:37:07.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.024+0000 7fc0527fc640 1 -- 172.21.15.90:0/2464001216 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc06c0a56a0 con 0x562948097f70 2024-03-11T18:37:07.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.024+0000 7fc07d3a2000 10 client.4722 fetch_fsmap learned FSMap version 19 2024-03-11T18:37:07.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.024+0000 7fc07d3a2000 1 -- 172.21.15.90:0/2464001216 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) v3 -- 0x562947e50e10 con 0x562948097f70 2024-03-11T18:37:07.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.024+0000 7fc0527fc640 1 -- 172.21.15.90:0/2464001216 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562947e30e80 con 0x562948097f70 2024-03-11T18:37:07.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.028+0000 7fc0527fc640 1 -- 172.21.15.90:0/2464001216 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 19) v1 ==== 4500+0+0 (secure 0 0 0) 0x7fc06c0a5800 con 0x562948097f70 2024-03-11T18:37:07.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.028+0000 7fc07d3a2000 10 client.4722 fetch_fsmap finished waiting for FSMap version 19 2024-03-11T18:37:07.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.028+0000 7fc07d3a2000 10 client.4722 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-11T18:37:07.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.028+0000 7fc07d3a2000 20 client.4722 populate_metadata read hostname 'smithi090' 2024-03-11T18:37:07.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.028+0000 7fc07d3a2000 1 --2- 172.21.15.90:0/2464001216 >> [v2:172.21.15.90:6836/3485943375,v1:172.21.15.90:6837/3485943375] conn(0x562948099bc0 0x5629480a8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:07.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.028+0000 7fc07d3a2000 4 client.4722 mds_command: new command op to 4255 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-11T18:37:07.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.028+0000 7fc07d3a2000 1 -- 172.21.15.90:0/2464001216 --> [v2:172.21.15.90:6836/3485943375,v1:172.21.15.90:6837/3485943375] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562947e73fa0 con 0x562948099bc0 2024-03-11T18:37:07.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.028+0000 7fc0737fe640 1 --2- 172.21.15.90:0/2464001216 >> [v2:172.21.15.90:6836/3485943375,v1:172.21.15.90:6837/3485943375] conn(0x562948099bc0 0x5629480a8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:07.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.032+0000 7fc0737fe640 1 --2- 172.21.15.90:0/2464001216 >> [v2:172.21.15.90:6836/3485943375,v1:172.21.15.90:6837/3485943375] conn(0x562948099bc0 0x5629480a8200 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d2a52c55697bfa1 server_cookie=9517f7665c895b9b in_seq=0 out_seq=0 2024-03-11T18:37:07.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.032+0000 7fc0527fc640 10 client.4722 ms_handle_connect on v2:172.21.15.90:6836/3485943375 2024-03-11T18:37:07.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.036+0000 7fc0527fc640 1 -- 172.21.15.90:0/2464001216 <== mds.0 v2:172.21.15.90:6836/3485943375 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x562947e73fa0 con 0x562948099bc0 2024-03-11T18:37:07.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.036+0000 7fc0527fc640 10 client.4722 handle_command_reply: tid=0 2024-03-11T18:37:07.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.036+0000 7fc07d3a2000 1 client.4722 shutdown 2024-03-11T18:37:07.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.036+0000 7fc07d3a2000 1 -- 172.21.15.90:0/2464001216 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562948097f70 msgr2=0x5629480a7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:07.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.036+0000 7fc07d3a2000 1 --2- 172.21.15.90:0/2464001216 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562948097f70 0x5629480a7840 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc06c0bb040 tx=0x7fc06c019850 comp rx=0 tx=0).stop 2024-03-11T18:37:07.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.036+0000 7fc07d3a2000 1 -- 172.21.15.90:0/2464001216 shutdown_connections 2024-03-11T18:37:07.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.036+0000 7fc07d3a2000 1 --2- 172.21.15.90:0/2464001216 >> [v2:172.21.15.90:6836/3485943375,v1:172.21.15.90:6837/3485943375] conn(0x562948099bc0 0x5629480a8200 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:07.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.036+0000 7fc07d3a2000 1 --2- 172.21.15.90:0/2464001216 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562948097f70 0x5629480a7840 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:07.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.036+0000 7fc07d3a2000 1 --2- 172.21.15.90:0/2464001216 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x56294805eff0 0x5629481003e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:07.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.036+0000 7fc0527fc640 0 client.4722 ms_handle_reset on v2:172.21.15.90:6836/3485943375 2024-03-11T18:37:07.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.036+0000 7fc07d3a2000 1 --2- 172.21.15.90:0/2464001216 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x56294805e890 0x5629480ffea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:07.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.036+0000 7fc07d3a2000 1 -- 172.21.15.90:0/2464001216 >> 172.21.15.90:0/2464001216 conn(0x5629480a5070 msgr2=0x562948098350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:37:07.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.036+0000 7fc07d3a2000 1 -- 172.21.15.90:0/2464001216 shutdown_connections 2024-03-11T18:37:07.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.036+0000 7fc07d3a2000 1 -- 172.21.15.90:0/2464001216 wait complete. 2024-03-11T18:37:07.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.036+0000 7fc07d3a2000 20 client.4722 trim_cache size 0 max 16384 2024-03-11T18:37:07.043 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-03-11T18:37:07.043 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "76d56b10-6c29-476a-804b-8482ea90f9e2", 2024-03-11T18:37:07.043 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 0, 2024-03-11T18:37:07.043 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 4255, 2024-03-11T18:37:07.043 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-03-11T18:37:07.043 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-03-11T18:37:07.043 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-03-11T18:37:07.044 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 67.814666924999997, 2024-03-11T18:37:07.044 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 19, 2024-03-11T18:37:07.044 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 30, 2024-03-11T18:37:07.044 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 28, 2024-03-11T18:37:07.044 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 85.694009374999993 2024-03-11T18:37:07.044 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-03-11T18:37:07.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.036+0000 7fc07a239640 1 -- 172.21.15.90:0/3258335372 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fc060066e40 msgr2=0x7fc060069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:07.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.036+0000 7fc07a239640 1 --2- 172.21.15.90:0/3258335372 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fc060066e40 0x7fc060069300 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc064007fd0 tx=0x7fc064009040 comp rx=0 tx=0).stop 2024-03-11T18:37:07.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.036+0000 7fc07a239640 1 -- 172.21.15.90:0/3258335372 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0741443b0 msgr2=0x7fc074145b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:07.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.036+0000 7fc07a239640 1 --2- 172.21.15.90:0/3258335372 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0741443b0 0x7fc074145b50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc0680079a0 tx=0x7fc06800a6c0 comp rx=0 tx=0).stop 2024-03-11T18:37:07.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.036+0000 7fc07a239640 1 -- 172.21.15.90:0/3258335372 shutdown_connections 2024-03-11T18:37:07.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.036+0000 7fc07a239640 1 --2- 172.21.15.90:0/3258335372 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fc060066e40 0x7fc060069300 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:07.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.036+0000 7fc07a239640 1 --2- 172.21.15.90:0/3258335372 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc074157310 0x7fc074148ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:07.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.036+0000 7fc07a239640 1 --2- 172.21.15.90:0/3258335372 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0741443b0 0x7fc074145b50 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:07.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.036+0000 7fc07a239640 1 --2- 172.21.15.90:0/3258335372 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc05c005190 0x7fc07413fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:07.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.036+0000 7fc07a239640 1 -- 172.21.15.90:0/3258335372 >> 172.21.15.90:0/3258335372 conn(0x7fc07406ac00 msgr2=0x7fc0740c5de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:37:07.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.036+0000 7fc07a239640 1 -- 172.21.15.90:0/3258335372 shutdown_connections 2024-03-11T18:37:07.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.036+0000 7fc07a239640 1 -- 172.21.15.90:0/3258335372 wait complete. 2024-03-11T18:37:07.059 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-03-11T18:37:07.071 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-03-11T18:37:07.071 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-03-11T18:37:07.071 INFO:tasks.ceph_fuse:client.0 config is {'valgrind': ['--tool=memcheck', '--leak-check=full', '--show-reachable=yes']} 2024-03-11T18:37:07.071 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-11T18:37:07.072 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-11T18:37:07.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T18:37:07.072 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list 2024-03-11T18:37:07.077 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-03-11T18:37:07.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T18:37:07.078 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-11T18:37:07.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T18:37:07.131 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link delete ceph-brx 2024-03-11T18:37:07.335 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-03-11T18:37:07.335 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-11T18:37:07.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.452+0000 7f3f11d9a640 1 Processor -- start 2024-03-11T18:37:07.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.452+0000 7f3f11d9a640 1 -- start start 2024-03-11T18:37:07.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.456+0000 7f3f11d9a640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3f0c0c6390 0x7f3f0c0c9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:07.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.456+0000 7f3f11d9a640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3f0c0ca2d0 0x7f3f0c0ca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:07.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.456+0000 7f3f11d9a640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f0c0c70e0 con 0x7f3f0c153db0 2024-03-11T18:37:07.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.456+0000 7f3f11d9a640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f0c0c73a0 con 0x7f3f0c0ca2d0 2024-03-11T18:37:07.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.456+0000 7f3f11d9a640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c0c7680 con 0x7f3f0c0c6390 2024-03-11T18:37:07.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.456+0000 7f3f0affd640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3f0c0c6390 0x7f3f0c0c9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:07.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.456+0000 7f3f0bfff640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3f0c0ca2d0 0x7f3f0c0ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:07.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.456+0000 7f3f0affd640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3f0c0c6390 0x7f3f0c0c9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.90:50264/0 (socket says 172.21.15.90:50264) 2024-03-11T18:37:07.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.456+0000 7f3f0affd640 1 -- 172.21.15.90:0/2188147521 learned_addr learned my addr 172.21.15.90:0/2188147521 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:37:07.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.464+0000 7f3f0affd640 1 -- 172.21.15.90:0/2188147521 >> v2:172.21.15.92:3300/0 conn(0x7f3f0c0ca2d0 msgr2=0x7f3f0c0ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T18:37:07.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.464+0000 7f3f0affd640 1 --2- 172.21.15.90:0/2188147521 >> v2:172.21.15.92:3300/0 conn(0x7f3f0c0ca2d0 0x7f3f0c0ca700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:07.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.464+0000 7f3f0affd640 1 -- 172.21.15.90:0/2188147521 >> v1:172.21.15.92:6789/0 conn(0x7f3f0c153db0 legacy=0x7f3f0c15dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:07.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.464+0000 7f3f0affd640 1 -- 172.21.15.90:0/2188147521 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f0c0c7960 con 0x7f3f0c0c6390 2024-03-11T18:37:07.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.468+0000 7f3f0affd640 1 --2- 172.21.15.90:0/2188147521 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3f0c0c6390 0x7f3f0c0c9790 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4009870 tx=0x7f3ef401c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a5f57a4496a9ae2 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:37:07.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.480+0000 7f3f0a7fc640 1 -- 172.21.15.90:0/2188147521 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef40a9620 con 0x7f3f0c0c6390 2024-03-11T18:37:07.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.480+0000 7f3f0a7fc640 1 -- 172.21.15.90:0/2188147521 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef40a9780 con 0x7f3f0c0c6390 2024-03-11T18:37:07.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.480+0000 7f3f0a7fc640 1 -- 172.21.15.90:0/2188147521 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef40a9b10 con 0x7f3f0c0c6390 2024-03-11T18:37:07.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.480+0000 7f3f11d9a640 1 -- 172.21.15.90:0/2188147521 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3f0c0c6390 msgr2=0x7f3f0c0c9790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:07.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.480+0000 7f3f11d9a640 1 --2- 172.21.15.90:0/2188147521 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3f0c0c6390 0x7f3f0c0c9790 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4009870 tx=0x7f3ef401c560 comp rx=0 tx=0).stop 2024-03-11T18:37:07.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.480+0000 7f3f11d9a640 1 -- 172.21.15.90:0/2188147521 shutdown_connections 2024-03-11T18:37:07.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.480+0000 7f3f11d9a640 1 --2- 172.21.15.90:0/2188147521 >> v2:172.21.15.92:3300/0 conn(0x7f3f0c0ca2d0 0x7f3f0c0ca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:07.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.480+0000 7f3f11d9a640 1 --2- 172.21.15.90:0/2188147521 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3f0c0c6390 0x7f3f0c0c9790 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:07.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.480+0000 7f3f11d9a640 1 -- 172.21.15.90:0/2188147521 >> 172.21.15.90:0/2188147521 conn(0x7f3f0c0c8490 msgr2=0x7f3f0c0c88a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:37:07.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.480+0000 7f3f11d9a640 1 -- 172.21.15.90:0/2188147521 shutdown_connections 2024-03-11T18:37:07.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.480+0000 7f3f11d9a640 1 -- 172.21.15.90:0/2188147521 wait complete. 2024-03-11T18:37:07.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.484+0000 7f3f11d9a640 1 Processor -- start 2024-03-11T18:37:07.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.484+0000 7f3f11d9a640 1 -- start start 2024-03-11T18:37:07.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.484+0000 7f3f11d9a640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f0c0c6390 0x7f3f0c15ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:07.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.484+0000 7f3f11d9a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f0c0ca2d0 0x7f3f0c1ec860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:07.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.484+0000 7f3f11d9a640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3f0c153db0 0x7f3f0c0c5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:07.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.484+0000 7f3f11d9a640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c1ec440 con 0x7f3f0c0c6390 2024-03-11T18:37:07.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.484+0000 7f3f11d9a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c0c5b90 con 0x7f3f0c0ca2d0 2024-03-11T18:37:07.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.484+0000 7f3f11d9a640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c14d0d0 con 0x7f3f0c153db0 2024-03-11T18:37:07.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.484+0000 7f3f0b7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f0c0c6390 0x7f3f0c15ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:07.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.484+0000 7f3f0b7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f0c0c6390 0x7f3f0c15ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44612/0 (socket says 172.21.15.90:44612) 2024-03-11T18:37:07.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.484+0000 7f3f0b7fe640 1 -- 172.21.15.90:0/2323225773 learned_addr learned my addr 172.21.15.90:0/2323225773 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T18:37:07.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.484+0000 7f3f0affd640 1 --2- 172.21.15.90:0/2323225773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f0c0ca2d0 0x7f3f0c1ec860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:07.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.484+0000 7f3f0bfff640 1 --2- 172.21.15.90:0/2323225773 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3f0c153db0 0x7f3f0c0c5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:07.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.488+0000 7f3f0b7fe640 1 -- 172.21.15.90:0/2323225773 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3f0c153db0 msgr2=0x7f3f0c0c5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:07.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.488+0000 7f3f0b7fe640 1 --2- 172.21.15.90:0/2323225773 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3f0c153db0 0x7f3f0c0c5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:07.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.488+0000 7f3f0b7fe640 1 -- 172.21.15.90:0/2323225773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f0c0ca2d0 msgr2=0x7f3f0c1ec860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:07.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.488+0000 7f3f0b7fe640 1 --2- 172.21.15.90:0/2323225773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f0c0ca2d0 0x7f3f0c1ec860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:07.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.488+0000 7f3f0b7fe640 1 -- 172.21.15.90:0/2323225773 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef40094d0 con 0x7f3f0c0c6390 2024-03-11T18:37:07.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.492+0000 7f3f0b7fe640 1 --2- 172.21.15.90:0/2323225773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f0c0c6390 0x7f3f0c15ae20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3efc00c420 tx=0x7f3efc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:37:07.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.496+0000 7f3f03fff640 1 -- 172.21.15.90:0/2323225773 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3efc0052d0 con 0x7f3f0c0c6390 2024-03-11T18:37:07.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.496+0000 7f3f03fff640 1 -- 172.21.15.90:0/2323225773 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3efc005430 con 0x7f3f0c0c6390 2024-03-11T18:37:07.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.496+0000 7f3f11d9a640 1 -- 172.21.15.90:0/2323225773 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f0c15cd20 con 0x7f3f0c0c6390 2024-03-11T18:37:07.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.496+0000 7f3f03fff640 1 -- 172.21.15.90:0/2323225773 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3efc007260 con 0x7f3f0c0c6390 2024-03-11T18:37:07.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.496+0000 7f3f11d9a640 1 -- 172.21.15.90:0/2323225773 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f0c1ebb30 con 0x7f3f0c0c6390 2024-03-11T18:37:07.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.500+0000 7f3f11d9a640 1 -- 172.21.15.90:0/2323225773 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f0c0c6d20 con 0x7f3f0c0c6390 2024-03-11T18:37:07.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.520+0000 7f3f03fff640 1 -- 172.21.15.90:0/2323225773 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f3efc0073c0 con 0x7f3f0c0c6390 2024-03-11T18:37:07.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.520+0000 7f3f03fff640 1 --2- 172.21.15.90:0/2323225773 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f3eec066e90 0x7f3eec069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T18:37:07.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.520+0000 7f3f0affd640 1 --2- 172.21.15.90:0/2323225773 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f3eec066e90 0x7f3eec069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T18:37:07.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.520+0000 7f3f0affd640 1 --2- 172.21.15.90:0/2323225773 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f3eec066e90 0x7f3eec069350 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3ef40ae000 tx=0x7f3ef4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T18:37:07.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.524+0000 7f3f03fff640 1 -- 172.21.15.90:0/2323225773 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f3efc007670 con 0x7f3f0c0c6390 2024-03-11T18:37:07.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:07.808+0000 7f3f03fff640 1 -- 172.21.15.90:0/2323225773 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3efc051440 con 0x7f3f0c0c6390 2024-03-11T18:37:08.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:08.012+0000 7f3f11d9a640 1 -- 172.21.15.90:0/2323225773 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3f0c04eef0 con 0x7f3f0c0c6390 2024-03-11T18:37:08.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:08.024+0000 7f3f03fff640 1 -- 172.21.15.90:0/2323225773 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v19) v1 ==== 53+0+88 (secure 0 0 0) 0x7f3efc020070 con 0x7f3f0c0c6390 2024-03-11T18:37:08.030 INFO:teuthology.orchestra.run.smithi090.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-11T18:37:08.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:08.024+0000 7f3f11d9a640 1 -- 172.21.15.90:0/2323225773 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f3eec066e90 msgr2=0x7f3eec069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:08.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:08.024+0000 7f3f11d9a640 1 --2- 172.21.15.90:0/2323225773 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f3eec066e90 0x7f3eec069350 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3ef40ae000 tx=0x7f3ef4009040 comp rx=0 tx=0).stop 2024-03-11T18:37:08.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:08.024+0000 7f3f11d9a640 1 -- 172.21.15.90:0/2323225773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f0c0c6390 msgr2=0x7f3f0c15ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T18:37:08.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:08.024+0000 7f3f11d9a640 1 --2- 172.21.15.90:0/2323225773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f0c0c6390 0x7f3f0c15ae20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3efc00c420 tx=0x7f3efc00c8e0 comp rx=0 tx=0).stop 2024-03-11T18:37:08.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:08.024+0000 7f3f11d9a640 1 -- 172.21.15.90:0/2323225773 shutdown_connections 2024-03-11T18:37:08.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:08.024+0000 7f3f11d9a640 1 --2- 172.21.15.90:0/2323225773 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f3eec066e90 0x7f3eec069350 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:08.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:08.024+0000 7f3f11d9a640 1 --2- 172.21.15.90:0/2323225773 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3f0c153db0 0x7f3f0c0c5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:08.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:08.024+0000 7f3f11d9a640 1 --2- 172.21.15.90:0/2323225773 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3f0c0ca2d0 0x7f3f0c1ec860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:08.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:08.024+0000 7f3f11d9a640 1 --2- 172.21.15.90:0/2323225773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f0c0c6390 0x7f3f0c15ae20 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T18:37:08.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:08.024+0000 7f3f11d9a640 1 -- 172.21.15.90:0/2323225773 >> 172.21.15.90:0/2323225773 conn(0x7f3f0c0c8490 msgr2=0x7f3f0c0c8870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T18:37:08.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:08.024+0000 7f3f11d9a640 1 -- 172.21.15.90:0/2323225773 shutdown_connections 2024-03-11T18:37:08.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:08.024+0000 7f3f11d9a640 1 -- 172.21.15.90:0/2323225773 wait complete. 2024-03-11T18:37:08.050 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-11T18:37:08.050 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-11T18:37:08.050 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi090.front.sepia.ceph.com 2024-03-11T18:37:08.050 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-11T18:37:08.050 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-11T18:37:08.050 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-11T18:37:08.050 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-11T18:37:08.050 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-11T18:37:08.050 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-11T18:37:08.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T18:37:08.051 DEBUG:teuthology.orchestra.run.smithi090:> ip addr 2024-03-11T18:37:08.056 INFO:teuthology.orchestra.run.smithi090.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-11T18:37:08.056 INFO:teuthology.orchestra.run.smithi090.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-11T18:37:08.056 INFO:teuthology.orchestra.run.smithi090.stdout: inet 127.0.0.1/8 scope host lo 2024-03-11T18:37:08.056 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-03-11T18:37:08.056 INFO:teuthology.orchestra.run.smithi090.stdout: inet6 ::1/128 scope host 2024-03-11T18:37:08.057 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-03-11T18:37:08.057 INFO:teuthology.orchestra.run.smithi090.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-11T18:37:08.057 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:d8:a1:a8 brd ff:ff:ff:ff:ff:ff 2024-03-11T18:37:08.057 INFO:teuthology.orchestra.run.smithi090.stdout: altname enp7s0f0 2024-03-11T18:37:08.057 INFO:teuthology.orchestra.run.smithi090.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-11T18:37:08.057 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:d8:a1:a9 brd ff:ff:ff:ff:ff:ff 2024-03-11T18:37:08.057 INFO:teuthology.orchestra.run.smithi090.stdout: altname enp7s0f1 2024-03-11T18:37:08.057 INFO:teuthology.orchestra.run.smithi090.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-11T18:37:08.057 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:8f:d7:66 brd ff:ff:ff:ff:ff:ff 2024-03-11T18:37:08.057 INFO:teuthology.orchestra.run.smithi090.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-11T18:37:08.057 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:8f:d7:67 brd ff:ff:ff:ff:ff:ff 2024-03-11T18:37:08.057 INFO:teuthology.orchestra.run.smithi090.stdout: inet 172.21.15.90/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-11T18:37:08.057 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft 42066sec preferred_lft 42066sec 2024-03-11T18:37:08.058 INFO:teuthology.orchestra.run.smithi090.stdout: inet6 fe80::ec4:7aff:fe8f:d767/64 scope link 2024-03-11T18:37:08.058 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-03-11T18:37:08.058 INFO:teuthology.orchestra.run.smithi090.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-11T18:37:08.058 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 02:42:26:17:e3:13 brd ff:ff:ff:ff:ff:ff 2024-03-11T18:37:08.058 INFO:teuthology.orchestra.run.smithi090.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-11T18:37:08.058 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-03-11T18:37:08.058 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-11T18:37:08.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T18:37:08.058 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec bash -c ' 2024-03-11T18:37:08.058 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-03-11T18:37:08.059 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link add name ceph-brx type bridge 2024-03-11T18:37:08.059 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip addr flush dev ceph-brx 2024-03-11T18:37:08.059 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link set ceph-brx up 2024-03-11T18:37:08.059 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-11T18:37:08.059 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-03-11T18:37:08.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T18:37:08.137 DEBUG:teuthology.orchestra.run.smithi090:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-11T18:37:08.193 INFO:teuthology.orchestra.run.smithi090.stdout:1 2024-03-11T18:37:08.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T18:37:08.195 DEBUG:teuthology.orchestra.run.smithi090:> route 2024-03-11T18:37:08.244 INFO:teuthology.orchestra.run.smithi090.stdout:Kernel IP routing table 2024-03-11T18:37:08.244 INFO:teuthology.orchestra.run.smithi090.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-11T18:37:08.244 INFO:teuthology.orchestra.run.smithi090.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-11T18:37:08.244 INFO:teuthology.orchestra.run.smithi090.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-11T18:37:08.244 INFO:teuthology.orchestra.run.smithi090.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-11T18:37:08.244 INFO:teuthology.orchestra.run.smithi090.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-11T18:37:08.244 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-11T18:37:08.244 INFO:teuthology.orchestra.run.smithi090.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-11T18:37:08.245 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-11T18:37:08.245 INFO:teuthology.orchestra.run.smithi090.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-11T18:37:08.245 INFO:teuthology.orchestra.run.smithi090.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-11T18:37:08.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T18:37:08.245 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec bash -c ' 2024-03-11T18:37:08.245 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-03-11T18:37:08.245 DEBUG:teuthology.orchestra.run.smithi090:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-11T18:37:08.245 DEBUG:teuthology.orchestra.run.smithi090:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-11T18:37:08.245 DEBUG:teuthology.orchestra.run.smithi090:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-11T18:37:08.245 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-03-11T18:37:08.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T18:37:08.316 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list 2024-03-11T18:37:08.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T18:37:08.367 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list-id 2024-03-11T18:37:08.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T18:37:08.415 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec bash -c ' 2024-03-11T18:37:08.415 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-03-11T18:37:08.415 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-11T18:37:08.415 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-11T18:37:08.415 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-03-11T18:37:08.478 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-11T18:37:08.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T18:37:08.479 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec bash -c ' 2024-03-11T18:37:08.479 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-03-11T18:37:08.479 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-11T18:37:08.479 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-11T18:37:08.479 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-11T18:37:08.479 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-11T18:37:08.479 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-11T18:37:08.479 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-03-11T18:37:08.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T18:37:08.575 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec bash -c ' 2024-03-11T18:37:08.575 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-03-11T18:37:08.575 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link set brx.0 up 2024-03-11T18:37:08.575 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link set dev brx.0 master ceph-brx 2024-03-11T18:37:08.575 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-03-11T18:37:08.594 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'valgrind': ['--tool=memcheck', '--leak-check=full', '--show-reachable=yes']} 2024-03-11T18:37:08.594 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-11T18:37:08.594 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-11T18:37:08.645 INFO:teuthology.orchestra.run.smithi090.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-11T18:37:08.645 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-11T18:37:08.645 DEBUG:teuthology.orchestra.run.smithi090:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-11T18:37:08.697 DEBUG:tasks.ceph_manager:running client.0 under valgrind with args ['sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/client.0.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2024-03-11T18:37:08.697 DEBUG:teuthology.orchestra.run.smithi090:> sudo modprobe fuse 2024-03-11T18:37:08.758 DEBUG:teuthology.orchestra.run.smithi090:> cat /proc/self/mounts | awk '{print $2}' 2024-03-11T18:37:08.806 INFO:teuthology.orchestra.run.smithi090.stdout:/sys 2024-03-11T18:37:08.806 INFO:teuthology.orchestra.run.smithi090.stdout:/proc 2024-03-11T18:37:08.806 INFO:teuthology.orchestra.run.smithi090.stdout:/dev 2024-03-11T18:37:08.806 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/pts 2024-03-11T18:37:08.806 INFO:teuthology.orchestra.run.smithi090.stdout:/run 2024-03-11T18:37:08.806 INFO:teuthology.orchestra.run.smithi090.stdout:/ 2024-03-11T18:37:08.806 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/security 2024-03-11T18:37:08.807 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/shm 2024-03-11T18:37:08.807 INFO:teuthology.orchestra.run.smithi090.stdout:/run/lock 2024-03-11T18:37:08.807 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/cgroup 2024-03-11T18:37:08.807 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/pstore 2024-03-11T18:37:08.807 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/bpf 2024-03-11T18:37:08.807 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/sys/fs/binfmt_misc 2024-03-11T18:37:08.807 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/hugepages 2024-03-11T18:37:08.807 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/mqueue 2024-03-11T18:37:08.807 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/debug 2024-03-11T18:37:08.807 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/tracing 2024-03-11T18:37:08.807 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/fs/nfsd 2024-03-11T18:37:08.807 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/fuse/connections 2024-03-11T18:37:08.807 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/config 2024-03-11T18:37:08.807 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-sysusers.service 2024-03-11T18:37:08.808 INFO:teuthology.orchestra.run.smithi090.stdout:/run/rpc_pipefs 2024-03-11T18:37:08.808 INFO:teuthology.orchestra.run.smithi090.stdout:/run/user/1000 2024-03-11T18:37:08.808 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph 2024-03-11T18:37:08.808 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph/osd/ceph-0 2024-03-11T18:37:08.808 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph/osd/ceph-1 2024-03-11T18:37:08.808 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph/osd/ceph-2 2024-03-11T18:37:08.808 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph/osd/ceph-3 2024-03-11T18:37:08.808 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns 2024-03-11T18:37:08.808 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-11T18:37:08.808 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-11T18:37:08.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T18:37:08.809 DEBUG:teuthology.orchestra.run.smithi090:> ls /sys/fs/fuse/connections 2024-03-11T18:37:08.858 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-11T18:37:08.858 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/client.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-11T18:37:08.902 DEBUG:teuthology.orchestra.run.smithi090:> sudo modprobe fuse 2024-03-11T18:37:08.916 DEBUG:teuthology.orchestra.run.smithi090:> cat /proc/self/mounts | awk '{print $2}' 2024-03-11T18:37:08.966 INFO:teuthology.orchestra.run.smithi090.stdout:/sys 2024-03-11T18:37:08.966 INFO:teuthology.orchestra.run.smithi090.stdout:/proc 2024-03-11T18:37:08.966 INFO:teuthology.orchestra.run.smithi090.stdout:/dev 2024-03-11T18:37:08.966 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/pts 2024-03-11T18:37:08.967 INFO:teuthology.orchestra.run.smithi090.stdout:/run 2024-03-11T18:37:08.967 INFO:teuthology.orchestra.run.smithi090.stdout:/ 2024-03-11T18:37:08.967 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/security 2024-03-11T18:37:08.967 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/shm 2024-03-11T18:37:08.967 INFO:teuthology.orchestra.run.smithi090.stdout:/run/lock 2024-03-11T18:37:08.967 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/cgroup 2024-03-11T18:37:08.967 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/pstore 2024-03-11T18:37:08.967 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/bpf 2024-03-11T18:37:08.967 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/sys/fs/binfmt_misc 2024-03-11T18:37:08.967 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/hugepages 2024-03-11T18:37:08.967 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/mqueue 2024-03-11T18:37:08.967 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/debug 2024-03-11T18:37:08.967 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/tracing 2024-03-11T18:37:08.967 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/fs/nfsd 2024-03-11T18:37:08.968 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/fuse/connections 2024-03-11T18:37:08.968 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/config 2024-03-11T18:37:08.968 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-sysusers.service 2024-03-11T18:37:08.968 INFO:teuthology.orchestra.run.smithi090.stdout:/run/rpc_pipefs 2024-03-11T18:37:08.968 INFO:teuthology.orchestra.run.smithi090.stdout:/run/user/1000 2024-03-11T18:37:08.968 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph 2024-03-11T18:37:08.968 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph/osd/ceph-0 2024-03-11T18:37:08.968 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph/osd/ceph-1 2024-03-11T18:37:08.968 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph/osd/ceph-2 2024-03-11T18:37:08.968 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph/osd/ceph-3 2024-03-11T18:37:08.968 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns 2024-03-11T18:37:08.968 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-11T18:37:08.968 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-11T18:37:08.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T18:37:08.969 DEBUG:teuthology.orchestra.run.smithi090:> ls /sys/fs/fuse/connections 2024-03-11T18:37:10.118 DEBUG:teuthology.orchestra.run.smithi090:> sudo modprobe fuse 2024-03-11T18:37:10.134 DEBUG:teuthology.orchestra.run.smithi090:> cat /proc/self/mounts | awk '{print $2}' 2024-03-11T18:37:10.183 INFO:teuthology.orchestra.run.smithi090.stdout:/sys 2024-03-11T18:37:10.183 INFO:teuthology.orchestra.run.smithi090.stdout:/proc 2024-03-11T18:37:10.183 INFO:teuthology.orchestra.run.smithi090.stdout:/dev 2024-03-11T18:37:10.183 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/pts 2024-03-11T18:37:10.184 INFO:teuthology.orchestra.run.smithi090.stdout:/run 2024-03-11T18:37:10.184 INFO:teuthology.orchestra.run.smithi090.stdout:/ 2024-03-11T18:37:10.184 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/security 2024-03-11T18:37:10.184 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/shm 2024-03-11T18:37:10.184 INFO:teuthology.orchestra.run.smithi090.stdout:/run/lock 2024-03-11T18:37:10.184 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/cgroup 2024-03-11T18:37:10.184 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/pstore 2024-03-11T18:37:10.184 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/bpf 2024-03-11T18:37:10.184 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/sys/fs/binfmt_misc 2024-03-11T18:37:10.184 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/hugepages 2024-03-11T18:37:10.184 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/mqueue 2024-03-11T18:37:10.184 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/debug 2024-03-11T18:37:10.184 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/tracing 2024-03-11T18:37:10.184 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/fs/nfsd 2024-03-11T18:37:10.185 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/fuse/connections 2024-03-11T18:37:10.185 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/config 2024-03-11T18:37:10.185 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-sysusers.service 2024-03-11T18:37:10.185 INFO:teuthology.orchestra.run.smithi090.stdout:/run/rpc_pipefs 2024-03-11T18:37:10.185 INFO:teuthology.orchestra.run.smithi090.stdout:/run/user/1000 2024-03-11T18:37:10.185 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph 2024-03-11T18:37:10.185 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph/osd/ceph-0 2024-03-11T18:37:10.185 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph/osd/ceph-1 2024-03-11T18:37:10.185 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph/osd/ceph-2 2024-03-11T18:37:10.185 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph/osd/ceph-3 2024-03-11T18:37:10.185 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns 2024-03-11T18:37:10.185 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-11T18:37:10.185 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-11T18:37:10.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T18:37:10.186 DEBUG:teuthology.orchestra.run.smithi090:> ls /sys/fs/fuse/connections 2024-03-11T18:37:11.334 DEBUG:teuthology.orchestra.run.smithi090:> sudo modprobe fuse 2024-03-11T18:37:11.349 DEBUG:teuthology.orchestra.run.smithi090:> cat /proc/self/mounts | awk '{print $2}' 2024-03-11T18:37:11.398 INFO:teuthology.orchestra.run.smithi090.stdout:/sys 2024-03-11T18:37:11.398 INFO:teuthology.orchestra.run.smithi090.stdout:/proc 2024-03-11T18:37:11.398 INFO:teuthology.orchestra.run.smithi090.stdout:/dev 2024-03-11T18:37:11.398 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/pts 2024-03-11T18:37:11.398 INFO:teuthology.orchestra.run.smithi090.stdout:/run 2024-03-11T18:37:11.398 INFO:teuthology.orchestra.run.smithi090.stdout:/ 2024-03-11T18:37:11.398 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/security 2024-03-11T18:37:11.398 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/shm 2024-03-11T18:37:11.398 INFO:teuthology.orchestra.run.smithi090.stdout:/run/lock 2024-03-11T18:37:11.399 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/cgroup 2024-03-11T18:37:11.399 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/pstore 2024-03-11T18:37:11.399 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/bpf 2024-03-11T18:37:11.399 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/sys/fs/binfmt_misc 2024-03-11T18:37:11.399 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/hugepages 2024-03-11T18:37:11.399 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/mqueue 2024-03-11T18:37:11.399 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/debug 2024-03-11T18:37:11.399 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/tracing 2024-03-11T18:37:11.399 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/fs/nfsd 2024-03-11T18:37:11.399 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/fuse/connections 2024-03-11T18:37:11.399 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/config 2024-03-11T18:37:11.399 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-sysusers.service 2024-03-11T18:37:11.399 INFO:teuthology.orchestra.run.smithi090.stdout:/run/rpc_pipefs 2024-03-11T18:37:11.399 INFO:teuthology.orchestra.run.smithi090.stdout:/run/user/1000 2024-03-11T18:37:11.400 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph 2024-03-11T18:37:11.400 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph/osd/ceph-0 2024-03-11T18:37:11.400 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph/osd/ceph-1 2024-03-11T18:37:11.400 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph/osd/ceph-2 2024-03-11T18:37:11.400 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph/osd/ceph-3 2024-03-11T18:37:11.400 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns 2024-03-11T18:37:11.400 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-11T18:37:11.400 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-11T18:37:11.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T18:37:11.401 DEBUG:teuthology.orchestra.run.smithi090:> ls /sys/fs/fuse/connections 2024-03-11T18:37:12.550 DEBUG:teuthology.orchestra.run.smithi090:> sudo modprobe fuse 2024-03-11T18:37:12.565 DEBUG:teuthology.orchestra.run.smithi090:> cat /proc/self/mounts | awk '{print $2}' 2024-03-11T18:37:12.614 INFO:teuthology.orchestra.run.smithi090.stdout:/sys 2024-03-11T18:37:12.614 INFO:teuthology.orchestra.run.smithi090.stdout:/proc 2024-03-11T18:37:12.614 INFO:teuthology.orchestra.run.smithi090.stdout:/dev 2024-03-11T18:37:12.614 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/pts 2024-03-11T18:37:12.614 INFO:teuthology.orchestra.run.smithi090.stdout:/run 2024-03-11T18:37:12.614 INFO:teuthology.orchestra.run.smithi090.stdout:/ 2024-03-11T18:37:12.614 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/security 2024-03-11T18:37:12.614 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/shm 2024-03-11T18:37:12.614 INFO:teuthology.orchestra.run.smithi090.stdout:/run/lock 2024-03-11T18:37:12.615 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/cgroup 2024-03-11T18:37:12.615 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/pstore 2024-03-11T18:37:12.615 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/bpf 2024-03-11T18:37:12.615 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/sys/fs/binfmt_misc 2024-03-11T18:37:12.615 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/hugepages 2024-03-11T18:37:12.615 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/mqueue 2024-03-11T18:37:12.615 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/debug 2024-03-11T18:37:12.615 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/tracing 2024-03-11T18:37:12.615 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/fs/nfsd 2024-03-11T18:37:12.615 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/fuse/connections 2024-03-11T18:37:12.615 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/config 2024-03-11T18:37:12.615 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-sysusers.service 2024-03-11T18:37:12.615 INFO:teuthology.orchestra.run.smithi090.stdout:/run/rpc_pipefs 2024-03-11T18:37:12.615 INFO:teuthology.orchestra.run.smithi090.stdout:/run/user/1000 2024-03-11T18:37:12.615 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph 2024-03-11T18:37:12.616 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph/osd/ceph-0 2024-03-11T18:37:12.616 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph/osd/ceph-1 2024-03-11T18:37:12.616 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph/osd/ceph-2 2024-03-11T18:37:12.616 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph/osd/ceph-3 2024-03-11T18:37:12.616 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns 2024-03-11T18:37:12.616 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-11T18:37:12.616 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-11T18:37:12.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T18:37:12.616 DEBUG:teuthology.orchestra.run.smithi090:> ls /sys/fs/fuse/connections 2024-03-11T18:37:13.762 DEBUG:teuthology.orchestra.run.smithi090:> sudo modprobe fuse 2024-03-11T18:37:13.776 DEBUG:teuthology.orchestra.run.smithi090:> cat /proc/self/mounts | awk '{print $2}' 2024-03-11T18:37:13.821 INFO:teuthology.orchestra.run.smithi090.stdout:/sys 2024-03-11T18:37:13.821 INFO:teuthology.orchestra.run.smithi090.stdout:/proc 2024-03-11T18:37:13.822 INFO:teuthology.orchestra.run.smithi090.stdout:/dev 2024-03-11T18:37:13.822 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/pts 2024-03-11T18:37:13.822 INFO:teuthology.orchestra.run.smithi090.stdout:/run 2024-03-11T18:37:13.822 INFO:teuthology.orchestra.run.smithi090.stdout:/ 2024-03-11T18:37:13.822 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/security 2024-03-11T18:37:13.822 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/shm 2024-03-11T18:37:13.822 INFO:teuthology.orchestra.run.smithi090.stdout:/run/lock 2024-03-11T18:37:13.822 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/cgroup 2024-03-11T18:37:13.822 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/pstore 2024-03-11T18:37:13.822 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/bpf 2024-03-11T18:37:13.822 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/sys/fs/binfmt_misc 2024-03-11T18:37:13.822 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/hugepages 2024-03-11T18:37:13.822 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/mqueue 2024-03-11T18:37:13.822 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/debug 2024-03-11T18:37:13.823 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/tracing 2024-03-11T18:37:13.823 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/fs/nfsd 2024-03-11T18:37:13.823 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/fuse/connections 2024-03-11T18:37:13.823 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/config 2024-03-11T18:37:13.823 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-sysusers.service 2024-03-11T18:37:13.823 INFO:teuthology.orchestra.run.smithi090.stdout:/run/rpc_pipefs 2024-03-11T18:37:13.823 INFO:teuthology.orchestra.run.smithi090.stdout:/run/user/1000 2024-03-11T18:37:13.823 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph 2024-03-11T18:37:13.823 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph/osd/ceph-0 2024-03-11T18:37:13.823 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph/osd/ceph-1 2024-03-11T18:37:13.823 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph/osd/ceph-2 2024-03-11T18:37:13.823 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph/osd/ceph-3 2024-03-11T18:37:13.823 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns 2024-03-11T18:37:13.823 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-11T18:37:13.823 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-11T18:37:13.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T18:37:13.824 DEBUG:teuthology.orchestra.run.smithi090:> ls /sys/fs/fuse/connections 2024-03-11T18:37:14.974 DEBUG:teuthology.orchestra.run.smithi090:> sudo modprobe fuse 2024-03-11T18:37:14.987 DEBUG:teuthology.orchestra.run.smithi090:> cat /proc/self/mounts | awk '{print $2}' 2024-03-11T18:37:15.033 INFO:teuthology.orchestra.run.smithi090.stdout:/sys 2024-03-11T18:37:15.034 INFO:teuthology.orchestra.run.smithi090.stdout:/proc 2024-03-11T18:37:15.034 INFO:teuthology.orchestra.run.smithi090.stdout:/dev 2024-03-11T18:37:15.034 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/pts 2024-03-11T18:37:15.034 INFO:teuthology.orchestra.run.smithi090.stdout:/run 2024-03-11T18:37:15.034 INFO:teuthology.orchestra.run.smithi090.stdout:/ 2024-03-11T18:37:15.034 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/security 2024-03-11T18:37:15.035 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/shm 2024-03-11T18:37:15.035 INFO:teuthology.orchestra.run.smithi090.stdout:/run/lock 2024-03-11T18:37:15.035 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/cgroup 2024-03-11T18:37:15.035 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/pstore 2024-03-11T18:37:15.035 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/bpf 2024-03-11T18:37:15.035 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/sys/fs/binfmt_misc 2024-03-11T18:37:15.035 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/hugepages 2024-03-11T18:37:15.035 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/mqueue 2024-03-11T18:37:15.035 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/debug 2024-03-11T18:37:15.035 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/tracing 2024-03-11T18:37:15.035 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/fs/nfsd 2024-03-11T18:37:15.035 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/fuse/connections 2024-03-11T18:37:15.035 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/config 2024-03-11T18:37:15.035 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-sysusers.service 2024-03-11T18:37:15.036 INFO:teuthology.orchestra.run.smithi090.stdout:/run/rpc_pipefs 2024-03-11T18:37:15.036 INFO:teuthology.orchestra.run.smithi090.stdout:/run/user/1000 2024-03-11T18:37:15.036 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph 2024-03-11T18:37:15.036 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph/osd/ceph-0 2024-03-11T18:37:15.036 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph/osd/ceph-1 2024-03-11T18:37:15.036 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph/osd/ceph-2 2024-03-11T18:37:15.036 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph/osd/ceph-3 2024-03-11T18:37:15.036 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns 2024-03-11T18:37:15.036 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-11T18:37:15.036 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-11T18:37:15.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T18:37:15.037 DEBUG:teuthology.orchestra.run.smithi090:> ls /sys/fs/fuse/connections 2024-03-11T18:37:15.452 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi090.stderr:ceph-fuse[36299]: starting ceph client 2024-03-11T18:37:15.543 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi090.stderr:2024-03-11T18:37:15.328+0000 68fe100 -1 init, newargv = 0x7e6ab60 newargc=15 2024-03-11T18:37:16.183 DEBUG:teuthology.orchestra.run.smithi090:> sudo modprobe fuse 2024-03-11T18:37:16.199 DEBUG:teuthology.orchestra.run.smithi090:> cat /proc/self/mounts | awk '{print $2}' 2024-03-11T18:37:16.246 INFO:teuthology.orchestra.run.smithi090.stdout:/sys 2024-03-11T18:37:16.247 INFO:teuthology.orchestra.run.smithi090.stdout:/proc 2024-03-11T18:37:16.247 INFO:teuthology.orchestra.run.smithi090.stdout:/dev 2024-03-11T18:37:16.247 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/pts 2024-03-11T18:37:16.247 INFO:teuthology.orchestra.run.smithi090.stdout:/run 2024-03-11T18:37:16.247 INFO:teuthology.orchestra.run.smithi090.stdout:/ 2024-03-11T18:37:16.247 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/security 2024-03-11T18:37:16.247 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/shm 2024-03-11T18:37:16.247 INFO:teuthology.orchestra.run.smithi090.stdout:/run/lock 2024-03-11T18:37:16.247 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/cgroup 2024-03-11T18:37:16.247 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/pstore 2024-03-11T18:37:16.247 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/bpf 2024-03-11T18:37:16.247 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/sys/fs/binfmt_misc 2024-03-11T18:37:16.247 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/hugepages 2024-03-11T18:37:16.248 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/mqueue 2024-03-11T18:37:16.248 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/debug 2024-03-11T18:37:16.248 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/tracing 2024-03-11T18:37:16.248 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/fs/nfsd 2024-03-11T18:37:16.248 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/fuse/connections 2024-03-11T18:37:16.248 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/config 2024-03-11T18:37:16.248 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-sysusers.service 2024-03-11T18:37:16.248 INFO:teuthology.orchestra.run.smithi090.stdout:/run/rpc_pipefs 2024-03-11T18:37:16.248 INFO:teuthology.orchestra.run.smithi090.stdout:/run/user/1000 2024-03-11T18:37:16.248 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph 2024-03-11T18:37:16.248 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph/osd/ceph-0 2024-03-11T18:37:16.248 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph/osd/ceph-1 2024-03-11T18:37:16.248 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph/osd/ceph-2 2024-03-11T18:37:16.248 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph/osd/ceph-3 2024-03-11T18:37:16.248 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns 2024-03-11T18:37:16.249 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-11T18:37:16.249 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-11T18:37:16.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T18:37:16.250 DEBUG:teuthology.orchestra.run.smithi090:> ls /sys/fs/fuse/connections 2024-03-11T18:37:17.398 DEBUG:teuthology.orchestra.run.smithi090:> sudo modprobe fuse 2024-03-11T18:37:17.418 DEBUG:teuthology.orchestra.run.smithi090:> cat /proc/self/mounts | awk '{print $2}' 2024-03-11T18:37:17.430 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi090.stderr:ceph-fuse[36299]: starting fuse 2024-03-11T18:37:17.443 INFO:teuthology.orchestra.run.smithi090.stdout:/sys 2024-03-11T18:37:17.444 INFO:teuthology.orchestra.run.smithi090.stdout:/proc 2024-03-11T18:37:17.444 INFO:teuthology.orchestra.run.smithi090.stdout:/dev 2024-03-11T18:37:17.444 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/pts 2024-03-11T18:37:17.444 INFO:teuthology.orchestra.run.smithi090.stdout:/run 2024-03-11T18:37:17.444 INFO:teuthology.orchestra.run.smithi090.stdout:/ 2024-03-11T18:37:17.444 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/security 2024-03-11T18:37:17.444 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/shm 2024-03-11T18:37:17.444 INFO:teuthology.orchestra.run.smithi090.stdout:/run/lock 2024-03-11T18:37:17.444 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/cgroup 2024-03-11T18:37:17.444 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/pstore 2024-03-11T18:37:17.444 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/bpf 2024-03-11T18:37:17.444 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/sys/fs/binfmt_misc 2024-03-11T18:37:17.445 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/hugepages 2024-03-11T18:37:17.445 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/mqueue 2024-03-11T18:37:17.445 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/debug 2024-03-11T18:37:17.445 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/tracing 2024-03-11T18:37:17.445 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/fs/nfsd 2024-03-11T18:37:17.445 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/fuse/connections 2024-03-11T18:37:17.445 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/config 2024-03-11T18:37:17.445 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-sysusers.service 2024-03-11T18:37:17.445 INFO:teuthology.orchestra.run.smithi090.stdout:/run/rpc_pipefs 2024-03-11T18:37:17.445 INFO:teuthology.orchestra.run.smithi090.stdout:/run/user/1000 2024-03-11T18:37:17.445 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph 2024-03-11T18:37:17.445 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph/osd/ceph-0 2024-03-11T18:37:17.445 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph/osd/ceph-1 2024-03-11T18:37:17.446 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph/osd/ceph-2 2024-03-11T18:37:17.446 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph/osd/ceph-3 2024-03-11T18:37:17.446 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns 2024-03-11T18:37:17.446 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-11T18:37:17.446 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-11T18:37:17.446 INFO:teuthology.orchestra.run.smithi090.stdout:/home/ubuntu/cephtest/mnt.0 2024-03-11T18:37:17.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T18:37:17.447 DEBUG:teuthology.orchestra.run.smithi090:> ls /sys/fs/fuse/connections 2024-03-11T18:37:17.494 INFO:teuthology.orchestra.run.smithi090.stdout:45 2024-03-11T18:37:17.495 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-03-11T18:37:17.495 DEBUG:teuthology.orchestra.run.smithi090:> sudo stdin-killer -- python3 -c ' 2024-03-11T18:37:17.495 DEBUG:teuthology.orchestra.run.smithi090:> import glob 2024-03-11T18:37:17.495 DEBUG:teuthology.orchestra.run.smithi090:> import re 2024-03-11T18:37:17.495 DEBUG:teuthology.orchestra.run.smithi090:> import os 2024-03-11T18:37:17.495 DEBUG:teuthology.orchestra.run.smithi090:> import subprocess 2024-03-11T18:37:17.496 DEBUG:teuthology.orchestra.run.smithi090:> 2024-03-11T18:37:17.496 DEBUG:teuthology.orchestra.run.smithi090:> def _find_admin_socket(client_name): 2024-03-11T18:37:17.496 DEBUG:teuthology.orchestra.run.smithi090:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-11T18:37:17.497 DEBUG:teuthology.orchestra.run.smithi090:> files = glob.glob(asok_path) 2024-03-11T18:37:17.497 DEBUG:teuthology.orchestra.run.smithi090:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-11T18:37:17.497 DEBUG:teuthology.orchestra.run.smithi090:> 2024-03-11T18:37:17.497 DEBUG:teuthology.orchestra.run.smithi090:> # Given a non-glob path, it better be there 2024-03-11T18:37:17.497 DEBUG:teuthology.orchestra.run.smithi090:> if "*" not in asok_path: 2024-03-11T18:37:17.497 DEBUG:teuthology.orchestra.run.smithi090:> assert(len(files) == 1) 2024-03-11T18:37:17.497 DEBUG:teuthology.orchestra.run.smithi090:> return files[0] 2024-03-11T18:37:17.497 DEBUG:teuthology.orchestra.run.smithi090:> 2024-03-11T18:37:17.498 DEBUG:teuthology.orchestra.run.smithi090:> for f in files: 2024-03-11T18:37:17.498 DEBUG:teuthology.orchestra.run.smithi090:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-11T18:37:17.498 DEBUG:teuthology.orchestra.run.smithi090:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-11T18:37:17.498 DEBUG:teuthology.orchestra.run.smithi090:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-11T18:37:17.498 DEBUG:teuthology.orchestra.run.smithi090:> contents = proc_f.read() 2024-03-11T18:37:17.498 DEBUG:teuthology.orchestra.run.smithi090:> if mountpoint in contents: 2024-03-11T18:37:17.498 DEBUG:teuthology.orchestra.run.smithi090:> return f 2024-03-11T18:37:17.498 DEBUG:teuthology.orchestra.run.smithi090:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-11T18:37:17.498 DEBUG:teuthology.orchestra.run.smithi090:> 2024-03-11T18:37:17.498 DEBUG:teuthology.orchestra.run.smithi090:> print(_find_admin_socket("client.0")) 2024-03-11T18:37:17.498 DEBUG:teuthology.orchestra.run.smithi090:> ' 2024-03-11T18:37:17.609 INFO:teuthology.orchestra.run.smithi090.stdout:/var/run/ceph/ceph-client.0.36299.asok 2024-03-11T18:37:17.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T18:37:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-11T18:37:17.620 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.36299.asok 2024-03-11T18:37:17.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T18:37:17.620 DEBUG:teuthology.orchestra.run.smithi090:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.36299.asok status 2024-03-11T18:37:17.935 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-03-11T18:37:17.935 INFO:teuthology.orchestra.run.smithi090.stdout: "metadata": { 2024-03-11T18:37:17.935 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_sha1": "94a539707733325322964c6a187c6a1b41be1daf", 2024-03-11T18:37:17.935 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_version": "ceph version 19.0.0-1607-g94a53970 (94a539707733325322964c6a187c6a1b41be1daf) squid (dev)", 2024-03-11T18:37:17.935 INFO:teuthology.orchestra.run.smithi090.stdout: "entity_id": "0", 2024-03-11T18:37:17.936 INFO:teuthology.orchestra.run.smithi090.stdout: "hostname": "smithi090", 2024-03-11T18:37:17.936 INFO:teuthology.orchestra.run.smithi090.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-11T18:37:17.936 INFO:teuthology.orchestra.run.smithi090.stdout: "pid": "36299", 2024-03-11T18:37:17.936 INFO:teuthology.orchestra.run.smithi090.stdout: "root": "/" 2024-03-11T18:37:17.936 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-03-11T18:37:17.936 INFO:teuthology.orchestra.run.smithi090.stdout: "dentry_count": 0, 2024-03-11T18:37:17.936 INFO:teuthology.orchestra.run.smithi090.stdout: "dentry_pinned_count": 0, 2024-03-11T18:37:17.936 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 4675, 2024-03-11T18:37:17.936 INFO:teuthology.orchestra.run.smithi090.stdout: "inst": { 2024-03-11T18:37:17.936 INFO:teuthology.orchestra.run.smithi090.stdout: "name": { 2024-03-11T18:37:17.937 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "client", 2024-03-11T18:37:17.937 INFO:teuthology.orchestra.run.smithi090.stdout: "num": 4675 2024-03-11T18:37:17.937 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-03-11T18:37:17.937 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": { 2024-03-11T18:37:17.937 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v1", 2024-03-11T18:37:17.937 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "172.21.15.90:0", 2024-03-11T18:37:17.937 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 3586776100 2024-03-11T18:37:17.937 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-03-11T18:37:17.937 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-03-11T18:37:17.938 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": { 2024-03-11T18:37:17.938 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v1", 2024-03-11T18:37:17.938 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "172.21.15.90:0", 2024-03-11T18:37:17.938 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 3586776100 2024-03-11T18:37:17.938 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-03-11T18:37:17.938 INFO:teuthology.orchestra.run.smithi090.stdout: "inst_str": "client.4675 172.21.15.90:0/3586776100", 2024-03-11T18:37:17.938 INFO:teuthology.orchestra.run.smithi090.stdout: "addr_str": "172.21.15.90:0/3586776100", 2024-03-11T18:37:17.938 INFO:teuthology.orchestra.run.smithi090.stdout: "inode_count": 1, 2024-03-11T18:37:17.938 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_epoch": 19, 2024-03-11T18:37:17.938 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_epoch": 30, 2024-03-11T18:37:17.938 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_epoch_barrier": 0, 2024-03-11T18:37:17.938 INFO:teuthology.orchestra.run.smithi090.stdout: "blocklisted": false, 2024-03-11T18:37:17.939 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs" 2024-03-11T18:37:17.939 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-03-11T18:37:17.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T18:37:17.954 DEBUG:teuthology.orchestra.run.smithi090:> stat --file-system '--printf=%T 2024-03-11T18:37:17.954 DEBUG:teuthology.orchestra.run.smithi090:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-11T18:37:18.074 INFO:teuthology.orchestra.run.smithi090.stdout:fuseblk 2024-03-11T18:37:18.074 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-11T18:37:18.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T18:37:18.075 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-11T18:37:18.151 INFO:teuthology.run_tasks:Running task workunit... 2024-03-11T18:37:18.400 INFO:tasks.workunit:Pulling workunits from ref b6a4997be2b8bb74411268cc42753f29ae9a78ad 2024-03-11T18:37:18.400 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-11T18:37:18.400 INFO:tasks.workunit:timeout=6h 2024-03-11T18:37:18.400 INFO:tasks.workunit:cleanup=True 2024-03-11T18:37:18.401 DEBUG:teuthology.orchestra.run.smithi090:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-11T18:37:18.409 INFO:teuthology.orchestra.run.smithi090.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-11T18:37:18.410 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-03-11T18:37:18.410 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 2dh/45d Inode: 1 Links: 2 2024-03-11T18:37:18.410 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-11T18:37:18.410 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-03-11T18:37:18.410 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-03-11 18:35:59.544487210 +0000 2024-03-11T18:37:18.410 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-03-11 18:36:15.574805993 +0000 2024-03-11T18:37:18.410 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: - 2024-03-11T18:37:18.411 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-11T18:37:18.411 DEBUG:teuthology.orchestra.run.smithi090:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-11T18:37:19.589 DEBUG:teuthology.orchestra.run.smithi090:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout b6a4997be2b8bb74411268cc42753f29ae9a78ad 2024-03-11T18:37:19.597 INFO:tasks.workunit.client.0.smithi090.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-11T18:37:34.238 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:37:34.241 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:38:04.483 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:38:04.486 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:38:21.468 INFO:tasks.workunit.client.0.smithi090.stderr:Note: switching to 'b6a4997be2b8bb74411268cc42753f29ae9a78ad'. 2024-03-11T18:38:21.468 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-03-11T18:38:21.468 INFO:tasks.workunit.client.0.smithi090.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-11T18:38:21.468 INFO:tasks.workunit.client.0.smithi090.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-11T18:38:21.469 INFO:tasks.workunit.client.0.smithi090.stderr:state without impacting any branches by switching back to a branch. 2024-03-11T18:38:21.469 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-03-11T18:38:21.469 INFO:tasks.workunit.client.0.smithi090.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-11T18:38:21.469 INFO:tasks.workunit.client.0.smithi090.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-11T18:38:21.469 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-03-11T18:38:21.469 INFO:tasks.workunit.client.0.smithi090.stderr: git switch -c 2024-03-11T18:38:21.469 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-03-11T18:38:21.469 INFO:tasks.workunit.client.0.smithi090.stderr:Or undo this operation with: 2024-03-11T18:38:21.469 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-03-11T18:38:21.470 INFO:tasks.workunit.client.0.smithi090.stderr: git switch - 2024-03-11T18:38:21.470 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-03-11T18:38:21.470 INFO:tasks.workunit.client.0.smithi090.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-11T18:38:21.470 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-03-11T18:38:21.470 INFO:tasks.workunit.client.0.smithi090.stderr:HEAD is now at b6a4997be2b Merge pull request #56060 from rhcs-dashboard/wip-64808-squid 2024-03-11T18:38:21.471 DEBUG:teuthology.orchestra.run.smithi090:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-11T18:38:21.477 INFO:tasks.workunit.client.0.smithi090.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-11T18:38:21.481 INFO:tasks.workunit.client.0.smithi090.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-11T18:38:21.481 INFO:tasks.workunit.client.0.smithi090.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-11T18:38:22.324 INFO:tasks.workunit.client.0.smithi090.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-11T18:38:22.439 INFO:tasks.workunit.client.0.smithi090.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-11T18:38:22.496 INFO:tasks.workunit.client.0.smithi090.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-11T18:38:22.497 INFO:tasks.workunit.client.0.smithi090.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-11T18:38:22.498 INFO:tasks.workunit.client.0.smithi090.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-11T18:38:22.540 INFO:tasks.workunit.client.0.smithi090.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-11T18:38:22.543 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-11T18:38:22.543 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-11T18:38:22.590 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-03-11T18:38:22.591 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-03-11T18:38:22.591 DEBUG:teuthology.orchestra.run.smithi090:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b6a4997be2b8bb74411268cc42753f29ae9a78ad TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-03-11T18:38:23.128 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir -p fsstress 2024-03-11T18:38:23.311 INFO:tasks.workunit.client.0.smithi090.stderr:+ pushd fsstress 2024-03-11T18:38:23.368 INFO:tasks.workunit.client.0.smithi090.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-11T18:38:23.369 INFO:tasks.workunit.client.0.smithi090.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-03-11T18:38:34.537 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:38:34.540 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:38:38.031 INFO:tasks.workunit.client.0.smithi090.stderr:+ tar xzf ltp-full.tgz 2024-03-11T18:39:04.582 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:39:04.585 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:39:34.650 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:39:34.659 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:40:04.798 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:40:04.802 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:40:34.927 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:40:34.930 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:41:04.972 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:41:04.977 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:41:35.102 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:41:35.105 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:42:05.178 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:42:05.181 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:42:35.307 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:42:35.310 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:43:05.419 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:43:05.422 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:43:35.546 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:43:35.549 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:44:05.701 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:44:05.707 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:44:08.891 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:09:28.647 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T18:44:09.006 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:09:28.762 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T18:44:09.091 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:09:28.794 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T18:44:09.092 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:09:28.787 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T18:44:09.235 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:09:28.938 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T18:44:09.238 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:09:28.933 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T18:44:09.502 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:09:29.258 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T18:44:09.722 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:09:29.426 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T18:44:09.757 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:09:29.452 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T18:44:35.770 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:44:35.773 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:45:05.823 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:45:05.826 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:45:35.868 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:45:35.871 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:46:05.922 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:46:05.925 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:46:36.075 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:46:36.078 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:46:39.607 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:11:59.364 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T18:46:39.620 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:11:59.376 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T18:46:39.658 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:11:59.353 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T18:46:39.659 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:11:59.362 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T18:46:39.668 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:11:59.360 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T18:46:39.669 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:11:59.368 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T18:46:39.772 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:11:59.474 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T18:46:39.817 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:11:59.512 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T18:46:39.818 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:11:59.571 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T18:47:06.154 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:47:06.157 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:47:36.266 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:47:36.269 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:48:06.352 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:48:06.354 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:48:36.472 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:48:36.474 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:49:06.617 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:49:06.620 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:49:36.670 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:49:36.673 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:50:06.898 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:50:06.901 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:50:36.944 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:50:36.948 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:50:47.761 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:16:07.518 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T18:50:47.767 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:16:07.525 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T18:50:47.882 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:16:07.578 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T18:50:47.888 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:16:07.584 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T18:50:47.888 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:16:07.592 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T18:50:47.894 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:16:07.598 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T18:50:47.974 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:16:07.732 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T18:50:48.038 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:16:07.734 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T18:50:48.062 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:16:07.766 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T18:51:06.988 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:51:06.991 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:51:37.034 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:51:37.037 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:52:07.112 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:52:07.115 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:52:32.158 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:17:51.915 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T18:52:32.162 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:17:51.919 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T18:52:32.275 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:17:51.972 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T18:52:32.278 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:17:51.975 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T18:52:32.280 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:17:51.985 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T18:52:32.283 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:17:51.988 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T18:52:32.307 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:17:52.064 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T18:52:32.349 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:17:52.054 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T18:52:32.353 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:17:52.050 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T18:52:37.250 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:52:37.253 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:53:07.303 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:53:07.307 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:53:37.383 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:53:37.386 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:54:07.518 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:54:07.522 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:54:37.562 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:54:37.566 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:54:55.023 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:20:14.779 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T18:54:55.038 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:20:14.795 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T18:54:55.144 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:20:14.840 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T18:54:55.149 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:20:14.853 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T18:54:55.153 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:20:14.849 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T18:54:55.158 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:20:14.862 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T18:54:55.159 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:20:14.916 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T18:54:55.268 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:20:14.972 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T18:54:55.339 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:20:15.035 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T18:55:07.666 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:55:07.670 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:55:37.787 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:55:37.791 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:56:07.832 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:56:07.835 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:56:37.886 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:56:37.891 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:57:07.957 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:57:07.960 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:57:38.094 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:57:38.097 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:58:08.214 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:58:08.217 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:58:38.292 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:58:38.295 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:59:08.421 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:59:08.425 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:59:24.648 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:24:44.405 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T18:59:24.652 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:24:44.409 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T18:59:24.769 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:24:44.465 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T18:59:24.774 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:24:44.462 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T18:59:24.781 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:24:44.477 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T18:59:24.789 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:24:44.478 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T18:59:24.816 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:24:44.574 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T18:59:24.940 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:24:44.636 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T18:59:24.972 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:24:44.660 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T18:59:38.490 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T18:59:38.494 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:00:08.543 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:00:08.547 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:00:38.664 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:00:38.668 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:00:55.389 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:26:15.147 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:00:55.397 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:26:15.155 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:00:55.444 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:26:15.138 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:00:55.452 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:26:15.146 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:00:55.471 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:26:15.229 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:00:55.497 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:26:15.190 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:00:55.511 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:26:15.197 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:00:55.514 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:26:15.200 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:00:55.574 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:26:15.260 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:01:08.767 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:01:08.770 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:01:38.871 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:01:38.875 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:02:08.941 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:02:08.945 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:02:38.995 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:02:38.999 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:03:08.903 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:28:28.661 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:03:08.922 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:28:28.680 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:03:08.944 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:28:28.635 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:03:08.954 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:28:28.636 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:03:08.959 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:28:28.650 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:03:08.961 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:28:28.644 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:03:09.048 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:03:09.051 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:03:09.051 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:28:28.809 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:03:09.062 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:28:28.745 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:03:09.094 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:28:28.784 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:03:39.409 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:03:39.413 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:04:09.463 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:04:09.466 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:04:39.534 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:04:39.537 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:05:09.653 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:05:09.657 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:05:39.781 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:05:39.787 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:06:09.895 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:06:09.898 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:06:40.040 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:06:40.043 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:07:10.369 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:07:10.373 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:07:19.048 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:32:38.807 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:07:19.069 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:32:38.829 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:07:19.097 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:32:38.786 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:07:19.101 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:32:38.782 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:07:19.115 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:32:38.804 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:07:19.117 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:32:38.798 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:07:19.232 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:32:38.992 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:07:19.274 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:32:38.955 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:07:19.337 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:32:39.026 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:07:40.490 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:07:40.494 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:07:42.888 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:33:02.648 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:07:42.990 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:33:02.749 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:07:43.039 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:33:02.799 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:07:43.102 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:33:02.791 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:07:43.105 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:33:02.786 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:07:43.118 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:33:02.807 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:07:43.125 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:33:02.805 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:07:43.160 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:33:02.841 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:07:43.172 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:33:02.861 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:08:10.541 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:08:10.544 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:08:40.603 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:08:40.606 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:09:10.662 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:09:10.666 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:09:40.726 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:09:40.735 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:10:10.846 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:10:10.849 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:10:40.892 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:10:40.895 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:11:11.163 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:11:11.167 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:11:26.671 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:36:46.431 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:11:26.715 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:36:46.397 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:11:26.716 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:36:46.406 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:11:26.723 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:36:46.404 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:11:26.723 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:36:46.413 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:11:26.767 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:36:46.528 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:11:26.895 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:36:46.576 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:11:26.897 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:36:46.587 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:11:26.906 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:36:46.667 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:11:41.216 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:11:41.219 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:12:11.285 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:12:11.288 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:12:41.332 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:12:41.336 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:13:11.460 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:13:11.463 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:13:41.588 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:13:41.591 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:14:11.724 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:14:11.727 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:14:41.786 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:14:41.789 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:15:11.832 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:15:11.836 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:15:35.313 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:40:55.074 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:15:35.320 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:40:55.081 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:15:35.354 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:40:55.038 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:15:35.366 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:40:55.058 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:15:35.369 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:40:55.052 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:15:35.378 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:40:55.069 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:15:35.503 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:40:55.264 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:15:35.533 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:40:55.217 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:15:35.537 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:40:55.229 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:15:41.885 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:15:41.888 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:16:06.528 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:41:26.289 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:16:06.535 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:41:26.296 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:16:06.586 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:41:26.347 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:16:06.628 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:41:26.319 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:16:06.634 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:41:26.326 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:16:06.636 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:41:26.320 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:16:06.646 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:41:26.330 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:16:06.699 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:41:26.391 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:16:06.710 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:41:26.393 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:16:12.022 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:16:12.025 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:16:42.101 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:16:42.104 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:17:12.171 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:17:12.173 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:17:42.225 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:17:42.227 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:17:42.310 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:43:02.071 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:17:42.312 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:43:02.072 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:17:42.369 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:43:02.130 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:17:42.427 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:43:02.111 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:17:42.431 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:43:02.115 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:17:42.438 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:43:02.130 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:17:42.441 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:43:02.134 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:17:42.523 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:43:02.207 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:17:42.528 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:43:02.221 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:18:12.384 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:18:12.388 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:18:42.455 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:18:42.458 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:19:12.544 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:19:12.547 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:19:41.502 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:45:01.262 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:19:41.532 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:45:01.292 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:19:41.547 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:45:01.232 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:19:41.557 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:45:01.242 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:19:41.561 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:45:01.254 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:19:41.571 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:45:01.265 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:19:41.676 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:45:01.437 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:19:41.699 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:45:01.384 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:19:41.708 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:45:01.402 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:19:42.663 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:19:42.666 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:20:12.993 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:20:12.996 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:20:43.037 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:20:43.040 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:21:13.098 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:21:13.101 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:21:43.151 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:21:43.155 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:22:10.015 INFO:tasks.workunit.client.0.smithi090.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-03-11T19:22:10.334 INFO:tasks.workunit.client.0.smithi090.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-11T19:22:10.334 INFO:tasks.workunit.client.0.smithi090.stderr:+ make 2024-03-11T19:22:12.613 INFO:tasks.workunit.client.0.smithi090.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-03-11T19:22:13.230 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:22:13.234 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:22:21.211 INFO:tasks.workunit.client.0.smithi090.stderr:++ readlink -f fsstress 2024-03-11T19:22:21.253 INFO:tasks.workunit.client.0.smithi090.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-03-11T19:22:21.253 INFO:tasks.workunit.client.0.smithi090.stderr:+ popd 2024-03-11T19:22:21.302 INFO:tasks.workunit.client.0.smithi090.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-11T19:22:21.302 INFO:tasks.workunit.client.0.smithi090.stderr:+ popd 2024-03-11T19:22:21.326 INFO:tasks.workunit.client.0.smithi090.stdout:~/cephtest/mnt.0/client.0/tmp 2024-03-11T19:22:21.327 INFO:tasks.workunit.client.0.smithi090.stderr:++ mktemp -d -p . 2024-03-11T19:22:21.410 INFO:tasks.workunit.client.0.smithi090.stderr:+ T=./tmp.t3HdW5tyEI 2024-03-11T19:22:21.410 INFO:tasks.workunit.client.0.smithi090.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.t3HdW5tyEI -l 1 -n 1000 -p 10 -v 2024-03-11T19:22:21.654 INFO:tasks.workunit.client.0.smithi090.stdout:seed = 1709615407 2024-03-11T19:22:21.915 INFO:tasks.workunit.client.0.smithi090.stdout:0/0: link - no file 2024-03-11T19:22:21.915 INFO:tasks.workunit.client.0.smithi090.stdout:0/1: write - no filename 2024-03-11T19:22:21.915 INFO:tasks.workunit.client.0.smithi090.stdout:0/2: write - no filename 2024-03-11T19:22:21.915 INFO:tasks.workunit.client.0.smithi090.stdout:0/3: read - no filename 2024-03-11T19:22:21.915 INFO:tasks.workunit.client.0.smithi090.stdout:0/4: dwrite - no filename 2024-03-11T19:22:21.915 INFO:tasks.workunit.client.0.smithi090.stdout:0/5: write - no filename 2024-03-11T19:22:21.915 INFO:tasks.workunit.client.0.smithi090.stdout:0/6: read - no filename 2024-03-11T19:22:21.915 INFO:tasks.workunit.client.0.smithi090.stdout:0/7: rmdir - no directory 2024-03-11T19:22:21.915 INFO:tasks.workunit.client.0.smithi090.stdout:0/8: dwrite - no filename 2024-03-11T19:22:21.915 INFO:tasks.workunit.client.0.smithi090.stdout:0/9: fsync - no filename 2024-03-11T19:22:21.916 INFO:tasks.workunit.client.0.smithi090.stdout:0/10: dread - no filename 2024-03-11T19:22:21.916 INFO:tasks.workunit.client.0.smithi090.stdout:0/11: unlink - no file 2024-03-11T19:22:21.979 INFO:tasks.workunit.client.0.smithi090.stdout:1/0: truncate - no filename 2024-03-11T19:22:22.097 INFO:tasks.workunit.client.0.smithi090.stdout:2/0: write - no filename 2024-03-11T19:22:22.098 INFO:tasks.workunit.client.0.smithi090.stdout:2/1: unlink - no file 2024-03-11T19:22:22.098 INFO:tasks.workunit.client.0.smithi090.stdout:2/2: dwrite - no filename 2024-03-11T19:22:22.098 INFO:tasks.workunit.client.0.smithi090.stdout:2/3: stat - no entries 2024-03-11T19:22:22.098 INFO:tasks.workunit.client.0.smithi090.stdout:2/4: write - no filename 2024-03-11T19:22:22.177 INFO:tasks.workunit.client.0.smithi090.stdout:0/12: creat f0 x:0 0 0 2024-03-11T19:22:22.190 INFO:tasks.workunit.client.0.smithi090.stdout:0/13: stat f0 0 2024-03-11T19:22:22.190 INFO:tasks.workunit.client.0.smithi090.stdout:0/14: readlink - no filename 2024-03-11T19:22:22.336 INFO:tasks.workunit.client.0.smithi090.stdout:1/1: mkdir d0 0 2024-03-11T19:22:22.339 INFO:tasks.workunit.client.0.smithi090.stdout:3/0: write - no filename 2024-03-11T19:22:22.339 INFO:tasks.workunit.client.0.smithi090.stdout:3/1: read - no filename 2024-03-11T19:22:22.339 INFO:tasks.workunit.client.0.smithi090.stdout:3/2: truncate - no filename 2024-03-11T19:22:22.354 INFO:tasks.workunit.client.0.smithi090.stdout:0/15: mknod c1 0 2024-03-11T19:22:22.369 INFO:tasks.workunit.client.0.smithi090.stdout:2/5: mkdir d0 0 2024-03-11T19:22:22.369 INFO:tasks.workunit.client.0.smithi090.stdout:2/6: dwrite - no filename 2024-03-11T19:22:22.369 INFO:tasks.workunit.client.0.smithi090.stdout:2/7: dread - no filename 2024-03-11T19:22:22.369 INFO:tasks.workunit.client.0.smithi090.stdout:2/8: write - no filename 2024-03-11T19:22:22.369 INFO:tasks.workunit.client.0.smithi090.stdout:2/9: truncate - no filename 2024-03-11T19:22:22.369 INFO:tasks.workunit.client.0.smithi090.stdout:2/10: truncate - no filename 2024-03-11T19:22:22.369 INFO:tasks.workunit.client.0.smithi090.stdout:2/11: dwrite - no filename 2024-03-11T19:22:22.369 INFO:tasks.workunit.client.0.smithi090.stdout:2/12: dread - no filename 2024-03-11T19:22:22.369 INFO:tasks.workunit.client.0.smithi090.stdout:2/13: write - no filename 2024-03-11T19:22:22.509 INFO:tasks.workunit.client.0.smithi090.stdout:0/16: mkdir d2 0 2024-03-11T19:22:22.649 INFO:tasks.workunit.client.0.smithi090.stdout:0/17: write f0 [958719,33929] 0 2024-03-11T19:22:22.793 INFO:tasks.workunit.client.0.smithi090.stdout:1/2: symlink d0/l1 0 2024-03-11T19:22:22.793 INFO:tasks.workunit.client.0.smithi090.stdout:1/3: fsync - no filename 2024-03-11T19:22:22.804 INFO:tasks.workunit.client.0.smithi090.stdout:3/3: creat f0 x:0 0 0 2024-03-11T19:22:22.823 INFO:tasks.workunit.client.0.smithi090.stdout:2/14: creat d0/f1 x:0 0 0 2024-03-11T19:22:22.880 INFO:tasks.workunit.client.0.smithi090.stdout:4/0: mknod c0 0 2024-03-11T19:22:22.881 INFO:tasks.workunit.client.0.smithi090.stdout:4/1: write - no filename 2024-03-11T19:22:22.905 INFO:tasks.workunit.client.0.smithi090.stdout:5/0: dread - no filename 2024-03-11T19:22:22.953 INFO:tasks.workunit.client.0.smithi090.stdout:3/4: symlink l1 0 2024-03-11T19:22:22.976 INFO:tasks.workunit.client.0.smithi090.stdout:6/0: rename - no filename 2024-03-11T19:22:22.976 INFO:tasks.workunit.client.0.smithi090.stdout:6/1: rename - no filename 2024-03-11T19:22:22.976 INFO:tasks.workunit.client.0.smithi090.stdout:6/2: readlink - no filename 2024-03-11T19:22:22.976 INFO:tasks.workunit.client.0.smithi090.stdout:6/3: readlink - no filename 2024-03-11T19:22:22.976 INFO:tasks.workunit.client.0.smithi090.stdout:6/4: write - no filename 2024-03-11T19:22:22.984 INFO:tasks.workunit.client.0.smithi090.stdout:6/5: chown . 1603 1 2024-03-11T19:22:23.011 INFO:tasks.workunit.client.0.smithi090.stdout:3/5: write f0 [193549,6853] 0 2024-03-11T19:22:23.038 INFO:tasks.workunit.client.0.smithi090.stdout:1/4: unlink d0/l1 0 2024-03-11T19:22:23.039 INFO:tasks.workunit.client.0.smithi090.stdout:1/5: truncate - no filename 2024-03-11T19:22:23.039 INFO:tasks.workunit.client.0.smithi090.stdout:1/6: dwrite - no filename 2024-03-11T19:22:23.078 INFO:tasks.workunit.client.0.smithi090.stdout:2/15: creat d0/f2 x:0 0 0 2024-03-11T19:22:23.115 INFO:tasks.workunit.client.0.smithi090.stdout:1/7: rename d0 to d0/d2 22 2024-03-11T19:22:23.116 INFO:tasks.workunit.client.0.smithi090.stdout:1/8: dread - no filename 2024-03-11T19:22:23.116 INFO:tasks.workunit.client.0.smithi090.stdout:1/9: unlink - no file 2024-03-11T19:22:23.230 INFO:tasks.workunit.client.0.smithi090.stdout:5/1: mknod c0 0 2024-03-11T19:22:23.271 INFO:tasks.workunit.client.0.smithi090.stdout:0/18: sync 2024-03-11T19:22:23.356 INFO:tasks.workunit.client.0.smithi090.stdout:6/6: creat f0 x:0 0 0 2024-03-11T19:22:23.430 INFO:tasks.workunit.client.0.smithi090.stdout:5/2: chown c0 8403740 1 2024-03-11T19:22:23.430 INFO:tasks.workunit.client.0.smithi090.stdout:5/3: dread - no filename 2024-03-11T19:22:23.466 INFO:tasks.workunit.client.0.smithi090.stdout:0/19: chown c1 21 1 2024-03-11T19:22:23.546 INFO:tasks.workunit.client.0.smithi090.stdout:3/6: dwrite f0 [0,4194304] 0 2024-03-11T19:22:23.750 INFO:tasks.workunit.client.0.smithi090.stdout:5/4: chown c0 2852895 1 2024-03-11T19:22:23.750 INFO:tasks.workunit.client.0.smithi090.stdout:5/5: write - no filename 2024-03-11T19:22:23.768 INFO:tasks.workunit.client.0.smithi090.stdout:6/7: mknod c1 0 2024-03-11T19:22:23.773 INFO:tasks.workunit.client.0.smithi090.stdout:8/0: dread - no filename 2024-03-11T19:22:23.799 INFO:tasks.workunit.client.0.smithi090.stdout:5/6: chown c0 645 1 2024-03-11T19:22:23.873 INFO:tasks.workunit.client.0.smithi090.stdout:6/8: sync 2024-03-11T19:22:23.922 INFO:tasks.workunit.client.0.smithi090.stdout:7/0: getdents . 0 2024-03-11T19:22:24.129 INFO:tasks.workunit.client.0.smithi090.stdout:9/0: truncate - no filename 2024-03-11T19:22:24.129 INFO:tasks.workunit.client.0.smithi090.stdout:9/1: truncate - no filename 2024-03-11T19:22:24.211 INFO:tasks.workunit.client.0.smithi090.stdout:3/7: mkdir d2 0 2024-03-11T19:22:24.377 INFO:tasks.workunit.client.0.smithi090.stdout:5/7: mkdir d1 0 2024-03-11T19:22:24.510 INFO:tasks.workunit.client.0.smithi090.stdout:3/8: dwrite f0 [0,4194304] 0 2024-03-11T19:22:24.674 INFO:tasks.workunit.client.0.smithi090.stdout:6/9: unlink f0 0 2024-03-11T19:22:24.771 INFO:tasks.workunit.client.0.smithi090.stdout:0/20: creat d2/f3 x:0 0 0 2024-03-11T19:22:24.899 INFO:tasks.workunit.client.0.smithi090.stdout:4/2: getdents . 0 2024-03-11T19:22:24.900 INFO:tasks.workunit.client.0.smithi090.stdout:4/3: dwrite - no filename 2024-03-11T19:22:25.011 INFO:tasks.workunit.client.0.smithi090.stdout:8/1: mkdir d0 0 2024-03-11T19:22:25.011 INFO:tasks.workunit.client.0.smithi090.stdout:8/2: dread - no filename 2024-03-11T19:22:25.209 INFO:tasks.workunit.client.0.smithi090.stdout:0/21: dwrite d2/f3 [0,4194304] 0 2024-03-11T19:22:25.342 INFO:tasks.workunit.client.0.smithi090.stdout:2/16: getdents d0 0 2024-03-11T19:22:25.426 INFO:tasks.workunit.client.0.smithi090.stdout:2/17: write d0/f1 [965081,129947] 0 2024-03-11T19:22:25.539 INFO:tasks.workunit.client.0.smithi090.stdout:7/1: mknod c0 0 2024-03-11T19:22:25.539 INFO:tasks.workunit.client.0.smithi090.stdout:7/2: write - no filename 2024-03-11T19:22:25.546 INFO:tasks.workunit.client.0.smithi090.stdout:6/10: mknod c2 0 2024-03-11T19:22:25.645 INFO:tasks.workunit.client.0.smithi090.stdout:9/2: creat f0 x:0 0 0 2024-03-11T19:22:25.698 INFO:tasks.workunit.client.0.smithi090.stdout:4/4: creat f1 x:0 0 0 2024-03-11T19:22:25.741 INFO:tasks.workunit.client.0.smithi090.stdout:2/18: write d0/f2 [258660,26702] 0 2024-03-11T19:22:25.753 INFO:tasks.workunit.client.0.smithi090.stdout:6/11: chown c2 108339 1 2024-03-11T19:22:25.753 INFO:tasks.workunit.client.0.smithi090.stdout:6/12: dread - no filename 2024-03-11T19:22:25.753 INFO:tasks.workunit.client.0.smithi090.stdout:6/13: write - no filename 2024-03-11T19:22:25.753 INFO:tasks.workunit.client.0.smithi090.stdout:6/14: dread - no filename 2024-03-11T19:22:25.876 INFO:tasks.workunit.client.0.smithi090.stdout:4/5: dread - f1 zero size 2024-03-11T19:22:25.939 INFO:tasks.workunit.client.0.smithi090.stdout:0/22: dwrite f0 [0,4194304] 0 2024-03-11T19:22:26.058 INFO:tasks.workunit.client.0.smithi090.stdout:5/8: symlink d1/l2 0 2024-03-11T19:22:26.069 INFO:tasks.workunit.client.0.smithi090.stdout:6/15: chown c1 28 1 2024-03-11T19:22:26.113 INFO:tasks.workunit.client.0.smithi090.stdout:4/6: chown c0 3501 1 2024-03-11T19:22:26.113 INFO:tasks.workunit.client.0.smithi090.stdout:4/7: rmdir - no directory 2024-03-11T19:22:26.115 INFO:tasks.workunit.client.0.smithi090.stdout:2/19: read d0/f1 [765207,73237] 0 2024-03-11T19:22:26.118 INFO:tasks.workunit.client.0.smithi090.stdout:6/16: stat c1 0 2024-03-11T19:22:26.118 INFO:tasks.workunit.client.0.smithi090.stdout:6/17: write - no filename 2024-03-11T19:22:26.119 INFO:tasks.workunit.client.0.smithi090.stdout:6/18: fdatasync - no filename 2024-03-11T19:22:26.119 INFO:tasks.workunit.client.0.smithi090.stdout:6/19: write - no filename 2024-03-11T19:22:26.119 INFO:tasks.workunit.client.0.smithi090.stdout:6/20: dwrite - no filename 2024-03-11T19:22:26.180 INFO:tasks.workunit.client.0.smithi090.stdout:4/8: dread - f1 zero size 2024-03-11T19:22:26.198 INFO:tasks.workunit.client.0.smithi090.stdout:6/21: chown c1 9346 1 2024-03-11T19:22:26.341 INFO:tasks.workunit.client.0.smithi090.stdout:3/9: creat d2/f3 x:0 0 0 2024-03-11T19:22:26.355 INFO:tasks.workunit.client.0.smithi090.stdout:4/9: write f1 [78123,89406] 0 2024-03-11T19:22:26.356 INFO:tasks.workunit.client.0.smithi090.stdout:4/10: readlink - no filename 2024-03-11T19:22:26.474 INFO:tasks.workunit.client.0.smithi090.stdout:7/3: creat f1 x:0 0 0 2024-03-11T19:22:26.485 INFO:tasks.workunit.client.0.smithi090.stdout:9/3: mkdir d1 0 2024-03-11T19:22:26.496 INFO:tasks.workunit.client.0.smithi090.stdout:2/20: dread d0/f1 [0,4194304] 0 2024-03-11T19:22:26.524 INFO:tasks.workunit.client.0.smithi090.stdout:8/3: symlink d0/l1 0 2024-03-11T19:22:26.525 INFO:tasks.workunit.client.0.smithi090.stdout:8/4: write - no filename 2024-03-11T19:22:26.554 INFO:tasks.workunit.client.0.smithi090.stdout:3/10: write f0 [4703637,58840] 0 2024-03-11T19:22:26.660 INFO:tasks.workunit.client.0.smithi090.stdout:1/10: rmdir d0 0 2024-03-11T19:22:26.687 INFO:tasks.workunit.client.0.smithi090.stdout:0/23: creat d2/f4 x:0 0 0 2024-03-11T19:22:26.743 INFO:tasks.workunit.client.0.smithi090.stdout:9/4: write f0 [88400,79267] 0 2024-03-11T19:22:26.789 INFO:tasks.workunit.client.0.smithi090.stdout:5/9: creat d1/f3 x:0 0 0 2024-03-11T19:22:26.832 INFO:tasks.workunit.client.0.smithi090.stdout:0/24: fdatasync d2/f4 0 2024-03-11T19:22:26.858 INFO:tasks.workunit.client.0.smithi090.stdout:5/10: stat c0 0 2024-03-11T19:22:26.903 INFO:tasks.workunit.client.0.smithi090.stdout:6/22: creat f3 x:0 0 0 2024-03-11T19:22:26.923 INFO:tasks.workunit.client.0.smithi090.stdout:9/5: dread f0 [0,4194304] 0 2024-03-11T19:22:26.947 INFO:tasks.workunit.client.0.smithi090.stdout:5/11: readlink d1/l2 0 2024-03-11T19:22:26.999 INFO:tasks.workunit.client.0.smithi090.stdout:5/12: stat d1/l2 0 2024-03-11T19:22:27.014 INFO:tasks.workunit.client.0.smithi090.stdout:6/23: write f3 [950282,51561] 0 2024-03-11T19:22:27.229 INFO:tasks.workunit.client.0.smithi090.stdout:4/11: mknod c2 0 2024-03-11T19:22:27.233 INFO:tasks.workunit.client.0.smithi090.stdout:9/6: fsync f0 0 2024-03-11T19:22:27.307 INFO:tasks.workunit.client.0.smithi090.stdout:4/12: write f1 [1123508,108860] 0 2024-03-11T19:22:27.343 INFO:tasks.workunit.client.0.smithi090.stdout:4/13: chown f1 29 1 2024-03-11T19:22:27.356 INFO:tasks.workunit.client.0.smithi090.stdout:4/14: chown f1 288 1 2024-03-11T19:22:27.435 INFO:tasks.workunit.client.0.smithi090.stdout:7/4: rename f1 to f2 0 2024-03-11T19:22:27.821 INFO:tasks.workunit.client.0.smithi090.stdout:8/5: mknod d0/c2 0 2024-03-11T19:22:27.842 INFO:tasks.workunit.client.0.smithi090.stdout:2/21: rename d0/f1 to d0/f3 0 2024-03-11T19:22:27.949 INFO:tasks.workunit.client.0.smithi090.stdout:1/11: symlink l3 0 2024-03-11T19:22:27.994 INFO:tasks.workunit.client.0.smithi090.stdout:3/11: mkdir d2/d4 0 2024-03-11T19:22:28.188 INFO:tasks.workunit.client.0.smithi090.stdout:0/25: creat d2/f5 x:0 0 0 2024-03-11T19:22:28.435 INFO:tasks.workunit.client.0.smithi090.stdout:5/13: mkdir d1/d4 0 2024-03-11T19:22:28.475 INFO:tasks.workunit.client.0.smithi090.stdout:5/14: write d1/f3 [166521,41107] 0 2024-03-11T19:22:28.687 INFO:tasks.workunit.client.0.smithi090.stdout:6/24: mkdir d4 0 2024-03-11T19:22:28.796 INFO:tasks.workunit.client.0.smithi090.stdout:5/15: dwrite d1/f3 [0,4194304] 0 2024-03-11T19:22:28.909 INFO:tasks.workunit.client.0.smithi090.stdout:5/16: readlink d1/l2 0 2024-03-11T19:22:28.944 INFO:tasks.workunit.client.0.smithi090.stdout:4/15: mknod c3 0 2024-03-11T19:22:28.955 INFO:tasks.workunit.client.0.smithi090.stdout:7/5: mkdir d3 0 2024-03-11T19:22:29.006 INFO:tasks.workunit.client.0.smithi090.stdout:4/16: fdatasync f1 0 2024-03-11T19:22:29.030 INFO:tasks.workunit.client.0.smithi090.stdout:6/25: dread f3 [0,4194304] 0 2024-03-11T19:22:29.232 INFO:tasks.workunit.client.0.smithi090.stdout:8/6: unlink d0/c2 0 2024-03-11T19:22:29.232 INFO:tasks.workunit.client.0.smithi090.stdout:8/7: write - no filename 2024-03-11T19:22:29.232 INFO:tasks.workunit.client.0.smithi090.stdout:8/8: dread - no filename 2024-03-11T19:22:29.252 INFO:tasks.workunit.client.0.smithi090.stdout:5/17: dread d1/f3 [0,4194304] 0 2024-03-11T19:22:29.268 INFO:tasks.workunit.client.0.smithi090.stdout:1/12: mknod c4 0 2024-03-11T19:22:29.268 INFO:tasks.workunit.client.0.smithi090.stdout:1/13: fsync - no filename 2024-03-11T19:22:29.357 INFO:tasks.workunit.client.0.smithi090.stdout:3/12: mknod d2/c5 0 2024-03-11T19:22:29.444 INFO:tasks.workunit.client.0.smithi090.stdout:8/9: chown d0/l1 1434487 1 2024-03-11T19:22:29.584 INFO:tasks.workunit.client.0.smithi090.stdout:3/13: write d2/f3 [627713,76391] 0 2024-03-11T19:22:29.620 INFO:tasks.workunit.client.0.smithi090.stdout:0/26: creat d2/f6 x:0 0 0 2024-03-11T19:22:29.725 INFO:tasks.workunit.client.0.smithi090.stdout:3/14: fdatasync f0 0 2024-03-11T19:22:29.731 INFO:tasks.workunit.client.0.smithi090.stdout:4/17: dwrite f1 [0,4194304] 0 2024-03-11T19:22:29.883 INFO:tasks.workunit.client.0.smithi090.stdout:9/7: mknod d1/c2 0 2024-03-11T19:22:29.924 INFO:tasks.workunit.client.0.smithi090.stdout:9/8: chown d1 12813228 1 2024-03-11T19:22:30.197 INFO:tasks.workunit.client.0.smithi090.stdout:1/14: mkdir d5 0 2024-03-11T19:22:30.197 INFO:tasks.workunit.client.0.smithi090.stdout:1/15: dread - no filename 2024-03-11T19:22:30.197 INFO:tasks.workunit.client.0.smithi090.stdout:1/16: write - no filename 2024-03-11T19:22:30.357 INFO:tasks.workunit.client.0.smithi090.stdout:5/18: rename c0 to d1/d4/c5 0 2024-03-11T19:22:30.427 INFO:tasks.workunit.client.0.smithi090.stdout:8/10: mkdir d0/d3 0 2024-03-11T19:22:30.675 INFO:tasks.workunit.client.0.smithi090.stdout:5/19: sync 2024-03-11T19:22:30.675 INFO:tasks.workunit.client.0.smithi090.stdout:8/11: sync 2024-03-11T19:22:30.715 INFO:tasks.workunit.client.0.smithi090.stdout:0/27: creat d2/f7 x:0 0 0 2024-03-11T19:22:30.846 INFO:tasks.workunit.client.0.smithi090.stdout:0/28: stat d2/f3 0 2024-03-11T19:22:30.854 INFO:tasks.workunit.client.0.smithi090.stdout:8/12: rename d0/d3 to d0/d3/d4 22 2024-03-11T19:22:30.880 INFO:tasks.workunit.client.0.smithi090.stdout:4/18: mkdir d4 0 2024-03-11T19:22:31.028 INFO:tasks.workunit.client.0.smithi090.stdout:2/22: dwrite d0/f3 [0,4194304] 0 2024-03-11T19:22:31.164 INFO:tasks.workunit.client.0.smithi090.stdout:4/19: write f1 [3456851,37020] 0 2024-03-11T19:22:31.176 INFO:tasks.workunit.client.0.smithi090.stdout:8/13: chown d0/l1 29020 1 2024-03-11T19:22:31.176 INFO:tasks.workunit.client.0.smithi090.stdout:8/14: write - no filename 2024-03-11T19:22:31.268 INFO:tasks.workunit.client.0.smithi090.stdout:8/15: stat d0/d3 0 2024-03-11T19:22:31.269 INFO:tasks.workunit.client.0.smithi090.stdout:8/16: dread - no filename 2024-03-11T19:22:31.269 INFO:tasks.workunit.client.0.smithi090.stdout:8/17: fdatasync - no filename 2024-03-11T19:22:31.372 INFO:tasks.workunit.client.0.smithi090.stdout:9/9: creat d1/f3 x:0 0 0 2024-03-11T19:22:31.397 INFO:tasks.workunit.client.0.smithi090.stdout:7/6: creat d3/f4 x:0 0 0 2024-03-11T19:22:31.604 INFO:tasks.workunit.client.0.smithi090.stdout:7/7: chown d3 373 1 2024-03-11T19:22:31.792 INFO:tasks.workunit.client.0.smithi090.stdout:0/29: dwrite d2/f7 [0,4194304] 0 2024-03-11T19:22:32.014 INFO:tasks.workunit.client.0.smithi090.stdout:3/15: link f0 d2/f6 0 2024-03-11T19:22:32.023 INFO:tasks.workunit.client.0.smithi090.stdout:7/8: dread - d3/f4 zero size 2024-03-11T19:22:32.099 INFO:tasks.workunit.client.0.smithi090.stdout:4/20: dread f1 [0,4194304] 0 2024-03-11T19:22:32.139 INFO:tasks.workunit.client.0.smithi090.stdout:0/30: stat d2/f5 0 2024-03-11T19:22:32.228 INFO:tasks.workunit.client.0.smithi090.stdout:7/9: dread - f2 zero size 2024-03-11T19:22:32.236 INFO:tasks.workunit.client.0.smithi090.stdout:7/10: readlink - no filename 2024-03-11T19:22:32.276 INFO:tasks.workunit.client.0.smithi090.stdout:6/26: link c1 d4/c5 0 2024-03-11T19:22:32.334 INFO:tasks.workunit.client.0.smithi090.stdout:2/23: dwrite d0/f2 [0,4194304] 0 2024-03-11T19:22:32.477 INFO:tasks.workunit.client.0.smithi090.stdout:1/17: creat d5/f6 x:0 0 0 2024-03-11T19:22:32.513 INFO:tasks.workunit.client.0.smithi090.stdout:8/18: unlink d0/l1 0 2024-03-11T19:22:32.513 INFO:tasks.workunit.client.0.smithi090.stdout:8/19: link - no file 2024-03-11T19:22:32.513 INFO:tasks.workunit.client.0.smithi090.stdout:8/20: dread - no filename 2024-03-11T19:22:32.513 INFO:tasks.workunit.client.0.smithi090.stdout:8/21: dread - no filename 2024-03-11T19:22:32.513 INFO:tasks.workunit.client.0.smithi090.stdout:8/22: dread - no filename 2024-03-11T19:22:32.525 INFO:tasks.workunit.client.0.smithi090.stdout:0/31: chown d2/f4 413297 1 2024-03-11T19:22:32.575 INFO:tasks.workunit.client.0.smithi090.stdout:5/20: dwrite d1/f3 [0,4194304] 0 2024-03-11T19:22:32.613 INFO:tasks.workunit.client.0.smithi090.stdout:9/10: mknod d1/c4 0 2024-03-11T19:22:32.614 INFO:tasks.workunit.client.0.smithi090.stdout:1/18: dread - d5/f6 zero size 2024-03-11T19:22:32.685 INFO:tasks.workunit.client.0.smithi090.stdout:2/24: chown d0/f3 3 1 2024-03-11T19:22:32.711 INFO:tasks.workunit.client.0.smithi090.stdout:0/32: dread - d2/f4 zero size 2024-03-11T19:22:32.765 INFO:tasks.workunit.client.0.smithi090.stdout:1/19: dread - d5/f6 zero size 2024-03-11T19:22:32.767 INFO:tasks.workunit.client.0.smithi090.stdout:2/25: fsync d0/f3 0 2024-03-11T19:22:32.768 INFO:tasks.workunit.client.0.smithi090.stdout:5/21: read d1/f3 [1179974,58615] 0 2024-03-11T19:22:32.853 INFO:tasks.workunit.client.0.smithi090.stdout:3/16: rename l1 to d2/d4/l7 0 2024-03-11T19:22:33.061 INFO:tasks.workunit.client.0.smithi090.stdout:7/11: mknod d3/c5 0 2024-03-11T19:22:33.173 INFO:tasks.workunit.client.0.smithi090.stdout:5/22: dread d1/f3 [0,4194304] 0 2024-03-11T19:22:33.177 INFO:tasks.workunit.client.0.smithi090.stdout:8/23: mknod d0/c5 0 2024-03-11T19:22:33.177 INFO:tasks.workunit.client.0.smithi090.stdout:8/24: write - no filename 2024-03-11T19:22:33.207 INFO:tasks.workunit.client.0.smithi090.stdout:9/11: fdatasync f0 0 2024-03-11T19:22:33.335 INFO:tasks.workunit.client.0.smithi090.stdout:9/12: stat d1/f3 0 2024-03-11T19:22:33.336 INFO:tasks.workunit.client.0.smithi090.stdout:8/25: chown d0/d3 127 1 2024-03-11T19:22:33.336 INFO:tasks.workunit.client.0.smithi090.stdout:8/26: dwrite - no filename 2024-03-11T19:22:33.342 INFO:tasks.workunit.client.0.smithi090.stdout:5/23: readlink d1/l2 0 2024-03-11T19:22:33.367 INFO:tasks.workunit.client.0.smithi090.stdout:2/26: dwrite d0/f2 [0,4194304] 0 2024-03-11T19:22:33.472 INFO:tasks.workunit.client.0.smithi090.stdout:4/21: getdents d4 0 2024-03-11T19:22:33.509 INFO:tasks.workunit.client.0.smithi090.stdout:5/24: sync 2024-03-11T19:22:33.645 INFO:tasks.workunit.client.0.smithi090.stdout:1/20: mknod d5/c7 0 2024-03-11T19:22:33.805 INFO:tasks.workunit.client.0.smithi090.stdout:3/17: creat d2/f8 x:0 0 0 2024-03-11T19:22:33.830 INFO:tasks.workunit.client.0.smithi090.stdout:1/21: dread - d5/f6 zero size 2024-03-11T19:22:33.892 INFO:tasks.workunit.client.0.smithi090.stdout:9/13: dwrite d1/f3 [0,4194304] 0 2024-03-11T19:22:34.018 INFO:tasks.workunit.client.0.smithi090.stdout:1/22: write d5/f6 [457418,45928] 0 2024-03-11T19:22:34.050 INFO:tasks.workunit.client.0.smithi090.stdout:9/14: write d1/f3 [3983876,20744] 0 2024-03-11T19:22:34.179 INFO:tasks.workunit.client.0.smithi090.stdout:8/27: symlink d0/l6 0 2024-03-11T19:22:34.179 INFO:tasks.workunit.client.0.smithi090.stdout:8/28: fsync - no filename 2024-03-11T19:22:34.179 INFO:tasks.workunit.client.0.smithi090.stdout:8/29: write - no filename 2024-03-11T19:22:34.179 INFO:tasks.workunit.client.0.smithi090.stdout:8/30: dwrite - no filename 2024-03-11T19:22:34.180 INFO:tasks.workunit.client.0.smithi090.stdout:8/31: write - no filename 2024-03-11T19:22:34.292 INFO:tasks.workunit.client.0.smithi090.stdout:6/27: write f3 [674784,23995] 0 2024-03-11T19:22:34.373 INFO:tasks.workunit.client.0.smithi090.stdout:7/12: rmdir d3 39 2024-03-11T19:22:34.524 INFO:tasks.workunit.client.0.smithi090.stdout:5/25: creat d1/d4/f6 x:0 0 0 2024-03-11T19:22:35.078 INFO:tasks.workunit.client.0.smithi090.stdout:1/23: symlink d5/l8 0 2024-03-11T19:22:35.215 INFO:tasks.workunit.client.0.smithi090.stdout:2/27: dwrite d0/f2 [4194304,4194304] 0 2024-03-11T19:22:35.326 INFO:tasks.workunit.client.0.smithi090.stdout:9/15: mknod d1/c5 0 2024-03-11T19:22:35.374 INFO:tasks.workunit.client.0.smithi090.stdout:4/22: creat d4/f5 x:0 0 0 2024-03-11T19:22:35.378 INFO:tasks.workunit.client.0.smithi090.stdout:8/32: creat d0/f7 x:0 0 0 2024-03-11T19:22:35.437 INFO:tasks.workunit.client.0.smithi090.stdout:8/33: stat d0/l6 0 2024-03-11T19:22:35.442 INFO:tasks.workunit.client.0.smithi090.stdout:4/23: rename d4 to d4/d6 22 2024-03-11T19:22:35.444 INFO:tasks.workunit.client.0.smithi090.stdout:1/24: dread d5/f6 [0,4194304] 0 2024-03-11T19:22:35.475 INFO:tasks.workunit.client.0.smithi090.stdout:6/28: rmdir d4 39 2024-03-11T19:22:35.574 INFO:tasks.workunit.client.0.smithi090.stdout:7/13: chown d3/c5 198342471 1 2024-03-11T19:22:35.661 INFO:tasks.workunit.client.0.smithi090.stdout:5/26: truncate d1/f3 1721795 0 2024-03-11T19:22:35.723 INFO:tasks.workunit.client.0.smithi090.stdout:5/27: truncate d1/d4/f6 564225 0 2024-03-11T19:22:35.977 INFO:tasks.workunit.client.0.smithi090.stdout:2/28: creat d0/f4 x:0 0 0 2024-03-11T19:22:36.143 INFO:tasks.workunit.client.0.smithi090.stdout:0/33: truncate d2/f7 4158142 0 2024-03-11T19:22:36.154 INFO:tasks.workunit.client.0.smithi090.stdout:9/16: unlink d1/c4 0 2024-03-11T19:22:36.577 INFO:tasks.workunit.client.0.smithi090.stdout:3/18: dwrite d2/f6 [0,4194304] 0 2024-03-11T19:22:36.688 INFO:tasks.workunit.client.0.smithi090.stdout:4/24: rename f1 to d4/f7 0 2024-03-11T19:22:36.731 INFO:tasks.workunit.client.0.smithi090.stdout:4/25: dread - d4/f5 zero size 2024-03-11T19:22:37.170 INFO:tasks.workunit.client.0.smithi090.stdout:8/34: rmdir d0 39 2024-03-11T19:22:37.193 INFO:tasks.workunit.client.0.smithi090.stdout:1/25: symlink d5/l9 0 2024-03-11T19:22:37.251 INFO:tasks.workunit.client.0.smithi090.stdout:7/14: symlink d3/l6 0 2024-03-11T19:22:37.290 INFO:tasks.workunit.client.0.smithi090.stdout:7/15: fdatasync d3/f4 0 2024-03-11T19:22:37.311 INFO:tasks.workunit.client.0.smithi090.stdout:7/16: read - d3/f4 zero size 2024-03-11T19:22:37.442 INFO:tasks.workunit.client.0.smithi090.stdout:2/29: creat d0/f5 x:0 0 0 2024-03-11T19:22:37.442 INFO:tasks.workunit.client.0.smithi090.stdout:2/30: readlink - no filename 2024-03-11T19:22:37.680 INFO:tasks.workunit.client.0.smithi090.stdout:9/17: creat d1/f6 x:0 0 0 2024-03-11T19:22:37.719 INFO:tasks.workunit.client.0.smithi090.stdout:9/18: read d1/f3 [1910965,115323] 0 2024-03-11T19:22:37.848 INFO:tasks.workunit.client.0.smithi090.stdout:4/26: fdatasync d4/f7 0 2024-03-11T19:22:38.181 INFO:tasks.workunit.client.0.smithi090.stdout:8/35: truncate d0/f7 163184 0 2024-03-11T19:22:38.206 INFO:tasks.workunit.client.0.smithi090.stdout:8/36: fsync d0/f7 0 2024-03-11T19:22:38.464 INFO:tasks.workunit.client.0.smithi090.stdout:1/26: write d5/f6 [19436,83700] 0 2024-03-11T19:22:38.660 INFO:tasks.workunit.client.0.smithi090.stdout:3/19: rmdir d2 39 2024-03-11T19:22:38.818 INFO:tasks.workunit.client.0.smithi090.stdout:7/17: dwrite f2 [0,4194304] 0 2024-03-11T19:22:38.966 INFO:tasks.workunit.client.0.smithi090.stdout:7/18: read f2 [1980612,102417] 0 2024-03-11T19:22:39.122 INFO:tasks.workunit.client.0.smithi090.stdout:2/31: rmdir d0 39 2024-03-11T19:22:39.442 INFO:tasks.workunit.client.0.smithi090.stdout:1/27: mkdir d5/da 0 2024-03-11T19:22:39.484 INFO:tasks.workunit.client.0.smithi090.stdout:1/28: dread d5/f6 [0,4194304] 0 2024-03-11T19:22:39.512 INFO:tasks.workunit.client.0.smithi090.stdout:6/29: truncate f3 464524 0 2024-03-11T19:22:39.535 INFO:tasks.workunit.client.0.smithi090.stdout:1/29: chown l3 48869323 1 2024-03-11T19:22:39.729 INFO:tasks.workunit.client.0.smithi090.stdout:3/20: chown d2/f3 55783736 1 2024-03-11T19:22:40.098 INFO:tasks.workunit.client.0.smithi090.stdout:0/34: rmdir d2 39 2024-03-11T19:22:40.371 INFO:tasks.workunit.client.0.smithi090.stdout:7/19: unlink d3/c5 0 2024-03-11T19:22:40.377 INFO:tasks.workunit.client.0.smithi090.stdout:9/19: rmdir d1 39 2024-03-11T19:22:40.692 INFO:tasks.workunit.client.0.smithi090.stdout:8/37: creat d0/d3/f8 x:0 0 0 2024-03-11T19:22:41.119 INFO:tasks.workunit.client.0.smithi090.stdout:8/38: dwrite d0/f7 [0,4194304] 0 2024-03-11T19:22:41.237 INFO:tasks.workunit.client.0.smithi090.stdout:1/30: mknod d5/cb 0 2024-03-11T19:22:41.261 INFO:tasks.workunit.client.0.smithi090.stdout:8/39: stat d0 0 2024-03-11T19:22:41.619 INFO:tasks.workunit.client.0.smithi090.stdout:7/20: dread f2 [0,4194304] 0 2024-03-11T19:22:41.649 INFO:tasks.workunit.client.0.smithi090.stdout:9/20: fsync f0 0 2024-03-11T19:22:41.806 INFO:tasks.workunit.client.0.smithi090.stdout:2/32: creat d0/f6 x:0 0 0 2024-03-11T19:22:41.811 INFO:tasks.workunit.client.0.smithi090.stdout:5/28: write d1/f3 [1993251,79580] 0 2024-03-11T19:22:42.638 INFO:tasks.workunit.client.0.smithi090.stdout:1/31: creat d5/fc x:0 0 0 2024-03-11T19:22:42.704 INFO:tasks.workunit.client.0.smithi090.stdout:0/35: fsync d2/f5 0 2024-03-11T19:22:42.939 INFO:tasks.workunit.client.0.smithi090.stdout:4/27: getdents d4 0 2024-03-11T19:22:42.990 INFO:tasks.workunit.client.0.smithi090.stdout:7/21: write f2 [1733062,24011] 0 2024-03-11T19:22:43.087 INFO:tasks.workunit.client.0.smithi090.stdout:2/33: creat d0/f7 x:0 0 0 2024-03-11T19:22:43.100 INFO:tasks.workunit.client.0.smithi090.stdout:9/21: chown d1/c5 5408684 1 2024-03-11T19:22:43.293 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:22:43.297 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:22:43.331 INFO:tasks.workunit.client.0.smithi090.stdout:7/22: dwrite f2 [0,4194304] 0 2024-03-11T19:22:43.457 INFO:tasks.workunit.client.0.smithi090.stdout:6/30: rename c1 to d4/c6 0 2024-03-11T19:22:43.506 INFO:tasks.workunit.client.0.smithi090.stdout:7/23: write f2 [4236987,81842] 0 2024-03-11T19:22:43.599 INFO:tasks.workunit.client.0.smithi090.stdout:5/29: truncate d1/d4/f6 1167663 0 2024-03-11T19:22:43.619 INFO:tasks.workunit.client.0.smithi090.stdout:5/30: fdatasync d1/f3 0 2024-03-11T19:22:43.870 INFO:tasks.workunit.client.0.smithi090.stdout:8/40: rmdir d0 39 2024-03-11T19:22:44.523 INFO:tasks.workunit.client.0.smithi090.stdout:4/28: creat d4/f8 x:0 0 0 2024-03-11T19:22:44.547 INFO:tasks.workunit.client.0.smithi090.stdout:4/29: fdatasync d4/f8 0 2024-03-11T19:22:44.586 INFO:tasks.workunit.client.0.smithi090.stdout:2/34: symlink d0/l8 0 2024-03-11T19:22:44.609 INFO:tasks.workunit.client.0.smithi090.stdout:4/30: write d4/f8 [1024873,126020] 0 2024-03-11T19:22:44.652 INFO:tasks.workunit.client.0.smithi090.stdout:4/31: chown d4/f7 178 1 2024-03-11T19:22:44.761 INFO:tasks.workunit.client.0.smithi090.stdout:9/22: symlink d1/l7 0 2024-03-11T19:22:44.800 INFO:tasks.workunit.client.0.smithi090.stdout:6/31: mknod d4/c7 0 2024-03-11T19:22:44.801 INFO:tasks.workunit.client.0.smithi090.stdout:7/24: creat d3/f7 x:0 0 0 2024-03-11T19:22:44.842 INFO:tasks.workunit.client.0.smithi090.stdout:1/32: mkdir d5/da/dd 0 2024-03-11T19:22:44.850 INFO:tasks.workunit.client.0.smithi090.stdout:3/21: getdents d2 0 2024-03-11T19:22:44.926 INFO:tasks.workunit.client.0.smithi090.stdout:0/36: unlink d2/f7 0 2024-03-11T19:22:44.989 INFO:tasks.workunit.client.0.smithi090.stdout:2/35: symlink d0/l9 0 2024-03-11T19:22:45.095 INFO:tasks.workunit.client.0.smithi090.stdout:4/32: rename c2 to d4/c9 0 2024-03-11T19:22:45.323 INFO:tasks.workunit.client.0.smithi090.stdout:7/25: creat d3/f8 x:0 0 0 2024-03-11T19:22:45.359 INFO:tasks.workunit.client.0.smithi090.stdout:3/22: creat d2/f9 x:0 0 0 2024-03-11T19:22:45.404 INFO:tasks.workunit.client.0.smithi090.stdout:8/41: symlink d0/l9 0 2024-03-11T19:22:45.452 INFO:tasks.workunit.client.0.smithi090.stdout:6/32: dread f3 [0,4194304] 0 2024-03-11T19:22:45.500 INFO:tasks.workunit.client.0.smithi090.stdout:1/33: dread d5/f6 [0,4194304] 0 2024-03-11T19:22:45.544 INFO:tasks.workunit.client.0.smithi090.stdout:2/36: symlink d0/la 0 2024-03-11T19:22:45.547 INFO:tasks.workunit.client.0.smithi090.stdout:7/26: dread - d3/f7 zero size 2024-03-11T19:22:45.612 INFO:tasks.workunit.client.0.smithi090.stdout:1/34: readlink d5/l8 0 2024-03-11T19:22:45.691 INFO:tasks.workunit.client.0.smithi090.stdout:0/37: rename d2/f6 to d2/f8 0 2024-03-11T19:22:45.736 INFO:tasks.workunit.client.0.smithi090.stdout:4/33: creat d4/fa x:0 0 0 2024-03-11T19:22:45.740 INFO:tasks.workunit.client.0.smithi090.stdout:2/37: chown d0/f6 466930991 1 2024-03-11T19:22:45.781 INFO:tasks.workunit.client.0.smithi090.stdout:3/23: creat d2/fa x:0 0 0 2024-03-11T19:22:45.955 INFO:tasks.workunit.client.0.smithi090.stdout:1/35: write d5/fc [29199,74827] 0 2024-03-11T19:22:46.110 INFO:tasks.workunit.client.0.smithi090.stdout:6/33: symlink d4/l8 0 2024-03-11T19:22:46.139 INFO:tasks.workunit.client.0.smithi090.stdout:7/27: dread f2 [0,4194304] 0 2024-03-11T19:22:46.279 INFO:tasks.workunit.client.0.smithi090.stdout:5/31: getdents d1 0 2024-03-11T19:22:46.286 INFO:tasks.workunit.client.0.smithi090.stdout:1/36: read d5/f6 [29554,103153] 0 2024-03-11T19:22:46.332 INFO:tasks.workunit.client.0.smithi090.stdout:8/42: dwrite d0/f7 [0,4194304] 0 2024-03-11T19:22:46.456 INFO:tasks.workunit.client.0.smithi090.stdout:7/28: dread - d3/f7 zero size 2024-03-11T19:22:46.548 INFO:tasks.workunit.client.0.smithi090.stdout:2/38: fsync d0/f5 0 2024-03-11T19:22:46.662 INFO:tasks.workunit.client.0.smithi090.stdout:4/34: mkdir d4/db 0 2024-03-11T19:22:46.758 INFO:tasks.workunit.client.0.smithi090.stdout:4/35: dread - d4/f5 zero size 2024-03-11T19:22:46.907 INFO:tasks.workunit.client.0.smithi090.stdout:5/32: dwrite d1/f3 [0,4194304] 0 2024-03-11T19:22:47.139 INFO:tasks.workunit.client.0.smithi090.stdout:9/23: dwrite d1/f6 [0,4194304] 0 2024-03-11T19:22:47.247 INFO:tasks.workunit.client.0.smithi090.stdout:4/36: dread d4/f8 [0,4194304] 0 2024-03-11T19:22:47.272 INFO:tasks.workunit.client.0.smithi090.stdout:3/24: truncate d2/f3 1210770 0 2024-03-11T19:22:47.291 INFO:tasks.workunit.client.0.smithi090.stdout:6/34: mknod d4/c9 0 2024-03-11T19:22:47.337 INFO:tasks.workunit.client.0.smithi090.stdout:3/25: stat d2/c5 0 2024-03-11T19:22:47.354 INFO:tasks.workunit.client.0.smithi090.stdout:4/37: fsync d4/f5 0 2024-03-11T19:22:47.523 INFO:tasks.workunit.client.0.smithi090.stdout:7/29: creat d3/f9 x:0 0 0 2024-03-11T19:22:47.788 INFO:tasks.workunit.client.0.smithi090.stdout:9/24: dwrite d1/f6 [0,4194304] 0 2024-03-11T19:22:48.127 INFO:tasks.workunit.client.0.smithi090.stdout:8/43: dwrite d0/d3/f8 [0,4194304] 0 2024-03-11T19:22:48.298 INFO:tasks.workunit.client.0.smithi090.stdout:8/44: write d0/f7 [1172518,46045] 0 2024-03-11T19:22:48.478 INFO:tasks.workunit.client.0.smithi090.stdout:6/35: mknod d4/ca 0 2024-03-11T19:22:48.488 INFO:tasks.workunit.client.0.smithi090.stdout:0/38: write f0 [1549170,34384] 0 2024-03-11T19:22:48.555 INFO:tasks.workunit.client.0.smithi090.stdout:4/38: chown d4/c9 3137622 1 2024-03-11T19:22:48.599 INFO:tasks.workunit.client.0.smithi090.stdout:3/26: mkdir d2/db 0 2024-03-11T19:22:48.677 INFO:tasks.workunit.client.0.smithi090.stdout:7/30: mknod d3/ca 0 2024-03-11T19:22:48.717 INFO:tasks.workunit.client.0.smithi090.stdout:7/31: write d3/f7 [463842,30747] 0 2024-03-11T19:22:48.821 INFO:tasks.workunit.client.0.smithi090.stdout:1/37: symlink d5/da/dd/le 0 2024-03-11T19:22:48.869 INFO:tasks.workunit.client.0.smithi090.stdout:1/38: dread d5/f6 [0,4194304] 0 2024-03-11T19:22:48.900 INFO:tasks.workunit.client.0.smithi090.stdout:1/39: fsync d5/fc 0 2024-03-11T19:22:48.946 INFO:tasks.workunit.client.0.smithi090.stdout:1/40: write d5/fc [164960,74070] 0 2024-03-11T19:22:49.478 INFO:tasks.workunit.client.0.smithi090.stdout:8/45: mkdir d0/d3/da 0 2024-03-11T19:22:49.574 INFO:tasks.workunit.client.0.smithi090.stdout:6/36: creat d4/fb x:0 0 0 2024-03-11T19:22:49.854 INFO:tasks.workunit.client.0.smithi090.stdout:2/39: write d0/f3 [407324,114330] 0 2024-03-11T19:22:49.903 INFO:tasks.workunit.client.0.smithi090.stdout:7/32: mknod d3/cb 0 2024-03-11T19:22:49.992 INFO:tasks.workunit.client.0.smithi090.stdout:0/39: dread d2/f3 [0,4194304] 0 2024-03-11T19:22:50.137 INFO:tasks.workunit.client.0.smithi090.stdout:6/37: dwrite d4/fb [0,4194304] 0 2024-03-11T19:22:50.157 INFO:tasks.workunit.client.0.smithi090.stdout:1/41: fsync d5/f6 0 2024-03-11T19:22:50.406 INFO:tasks.workunit.client.0.smithi090.stdout:2/40: dread d0/f3 [0,4194304] 0 2024-03-11T19:22:50.590 INFO:tasks.workunit.client.0.smithi090.stdout:8/46: creat d0/d3/fb x:0 0 0 2024-03-11T19:22:50.946 INFO:tasks.workunit.client.0.smithi090.stdout:3/27: fdatasync d2/f3 0 2024-03-11T19:22:51.055 INFO:tasks.workunit.client.0.smithi090.stdout:7/33: mkdir d3/dc 0 2024-03-11T19:22:51.080 INFO:tasks.workunit.client.0.smithi090.stdout:7/34: dread - d3/f8 zero size 2024-03-11T19:22:51.370 INFO:tasks.workunit.client.0.smithi090.stdout:5/33: write d1/d4/f6 [226231,54161] 0 2024-03-11T19:22:51.419 INFO:tasks.workunit.client.0.smithi090.stdout:1/42: rename d5/da/dd/le to d5/da/lf 0 2024-03-11T19:22:51.478 INFO:tasks.workunit.client.0.smithi090.stdout:5/34: rename d1 to d1/d7 22 2024-03-11T19:22:51.507 INFO:tasks.workunit.client.0.smithi090.stdout:9/25: dwrite f0 [0,4194304] 0 2024-03-11T19:22:51.661 INFO:tasks.workunit.client.0.smithi090.stdout:8/47: unlink d0/c5 0 2024-03-11T19:22:51.745 INFO:tasks.workunit.client.0.smithi090.stdout:4/39: rmdir d4/db 0 2024-03-11T19:22:51.966 INFO:tasks.workunit.client.0.smithi090.stdout:7/35: fdatasync f2 0 2024-03-11T19:22:52.054 INFO:tasks.workunit.client.0.smithi090.stdout:8/48: dread d0/f7 [0,4194304] 0 2024-03-11T19:22:52.120 INFO:tasks.workunit.client.0.smithi090.stdout:5/35: dwrite d1/d4/f6 [0,4194304] 0 2024-03-11T19:22:52.284 INFO:tasks.workunit.client.0.smithi090.stdout:5/36: readlink d1/l2 0 2024-03-11T19:22:52.290 INFO:tasks.workunit.client.0.smithi090.stdout:8/49: chown d0/l9 454879 1 2024-03-11T19:22:52.740 INFO:tasks.workunit.client.0.smithi090.stdout:1/43: symlink d5/l10 0 2024-03-11T19:22:52.850 INFO:tasks.workunit.client.0.smithi090.stdout:8/50: sync 2024-03-11T19:22:52.871 INFO:tasks.workunit.client.0.smithi090.stdout:9/26: dread d1/f3 [0,4194304] 0 2024-03-11T19:22:53.269 INFO:tasks.workunit.client.0.smithi090.stdout:7/36: mkdir d3/dd 0 2024-03-11T19:22:53.327 INFO:tasks.workunit.client.0.smithi090.stdout:1/44: symlink d5/da/l11 0 2024-03-11T19:22:53.803 INFO:tasks.workunit.client.0.smithi090.stdout:2/41: truncate d0/f3 3629588 0 2024-03-11T19:22:53.943 INFO:tasks.workunit.client.0.smithi090.stdout:9/27: write d1/f3 [3858202,9134] 0 2024-03-11T19:22:54.140 INFO:tasks.workunit.client.0.smithi090.stdout:2/42: chown d0 147 1 2024-03-11T19:22:54.147 INFO:tasks.workunit.client.0.smithi090.stdout:5/37: dwrite d1/f3 [0,4194304] 0 2024-03-11T19:22:54.389 INFO:tasks.workunit.client.0.smithi090.stdout:6/38: getdents d4 0 2024-03-11T19:22:54.404 INFO:tasks.workunit.client.0.smithi090.stdout:8/51: mkdir d0/d3/da/dc 0 2024-03-11T19:22:54.941 INFO:tasks.workunit.client.0.smithi090.stdout:7/37: dwrite d3/f8 [0,4194304] 0 2024-03-11T19:22:55.104 INFO:tasks.workunit.client.0.smithi090.stdout:1/45: dwrite d5/fc [0,4194304] 0 2024-03-11T19:22:55.127 INFO:tasks.workunit.client.0.smithi090.stdout:5/38: rmdir d1 39 2024-03-11T19:22:55.484 INFO:tasks.workunit.client.0.smithi090.stdout:6/39: mknod d4/cc 0 2024-03-11T19:22:55.586 INFO:tasks.workunit.client.0.smithi090.stdout:7/38: chown c0 55634622 1 2024-03-11T19:22:55.601 INFO:tasks.workunit.client.0.smithi090.stdout:3/28: dwrite d2/f8 [0,4194304] 0 2024-03-11T19:22:55.822 INFO:tasks.workunit.client.0.smithi090.stdout:0/40: dwrite d2/f8 [0,4194304] 0 2024-03-11T19:22:56.118 INFO:tasks.workunit.client.0.smithi090.stdout:6/40: symlink d4/ld 0 2024-03-11T19:22:56.131 INFO:tasks.workunit.client.0.smithi090.stdout:8/52: creat d0/d3/da/dc/fd x:0 0 0 2024-03-11T19:22:56.220 INFO:tasks.workunit.client.0.smithi090.stdout:5/39: mknod d1/c8 0 2024-03-11T19:22:56.267 INFO:tasks.workunit.client.0.smithi090.stdout:4/40: dwrite d4/f7 [0,4194304] 0 2024-03-11T19:22:56.586 INFO:tasks.workunit.client.0.smithi090.stdout:9/28: dwrite d1/f3 [4194304,4194304] 0 2024-03-11T19:22:56.618 INFO:tasks.workunit.client.0.smithi090.stdout:0/41: chown d2/f3 1176231 1 2024-03-11T19:22:56.678 INFO:tasks.workunit.client.0.smithi090.stdout:4/41: fdatasync d4/f7 0 2024-03-11T19:22:56.788 INFO:tasks.workunit.client.0.smithi090.stdout:2/43: dread d0/f3 [0,4194304] 0 2024-03-11T19:22:56.870 INFO:tasks.workunit.client.0.smithi090.stdout:6/41: creat d4/fe x:0 0 0 2024-03-11T19:22:57.231 INFO:tasks.workunit.client.0.smithi090.stdout:5/40: mknod d1/c9 0 2024-03-11T19:22:57.414 INFO:tasks.workunit.client.0.smithi090.stdout:9/29: symlink d1/l8 0 2024-03-11T19:22:57.450 INFO:tasks.workunit.client.0.smithi090.stdout:2/44: creat d0/fb x:0 0 0 2024-03-11T19:22:57.567 INFO:tasks.workunit.client.0.smithi090.stdout:4/42: mknod d4/cc 0 2024-03-11T19:22:57.567 INFO:tasks.workunit.client.0.smithi090.stdout:1/46: dwrite d5/fc [4194304,4194304] 0 2024-03-11T19:22:57.899 INFO:tasks.workunit.client.0.smithi090.stdout:6/42: creat d4/ff x:0 0 0 2024-03-11T19:22:57.975 INFO:tasks.workunit.client.0.smithi090.stdout:9/30: fsync d1/f3 0 2024-03-11T19:22:57.987 INFO:tasks.workunit.client.0.smithi090.stdout:7/39: dwrite d3/f9 [0,4194304] 0 2024-03-11T19:22:58.109 INFO:tasks.workunit.client.0.smithi090.stdout:5/41: symlink d1/la 0 2024-03-11T19:22:58.227 INFO:tasks.workunit.client.0.smithi090.stdout:9/31: sync 2024-03-11T19:22:58.460 INFO:tasks.workunit.client.0.smithi090.stdout:7/40: dread - d3/f4 zero size 2024-03-11T19:22:58.476 INFO:tasks.workunit.client.0.smithi090.stdout:3/29: dwrite d2/fa [0,4194304] 0 2024-03-11T19:22:58.686 INFO:tasks.workunit.client.0.smithi090.stdout:6/43: creat d4/f10 x:0 0 0 2024-03-11T19:22:58.795 INFO:tasks.workunit.client.0.smithi090.stdout:0/42: dwrite d2/f8 [0,4194304] 0 2024-03-11T19:22:59.090 INFO:tasks.workunit.client.0.smithi090.stdout:1/47: rename d5/da/l11 to d5/da/dd/l12 0 2024-03-11T19:22:59.191 INFO:tasks.workunit.client.0.smithi090.stdout:8/53: dwrite d0/d3/fb [0,4194304] 0 2024-03-11T19:22:59.206 INFO:tasks.workunit.client.0.smithi090.stdout:9/32: write d1/f3 [7250180,129435] 0 2024-03-11T19:22:59.226 INFO:tasks.workunit.client.0.smithi090.stdout:2/45: dread d0/f2 [4194304,4194304] 0 2024-03-11T19:22:59.404 INFO:tasks.workunit.client.0.smithi090.stdout:6/44: creat d4/f11 x:0 0 0 2024-03-11T19:22:59.436 INFO:tasks.workunit.client.0.smithi090.stdout:0/43: chown f0 437638 1 2024-03-11T19:22:59.436 INFO:tasks.workunit.client.0.smithi090.stdout:0/44: readlink - no filename 2024-03-11T19:22:59.543 INFO:tasks.workunit.client.0.smithi090.stdout:1/48: chown d5/da 127577 1 2024-03-11T19:22:59.711 INFO:tasks.workunit.client.0.smithi090.stdout:9/33: creat d1/f9 x:0 0 0 2024-03-11T19:22:59.778 INFO:tasks.workunit.client.0.smithi090.stdout:3/30: truncate d2/f9 450204 0 2024-03-11T19:22:59.917 INFO:tasks.workunit.client.0.smithi090.stdout:2/46: write d0/fb [271724,3827] 0 2024-03-11T19:23:00.076 INFO:tasks.workunit.client.0.smithi090.stdout:4/43: dwrite d4/fa [0,4194304] 0 2024-03-11T19:23:00.132 INFO:tasks.workunit.client.0.smithi090.stdout:1/49: read d5/fc [4463982,43232] 0 2024-03-11T19:23:00.359 INFO:tasks.workunit.client.0.smithi090.stdout:6/45: link d4/ff d4/f12 0 2024-03-11T19:23:00.532 INFO:tasks.workunit.client.0.smithi090.stdout:9/34: write d1/f9 [1017598,343] 0 2024-03-11T19:23:00.558 INFO:tasks.workunit.client.0.smithi090.stdout:3/31: creat d2/d4/fc x:0 0 0 2024-03-11T19:23:00.626 INFO:tasks.workunit.client.0.smithi090.stdout:4/44: write d4/f7 [1477711,112686] 0 2024-03-11T19:23:00.630 INFO:tasks.workunit.client.0.smithi090.stdout:5/42: dwrite d1/d4/f6 [0,4194304] 0 2024-03-11T19:23:00.788 INFO:tasks.workunit.client.0.smithi090.stdout:7/41: dwrite d3/f4 [0,4194304] 0 2024-03-11T19:23:01.046 INFO:tasks.workunit.client.0.smithi090.stdout:0/45: dread f0 [0,4194304] 0 2024-03-11T19:23:01.072 INFO:tasks.workunit.client.0.smithi090.stdout:6/46: fsync d4/fb 0 2024-03-11T19:23:01.095 INFO:tasks.workunit.client.0.smithi090.stdout:1/50: write d5/fc [4438486,46832] 0 2024-03-11T19:23:01.323 INFO:tasks.workunit.client.0.smithi090.stdout:5/43: symlink d1/d4/lb 0 2024-03-11T19:23:01.410 INFO:tasks.workunit.client.0.smithi090.stdout:4/45: rename c0 to d4/cd 0 2024-03-11T19:23:01.417 INFO:tasks.workunit.client.0.smithi090.stdout:8/54: dwrite d0/d3/fb [0,4194304] 0 2024-03-11T19:23:01.698 INFO:tasks.workunit.client.0.smithi090.stdout:6/47: fdatasync f3 0 2024-03-11T19:23:01.763 INFO:tasks.workunit.client.0.smithi090.stdout:1/51: read d5/fc [3638659,111083] 0 2024-03-11T19:23:01.824 INFO:tasks.workunit.client.0.smithi090.stdout:8/55: chown d0 3916378 1 2024-03-11T19:23:02.195 INFO:tasks.workunit.client.0.smithi090.stdout:7/42: dread d3/f9 [0,4194304] 0 2024-03-11T19:23:02.332 INFO:tasks.workunit.client.0.smithi090.stdout:2/47: dwrite d0/f7 [0,4194304] 0 2024-03-11T19:23:02.358 INFO:tasks.workunit.client.0.smithi090.stdout:0/46: rename d2/f4 to d2/f9 0 2024-03-11T19:23:02.469 INFO:tasks.workunit.client.0.smithi090.stdout:5/44: creat d1/d4/fc x:0 0 0 2024-03-11T19:23:02.489 INFO:tasks.workunit.client.0.smithi090.stdout:3/32: dwrite d2/f3 [0,4194304] 0 2024-03-11T19:23:02.629 INFO:tasks.workunit.client.0.smithi090.stdout:6/48: unlink d4/f12 0 2024-03-11T19:23:02.637 INFO:tasks.workunit.client.0.smithi090.stdout:9/35: dwrite f0 [0,4194304] 0 2024-03-11T19:23:02.836 INFO:tasks.workunit.client.0.smithi090.stdout:6/49: sync 2024-03-11T19:23:02.864 INFO:tasks.workunit.client.0.smithi090.stdout:9/36: truncate d1/f3 8537018 0 2024-03-11T19:23:02.991 INFO:tasks.workunit.client.0.smithi090.stdout:9/37: dread f0 [0,4194304] 0 2024-03-11T19:23:03.262 INFO:tasks.workunit.client.0.smithi090.stdout:3/33: readlink d2/d4/l7 0 2024-03-11T19:23:03.287 INFO:tasks.workunit.client.0.smithi090.stdout:3/34: read - d2/d4/fc zero size 2024-03-11T19:23:03.449 INFO:tasks.workunit.client.0.smithi090.stdout:3/35: dread d2/fa [0,4194304] 0 2024-03-11T19:23:03.595 INFO:tasks.workunit.client.0.smithi090.stdout:6/50: unlink d4/ld 0 2024-03-11T19:23:03.759 INFO:tasks.workunit.client.0.smithi090.stdout:8/56: write d0/d3/f8 [3985348,50720] 0 2024-03-11T19:23:03.839 INFO:tasks.workunit.client.0.smithi090.stdout:4/46: link d4/f8 d4/fe 0 2024-03-11T19:23:03.890 INFO:tasks.workunit.client.0.smithi090.stdout:9/38: symlink d1/la 0 2024-03-11T19:23:04.142 INFO:tasks.workunit.client.0.smithi090.stdout:1/52: dwrite d5/f6 [0,4194304] 0 2024-03-11T19:23:04.337 INFO:tasks.workunit.client.0.smithi090.stdout:6/51: mkdir d4/d13 0 2024-03-11T19:23:04.436 INFO:tasks.workunit.client.0.smithi090.stdout:0/47: dwrite f0 [0,4194304] 0 2024-03-11T19:23:04.447 INFO:tasks.workunit.client.0.smithi090.stdout:6/52: write d4/fb [1898157,50589] 0 2024-03-11T19:23:04.586 INFO:tasks.workunit.client.0.smithi090.stdout:0/48: sync 2024-03-11T19:23:04.686 INFO:tasks.workunit.client.0.smithi090.stdout:7/43: link d3/f7 d3/dc/fe 0 2024-03-11T19:23:04.799 INFO:tasks.workunit.client.0.smithi090.stdout:8/57: symlink d0/d3/da/le 0 2024-03-11T19:23:04.806 INFO:tasks.workunit.client.0.smithi090.stdout:0/49: dread d2/f8 [0,4194304] 0 2024-03-11T19:23:04.832 INFO:tasks.workunit.client.0.smithi090.stdout:8/58: chown d0 102032 1 2024-03-11T19:23:05.000 INFO:tasks.workunit.client.0.smithi090.stdout:9/39: mknod d1/cb 0 2024-03-11T19:23:05.006 INFO:tasks.workunit.client.0.smithi090.stdout:0/50: dread d2/f3 [0,4194304] 0 2024-03-11T19:23:05.021 INFO:tasks.workunit.client.0.smithi090.stdout:5/45: link d1/d4/lb d1/ld 0 2024-03-11T19:23:05.099 INFO:tasks.workunit.client.0.smithi090.stdout:5/46: fsync d1/d4/f6 0 2024-03-11T19:23:05.153 INFO:tasks.workunit.client.0.smithi090.stdout:3/36: symlink d2/db/ld 0 2024-03-11T19:23:05.332 INFO:tasks.workunit.client.0.smithi090.stdout:1/53: mknod d5/c13 0 2024-03-11T19:23:05.387 INFO:tasks.workunit.client.0.smithi090.stdout:6/53: mkdir d4/d14 0 2024-03-11T19:23:05.402 INFO:tasks.workunit.client.0.smithi090.stdout:2/48: getdents d0 0 2024-03-11T19:23:05.427 INFO:tasks.workunit.client.0.smithi090.stdout:1/54: stat d5/fc 0 2024-03-11T19:23:05.709 INFO:tasks.workunit.client.0.smithi090.stdout:7/44: write d3/f9 [1951379,129739] 0 2024-03-11T19:23:05.784 INFO:tasks.workunit.client.0.smithi090.stdout:0/51: mknod d2/ca 0 2024-03-11T19:23:05.832 INFO:tasks.workunit.client.0.smithi090.stdout:3/37: mkdir d2/de 0 2024-03-11T19:23:05.867 INFO:tasks.workunit.client.0.smithi090.stdout:6/54: mkdir d4/d15 0 2024-03-11T19:23:06.031 INFO:tasks.workunit.client.0.smithi090.stdout:1/55: creat d5/da/dd/f14 x:0 0 0 2024-03-11T19:23:06.045 INFO:tasks.workunit.client.0.smithi090.stdout:9/40: write d1/f6 [5050031,59713] 0 2024-03-11T19:23:06.113 INFO:tasks.workunit.client.0.smithi090.stdout:3/38: dread - d2/d4/fc zero size 2024-03-11T19:23:06.133 INFO:tasks.workunit.client.0.smithi090.stdout:8/59: dread d0/f7 [0,4194304] 0 2024-03-11T19:23:06.207 INFO:tasks.workunit.client.0.smithi090.stdout:7/45: dread d3/f7 [0,4194304] 0 2024-03-11T19:23:06.294 INFO:tasks.workunit.client.0.smithi090.stdout:4/47: link d4/fe d4/ff 0 2024-03-11T19:23:06.319 INFO:tasks.workunit.client.0.smithi090.stdout:0/52: mknod d2/cb 0 2024-03-11T19:23:06.361 INFO:tasks.workunit.client.0.smithi090.stdout:2/49: link d0/fb d0/fc 0 2024-03-11T19:23:06.498 INFO:tasks.workunit.client.0.smithi090.stdout:1/56: truncate d5/da/dd/f14 339778 0 2024-03-11T19:23:06.552 INFO:tasks.workunit.client.0.smithi090.stdout:9/41: read d1/f9 [477670,105609] 0 2024-03-11T19:23:06.576 INFO:tasks.workunit.client.0.smithi090.stdout:8/60: dread d0/f7 [0,4194304] 0 2024-03-11T19:23:06.634 INFO:tasks.workunit.client.0.smithi090.stdout:2/50: dread d0/fb [0,4194304] 0 2024-03-11T19:23:06.660 INFO:tasks.workunit.client.0.smithi090.stdout:2/51: chown d0/l8 2 1 2024-03-11T19:23:07.153 INFO:tasks.workunit.client.0.smithi090.stdout:2/52: dwrite d0/f7 [4194304,4194304] 0 2024-03-11T19:23:07.311 INFO:tasks.workunit.client.0.smithi090.stdout:5/47: getdents d1 0 2024-03-11T19:23:07.362 INFO:tasks.workunit.client.0.smithi090.stdout:0/53: mknod d2/cc 0 2024-03-11T19:23:07.382 INFO:tasks.workunit.client.0.smithi090.stdout:9/42: creat d1/fc x:0 0 0 2024-03-11T19:23:07.511 INFO:tasks.workunit.client.0.smithi090.stdout:3/39: dwrite d2/f6 [0,4194304] 0 2024-03-11T19:23:07.634 INFO:tasks.workunit.client.0.smithi090.stdout:4/48: unlink d4/fe 0 2024-03-11T19:23:07.765 INFO:tasks.workunit.client.0.smithi090.stdout:3/40: write d2/f6 [2098029,77508] 0 2024-03-11T19:23:08.087 INFO:tasks.workunit.client.0.smithi090.stdout:6/55: dwrite d4/ff [0,4194304] 0 2024-03-11T19:23:08.252 INFO:tasks.workunit.client.0.smithi090.stdout:9/43: mkdir d1/dd 0 2024-03-11T19:23:08.255 INFO:tasks.workunit.client.0.smithi090.stdout:5/48: fsync d1/f3 0 2024-03-11T19:23:08.287 INFO:tasks.workunit.client.0.smithi090.stdout:1/57: creat d5/f15 x:0 0 0 2024-03-11T19:23:08.380 INFO:tasks.workunit.client.0.smithi090.stdout:0/54: mknod d2/cd 0 2024-03-11T19:23:08.399 INFO:tasks.workunit.client.0.smithi090.stdout:7/46: rename f2 to d3/dd/ff 0 2024-03-11T19:23:08.412 INFO:tasks.workunit.client.0.smithi090.stdout:1/58: write d5/da/dd/f14 [941073,6363] 0 2024-03-11T19:23:08.501 INFO:tasks.workunit.client.0.smithi090.stdout:7/47: readlink d3/l6 0 2024-03-11T19:23:08.516 INFO:tasks.workunit.client.0.smithi090.stdout:1/59: fdatasync d5/f15 0 2024-03-11T19:23:08.540 INFO:tasks.workunit.client.0.smithi090.stdout:4/49: rename d4/f7 to d4/f10 0 2024-03-11T19:23:08.604 INFO:tasks.workunit.client.0.smithi090.stdout:0/55: sync 2024-03-11T19:23:08.673 INFO:tasks.workunit.client.0.smithi090.stdout:1/60: read d5/f6 [1407654,60432] 0 2024-03-11T19:23:08.778 INFO:tasks.workunit.client.0.smithi090.stdout:5/49: dwrite d1/d4/f6 [0,4194304] 0 2024-03-11T19:23:09.112 INFO:tasks.workunit.client.0.smithi090.stdout:7/48: symlink d3/l10 0 2024-03-11T19:23:09.164 INFO:tasks.workunit.client.0.smithi090.stdout:4/50: creat d4/f11 x:0 0 0 2024-03-11T19:23:09.242 INFO:tasks.workunit.client.0.smithi090.stdout:2/53: dwrite d0/f6 [0,4194304] 0 2024-03-11T19:23:09.476 INFO:tasks.workunit.client.0.smithi090.stdout:6/56: write d4/f11 [971402,55680] 0 2024-03-11T19:23:09.512 INFO:tasks.workunit.client.0.smithi090.stdout:1/61: mknod d5/c16 0 2024-03-11T19:23:09.622 INFO:tasks.workunit.client.0.smithi090.stdout:6/57: dread d4/f11 [0,4194304] 0 2024-03-11T19:23:09.806 INFO:tasks.workunit.client.0.smithi090.stdout:5/50: rmdir d1 39 2024-03-11T19:23:09.817 INFO:tasks.workunit.client.0.smithi090.stdout:8/61: getdents d0/d3/da 0 2024-03-11T19:23:09.847 INFO:tasks.workunit.client.0.smithi090.stdout:8/62: chown d0/d3/da/le 1 1 2024-03-11T19:23:09.882 INFO:tasks.workunit.client.0.smithi090.stdout:3/41: link d2/f6 d2/ff 0 2024-03-11T19:23:09.964 INFO:tasks.workunit.client.0.smithi090.stdout:2/54: creat d0/fd x:0 0 0 2024-03-11T19:23:10.019 INFO:tasks.workunit.client.0.smithi090.stdout:4/51: dread d4/ff [0,4194304] 0 2024-03-11T19:23:10.045 INFO:tasks.workunit.client.0.smithi090.stdout:4/52: chown d4/f5 13747 1 2024-03-11T19:23:10.148 INFO:tasks.workunit.client.0.smithi090.stdout:1/62: fsync d5/f6 0 2024-03-11T19:23:10.578 INFO:tasks.workunit.client.0.smithi090.stdout:0/56: link d2/f8 d2/fe 0 2024-03-11T19:23:10.611 INFO:tasks.workunit.client.0.smithi090.stdout:9/44: truncate d1/f3 5693324 0 2024-03-11T19:23:10.615 INFO:tasks.workunit.client.0.smithi090.stdout:8/63: dread - d0/d3/da/dc/fd zero size 2024-03-11T19:23:10.696 INFO:tasks.workunit.client.0.smithi090.stdout:9/45: write d1/fc [442466,23888] 0 2024-03-11T19:23:10.747 INFO:tasks.workunit.client.0.smithi090.stdout:9/46: write d1/f6 [12584,37425] 0 2024-03-11T19:23:10.936 INFO:tasks.workunit.client.0.smithi090.stdout:9/47: dread d1/f6 [0,4194304] 0 2024-03-11T19:23:11.114 INFO:tasks.workunit.client.0.smithi090.stdout:9/48: sync 2024-03-11T19:23:11.216 INFO:tasks.workunit.client.0.smithi090.stdout:6/58: dwrite d4/fe [0,4194304] 0 2024-03-11T19:23:11.342 INFO:tasks.workunit.client.0.smithi090.stdout:7/49: truncate d3/f9 3179028 0 2024-03-11T19:23:11.389 INFO:tasks.workunit.client.0.smithi090.stdout:1/63: rmdir d5/da/dd 39 2024-03-11T19:23:11.455 INFO:tasks.workunit.client.0.smithi090.stdout:3/42: write d2/f3 [2724352,100024] 0 2024-03-11T19:23:11.502 INFO:tasks.workunit.client.0.smithi090.stdout:3/43: read d2/f6 [2920123,41449] 0 2024-03-11T19:23:11.507 INFO:tasks.workunit.client.0.smithi090.stdout:5/51: rmdir d1/d4 39 2024-03-11T19:23:11.536 INFO:tasks.workunit.client.0.smithi090.stdout:5/52: chown d1 84784716 1 2024-03-11T19:23:11.643 INFO:tasks.workunit.client.0.smithi090.stdout:8/64: mknod d0/d3/cf 0 2024-03-11T19:23:11.708 INFO:tasks.workunit.client.0.smithi090.stdout:3/44: dread f0 [0,4194304] 0 2024-03-11T19:23:12.659 INFO:tasks.workunit.client.0.smithi090.stdout:4/53: truncate d4/f10 3503738 0 2024-03-11T19:23:12.661 INFO:tasks.workunit.client.0.smithi090.stdout:9/49: creat d1/fe x:0 0 0 2024-03-11T19:23:12.739 INFO:tasks.workunit.client.0.smithi090.stdout:7/50: creat d3/dd/f11 x:0 0 0 2024-03-11T19:23:12.851 INFO:tasks.workunit.client.0.smithi090.stdout:6/59: rename d4/c7 to d4/c16 0 2024-03-11T19:23:13.069 INFO:tasks.workunit.client.0.smithi090.stdout:1/64: unlink d5/f6 0 2024-03-11T19:23:13.106 INFO:tasks.workunit.client.0.smithi090.stdout:6/60: dwrite d4/fe [0,4194304] 0 2024-03-11T19:23:13.212 INFO:tasks.workunit.client.0.smithi090.stdout:1/65: readlink l3 0 2024-03-11T19:23:13.420 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:23:13.423 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:23:13.443 INFO:tasks.workunit.client.0.smithi090.stdout:8/65: creat d0/d3/f10 x:0 0 0 2024-03-11T19:23:13.483 INFO:tasks.workunit.client.0.smithi090.stdout:8/66: dread - d0/d3/da/dc/fd zero size 2024-03-11T19:23:14.890 INFO:tasks.workunit.client.0.smithi090.stdout:5/53: truncate d1/d4/fc 304317 0 2024-03-11T19:23:15.139 INFO:tasks.workunit.client.0.smithi090.stdout:8/67: creat d0/f11 x:0 0 0 2024-03-11T19:23:15.197 INFO:tasks.workunit.client.0.smithi090.stdout:0/57: link d2/cc d2/cf 0 2024-03-11T19:23:15.272 INFO:tasks.workunit.client.0.smithi090.stdout:0/58: chown c1 4844 1 2024-03-11T19:23:15.336 INFO:tasks.workunit.client.0.smithi090.stdout:9/50: dread d1/f3 [0,4194304] 0 2024-03-11T19:23:15.630 INFO:tasks.workunit.client.0.smithi090.stdout:2/55: getdents d0 0 2024-03-11T19:23:15.711 INFO:tasks.workunit.client.0.smithi090.stdout:9/51: dwrite d1/fc [0,4194304] 0 2024-03-11T19:23:15.835 INFO:tasks.workunit.client.0.smithi090.stdout:7/51: link d3/dd/f11 d3/dd/f12 0 2024-03-11T19:23:16.032 INFO:tasks.workunit.client.0.smithi090.stdout:1/66: read d5/da/dd/f14 [888304,31390] 0 2024-03-11T19:23:16.672 INFO:tasks.workunit.client.0.smithi090.stdout:2/56: fdatasync d0/f3 0 2024-03-11T19:23:16.793 INFO:tasks.workunit.client.0.smithi090.stdout:3/45: getdents d2 0 2024-03-11T19:23:16.811 INFO:tasks.workunit.client.0.smithi090.stdout:5/54: mkdir d1/de 0 2024-03-11T19:23:16.827 INFO:tasks.workunit.client.0.smithi090.stdout:1/67: rmdir d5 39 2024-03-11T19:23:16.946 INFO:tasks.workunit.client.0.smithi090.stdout:2/57: chown d0/fb 39639651 1 2024-03-11T19:23:16.983 INFO:tasks.workunit.client.0.smithi090.stdout:3/46: chown d2/ff 144889969 1 2024-03-11T19:23:17.081 INFO:tasks.workunit.client.0.smithi090.stdout:6/61: dwrite d4/fb [4194304,4194304] 0 2024-03-11T19:23:18.047 INFO:tasks.workunit.client.0.smithi090.stdout:8/68: dwrite d0/d3/f8 [0,4194304] 0 2024-03-11T19:23:18.056 INFO:tasks.workunit.client.0.smithi090.stdout:5/55: dread d1/f3 [0,4194304] 0 2024-03-11T19:23:18.309 INFO:tasks.workunit.client.0.smithi090.stdout:6/62: dread - d4/f10 zero size 2024-03-11T19:23:18.326 INFO:tasks.workunit.client.0.smithi090.stdout:7/52: write d3/dd/ff [1421011,103599] 0 2024-03-11T19:23:18.565 INFO:tasks.workunit.client.0.smithi090.stdout:0/59: getdents d2 0 2024-03-11T19:23:18.630 INFO:tasks.workunit.client.0.smithi090.stdout:4/54: dread d4/f10 [0,4194304] 0 2024-03-11T19:23:18.671 INFO:tasks.workunit.client.0.smithi090.stdout:7/53: dread - d3/dd/f12 zero size 2024-03-11T19:23:18.759 INFO:tasks.workunit.client.0.smithi090.stdout:9/52: dwrite d1/f6 [0,4194304] 0 2024-03-11T19:23:18.766 INFO:tasks.workunit.client.0.smithi090.stdout:4/55: read d4/f8 [260453,6300] 0 2024-03-11T19:23:18.904 INFO:tasks.workunit.client.0.smithi090.stdout:3/47: symlink d2/de/l10 0 2024-03-11T19:23:19.033 INFO:tasks.workunit.client.0.smithi090.stdout:6/63: fdatasync d4/f11 0 2024-03-11T19:23:19.322 INFO:tasks.workunit.client.0.smithi090.stdout:8/69: truncate d0/d3/fb 4916943 0 2024-03-11T19:23:19.366 INFO:tasks.workunit.client.0.smithi090.stdout:2/58: dwrite d0/fc [0,4194304] 0 2024-03-11T19:23:19.391 INFO:tasks.workunit.client.0.smithi090.stdout:0/60: creat d2/f10 x:0 0 0 2024-03-11T19:23:19.976 INFO:tasks.workunit.client.0.smithi090.stdout:0/61: dread d2/f3 [0,4194304] 0 2024-03-11T19:23:20.026 INFO:tasks.workunit.client.0.smithi090.stdout:9/53: dread f0 [0,4194304] 0 2024-03-11T19:23:20.035 INFO:tasks.workunit.client.0.smithi090.stdout:5/56: dwrite d1/d4/f6 [0,4194304] 0 2024-03-11T19:23:20.113 INFO:tasks.workunit.client.0.smithi090.stdout:3/48: truncate d2/f3 2789704 0 2024-03-11T19:23:20.152 INFO:tasks.workunit.client.0.smithi090.stdout:7/54: mkdir d3/dc/d13 0 2024-03-11T19:23:20.166 INFO:tasks.workunit.client.0.smithi090.stdout:2/59: fsync d0/f3 0 2024-03-11T19:23:20.209 INFO:tasks.workunit.client.0.smithi090.stdout:8/70: rename d0/f11 to d0/d3/da/dc/f12 0 2024-03-11T19:23:20.280 INFO:tasks.workunit.client.0.smithi090.stdout:2/60: read d0/f7 [1243017,21737] 0 2024-03-11T19:23:20.466 INFO:tasks.workunit.client.0.smithi090.stdout:6/64: link d4/f11 d4/d13/f17 0 2024-03-11T19:23:20.474 INFO:tasks.workunit.client.0.smithi090.stdout:1/68: link d5/da/lf d5/da/dd/l17 0 2024-03-11T19:23:21.030 INFO:tasks.workunit.client.0.smithi090.stdout:7/55: truncate d3/f4 3426730 0 2024-03-11T19:23:21.044 INFO:tasks.workunit.client.0.smithi090.stdout:8/71: symlink d0/d3/da/l13 0 2024-03-11T19:23:21.054 INFO:tasks.workunit.client.0.smithi090.stdout:2/61: mkdir d0/de 0 2024-03-11T19:23:21.224 INFO:tasks.workunit.client.0.smithi090.stdout:6/65: read d4/ff [263824,98144] 0 2024-03-11T19:23:21.283 INFO:tasks.workunit.client.0.smithi090.stdout:6/66: readlink d4/l8 0 2024-03-11T19:23:21.330 INFO:tasks.workunit.client.0.smithi090.stdout:1/69: symlink d5/da/dd/l18 0 2024-03-11T19:23:21.334 INFO:tasks.workunit.client.0.smithi090.stdout:1/70: stat c4 0 2024-03-11T19:23:21.461 INFO:tasks.workunit.client.0.smithi090.stdout:4/56: getdents d4 0 2024-03-11T19:23:21.650 INFO:tasks.workunit.client.0.smithi090.stdout:9/54: dwrite f0 [0,4194304] 0 2024-03-11T19:23:21.654 INFO:tasks.workunit.client.0.smithi090.stdout:3/49: write d2/d4/fc [703325,31824] 0 2024-03-11T19:23:21.785 INFO:tasks.workunit.client.0.smithi090.stdout:5/57: mknod d1/cf 0 2024-03-11T19:23:22.131 INFO:tasks.workunit.client.0.smithi090.stdout:0/62: dwrite d2/fe [0,4194304] 0 2024-03-11T19:23:23.013 INFO:tasks.workunit.client.0.smithi090.stdout:8/72: write d0/d3/f10 [490285,117238] 0 2024-03-11T19:23:23.029 INFO:tasks.workunit.client.0.smithi090.stdout:3/50: rename d2/db/ld to d2/de/l11 0 2024-03-11T19:23:23.192 INFO:tasks.workunit.client.0.smithi090.stdout:3/51: chown d2/c5 260 1 2024-03-11T19:23:23.221 INFO:tasks.workunit.client.0.smithi090.stdout:7/56: symlink d3/dc/d13/l14 0 2024-03-11T19:23:23.364 INFO:tasks.workunit.client.0.smithi090.stdout:2/62: dwrite d0/f2 [0,4194304] 0 2024-03-11T19:23:23.544 INFO:tasks.workunit.client.0.smithi090.stdout:6/67: mkdir d4/d15/d18 0 2024-03-11T19:23:23.928 INFO:tasks.workunit.client.0.smithi090.stdout:5/58: fdatasync d1/d4/fc 0 2024-03-11T19:23:23.993 INFO:tasks.workunit.client.0.smithi090.stdout:9/55: mkdir d1/dd/df 0 2024-03-11T19:23:24.082 INFO:tasks.workunit.client.0.smithi090.stdout:8/73: fdatasync d0/f7 0 2024-03-11T19:23:24.289 INFO:tasks.workunit.client.0.smithi090.stdout:4/57: dwrite d4/fa [0,4194304] 0 2024-03-11T19:23:24.661 INFO:tasks.workunit.client.0.smithi090.stdout:1/71: creat d5/da/f19 x:0 0 0 2024-03-11T19:23:24.681 INFO:tasks.workunit.client.0.smithi090.stdout:1/72: readlink d5/l9 0 2024-03-11T19:23:24.703 INFO:tasks.workunit.client.0.smithi090.stdout:1/73: stat d5/da/dd/f14 0 2024-03-11T19:23:25.506 INFO:tasks.workunit.client.0.smithi090.stdout:8/74: rename d0/d3/da/le to d0/d3/da/l14 0 2024-03-11T19:23:25.523 INFO:tasks.workunit.client.0.smithi090.stdout:3/52: write d2/f9 [1050930,92299] 0 2024-03-11T19:23:25.587 INFO:tasks.workunit.client.0.smithi090.stdout:2/63: mknod d0/de/cf 0 2024-03-11T19:23:25.818 INFO:tasks.workunit.client.0.smithi090.stdout:1/74: truncate d5/da/dd/f14 728019 0 2024-03-11T19:23:25.973 INFO:tasks.workunit.client.0.smithi090.stdout:9/56: creat d1/dd/df/f10 x:0 0 0 2024-03-11T19:23:26.159 INFO:tasks.workunit.client.0.smithi090.stdout:5/59: truncate d1/d4/fc 923205 0 2024-03-11T19:23:26.215 INFO:tasks.workunit.client.0.smithi090.stdout:0/63: getdents d2 0 2024-03-11T19:23:26.390 INFO:tasks.workunit.client.0.smithi090.stdout:7/57: truncate d3/dd/ff 3670268 0 2024-03-11T19:23:26.480 INFO:tasks.workunit.client.0.smithi090.stdout:7/58: readlink d3/l6 0 2024-03-11T19:23:26.670 INFO:tasks.workunit.client.0.smithi090.stdout:4/58: dwrite d4/f5 [0,4194304] 0 2024-03-11T19:23:26.787 INFO:tasks.workunit.client.0.smithi090.stdout:8/75: truncate d0/f7 2766921 0 2024-03-11T19:23:26.820 INFO:tasks.workunit.client.0.smithi090.stdout:6/68: rmdir d4/d15/d18 0 2024-03-11T19:23:26.975 INFO:tasks.workunit.client.0.smithi090.stdout:4/59: dread d4/fa [0,4194304] 0 2024-03-11T19:23:27.144 INFO:tasks.workunit.client.0.smithi090.stdout:9/57: creat d1/dd/f11 x:0 0 0 2024-03-11T19:23:27.157 INFO:tasks.workunit.client.0.smithi090.stdout:0/64: mkdir d2/d11 0 2024-03-11T19:23:27.223 INFO:tasks.workunit.client.0.smithi090.stdout:5/60: fsync d1/d4/f6 0 2024-03-11T19:23:27.282 INFO:tasks.workunit.client.0.smithi090.stdout:2/64: write d0/f7 [7363948,127429] 0 2024-03-11T19:23:27.494 INFO:tasks.workunit.client.0.smithi090.stdout:2/65: chown d0/f5 14305545 1 2024-03-11T19:23:27.667 INFO:tasks.workunit.client.0.smithi090.stdout:3/53: dwrite d2/ff [0,4194304] 0 2024-03-11T19:23:27.788 INFO:tasks.workunit.client.0.smithi090.stdout:7/59: creat d3/f15 x:0 0 0 2024-03-11T19:23:27.971 INFO:tasks.workunit.client.0.smithi090.stdout:0/65: dwrite d2/f10 [0,4194304] 0 2024-03-11T19:23:28.266 INFO:tasks.workunit.client.0.smithi090.stdout:0/66: dread d2/f10 [0,4194304] 0 2024-03-11T19:23:28.267 INFO:tasks.workunit.client.0.smithi090.stdout:0/67: readlink - no filename 2024-03-11T19:23:28.326 INFO:tasks.workunit.client.0.smithi090.stdout:6/69: fsync d4/fb 0 2024-03-11T19:23:28.444 INFO:tasks.workunit.client.0.smithi090.stdout:4/60: creat d4/f12 x:0 0 0 2024-03-11T19:23:28.578 INFO:tasks.workunit.client.0.smithi090.stdout:9/58: rmdir d1/dd 39 2024-03-11T19:23:28.894 INFO:tasks.workunit.client.0.smithi090.stdout:2/66: mkdir d0/d10 0 2024-03-11T19:23:28.920 INFO:tasks.workunit.client.0.smithi090.stdout:6/70: sync 2024-03-11T19:23:29.133 INFO:tasks.workunit.client.0.smithi090.stdout:3/54: creat d2/f12 x:0 0 0 2024-03-11T19:23:29.248 INFO:tasks.workunit.client.0.smithi090.stdout:7/60: mknod d3/c16 0 2024-03-11T19:23:29.872 INFO:tasks.workunit.client.0.smithi090.stdout:0/68: write d2/fe [1873539,29815] 0 2024-03-11T19:23:30.079 INFO:tasks.workunit.client.0.smithi090.stdout:1/75: dwrite d5/da/dd/f14 [0,4194304] 0 2024-03-11T19:23:30.396 INFO:tasks.workunit.client.0.smithi090.stdout:5/61: truncate d1/d4/fc 932453 0 2024-03-11T19:23:30.486 INFO:tasks.workunit.client.0.smithi090.stdout:2/67: symlink d0/l11 0 2024-03-11T19:23:30.611 INFO:tasks.workunit.client.0.smithi090.stdout:1/76: dwrite d5/da/dd/f14 [0,4194304] 0 2024-03-11T19:23:30.750 INFO:tasks.workunit.client.0.smithi090.stdout:1/77: fsync d5/da/dd/f14 0 2024-03-11T19:23:30.803 INFO:tasks.workunit.client.0.smithi090.stdout:3/55: truncate d2/d4/fc 1468423 0 2024-03-11T19:23:30.818 INFO:tasks.workunit.client.0.smithi090.stdout:7/61: mkdir d3/dc/d17 0 2024-03-11T19:23:30.845 INFO:tasks.workunit.client.0.smithi090.stdout:6/71: chown d4/c5 1027 1 2024-03-11T19:23:30.849 INFO:tasks.workunit.client.0.smithi090.stdout:7/62: readlink d3/l6 0 2024-03-11T19:23:31.119 INFO:tasks.workunit.client.0.smithi090.stdout:3/56: dwrite d2/f6 [0,4194304] 0 2024-03-11T19:23:31.356 INFO:tasks.workunit.client.0.smithi090.stdout:9/59: write d1/f6 [5721977,55577] 0 2024-03-11T19:23:31.376 INFO:tasks.workunit.client.0.smithi090.stdout:0/69: unlink d2/f9 0 2024-03-11T19:23:31.412 INFO:tasks.workunit.client.0.smithi090.stdout:1/78: fdatasync d5/da/f19 0 2024-03-11T19:23:31.623 INFO:tasks.workunit.client.0.smithi090.stdout:7/63: creat d3/dc/d13/f18 x:0 0 0 2024-03-11T19:23:31.643 INFO:tasks.workunit.client.0.smithi090.stdout:3/57: mknod d2/c13 0 2024-03-11T19:23:31.724 INFO:tasks.workunit.client.0.smithi090.stdout:8/76: dread d0/f7 [0,4194304] 0 2024-03-11T19:23:31.783 INFO:tasks.workunit.client.0.smithi090.stdout:3/58: sync 2024-03-11T19:23:31.903 INFO:tasks.workunit.client.0.smithi090.stdout:5/62: mknod d1/c10 0 2024-03-11T19:23:31.995 INFO:tasks.workunit.client.0.smithi090.stdout:1/79: creat d5/da/dd/f1a x:0 0 0 2024-03-11T19:23:32.016 INFO:tasks.workunit.client.0.smithi090.stdout:4/61: getdents d4 0 2024-03-11T19:23:32.027 INFO:tasks.workunit.client.0.smithi090.stdout:5/63: chown d1/c8 2659 1 2024-03-11T19:23:32.201 INFO:tasks.workunit.client.0.smithi090.stdout:7/64: creat d3/dc/f19 x:0 0 0 2024-03-11T19:23:32.226 INFO:tasks.workunit.client.0.smithi090.stdout:7/65: chown d3/f15 2036 1 2024-03-11T19:23:32.251 INFO:tasks.workunit.client.0.smithi090.stdout:8/77: truncate d0/f7 2659784 0 2024-03-11T19:23:32.266 INFO:tasks.workunit.client.0.smithi090.stdout:7/66: chown d3/dc/d17 18308025 1 2024-03-11T19:23:32.497 INFO:tasks.workunit.client.0.smithi090.stdout:3/59: mknod d2/c14 0 2024-03-11T19:23:32.659 INFO:tasks.workunit.client.0.smithi090.stdout:0/70: mknod d2/d11/c12 0 2024-03-11T19:23:32.675 INFO:tasks.workunit.client.0.smithi090.stdout:0/71: stat c1 0 2024-03-11T19:23:32.759 INFO:tasks.workunit.client.0.smithi090.stdout:1/80: fdatasync d5/fc 0 2024-03-11T19:23:33.158 INFO:tasks.workunit.client.0.smithi090.stdout:2/68: dwrite d0/f4 [0,4194304] 0 2024-03-11T19:23:33.452 INFO:tasks.workunit.client.0.smithi090.stdout:7/67: fsync d3/dd/f11 0 2024-03-11T19:23:33.820 INFO:tasks.workunit.client.0.smithi090.stdout:1/81: dwrite d5/da/dd/f1a [0,4194304] 0 2024-03-11T19:23:34.050 INFO:tasks.workunit.client.0.smithi090.stdout:6/72: dwrite d4/f11 [0,4194304] 0 2024-03-11T19:23:34.190 INFO:tasks.workunit.client.0.smithi090.stdout:0/72: fdatasync d2/f3 0 2024-03-11T19:23:34.347 INFO:tasks.workunit.client.0.smithi090.stdout:3/60: dread d2/d4/fc [0,4194304] 0 2024-03-11T19:23:34.356 INFO:tasks.workunit.client.0.smithi090.stdout:1/82: fdatasync d5/da/dd/f14 0 2024-03-11T19:23:34.449 INFO:tasks.workunit.client.0.smithi090.stdout:4/62: getdents d4 0 2024-03-11T19:23:34.903 INFO:tasks.workunit.client.0.smithi090.stdout:8/78: write d0/d3/f10 [1560615,99071] 0 2024-03-11T19:23:34.952 INFO:tasks.workunit.client.0.smithi090.stdout:2/69: write d0/f7 [7344615,107759] 0 2024-03-11T19:23:35.181 INFO:tasks.workunit.client.0.smithi090.stdout:8/79: dread d0/d3/f10 [0,4194304] 0 2024-03-11T19:23:35.215 INFO:tasks.workunit.client.0.smithi090.stdout:8/80: rename d0/d3 to d0/d3/da/d15 22 2024-03-11T19:23:35.278 INFO:tasks.workunit.client.0.smithi090.stdout:9/60: link d1/l8 d1/dd/df/l12 0 2024-03-11T19:23:35.681 INFO:tasks.workunit.client.0.smithi090.stdout:6/73: dread d4/fb [4194304,4194304] 0 2024-03-11T19:23:35.842 INFO:tasks.workunit.client.0.smithi090.stdout:1/83: read d5/fc [1566911,16186] 0 2024-03-11T19:23:35.930 INFO:tasks.workunit.client.0.smithi090.stdout:4/63: symlink d4/l13 0 2024-03-11T19:23:35.979 INFO:tasks.workunit.client.0.smithi090.stdout:6/74: dwrite d4/f11 [0,4194304] 0 2024-03-11T19:23:35.980 INFO:tasks.workunit.client.0.smithi090.stdout:4/64: chown d4 8 1 2024-03-11T19:23:36.104 INFO:tasks.workunit.client.0.smithi090.stdout:5/64: write d1/d4/fc [331021,60724] 0 2024-03-11T19:23:36.172 INFO:tasks.workunit.client.0.smithi090.stdout:4/65: chown d4/f10 1822 1 2024-03-11T19:23:36.191 INFO:tasks.workunit.client.0.smithi090.stdout:4/66: chown c3 35845225 1 2024-03-11T19:23:36.254 INFO:tasks.workunit.client.0.smithi090.stdout:2/70: mkdir d0/de/d12 0 2024-03-11T19:23:36.364 INFO:tasks.workunit.client.0.smithi090.stdout:8/81: symlink d0/d3/da/dc/l16 0 2024-03-11T19:23:36.396 INFO:tasks.workunit.client.0.smithi090.stdout:7/68: getdents d3/dc/d17 0 2024-03-11T19:23:36.457 INFO:tasks.workunit.client.0.smithi090.stdout:9/61: truncate d1/f3 6558978 0 2024-03-11T19:23:36.980 INFO:tasks.workunit.client.0.smithi090.stdout:1/84: creat d5/f1b x:0 0 0 2024-03-11T19:23:37.077 INFO:tasks.workunit.client.0.smithi090.stdout:3/61: dwrite d2/f8 [0,4194304] 0 2024-03-11T19:23:37.285 INFO:tasks.workunit.client.0.smithi090.stdout:4/67: rename d4/cd to d4/c14 0 2024-03-11T19:23:37.574 INFO:tasks.workunit.client.0.smithi090.stdout:7/69: dread d3/dd/ff [0,4194304] 0 2024-03-11T19:23:38.035 INFO:tasks.workunit.client.0.smithi090.stdout:0/73: truncate d2/f8 2738479 0 2024-03-11T19:23:38.542 INFO:tasks.workunit.client.0.smithi090.stdout:5/65: dwrite d1/d4/f6 [0,4194304] 0 2024-03-11T19:23:38.739 INFO:tasks.workunit.client.0.smithi090.stdout:6/75: symlink d4/d14/l19 0 2024-03-11T19:23:38.759 INFO:tasks.workunit.client.0.smithi090.stdout:6/76: dread - d4/f10 zero size 2024-03-11T19:23:38.824 INFO:tasks.workunit.client.0.smithi090.stdout:6/77: write d4/d13/f17 [946421,59391] 0 2024-03-11T19:23:38.903 INFO:tasks.workunit.client.0.smithi090.stdout:3/62: write d2/ff [3940171,50833] 0 2024-03-11T19:23:39.119 INFO:tasks.workunit.client.0.smithi090.stdout:4/68: rename d4/f11 to d4/f15 0 2024-03-11T19:23:39.934 INFO:tasks.workunit.client.0.smithi090.stdout:0/74: symlink d2/d11/l13 0 2024-03-11T19:23:40.192 INFO:tasks.workunit.client.0.smithi090.stdout:0/75: sync 2024-03-11T19:23:40.263 INFO:tasks.workunit.client.0.smithi090.stdout:1/85: mkdir d5/da/d1c 0 2024-03-11T19:23:40.384 INFO:tasks.workunit.client.0.smithi090.stdout:1/86: dread d5/fc [4194304,4194304] 0 2024-03-11T19:23:40.598 INFO:tasks.workunit.client.0.smithi090.stdout:7/70: truncate d3/f4 3842997 0 2024-03-11T19:23:40.609 INFO:tasks.workunit.client.0.smithi090.stdout:4/69: dread d4/f5 [0,4194304] 0 2024-03-11T19:23:40.757 INFO:tasks.workunit.client.0.smithi090.stdout:4/70: dread d4/f10 [0,4194304] 0 2024-03-11T19:23:40.788 INFO:tasks.workunit.client.0.smithi090.stdout:8/82: link d0/d3/da/dc/f12 d0/f17 0 2024-03-11T19:23:40.844 INFO:tasks.workunit.client.0.smithi090.stdout:9/62: link d1/fc d1/dd/f13 0 2024-03-11T19:23:40.953 INFO:tasks.workunit.client.0.smithi090.stdout:9/63: dread d1/f9 [0,4194304] 0 2024-03-11T19:23:40.981 INFO:tasks.workunit.client.0.smithi090.stdout:2/71: truncate d0/f7 7897264 0 2024-03-11T19:23:41.101 INFO:tasks.workunit.client.0.smithi090.stdout:6/78: mkdir d4/d13/d1a 0 2024-03-11T19:23:41.404 INFO:tasks.workunit.client.0.smithi090.stdout:7/71: symlink d3/dc/l1a 0 2024-03-11T19:23:41.764 INFO:tasks.workunit.client.0.smithi090.stdout:8/83: mknod d0/d3/c18 0 2024-03-11T19:23:41.782 INFO:tasks.workunit.client.0.smithi090.stdout:2/72: fsync d0/f5 0 2024-03-11T19:23:41.864 INFO:tasks.workunit.client.0.smithi090.stdout:9/64: dread - d1/dd/df/f10 zero size 2024-03-11T19:23:41.910 INFO:tasks.workunit.client.0.smithi090.stdout:5/66: dwrite d1/f3 [0,4194304] 0 2024-03-11T19:23:42.046 INFO:tasks.workunit.client.0.smithi090.stdout:2/73: write d0/fd [242482,25283] 0 2024-03-11T19:23:42.122 INFO:tasks.workunit.client.0.smithi090.stdout:6/79: creat d4/d13/f1b x:0 0 0 2024-03-11T19:23:42.133 INFO:tasks.workunit.client.0.smithi090.stdout:3/63: write d2/f9 [580897,84505] 0 2024-03-11T19:23:42.174 INFO:tasks.workunit.client.0.smithi090.stdout:1/87: mknod d5/da/d1c/c1d 0 2024-03-11T19:23:42.590 INFO:tasks.workunit.client.0.smithi090.stdout:9/65: symlink d1/dd/l14 0 2024-03-11T19:23:43.154 INFO:tasks.workunit.client.0.smithi090.stdout:2/74: dread d0/fc [0,4194304] 0 2024-03-11T19:23:43.161 INFO:tasks.workunit.client.0.smithi090.stdout:6/80: fdatasync d4/ff 0 2024-03-11T19:23:43.174 INFO:tasks.workunit.client.0.smithi090.stdout:4/71: write d4/f12 [777429,127083] 0 2024-03-11T19:23:43.246 INFO:tasks.workunit.client.0.smithi090.stdout:7/72: creat d3/dc/d17/f1b x:0 0 0 2024-03-11T19:23:43.391 INFO:tasks.workunit.client.0.smithi090.stdout:3/64: mkdir d2/db/d15 0 2024-03-11T19:23:43.508 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:23:43.511 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:23:43.570 INFO:tasks.workunit.client.0.smithi090.stdout:4/72: dwrite d4/f12 [0,4194304] 0 2024-03-11T19:23:43.649 INFO:tasks.workunit.client.0.smithi090.stdout:8/84: unlink d0/f7 0 2024-03-11T19:23:43.732 INFO:tasks.workunit.client.0.smithi090.stdout:1/88: write d5/fc [115289,89478] 0 2024-03-11T19:23:43.890 INFO:tasks.workunit.client.0.smithi090.stdout:4/73: dread d4/fa [0,4194304] 0 2024-03-11T19:23:44.376 INFO:tasks.workunit.client.0.smithi090.stdout:7/73: unlink c0 0 2024-03-11T19:23:44.940 INFO:tasks.workunit.client.0.smithi090.stdout:3/65: dwrite d2/f8 [0,4194304] 0 2024-03-11T19:23:45.108 INFO:tasks.workunit.client.0.smithi090.stdout:0/76: write d2/f8 [303035,105940] 0 2024-03-11T19:23:45.286 INFO:tasks.workunit.client.0.smithi090.stdout:6/81: mknod d4/d13/d1a/c1c 0 2024-03-11T19:23:45.456 INFO:tasks.workunit.client.0.smithi090.stdout:2/75: symlink d0/de/d12/l13 0 2024-03-11T19:23:45.805 INFO:tasks.workunit.client.0.smithi090.stdout:2/76: sync 2024-03-11T19:23:45.945 INFO:tasks.workunit.client.0.smithi090.stdout:8/85: fsync d0/d3/fb 0 2024-03-11T19:23:46.020 INFO:tasks.workunit.client.0.smithi090.stdout:4/74: truncate d4/f15 126582 0 2024-03-11T19:23:46.036 INFO:tasks.workunit.client.0.smithi090.stdout:4/75: chown c3 28 1 2024-03-11T19:23:46.205 INFO:tasks.workunit.client.0.smithi090.stdout:9/66: mknod d1/dd/c15 0 2024-03-11T19:23:46.658 INFO:tasks.workunit.client.0.smithi090.stdout:8/86: read d0/d3/f10 [396384,122664] 0 2024-03-11T19:23:47.024 INFO:tasks.workunit.client.0.smithi090.stdout:7/74: write d3/dc/d13/f18 [192320,32929] 0 2024-03-11T19:23:47.104 INFO:tasks.workunit.client.0.smithi090.stdout:6/82: rmdir d4/d13 39 2024-03-11T19:23:47.282 INFO:tasks.workunit.client.0.smithi090.stdout:7/75: chown d3/dc/d13/l14 1 1 2024-03-11T19:23:47.310 INFO:tasks.workunit.client.0.smithi090.stdout:1/89: dwrite d5/da/dd/f14 [0,4194304] 0 2024-03-11T19:23:47.456 INFO:tasks.workunit.client.0.smithi090.stdout:9/67: truncate d1/dd/f13 2981037 0 2024-03-11T19:23:47.646 INFO:tasks.workunit.client.0.smithi090.stdout:7/76: mknod d3/c1c 0 2024-03-11T19:23:47.828 INFO:tasks.workunit.client.0.smithi090.stdout:1/90: dwrite d5/da/dd/f14 [0,4194304] 0 2024-03-11T19:23:48.207 INFO:tasks.workunit.client.0.smithi090.stdout:7/77: mknod d3/dd/c1d 0 2024-03-11T19:23:48.293 INFO:tasks.workunit.client.0.smithi090.stdout:3/66: dwrite d2/f12 [0,4194304] 0 2024-03-11T19:23:48.624 INFO:tasks.workunit.client.0.smithi090.stdout:0/77: truncate d2/f8 596527 0 2024-03-11T19:23:48.725 INFO:tasks.workunit.client.0.smithi090.stdout:8/87: write d0/d3/da/dc/fd [34625,108140] 0 2024-03-11T19:23:49.062 INFO:tasks.workunit.client.0.smithi090.stdout:6/83: write d4/f10 [803463,79263] 0 2024-03-11T19:23:49.118 INFO:tasks.workunit.client.0.smithi090.stdout:2/77: dwrite d0/fb [4194304,4194304] 0 2024-03-11T19:23:49.261 INFO:tasks.workunit.client.0.smithi090.stdout:4/76: dwrite d4/f12 [4194304,4194304] 0 2024-03-11T19:23:49.443 INFO:tasks.workunit.client.0.smithi090.stdout:7/78: fsync d3/dd/f11 0 2024-03-11T19:23:49.515 INFO:tasks.workunit.client.0.smithi090.stdout:5/67: rmdir d1/de 0 2024-03-11T19:23:49.530 INFO:tasks.workunit.client.0.smithi090.stdout:9/68: dread d1/fc [0,4194304] 0 2024-03-11T19:23:49.607 INFO:tasks.workunit.client.0.smithi090.stdout:3/67: rename d2/c14 to d2/c16 0 2024-03-11T19:23:49.614 INFO:tasks.workunit.client.0.smithi090.stdout:9/69: chown d1/dd 1 1 2024-03-11T19:23:49.700 INFO:tasks.workunit.client.0.smithi090.stdout:6/84: dread d4/f10 [0,4194304] 0 2024-03-11T19:23:50.637 INFO:tasks.workunit.client.0.smithi090.stdout:5/68: fdatasync d1/f3 0 2024-03-11T19:23:50.723 INFO:tasks.workunit.client.0.smithi090.stdout:4/77: rename d4/c9 to d4/c16 0 2024-03-11T19:23:50.784 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:49:10.543 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:23:50.802 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:49:10.561 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:23:50.859 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:49:10.546 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:23:50.885 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:49:10.572 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:23:50.979 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:49:10.673 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:23:50.995 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:49:10.690 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:23:51.022 INFO:tasks.workunit.client.0.smithi090.stdout:9/70: fdatasync d1/dd/df/f10 0 2024-03-11T19:23:51.085 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:49:10.844 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:23:51.119 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:49:10.806 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:23:51.121 INFO:tasks.workunit.client.0.smithi090.stdout:9/71: dread d1/dd/f13 [0,4194304] 0 2024-03-11T19:23:51.133 INFO:tasks.workunit.client.0.smithi090.stdout:3/68: truncate d2/ff 2383 0 2024-03-11T19:23:51.178 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:49:10.872 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:23:51.360 INFO:tasks.workunit.client.0.smithi090.stdout:2/78: symlink d0/d10/l14 0 2024-03-11T19:23:51.393 INFO:tasks.workunit.client.0.smithi090.stdout:2/79: write d0/f6 [2570793,88136] 0 2024-03-11T19:23:51.437 INFO:tasks.workunit.client.0.smithi090.stdout:1/91: write d5/f15 [1026192,75292] 0 2024-03-11T19:23:51.519 INFO:tasks.workunit.client.0.smithi090.stdout:5/69: dread d1/d4/fc [0,4194304] 0 2024-03-11T19:23:51.553 INFO:tasks.workunit.client.0.smithi090.stdout:6/85: symlink d4/d13/l1d 0 2024-03-11T19:23:52.178 INFO:tasks.workunit.client.0.smithi090.stdout:1/92: symlink d5/da/dd/l1e 0 2024-03-11T19:23:52.393 INFO:tasks.workunit.client.0.smithi090.stdout:1/93: sync 2024-03-11T19:23:52.450 INFO:tasks.workunit.client.0.smithi090.stdout:5/70: creat d1/d4/f11 x:0 0 0 2024-03-11T19:23:52.488 INFO:tasks.workunit.client.0.smithi090.stdout:8/88: rename d0/d3/fb to d0/f19 0 2024-03-11T19:23:52.507 INFO:tasks.workunit.client.0.smithi090.stdout:3/69: creat d2/db/d15/f17 x:0 0 0 2024-03-11T19:23:52.630 INFO:tasks.workunit.client.0.smithi090.stdout:9/72: link d1/dd/f13 d1/dd/df/f16 0 2024-03-11T19:23:52.794 INFO:tasks.workunit.client.0.smithi090.stdout:1/94: creat d5/da/dd/f1f x:0 0 0 2024-03-11T19:23:52.993 INFO:tasks.workunit.client.0.smithi090.stdout:7/79: rename d3/dd to d3/dc/d1e 0 2024-03-11T19:23:52.994 INFO:tasks.workunit.client.0.smithi090.stdout:0/78: read d2/fe [183424,57693] 0 2024-03-11T19:23:53.199 INFO:tasks.workunit.client.0.smithi090.stdout:3/70: truncate d2/f9 914621 0 2024-03-11T19:23:53.617 INFO:tasks.workunit.client.0.smithi090.stdout:5/71: dwrite d1/f3 [4194304,4194304] 0 2024-03-11T19:23:53.858 INFO:tasks.workunit.client.0.smithi090.stdout:7/80: dread - d3/f15 zero size 2024-03-11T19:23:54.226 INFO:tasks.workunit.client.0.smithi090.stdout:6/86: rename d4/c9 to d4/c1e 0 2024-03-11T19:23:54.377 INFO:tasks.workunit.client.0.smithi090.stdout:0/79: creat d2/f14 x:0 0 0 2024-03-11T19:23:54.526 INFO:tasks.workunit.client.0.smithi090.stdout:0/80: readlink d2/d11/l13 0 2024-03-11T19:23:54.635 INFO:tasks.workunit.client.0.smithi090.stdout:0/81: dread - d2/f5 zero size 2024-03-11T19:23:54.700 INFO:tasks.workunit.client.0.smithi090.stdout:1/95: mknod d5/da/c20 0 2024-03-11T19:23:54.749 INFO:tasks.workunit.client.0.smithi090.stdout:2/80: dwrite d0/fd [0,4194304] 0 2024-03-11T19:23:54.879 INFO:tasks.workunit.client.0.smithi090.stdout:2/81: chown d0 172448 1 2024-03-11T19:23:55.006 INFO:tasks.workunit.client.0.smithi090.stdout:2/82: write d0/fb [3079735,52733] 0 2024-03-11T19:23:55.180 INFO:tasks.workunit.client.0.smithi090.stdout:9/73: write d1/fc [3676790,116274] 0 2024-03-11T19:23:55.289 INFO:tasks.workunit.client.0.smithi090.stdout:6/87: read d4/f11 [274805,129100] 0 2024-03-11T19:23:55.370 INFO:tasks.workunit.client.0.smithi090.stdout:4/78: rename c3 to d4/c17 0 2024-03-11T19:23:55.580 INFO:tasks.workunit.client.0.smithi090.stdout:0/82: mkdir d2/d11/d15 0 2024-03-11T19:23:55.726 INFO:tasks.workunit.client.0.smithi090.stdout:1/96: creat d5/da/d1c/f21 x:0 0 0 2024-03-11T19:23:55.754 INFO:tasks.workunit.client.0.smithi090.stdout:1/97: chown d5/da/lf 15009299 1 2024-03-11T19:23:56.379 INFO:tasks.workunit.client.0.smithi090.stdout:4/79: fsync d4/ff 0 2024-03-11T19:23:56.403 INFO:tasks.workunit.client.0.smithi090.stdout:4/80: stat d4/ff 0 2024-03-11T19:23:56.737 INFO:tasks.workunit.client.0.smithi090.stdout:0/83: symlink d2/d11/l16 0 2024-03-11T19:23:56.956 INFO:tasks.workunit.client.0.smithi090.stdout:5/72: rename d1/l2 to d1/l12 0 2024-03-11T19:23:56.974 INFO:tasks.workunit.client.0.smithi090.stdout:5/73: chown d1/ld 9699673 1 2024-03-11T19:23:57.434 INFO:tasks.workunit.client.0.smithi090.stdout:7/81: symlink d3/l1f 0 2024-03-11T19:23:57.455 INFO:tasks.workunit.client.0.smithi090.stdout:7/82: chown d3/dc/d13 407100763 1 2024-03-11T19:23:57.491 INFO:tasks.workunit.client.0.smithi090.stdout:4/81: rmdir d4 39 2024-03-11T19:23:57.731 INFO:tasks.workunit.client.0.smithi090.stdout:0/84: truncate d2/f5 323768 0 2024-03-11T19:23:57.746 INFO:tasks.workunit.client.0.smithi090.stdout:1/98: mknod d5/da/c22 0 2024-03-11T19:23:57.809 INFO:tasks.workunit.client.0.smithi090.stdout:3/71: write d2/f3 [3397009,78342] 0 2024-03-11T19:23:57.836 INFO:tasks.workunit.client.0.smithi090.stdout:2/83: mknod d0/c15 0 2024-03-11T19:23:57.983 INFO:tasks.workunit.client.0.smithi090.stdout:8/89: rename d0/l9 to d0/d3/l1a 0 2024-03-11T19:23:58.004 INFO:tasks.workunit.client.0.smithi090.stdout:9/74: rmdir d1 39 2024-03-11T19:23:58.147 INFO:tasks.workunit.client.0.smithi090.stdout:2/84: write d0/fb [3091135,105855] 0 2024-03-11T19:23:58.156 INFO:tasks.workunit.client.0.smithi090.stdout:7/83: creat d3/dc/d13/f20 x:0 0 0 2024-03-11T19:23:58.424 INFO:tasks.workunit.client.0.smithi090.stdout:7/84: stat d3/dc/d13/f18 0 2024-03-11T19:23:58.612 INFO:tasks.workunit.client.0.smithi090.stdout:6/88: dwrite d4/d13/f17 [4194304,4194304] 0 2024-03-11T19:23:58.622 INFO:tasks.workunit.client.0.smithi090.stdout:0/85: dwrite d2/f14 [0,4194304] 0 2024-03-11T19:23:58.843 INFO:tasks.workunit.client.0.smithi090.stdout:5/74: unlink d1/f3 0 2024-03-11T19:23:58.992 INFO:tasks.workunit.client.0.smithi090.stdout:2/85: dread d0/fb [0,4194304] 0 2024-03-11T19:23:59.105 INFO:tasks.workunit.client.0.smithi090.stdout:9/75: read d1/dd/df/f16 [1685725,55825] 0 2024-03-11T19:23:59.192 INFO:tasks.workunit.client.0.smithi090.stdout:2/86: dread d0/fd [0,4194304] 0 2024-03-11T19:23:59.205 INFO:tasks.workunit.client.0.smithi090.stdout:2/87: chown d0/d10 134 1 2024-03-11T19:23:59.365 INFO:tasks.workunit.client.0.smithi090.stdout:6/89: fdatasync f3 0 2024-03-11T19:23:59.562 INFO:tasks.workunit.client.0.smithi090.stdout:0/86: creat d2/d11/f17 x:0 0 0 2024-03-11T19:23:59.589 INFO:tasks.workunit.client.0.smithi090.stdout:0/87: chown d2/f14 15 1 2024-03-11T19:24:00.454 INFO:tasks.workunit.client.0.smithi090.stdout:0/88: creat d2/d11/f18 x:0 0 0 2024-03-11T19:24:00.545 INFO:tasks.workunit.client.0.smithi090.stdout:1/99: dwrite d5/da/dd/f14 [4194304,4194304] 0 2024-03-11T19:24:00.561 INFO:tasks.workunit.client.0.smithi090.stdout:4/82: write d4/fa [5095616,61960] 0 2024-03-11T19:24:00.767 INFO:tasks.workunit.client.0.smithi090.stdout:0/89: stat d2/d11/f17 0 2024-03-11T19:24:00.913 INFO:tasks.workunit.client.0.smithi090.stdout:3/72: getdents d2/db/d15 0 2024-03-11T19:24:01.260 INFO:tasks.workunit.client.0.smithi090.stdout:1/100: fdatasync d5/f1b 0 2024-03-11T19:24:01.345 INFO:tasks.workunit.client.0.smithi090.stdout:8/90: dwrite d0/d3/da/dc/fd [0,4194304] 0 2024-03-11T19:24:01.499 INFO:tasks.workunit.client.0.smithi090.stdout:4/83: mkdir d4/d18 0 2024-03-11T19:24:01.521 INFO:tasks.workunit.client.0.smithi090.stdout:7/85: dwrite d3/f8 [0,4194304] 0 2024-03-11T19:24:01.636 INFO:tasks.workunit.client.0.smithi090.stdout:4/84: readlink d4/l13 0 2024-03-11T19:24:01.706 INFO:tasks.workunit.client.0.smithi090.stdout:9/76: creat d1/f17 x:0 0 0 2024-03-11T19:24:01.865 INFO:tasks.workunit.client.0.smithi090.stdout:3/73: creat d2/db/d15/f18 x:0 0 0 2024-03-11T19:24:01.977 INFO:tasks.workunit.client.0.smithi090.stdout:9/77: sync 2024-03-11T19:24:02.196 INFO:tasks.workunit.client.0.smithi090.stdout:0/90: creat d2/d11/d15/f19 x:0 0 0 2024-03-11T19:24:02.380 INFO:tasks.workunit.client.0.smithi090.stdout:6/90: write d4/fb [1892232,45918] 0 2024-03-11T19:24:02.396 INFO:tasks.workunit.client.0.smithi090.stdout:3/74: read d2/f8 [3350173,120371] 0 2024-03-11T19:24:02.437 INFO:tasks.workunit.client.0.smithi090.stdout:8/91: fdatasync d0/f19 0 2024-03-11T19:24:02.621 INFO:tasks.workunit.client.0.smithi090.stdout:9/78: dread f0 [0,4194304] 0 2024-03-11T19:24:02.789 INFO:tasks.workunit.client.0.smithi090.stdout:2/88: truncate d0/fc 1874544 0 2024-03-11T19:24:02.951 INFO:tasks.workunit.client.0.smithi090.stdout:4/85: unlink d4/f15 0 2024-03-11T19:24:02.991 INFO:tasks.workunit.client.0.smithi090.stdout:4/86: write d4/fa [4262304,61810] 0 2024-03-11T19:24:03.031 INFO:tasks.workunit.client.0.smithi090.stdout:3/75: stat f0 0 2024-03-11T19:24:03.088 INFO:tasks.workunit.client.0.smithi090.stdout:3/76: dread - d2/db/d15/f17 zero size 2024-03-11T19:24:03.327 INFO:tasks.workunit.client.0.smithi090.stdout:1/101: mkdir d5/da/d23 0 2024-03-11T19:24:03.358 INFO:tasks.workunit.client.0.smithi090.stdout:8/92: creat d0/d3/da/f1b x:0 0 0 2024-03-11T19:24:03.368 INFO:tasks.workunit.client.0.smithi090.stdout:6/91: dread - d4/d13/f1b zero size 2024-03-11T19:24:03.550 INFO:tasks.workunit.client.0.smithi090.stdout:9/79: mkdir d1/dd/d18 0 2024-03-11T19:24:03.709 INFO:tasks.workunit.client.0.smithi090.stdout:6/92: dwrite d4/d13/f17 [4194304,4194304] 0 2024-03-11T19:24:03.943 INFO:tasks.workunit.client.0.smithi090.stdout:4/87: mknod d4/c19 0 2024-03-11T19:24:04.266 INFO:tasks.workunit.client.0.smithi090.stdout:8/93: mknod d0/d3/da/c1c 0 2024-03-11T19:24:04.385 INFO:tasks.workunit.client.0.smithi090.stdout:5/75: rename d1/cf to d1/d4/c13 0 2024-03-11T19:24:04.585 INFO:tasks.workunit.client.0.smithi090.stdout:9/80: unlink d1/fc 0 2024-03-11T19:24:04.608 INFO:tasks.workunit.client.0.smithi090.stdout:0/91: write d2/fe [667314,59159] 0 2024-03-11T19:24:04.901 INFO:tasks.workunit.client.0.smithi090.stdout:4/88: rename d4/cc to d4/c1a 0 2024-03-11T19:24:05.132 INFO:tasks.workunit.client.0.smithi090.stdout:7/86: dwrite d3/f4 [0,4194304] 0 2024-03-11T19:24:05.268 INFO:tasks.workunit.client.0.smithi090.stdout:8/94: symlink d0/d3/l1d 0 2024-03-11T19:24:05.295 INFO:tasks.workunit.client.0.smithi090.stdout:5/76: dread - d1/d4/f11 zero size 2024-03-11T19:24:05.347 INFO:tasks.workunit.client.0.smithi090.stdout:5/77: chown d1/d4/lb 206 1 2024-03-11T19:24:05.366 INFO:tasks.workunit.client.0.smithi090.stdout:5/78: read - d1/d4/f11 zero size 2024-03-11T19:24:05.602 INFO:tasks.workunit.client.0.smithi090.stdout:0/92: dread f0 [0,4194304] 0 2024-03-11T19:24:05.791 INFO:tasks.workunit.client.0.smithi090.stdout:9/81: mkdir d1/dd/d19 0 2024-03-11T19:24:05.978 INFO:tasks.workunit.client.0.smithi090.stdout:4/89: fsync d4/f12 0 2024-03-11T19:24:06.237 INFO:tasks.workunit.client.0.smithi090.stdout:8/95: symlink d0/d3/da/l1e 0 2024-03-11T19:24:06.319 INFO:tasks.workunit.client.0.smithi090.stdout:7/87: creat d3/dc/d13/f21 x:0 0 0 2024-03-11T19:24:06.448 INFO:tasks.workunit.client.0.smithi090.stdout:1/102: link d5/da/c22 d5/da/d23/c24 0 2024-03-11T19:24:06.503 INFO:tasks.workunit.client.0.smithi090.stdout:3/77: write d2/f8 [1215850,3632] 0 2024-03-11T19:24:06.678 INFO:tasks.workunit.client.0.smithi090.stdout:6/93: rmdir d4/d15 0 2024-03-11T19:24:06.684 INFO:tasks.workunit.client.0.smithi090.stdout:0/93: write d2/d11/f17 [581921,95868] 0 2024-03-11T19:24:06.789 INFO:tasks.workunit.client.0.smithi090.stdout:2/89: dwrite d0/f5 [0,4194304] 0 2024-03-11T19:24:06.918 INFO:tasks.workunit.client.0.smithi090.stdout:9/82: read d1/f3 [901976,40163] 0 2024-03-11T19:24:06.938 INFO:tasks.workunit.client.0.smithi090.stdout:4/90: unlink d4/f10 0 2024-03-11T19:24:07.011 INFO:tasks.workunit.client.0.smithi090.stdout:4/91: chown d4/f12 43 1 2024-03-11T19:24:07.032 INFO:tasks.workunit.client.0.smithi090.stdout:5/79: rmdir d1/d4 39 2024-03-11T19:24:07.275 INFO:tasks.workunit.client.0.smithi090.stdout:8/96: unlink d0/d3/l1d 0 2024-03-11T19:24:07.363 INFO:tasks.workunit.client.0.smithi090.stdout:0/94: dwrite d2/d11/f17 [0,4194304] 0 2024-03-11T19:24:07.405 INFO:tasks.workunit.client.0.smithi090.stdout:4/92: dread d4/fa [4194304,4194304] 0 2024-03-11T19:24:07.572 INFO:tasks.workunit.client.0.smithi090.stdout:7/88: fdatasync d3/dc/d13/f20 0 2024-03-11T19:24:07.846 INFO:tasks.workunit.client.0.smithi090.stdout:3/78: creat d2/f19 x:0 0 0 2024-03-11T19:24:07.955 INFO:tasks.workunit.client.0.smithi090.stdout:6/94: creat d4/d13/f1f x:0 0 0 2024-03-11T19:24:09.510 INFO:tasks.workunit.client.0.smithi090.stdout:0/95: dread d2/f5 [0,4194304] 0 2024-03-11T19:24:09.547 INFO:tasks.workunit.client.0.smithi090.stdout:4/93: mknod d4/c1b 0 2024-03-11T19:24:09.793 INFO:tasks.workunit.client.0.smithi090.stdout:7/89: fsync d3/dc/f19 0 2024-03-11T19:24:09.829 INFO:tasks.workunit.client.0.smithi090.stdout:7/90: read - d3/dc/d1e/f11 zero size 2024-03-11T19:24:09.857 INFO:tasks.workunit.client.0.smithi090.stdout:7/91: chown d3/dc/l1a 107147 1 2024-03-11T19:24:10.078 INFO:tasks.workunit.client.0.smithi090.stdout:6/95: symlink d4/d13/l20 0 2024-03-11T19:24:10.087 INFO:tasks.workunit.client.0.smithi090.stdout:3/79: dread - d2/db/d15/f18 zero size 2024-03-11T19:24:10.260 INFO:tasks.workunit.client.0.smithi090.stdout:5/80: symlink d1/l14 0 2024-03-11T19:24:10.600 INFO:tasks.workunit.client.0.smithi090.stdout:0/96: creat d2/d11/f1a x:0 0 0 2024-03-11T19:24:10.625 INFO:tasks.workunit.client.0.smithi090.stdout:4/94: mknod d4/c1c 0 2024-03-11T19:24:11.161 INFO:tasks.workunit.client.0.smithi090.stdout:6/96: creat d4/d14/f21 x:0 0 0 2024-03-11T19:24:11.248 INFO:tasks.workunit.client.0.smithi090.stdout:1/103: dwrite d5/da/dd/f1a [4194304,4194304] 0 2024-03-11T19:24:11.417 INFO:tasks.workunit.client.0.smithi090.stdout:3/80: dread d2/f9 [0,4194304] 0 2024-03-11T19:24:11.636 INFO:tasks.workunit.client.0.smithi090.stdout:1/104: chown d5/da/dd/f14 15691 1 2024-03-11T19:24:11.803 INFO:tasks.workunit.client.0.smithi090.stdout:9/83: dwrite d1/dd/f11 [0,4194304] 0 2024-03-11T19:24:12.077 INFO:tasks.workunit.client.0.smithi090.stdout:4/95: mkdir d4/d18/d1d 0 2024-03-11T19:24:12.353 INFO:tasks.workunit.client.0.smithi090.stdout:6/97: rename d4/d13/d1a to d4/d13/d22 0 2024-03-11T19:24:12.380 INFO:tasks.workunit.client.0.smithi090.stdout:4/96: sync 2024-03-11T19:24:12.495 INFO:tasks.workunit.client.0.smithi090.stdout:1/105: mknod d5/da/d1c/c25 0 2024-03-11T19:24:12.508 INFO:tasks.workunit.client.0.smithi090.stdout:2/90: truncate d0/fc 2089463 0 2024-03-11T19:24:12.872 INFO:tasks.workunit.client.0.smithi090.stdout:8/97: unlink d0/d3/da/l13 0 2024-03-11T19:24:12.916 INFO:tasks.workunit.client.0.smithi090.stdout:7/92: getdents d3 0 2024-03-11T19:24:13.499 INFO:tasks.workunit.client.0.smithi090.stdout:1/106: fsync d5/f15 0 2024-03-11T19:24:13.644 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:24:13.647 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:24:14.077 INFO:tasks.workunit.client.0.smithi090.stdout:5/81: getdents d1/d4 0 2024-03-11T19:24:15.175 INFO:tasks.workunit.client.0.smithi090.stdout:3/81: dwrite f0 [0,4194304] 0 2024-03-11T19:24:15.365 INFO:tasks.workunit.client.0.smithi090.stdout:6/98: write d4/d13/f1b [657136,5045] 0 2024-03-11T19:24:15.461 INFO:tasks.workunit.client.0.smithi090.stdout:4/97: write d4/fa [1816032,124389] 0 2024-03-11T19:24:15.686 INFO:tasks.workunit.client.0.smithi090.stdout:3/82: mkdir d2/db/d15/d1a 0 2024-03-11T19:24:15.927 INFO:tasks.workunit.client.0.smithi090.stdout:4/98: truncate d4/f8 1781757 0 2024-03-11T19:24:15.941 INFO:tasks.workunit.client.0.smithi090.stdout:5/82: symlink d1/l15 0 2024-03-11T19:24:16.127 INFO:tasks.workunit.client.0.smithi090.stdout:6/99: rename f3 to d4/d14/f23 0 2024-03-11T19:24:16.153 INFO:tasks.workunit.client.0.smithi090.stdout:0/97: creat d2/f1b x:0 0 0 2024-03-11T19:24:16.175 INFO:tasks.workunit.client.0.smithi090.stdout:7/93: truncate d3/f4 2995649 0 2024-03-11T19:24:16.275 INFO:tasks.workunit.client.0.smithi090.stdout:4/99: sync 2024-03-11T19:24:16.457 INFO:tasks.workunit.client.0.smithi090.stdout:9/84: fdatasync d1/dd/df/f10 0 2024-03-11T19:24:16.735 INFO:tasks.workunit.client.0.smithi090.stdout:3/83: creat d2/f1b x:0 0 0 2024-03-11T19:24:16.943 INFO:tasks.workunit.client.0.smithi090.stdout:5/83: fdatasync d1/d4/f11 0 2024-03-11T19:24:17.000 INFO:tasks.workunit.client.0.smithi090.stdout:0/98: creat d2/f1c x:0 0 0 2024-03-11T19:24:17.319 INFO:tasks.workunit.client.0.smithi090.stdout:8/98: dwrite d0/d3/da/dc/fd [0,4194304] 0 2024-03-11T19:24:17.345 INFO:tasks.workunit.client.0.smithi090.stdout:1/107: dwrite d5/f1b [0,4194304] 0 2024-03-11T19:24:17.397 INFO:tasks.workunit.client.0.smithi090.stdout:2/91: dwrite d0/f7 [8388608,4194304] 0 2024-03-11T19:24:17.499 INFO:tasks.workunit.client.0.smithi090.stdout:0/99: chown d2/f14 2655 1 2024-03-11T19:24:17.534 INFO:tasks.workunit.client.0.smithi090.stdout:5/84: dread d1/d4/fc [0,4194304] 0 2024-03-11T19:24:17.648 INFO:tasks.workunit.client.0.smithi090.stdout:6/100: mkdir d4/d24 0 2024-03-11T19:24:17.822 INFO:tasks.workunit.client.0.smithi090.stdout:6/101: sync 2024-03-11T19:24:18.173 INFO:tasks.workunit.client.0.smithi090.stdout:4/100: dwrite d4/f5 [0,4194304] 0 2024-03-11T19:24:18.345 INFO:tasks.workunit.client.0.smithi090.stdout:2/92: creat d0/de/f16 x:0 0 0 2024-03-11T19:24:18.492 INFO:tasks.workunit.client.0.smithi090.stdout:8/99: symlink d0/d3/da/dc/l1f 0 2024-03-11T19:24:18.614 INFO:tasks.workunit.client.0.smithi090.stdout:8/100: sync 2024-03-11T19:24:18.675 INFO:tasks.workunit.client.0.smithi090.stdout:9/85: mknod d1/dd/d19/c1a 0 2024-03-11T19:24:18.890 INFO:tasks.workunit.client.0.smithi090.stdout:6/102: mkdir d4/d14/d25 0 2024-03-11T19:24:18.897 INFO:tasks.workunit.client.0.smithi090.stdout:7/94: dwrite d3/dc/d13/f21 [0,4194304] 0 2024-03-11T19:24:18.994 INFO:tasks.workunit.client.0.smithi090.stdout:6/103: chown d4/d13/l20 413 1 2024-03-11T19:24:19.067 INFO:tasks.workunit.client.0.smithi090.stdout:7/95: sync 2024-03-11T19:24:19.138 INFO:tasks.workunit.client.0.smithi090.stdout:7/96: read d3/dc/fe [33960,34680] 0 2024-03-11T19:24:19.177 INFO:tasks.workunit.client.0.smithi090.stdout:6/104: sync 2024-03-11T19:24:19.379 INFO:tasks.workunit.client.0.smithi090.stdout:0/100: write d2/f8 [772949,89726] 0 2024-03-11T19:24:19.529 INFO:tasks.workunit.client.0.smithi090.stdout:3/84: dwrite d2/f8 [0,4194304] 0 2024-03-11T19:24:19.705 INFO:tasks.workunit.client.0.smithi090.stdout:2/93: dread d0/f5 [0,4194304] 0 2024-03-11T19:24:19.968 INFO:tasks.workunit.client.0.smithi090.stdout:4/101: write d4/f12 [158683,5361] 0 2024-03-11T19:24:20.062 INFO:tasks.workunit.client.0.smithi090.stdout:4/102: read d4/fa [1655877,50237] 0 2024-03-11T19:24:20.099 INFO:tasks.workunit.client.0.smithi090.stdout:9/86: dread d1/dd/f11 [0,4194304] 0 2024-03-11T19:24:20.216 INFO:tasks.workunit.client.0.smithi090.stdout:6/105: dread - d4/d13/f1f zero size 2024-03-11T19:24:20.368 INFO:tasks.workunit.client.0.smithi090.stdout:5/85: creat d1/f16 x:0 0 0 2024-03-11T19:24:20.472 INFO:tasks.workunit.client.0.smithi090.stdout:8/101: unlink d0/l6 0 2024-03-11T19:24:20.529 INFO:tasks.workunit.client.0.smithi090.stdout:1/108: link d5/l10 d5/da/d1c/l26 0 2024-03-11T19:24:20.556 INFO:tasks.workunit.client.0.smithi090.stdout:1/109: chown d5/da/c20 19914201 1 2024-03-11T19:24:20.739 INFO:tasks.workunit.client.0.smithi090.stdout:3/85: rmdir d2/d4 39 2024-03-11T19:24:20.845 INFO:tasks.workunit.client.0.smithi090.stdout:2/94: creat d0/de/f17 x:0 0 0 2024-03-11T19:24:21.146 INFO:tasks.workunit.client.0.smithi090.stdout:4/103: creat d4/d18/f1e x:0 0 0 2024-03-11T19:24:21.227 INFO:tasks.workunit.client.0.smithi090.stdout:9/87: mkdir d1/dd/d1b 0 2024-03-11T19:24:21.355 INFO:tasks.workunit.client.0.smithi090.stdout:6/106: fsync d4/d13/f1f 0 2024-03-11T19:24:21.486 INFO:tasks.workunit.client.0.smithi090.stdout:8/102: creat d0/f20 x:0 0 0 2024-03-11T19:24:21.547 INFO:tasks.workunit.client.0.smithi090.stdout:1/110: dread - d5/da/d1c/f21 zero size 2024-03-11T19:24:22.103 INFO:tasks.workunit.client.0.smithi090.stdout:7/97: creat d3/f22 x:0 0 0 2024-03-11T19:24:22.126 INFO:tasks.workunit.client.0.smithi090.stdout:7/98: chown d3/c1c 8833781 1 2024-03-11T19:24:22.254 INFO:tasks.workunit.client.0.smithi090.stdout:9/88: read d1/f6 [2700260,75692] 0 2024-03-11T19:24:22.607 INFO:tasks.workunit.client.0.smithi090.stdout:5/86: mkdir d1/d17 0 2024-03-11T19:24:22.667 INFO:tasks.workunit.client.0.smithi090.stdout:8/103: read - d0/d3/da/f1b zero size 2024-03-11T19:24:22.848 INFO:tasks.workunit.client.0.smithi090.stdout:3/86: write d2/db/d15/f18 [762539,30662] 0 2024-03-11T19:24:22.915 INFO:tasks.workunit.client.0.smithi090.stdout:2/95: write d0/f4 [4394885,70005] 0 2024-03-11T19:24:23.283 INFO:tasks.workunit.client.0.smithi090.stdout:0/101: getdents d2/d11 0 2024-03-11T19:24:23.294 INFO:tasks.workunit.client.0.smithi090.stdout:5/87: mknod d1/d4/c18 0 2024-03-11T19:24:23.318 INFO:tasks.workunit.client.0.smithi090.stdout:8/104: rmdir d0 39 2024-03-11T19:24:23.361 INFO:tasks.workunit.client.0.smithi090.stdout:2/96: dwrite d0/de/f16 [0,4194304] 0 2024-03-11T19:24:23.522 INFO:tasks.workunit.client.0.smithi090.stdout:9/89: dread d1/f9 [0,4194304] 0 2024-03-11T19:24:23.694 INFO:tasks.workunit.client.0.smithi090.stdout:1/111: write d5/da/d1c/f21 [657637,75271] 0 2024-03-11T19:24:23.782 INFO:tasks.workunit.client.0.smithi090.stdout:1/112: write d5/da/d1c/f21 [1259600,33631] 0 2024-03-11T19:24:24.270 INFO:tasks.workunit.client.0.smithi090.stdout:7/99: dwrite d3/dc/d17/f1b [0,4194304] 0 2024-03-11T19:24:24.400 INFO:tasks.workunit.client.0.smithi090.stdout:4/104: rmdir d4/d18/d1d 0 2024-03-11T19:24:24.822 INFO:tasks.workunit.client.0.smithi090.stdout:5/88: mkdir d1/d4/d19 0 2024-03-11T19:24:25.050 INFO:tasks.workunit.client.0.smithi090.stdout:0/102: truncate d2/d11/f18 345603 0 2024-03-11T19:24:25.073 INFO:tasks.workunit.client.0.smithi090.stdout:0/103: dread - d2/d11/f1a zero size 2024-03-11T19:24:25.345 INFO:tasks.workunit.client.0.smithi090.stdout:1/113: symlink d5/da/d1c/l27 0 2024-03-11T19:24:25.372 INFO:tasks.workunit.client.0.smithi090.stdout:1/114: chown d5/da/f19 1186 1 2024-03-11T19:24:25.828 INFO:tasks.workunit.client.0.smithi090.stdout:4/105: symlink d4/d18/l1f 0 2024-03-11T19:24:26.138 INFO:tasks.workunit.client.0.smithi090.stdout:6/107: getdents d4/d14 0 2024-03-11T19:24:26.159 INFO:tasks.workunit.client.0.smithi090.stdout:8/105: fsync d0/d3/f10 0 2024-03-11T19:24:26.176 INFO:tasks.workunit.client.0.smithi090.stdout:8/106: chown d0 171293 1 2024-03-11T19:24:26.392 INFO:tasks.workunit.client.0.smithi090.stdout:6/108: sync 2024-03-11T19:24:26.496 INFO:tasks.workunit.client.0.smithi090.stdout:5/89: rename d1/d4/c13 to d1/d4/c1a 0 2024-03-11T19:24:26.590 INFO:tasks.workunit.client.0.smithi090.stdout:9/90: mkdir d1/dd/df/d1c 0 2024-03-11T19:24:26.664 INFO:tasks.workunit.client.0.smithi090.stdout:2/97: write d0/f5 [513910,56163] 0 2024-03-11T19:24:26.781 INFO:tasks.workunit.client.0.smithi090.stdout:9/91: dread d1/f3 [0,4194304] 0 2024-03-11T19:24:26.802 INFO:tasks.workunit.client.0.smithi090.stdout:0/104: symlink d2/d11/l1d 0 2024-03-11T19:24:26.888 INFO:tasks.workunit.client.0.smithi090.stdout:1/115: creat d5/da/d1c/f28 x:0 0 0 2024-03-11T19:24:26.921 INFO:tasks.workunit.client.0.smithi090.stdout:1/116: chown d5/da/d23 434 1 2024-03-11T19:24:27.226 INFO:tasks.workunit.client.0.smithi090.stdout:4/106: symlink d4/d18/l20 0 2024-03-11T19:24:27.551 INFO:tasks.workunit.client.0.smithi090.stdout:7/100: dwrite d3/dc/d13/f20 [0,4194304] 0 2024-03-11T19:24:27.686 INFO:tasks.workunit.client.0.smithi090.stdout:5/90: creat d1/d4/f1b x:0 0 0 2024-03-11T19:24:27.966 INFO:tasks.workunit.client.0.smithi090.stdout:3/87: getdents d2/de 0 2024-03-11T19:24:28.197 INFO:tasks.workunit.client.0.smithi090.stdout:2/98: unlink d0/f5 0 2024-03-11T19:24:28.636 INFO:tasks.workunit.client.0.smithi090.stdout:8/107: rename d0/d3/l1a to d0/d3/l21 0 2024-03-11T19:24:28.809 INFO:tasks.workunit.client.0.smithi090.stdout:4/107: dwrite d4/d18/f1e [0,4194304] 0 2024-03-11T19:24:28.967 INFO:tasks.workunit.client.0.smithi090.stdout:4/108: chown d4/d18/l1f 0 1 2024-03-11T19:24:29.119 INFO:tasks.workunit.client.0.smithi090.stdout:3/88: creat d2/db/f1c x:0 0 0 2024-03-11T19:24:29.219 INFO:tasks.workunit.client.0.smithi090.stdout:6/109: write d4/d14/f23 [878604,64147] 0 2024-03-11T19:24:29.252 INFO:tasks.workunit.client.0.smithi090.stdout:6/110: chown d4 1 1 2024-03-11T19:24:29.527 INFO:tasks.workunit.client.0.smithi090.stdout:0/105: write f0 [2575673,56111] 0 2024-03-11T19:24:29.683 INFO:tasks.workunit.client.0.smithi090.stdout:0/106: sync 2024-03-11T19:24:29.687 INFO:tasks.workunit.client.0.smithi090.stdout:9/92: dwrite d1/dd/f11 [0,4194304] 0 2024-03-11T19:24:29.772 INFO:tasks.workunit.client.0.smithi090.stdout:8/108: mknod d0/d3/da/c22 0 2024-03-11T19:24:29.954 INFO:tasks.workunit.client.0.smithi090.stdout:7/101: write d3/dc/d1e/ff [1379602,686] 0 2024-03-11T19:24:30.040 INFO:tasks.workunit.client.0.smithi090.stdout:8/109: chown d0/f17 0 1 2024-03-11T19:24:30.273 INFO:tasks.workunit.client.0.smithi090.stdout:5/91: mknod d1/d17/c1c 0 2024-03-11T19:24:30.386 INFO:tasks.workunit.client.0.smithi090.stdout:0/107: dread d2/d11/f17 [0,4194304] 0 2024-03-11T19:24:30.408 INFO:tasks.workunit.client.0.smithi090.stdout:1/117: dwrite d5/f1b [4194304,4194304] 0 2024-03-11T19:24:30.641 INFO:tasks.workunit.client.0.smithi090.stdout:2/99: dwrite d0/f2 [4194304,4194304] 0 2024-03-11T19:24:30.759 INFO:tasks.workunit.client.0.smithi090.stdout:1/118: truncate d5/da/d1c/f28 543308 0 2024-03-11T19:24:30.892 INFO:tasks.workunit.client.0.smithi090.stdout:1/119: sync 2024-03-11T19:24:31.023 INFO:tasks.workunit.client.0.smithi090.stdout:5/92: readlink d1/l12 0 2024-03-11T19:24:31.036 INFO:tasks.workunit.client.0.smithi090.stdout:5/93: stat d1/ld 0 2024-03-11T19:24:31.472 INFO:tasks.workunit.client.0.smithi090.stdout:4/109: mknod d4/c21 0 2024-03-11T19:24:31.833 INFO:tasks.workunit.client.0.smithi090.stdout:6/111: dwrite d4/fb [0,4194304] 0 2024-03-11T19:24:31.931 INFO:tasks.workunit.client.0.smithi090.stdout:7/102: unlink d3/cb 0 2024-03-11T19:24:32.039 INFO:tasks.workunit.client.0.smithi090.stdout:4/110: mknod d4/c22 0 2024-03-11T19:24:32.280 INFO:tasks.workunit.client.0.smithi090.stdout:9/93: rename d1/dd/d1b to d1/d1d 0 2024-03-11T19:24:32.322 INFO:tasks.workunit.client.0.smithi090.stdout:5/94: symlink d1/d4/d19/l1d 0 2024-03-11T19:24:32.336 INFO:tasks.workunit.client.0.smithi090.stdout:3/89: rename d2/d4 to d2/d1d 0 2024-03-11T19:24:32.570 INFO:tasks.workunit.client.0.smithi090.stdout:2/100: dwrite d0/f4 [0,4194304] 0 2024-03-11T19:24:32.737 INFO:tasks.workunit.client.0.smithi090.stdout:7/103: symlink d3/l23 0 2024-03-11T19:24:32.763 INFO:tasks.workunit.client.0.smithi090.stdout:4/111: creat d4/f23 x:0 0 0 2024-03-11T19:24:32.791 INFO:tasks.workunit.client.0.smithi090.stdout:8/110: getdents d0/d3 0 2024-03-11T19:24:32.795 INFO:tasks.workunit.client.0.smithi090.stdout:1/120: creat d5/da/f29 x:0 0 0 2024-03-11T19:24:33.059 INFO:tasks.workunit.client.0.smithi090.stdout:0/108: getdents d2/d11/d15 0 2024-03-11T19:24:33.087 INFO:tasks.workunit.client.0.smithi090.stdout:6/112: rename d4/d13/f17 to d4/d24/f26 0 2024-03-11T19:24:33.113 INFO:tasks.workunit.client.0.smithi090.stdout:9/94: creat d1/dd/f1e x:0 0 0 2024-03-11T19:24:33.159 INFO:tasks.workunit.client.0.smithi090.stdout:2/101: creat d0/f18 x:0 0 0 2024-03-11T19:24:33.241 INFO:tasks.workunit.client.0.smithi090.stdout:9/95: stat d1/dd/df/d1c 0 2024-03-11T19:24:33.410 INFO:tasks.workunit.client.0.smithi090.stdout:5/95: write d1/d4/fc [193334,93456] 0 2024-03-11T19:24:33.449 INFO:tasks.workunit.client.0.smithi090.stdout:8/111: creat d0/d3/da/f23 x:0 0 0 2024-03-11T19:24:33.482 INFO:tasks.workunit.client.0.smithi090.stdout:4/112: creat d4/f24 x:0 0 0 2024-03-11T19:24:33.566 INFO:tasks.workunit.client.0.smithi090.stdout:5/96: stat d1/f16 0 2024-03-11T19:24:33.711 INFO:tasks.workunit.client.0.smithi090.stdout:3/90: dwrite d2/d1d/fc [0,4194304] 0 2024-03-11T19:24:33.716 INFO:tasks.workunit.client.0.smithi090.stdout:5/97: chown d1/d4/c5 23971 1 2024-03-11T19:24:33.807 INFO:tasks.workunit.client.0.smithi090.stdout:5/98: chown d1/ld 29126 1 2024-03-11T19:24:33.864 INFO:tasks.workunit.client.0.smithi090.stdout:0/109: symlink d2/l1e 0 2024-03-11T19:24:33.867 INFO:tasks.workunit.client.0.smithi090.stdout:6/113: rmdir d4/d13/d22 39 2024-03-11T19:24:33.884 INFO:tasks.workunit.client.0.smithi090.stdout:2/102: symlink d0/de/d12/l19 0 2024-03-11T19:24:33.902 INFO:tasks.workunit.client.0.smithi090.stdout:9/96: mknod d1/d1d/c1f 0 2024-03-11T19:24:33.971 INFO:tasks.workunit.client.0.smithi090.stdout:5/99: chown d1/d4/f6 4887600 1 2024-03-11T19:24:34.159 INFO:tasks.workunit.client.0.smithi090.stdout:5/100: chown d1/l15 29618716 1 2024-03-11T19:24:34.262 INFO:tasks.workunit.client.0.smithi090.stdout:5/101: chown d1/c10 182694013 1 2024-03-11T19:24:34.272 INFO:tasks.workunit.client.0.smithi090.stdout:7/104: dwrite d3/dc/d1e/f11 [0,4194304] 0 2024-03-11T19:24:34.448 INFO:tasks.workunit.client.0.smithi090.stdout:4/113: unlink d4/f23 0 2024-03-11T19:24:34.520 INFO:tasks.workunit.client.0.smithi090.stdout:8/112: mknod d0/d3/c24 0 2024-03-11T19:24:34.570 INFO:tasks.workunit.client.0.smithi090.stdout:6/114: fsync d4/d13/f1f 0 2024-03-11T19:24:34.571 INFO:tasks.workunit.client.0.smithi090.stdout:0/110: mkdir d2/d11/d15/d1f 0 2024-03-11T19:24:34.581 INFO:tasks.workunit.client.0.smithi090.stdout:2/103: creat d0/de/d12/f1a x:0 0 0 2024-03-11T19:24:34.769 INFO:tasks.workunit.client.0.smithi090.stdout:7/105: creat d3/dc/d13/f24 x:0 0 0 2024-03-11T19:24:34.959 INFO:tasks.workunit.client.0.smithi090.stdout:1/121: link d5/da/d1c/f21 d5/da/d23/f2a 0 2024-03-11T19:24:35.203 INFO:tasks.workunit.client.0.smithi090.stdout:4/114: write d4/d18/f1e [163864,71407] 0 2024-03-11T19:24:35.493 INFO:tasks.workunit.client.0.smithi090.stdout:9/97: symlink d1/l20 0 2024-03-11T19:24:35.807 INFO:tasks.workunit.client.0.smithi090.stdout:2/104: dread d0/f7 [8388608,4194304] 0 2024-03-11T19:24:37.055 INFO:tasks.workunit.client.0.smithi090.stdout:3/91: link d2/db/d15/f18 d2/db/d15/f1e 0 2024-03-11T19:24:37.203 INFO:tasks.workunit.client.0.smithi090.stdout:5/102: dwrite d1/d4/f1b [0,4194304] 0 2024-03-11T19:24:37.444 INFO:tasks.workunit.client.0.smithi090.stdout:0/111: creat d2/d11/d15/d1f/f20 x:0 0 0 2024-03-11T19:24:37.460 INFO:tasks.workunit.client.0.smithi090.stdout:9/98: dread d1/dd/f13 [0,4194304] 0 2024-03-11T19:24:37.564 INFO:tasks.workunit.client.0.smithi090.stdout:9/99: dread d1/f3 [4194304,4194304] 0 2024-03-11T19:24:37.766 INFO:tasks.workunit.client.0.smithi090.stdout:6/115: fdatasync d4/d24/f26 0 2024-03-11T19:24:37.802 INFO:tasks.workunit.client.0.smithi090.stdout:6/116: chown d4/fb 278 1 2024-03-11T19:24:37.825 INFO:tasks.workunit.client.0.smithi090.stdout:6/117: dread - d4/d14/f21 zero size 2024-03-11T19:24:37.920 INFO:tasks.workunit.client.0.smithi090.stdout:2/105: creat d0/f1b x:0 0 0 2024-03-11T19:24:38.232 INFO:tasks.workunit.client.0.smithi090.stdout:1/122: creat d5/da/d23/f2b x:0 0 0 2024-03-11T19:24:38.290 INFO:tasks.workunit.client.0.smithi090.stdout:2/106: dwrite d0/f18 [0,4194304] 0 2024-03-11T19:24:38.714 INFO:tasks.workunit.client.0.smithi090.stdout:8/113: link d0/d3/da/l1e d0/d3/l25 0 2024-03-11T19:24:38.774 INFO:tasks.workunit.client.0.smithi090.stdout:2/107: dwrite d0/f18 [0,4194304] 0 2024-03-11T19:24:38.935 INFO:tasks.workunit.client.0.smithi090.stdout:0/112: dread f0 [0,4194304] 0 2024-03-11T19:24:38.977 INFO:tasks.workunit.client.0.smithi090.stdout:8/114: truncate d0/f17 107984 0 2024-03-11T19:24:38.997 INFO:tasks.workunit.client.0.smithi090.stdout:0/113: chown d2/f8 127894565 1 2024-03-11T19:24:39.108 INFO:tasks.workunit.client.0.smithi090.stdout:7/106: write d3/dc/f19 [150895,61991] 0 2024-03-11T19:24:39.793 INFO:tasks.workunit.client.0.smithi090.stdout:1/123: creat d5/da/dd/f2c x:0 0 0 2024-03-11T19:24:39.815 INFO:tasks.workunit.client.0.smithi090.stdout:1/124: chown d5/f15 4566 1 2024-03-11T19:24:39.862 INFO:tasks.workunit.client.0.smithi090.stdout:1/125: write d5/da/dd/f2c [715876,66045] 0 2024-03-11T19:24:39.978 INFO:tasks.workunit.client.0.smithi090.stdout:5/103: rmdir d1/d4 39 2024-03-11T19:24:40.111 INFO:tasks.workunit.client.0.smithi090.stdout:9/100: mkdir d1/dd/df/d1c/d21 0 2024-03-11T19:24:40.376 INFO:tasks.workunit.client.0.smithi090.stdout:8/115: readlink d0/d3/da/l1e 0 2024-03-11T19:24:40.524 INFO:tasks.workunit.client.0.smithi090.stdout:2/108: dwrite d0/f4 [0,4194304] 0 2024-03-11T19:24:40.530 INFO:tasks.workunit.client.0.smithi090.stdout:8/116: truncate d0/f17 411443 0 2024-03-11T19:24:40.664 INFO:tasks.workunit.client.0.smithi090.stdout:0/114: mknod d2/d11/d15/c21 0 2024-03-11T19:24:40.705 INFO:tasks.workunit.client.0.smithi090.stdout:7/107: creat d3/dc/d13/f25 x:0 0 0 2024-03-11T19:24:41.068 INFO:tasks.workunit.client.0.smithi090.stdout:4/115: dwrite d4/ff [0,4194304] 0 2024-03-11T19:24:41.400 INFO:tasks.workunit.client.0.smithi090.stdout:5/104: dread - d1/d4/f11 zero size 2024-03-11T19:24:41.634 INFO:tasks.workunit.client.0.smithi090.stdout:6/118: dwrite d4/d14/f21 [0,4194304] 0 2024-03-11T19:24:41.802 INFO:tasks.workunit.client.0.smithi090.stdout:9/101: dread d1/dd/f11 [0,4194304] 0 2024-03-11T19:24:41.941 INFO:tasks.workunit.client.0.smithi090.stdout:2/109: fdatasync d0/f3 0 2024-03-11T19:24:42.200 INFO:tasks.workunit.client.0.smithi090.stdout:9/102: sync 2024-03-11T19:24:42.256 INFO:tasks.workunit.client.0.smithi090.stdout:8/117: rename d0/d3/da to d0/d3/d26 0 2024-03-11T19:24:42.505 INFO:tasks.workunit.client.0.smithi090.stdout:4/116: creat d4/d18/f25 x:0 0 0 2024-03-11T19:24:42.576 INFO:tasks.workunit.client.0.smithi090.stdout:3/92: write d2/db/d15/f1e [1251100,10278] 0 2024-03-11T19:24:42.666 INFO:tasks.workunit.client.0.smithi090.stdout:5/105: write d1/d4/fc [1706177,72302] 0 2024-03-11T19:24:42.875 INFO:tasks.workunit.client.0.smithi090.stdout:2/110: creat d0/d10/f1c x:0 0 0 2024-03-11T19:24:43.158 INFO:tasks.workunit.client.0.smithi090.stdout:0/115: truncate d2/d11/f18 1144051 0 2024-03-11T19:24:43.172 INFO:tasks.workunit.client.0.smithi090.stdout:9/103: symlink d1/d1d/l22 0 2024-03-11T19:24:43.782 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:24:43.785 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:24:43.896 INFO:tasks.workunit.client.0.smithi090.stdout:2/111: creat d0/de/d12/f1d x:0 0 0 2024-03-11T19:24:44.142 INFO:tasks.workunit.client.0.smithi090.stdout:1/126: dwrite d5/da/dd/f1f [0,4194304] 0 2024-03-11T19:24:44.401 INFO:tasks.workunit.client.0.smithi090.stdout:3/93: unlink d2/fa 0 2024-03-11T19:24:44.416 INFO:tasks.workunit.client.0.smithi090.stdout:0/116: symlink d2/d11/l22 0 2024-03-11T19:24:44.439 INFO:tasks.workunit.client.0.smithi090.stdout:0/117: readlink d2/l1e 0 2024-03-11T19:24:44.876 INFO:tasks.workunit.client.0.smithi090.stdout:4/117: truncate d4/d18/f1e 2329575 0 2024-03-11T19:24:44.909 INFO:tasks.workunit.client.0.smithi090.stdout:9/104: unlink d1/fe 0 2024-03-11T19:24:45.042 INFO:tasks.workunit.client.0.smithi090.stdout:7/108: getdents d3/dc/d17 0 2024-03-11T19:24:45.747 INFO:tasks.workunit.client.0.smithi090.stdout:9/105: dread - d1/f17 zero size 2024-03-11T19:24:45.760 INFO:tasks.workunit.client.0.smithi090.stdout:6/119: write d4/f11 [2764639,2619] 0 2024-03-11T19:24:45.828 INFO:tasks.workunit.client.0.smithi090.stdout:4/118: symlink d4/d18/l26 0 2024-03-11T19:24:45.969 INFO:tasks.workunit.client.0.smithi090.stdout:0/118: dwrite d2/f10 [4194304,4194304] 0 2024-03-11T19:24:46.171 INFO:tasks.workunit.client.0.smithi090.stdout:3/94: mkdir d2/d1f 0 2024-03-11T19:24:46.651 INFO:tasks.workunit.client.0.smithi090.stdout:8/118: dwrite d0/f19 [0,4194304] 0 2024-03-11T19:24:46.829 INFO:tasks.workunit.client.0.smithi090.stdout:4/119: dread d4/fa [4194304,4194304] 0 2024-03-11T19:24:46.935 INFO:tasks.workunit.client.0.smithi090.stdout:5/106: dwrite d1/d4/f6 [0,4194304] 0 2024-03-11T19:24:47.088 INFO:tasks.workunit.client.0.smithi090.stdout:8/119: chown d0/d3/d26/f23 51202039 1 2024-03-11T19:24:47.088 INFO:tasks.workunit.client.0.smithi090.stdout:3/95: dread - d2/db/f1c zero size 2024-03-11T19:24:47.122 INFO:tasks.workunit.client.0.smithi090.stdout:0/119: dread d2/fe [0,4194304] 0 2024-03-11T19:24:47.460 INFO:tasks.workunit.client.0.smithi090.stdout:5/107: dwrite d1/d4/fc [0,4194304] 0 2024-03-11T19:24:47.566 INFO:tasks.workunit.client.0.smithi090.stdout:6/120: creat d4/d13/d22/f27 x:0 0 0 2024-03-11T19:24:48.041 INFO:tasks.workunit.client.0.smithi090.stdout:0/120: creat d2/d11/d15/f23 x:0 0 0 2024-03-11T19:24:48.225 INFO:tasks.workunit.client.0.smithi090.stdout:5/108: rmdir d1/d17 39 2024-03-11T19:24:48.764 INFO:tasks.workunit.client.0.smithi090.stdout:6/121: symlink d4/d13/l28 0 2024-03-11T19:24:48.829 INFO:tasks.workunit.client.0.smithi090.stdout:7/109: write d3/f7 [796084,105799] 0 2024-03-11T19:24:48.899 INFO:tasks.workunit.client.0.smithi090.stdout:2/112: dwrite d0/fb [0,4194304] 0 2024-03-11T19:24:49.011 INFO:tasks.workunit.client.0.smithi090.stdout:1/127: write d5/da/d1c/f21 [998059,55128] 0 2024-03-11T19:24:49.134 INFO:tasks.workunit.client.0.smithi090.stdout:6/122: sync 2024-03-11T19:24:49.148 INFO:tasks.workunit.client.0.smithi090.stdout:6/123: stat d4/d13 0 2024-03-11T19:24:49.362 INFO:tasks.workunit.client.0.smithi090.stdout:9/106: write d1/f9 [1240671,69864] 0 2024-03-11T19:24:49.649 INFO:tasks.workunit.client.0.smithi090.stdout:7/110: creat d3/dc/d1e/f26 x:0 0 0 2024-03-11T19:24:49.732 INFO:tasks.workunit.client.0.smithi090.stdout:7/111: dread d3/f7 [0,4194304] 0 2024-03-11T19:24:49.752 INFO:tasks.workunit.client.0.smithi090.stdout:2/113: fdatasync d0/f4 0 2024-03-11T19:24:49.873 INFO:tasks.workunit.client.0.smithi090.stdout:1/128: creat d5/da/d23/f2d x:0 0 0 2024-03-11T19:24:49.970 INFO:tasks.workunit.client.0.smithi090.stdout:2/114: dread d0/fb [0,4194304] 0 2024-03-11T19:24:50.005 INFO:tasks.workunit.client.0.smithi090.stdout:6/124: mknod d4/d13/d22/c29 0 2024-03-11T19:24:50.301 INFO:tasks.workunit.client.0.smithi090.stdout:2/115: dread d0/f7 [4194304,4194304] 0 2024-03-11T19:24:50.479 INFO:tasks.workunit.client.0.smithi090.stdout:7/112: creat d3/f27 x:0 0 0 2024-03-11T19:24:50.642 INFO:tasks.workunit.client.0.smithi090.stdout:1/129: dread - d5/da/f19 zero size 2024-03-11T19:24:50.934 INFO:tasks.workunit.client.0.smithi090.stdout:4/120: mkdir d4/d27 0 2024-03-11T19:24:51.097 INFO:tasks.workunit.client.0.smithi090.stdout:9/107: symlink d1/dd/d18/l23 0 2024-03-11T19:24:51.172 INFO:tasks.workunit.client.0.smithi090.stdout:8/120: mknod d0/c27 0 2024-03-11T19:24:51.666 INFO:tasks.workunit.client.0.smithi090.stdout:3/96: dwrite d2/f3 [0,4194304] 0 2024-03-11T19:24:51.841 INFO:tasks.workunit.client.0.smithi090.stdout:4/121: unlink d4/c1b 0 2024-03-11T19:24:51.849 INFO:tasks.workunit.client.0.smithi090.stdout:7/113: read d3/dc/d13/f20 [3344620,130936] 0 2024-03-11T19:24:51.877 INFO:tasks.workunit.client.0.smithi090.stdout:5/109: dwrite d1/d4/f1b [0,4194304] 0 2024-03-11T19:24:51.899 INFO:tasks.workunit.client.0.smithi090.stdout:0/121: dwrite d2/f3 [0,4194304] 0 2024-03-11T19:24:52.060 INFO:tasks.workunit.client.0.smithi090.stdout:1/130: rmdir d5/da/d23 39 2024-03-11T19:24:52.263 INFO:tasks.workunit.client.0.smithi090.stdout:0/122: dread - d2/d11/d15/f23 zero size 2024-03-11T19:24:52.423 INFO:tasks.workunit.client.0.smithi090.stdout:8/121: dread d0/d3/f8 [0,4194304] 0 2024-03-11T19:24:52.446 INFO:tasks.workunit.client.0.smithi090.stdout:9/108: truncate d1/f6 5395396 0 2024-03-11T19:24:52.485 INFO:tasks.workunit.client.0.smithi090.stdout:6/125: write d4/ff [3504438,45965] 0 2024-03-11T19:24:53.153 INFO:tasks.workunit.client.0.smithi090.stdout:5/110: unlink d1/d4/f1b 0 2024-03-11T19:24:53.205 INFO:tasks.workunit.client.0.smithi090.stdout:5/111: chown d1/d4/fc 3655 1 2024-03-11T19:24:53.456 INFO:tasks.workunit.client.0.smithi090.stdout:6/126: creat d4/d14/f2a x:0 0 0 2024-03-11T19:24:53.662 INFO:tasks.workunit.client.0.smithi090.stdout:7/114: write d3/f15 [458458,77352] 0 2024-03-11T19:24:53.672 INFO:tasks.workunit.client.0.smithi090.stdout:3/97: write d2/db/d15/f17 [61824,130693] 0 2024-03-11T19:24:53.720 INFO:tasks.workunit.client.0.smithi090.stdout:2/116: getdents d0/d10 0 2024-03-11T19:24:53.725 INFO:tasks.workunit.client.0.smithi090.stdout:1/131: symlink d5/da/l2e 0 2024-03-11T19:24:53.755 INFO:tasks.workunit.client.0.smithi090.stdout:1/132: readlink d5/l9 0 2024-03-11T19:24:53.770 INFO:tasks.workunit.client.0.smithi090.stdout:4/122: mkdir d4/d27/d28 0 2024-03-11T19:24:53.978 INFO:tasks.workunit.client.0.smithi090.stdout:0/123: write f0 [1038544,46168] 0 2024-03-11T19:24:54.122 INFO:tasks.workunit.client.0.smithi090.stdout:8/122: write d0/d3/f8 [2525963,109195] 0 2024-03-11T19:24:54.189 INFO:tasks.workunit.client.0.smithi090.stdout:9/109: write d1/dd/f13 [1387643,55549] 0 2024-03-11T19:24:54.194 INFO:tasks.workunit.client.0.smithi090.stdout:2/117: dread d0/f3 [0,4194304] 0 2024-03-11T19:24:54.284 INFO:tasks.workunit.client.0.smithi090.stdout:2/118: readlink d0/de/d12/l19 0 2024-03-11T19:24:54.302 INFO:tasks.workunit.client.0.smithi090.stdout:2/119: chown d0/d10/l14 4044495 1 2024-03-11T19:24:54.328 INFO:tasks.workunit.client.0.smithi090.stdout:3/98: mknod d2/de/c20 0 2024-03-11T19:24:54.638 INFO:tasks.workunit.client.0.smithi090.stdout:4/123: mknod d4/d18/c29 0 2024-03-11T19:24:54.730 INFO:tasks.workunit.client.0.smithi090.stdout:1/133: dread d5/da/d23/f2a [0,4194304] 0 2024-03-11T19:24:55.082 INFO:tasks.workunit.client.0.smithi090.stdout:0/124: truncate d2/fe 705750 0 2024-03-11T19:24:55.230 INFO:tasks.workunit.client.0.smithi090.stdout:9/110: chown d1/l8 5497 1 2024-03-11T19:24:55.276 INFO:tasks.workunit.client.0.smithi090.stdout:2/120: write d0/fb [351285,61818] 0 2024-03-11T19:24:56.031 INFO:tasks.workunit.client.0.smithi090.stdout:7/115: dwrite d3/dc/d1e/f12 [0,4194304] 0 2024-03-11T19:24:56.039 INFO:tasks.workunit.client.0.smithi090.stdout:4/124: creat d4/d18/f2a x:0 0 0 2024-03-11T19:24:56.185 INFO:tasks.workunit.client.0.smithi090.stdout:5/112: creat d1/f1e x:0 0 0 2024-03-11T19:24:56.373 INFO:tasks.workunit.client.0.smithi090.stdout:0/125: creat d2/d11/f24 x:0 0 0 2024-03-11T19:24:56.584 INFO:tasks.workunit.client.0.smithi090.stdout:5/113: dwrite d1/f1e [0,4194304] 0 2024-03-11T19:24:57.361 INFO:tasks.workunit.client.0.smithi090.stdout:3/99: dwrite d2/f1b [0,4194304] 0 2024-03-11T19:24:57.470 INFO:tasks.workunit.client.0.smithi090.stdout:7/116: mknod d3/dc/d17/c28 0 2024-03-11T19:24:57.516 INFO:tasks.workunit.client.0.smithi090.stdout:7/117: readlink d3/dc/l1a 0 2024-03-11T19:24:57.556 INFO:tasks.workunit.client.0.smithi090.stdout:6/127: rename d4/c1e to d4/d13/c2b 0 2024-03-11T19:24:57.685 INFO:tasks.workunit.client.0.smithi090.stdout:1/134: write d5/fc [8115353,72976] 0 2024-03-11T19:24:57.846 INFO:tasks.workunit.client.0.smithi090.stdout:5/114: truncate d1/d4/fc 3895703 0 2024-03-11T19:24:58.011 INFO:tasks.workunit.client.0.smithi090.stdout:3/100: symlink d2/de/l21 0 2024-03-11T19:24:58.020 INFO:tasks.workunit.client.0.smithi090.stdout:9/111: write f0 [5156248,44538] 0 2024-03-11T19:24:58.128 INFO:tasks.workunit.client.0.smithi090.stdout:8/123: rename d0/d3/f10 to d0/d3/d26/dc/f28 0 2024-03-11T19:24:58.251 INFO:tasks.workunit.client.0.smithi090.stdout:2/121: write d0/d10/f1c [278951,120500] 0 2024-03-11T19:24:58.256 INFO:tasks.workunit.client.0.smithi090.stdout:7/118: truncate d3/dc/d17/f1b 1298921 0 2024-03-11T19:24:58.315 INFO:tasks.workunit.client.0.smithi090.stdout:2/122: chown d0/fc 1038645636 1 2024-03-11T19:24:58.326 INFO:tasks.workunit.client.0.smithi090.stdout:2/123: readlink d0/l8 0 2024-03-11T19:24:58.423 INFO:tasks.workunit.client.0.smithi090.stdout:4/125: rmdir d4 39 2024-03-11T19:24:58.960 INFO:tasks.workunit.client.0.smithi090.stdout:1/135: fsync d5/da/dd/f1a 0 2024-03-11T19:24:59.038 INFO:tasks.workunit.client.0.smithi090.stdout:7/119: mkdir d3/dc/d29 0 2024-03-11T19:24:59.062 INFO:tasks.workunit.client.0.smithi090.stdout:8/124: dread - d0/d3/d26/f23 zero size 2024-03-11T19:24:59.335 INFO:tasks.workunit.client.0.smithi090.stdout:3/101: getdents d2/d1f 0 2024-03-11T19:24:59.484 INFO:tasks.workunit.client.0.smithi090.stdout:8/125: dwrite d0/d3/f8 [4194304,4194304] 0 2024-03-11T19:24:59.667 INFO:tasks.workunit.client.0.smithi090.stdout:5/115: rmdir d1 39 2024-03-11T19:24:59.694 INFO:tasks.workunit.client.0.smithi090.stdout:3/102: truncate d2/db/d15/f17 1055383 0 2024-03-11T19:24:59.817 INFO:tasks.workunit.client.0.smithi090.stdout:0/126: dwrite d2/d11/f17 [0,4194304] 0 2024-03-11T19:24:59.906 INFO:tasks.workunit.client.0.smithi090.stdout:0/127: chown d2/d11/d15/c21 2 1 2024-03-11T19:24:59.964 INFO:tasks.workunit.client.0.smithi090.stdout:4/126: fsync d4/f8 0 2024-03-11T19:25:00.083 INFO:tasks.workunit.client.0.smithi090.stdout:7/120: mkdir d3/dc/d1e/d2a 0 2024-03-11T19:25:00.221 INFO:tasks.workunit.client.0.smithi090.stdout:6/128: rename d4/d13/d22/c1c to d4/c2c 0 2024-03-11T19:25:00.234 INFO:tasks.workunit.client.0.smithi090.stdout:6/129: readlink d4/d14/l19 0 2024-03-11T19:25:00.629 INFO:tasks.workunit.client.0.smithi090.stdout:1/136: dwrite d5/da/dd/f1a [8388608,4194304] 0 2024-03-11T19:25:00.802 INFO:tasks.workunit.client.0.smithi090.stdout:3/103: symlink d2/d1d/l22 0 2024-03-11T19:25:01.164 INFO:tasks.workunit.client.0.smithi090.stdout:4/127: read d4/d18/f1e [1677646,90049] 0 2024-03-11T19:25:01.284 INFO:tasks.workunit.client.0.smithi090.stdout:9/112: dwrite d1/dd/f1e [0,4194304] 0 2024-03-11T19:25:01.320 INFO:tasks.workunit.client.0.smithi090.stdout:6/130: mknod d4/d13/d22/c2d 0 2024-03-11T19:25:01.576 INFO:tasks.workunit.client.0.smithi090.stdout:2/124: dwrite d0/de/f17 [0,4194304] 0 2024-03-11T19:25:01.743 INFO:tasks.workunit.client.0.smithi090.stdout:4/128: fsync d4/f8 0 2024-03-11T19:25:01.969 INFO:tasks.workunit.client.0.smithi090.stdout:5/116: mkdir d1/d17/d1f 0 2024-03-11T19:25:02.206 INFO:tasks.workunit.client.0.smithi090.stdout:9/113: symlink d1/dd/d18/l24 0 2024-03-11T19:25:02.369 INFO:tasks.workunit.client.0.smithi090.stdout:6/131: symlink d4/d24/l2e 0 2024-03-11T19:25:02.388 INFO:tasks.workunit.client.0.smithi090.stdout:6/132: readlink d4/d14/l19 0 2024-03-11T19:25:02.701 INFO:tasks.workunit.client.0.smithi090.stdout:8/126: write d0/f19 [2030662,84982] 0 2024-03-11T19:25:02.922 INFO:tasks.workunit.client.0.smithi090.stdout:2/125: dread d0/f2 [0,4194304] 0 2024-03-11T19:25:02.962 INFO:tasks.workunit.client.0.smithi090.stdout:2/126: chown d0/de/cf 12 1 2024-03-11T19:25:03.006 INFO:tasks.workunit.client.0.smithi090.stdout:2/127: write d0/d10/f1c [775427,52638] 0 2024-03-11T19:25:03.305 INFO:tasks.workunit.client.0.smithi090.stdout:0/128: getdents d2/d11 0 2024-03-11T19:25:03.316 INFO:tasks.workunit.client.0.smithi090.stdout:5/117: dread - d1/f16 zero size 2024-03-11T19:25:03.427 INFO:tasks.workunit.client.0.smithi090.stdout:9/114: mkdir d1/dd/d18/d25 0 2024-03-11T19:25:03.601 INFO:tasks.workunit.client.0.smithi090.stdout:0/129: dwrite d2/d11/f17 [0,4194304] 0 2024-03-11T19:25:03.768 INFO:tasks.workunit.client.0.smithi090.stdout:7/121: getdents d3/dc/d17 0 2024-03-11T19:25:03.931 INFO:tasks.workunit.client.0.smithi090.stdout:8/127: mknod d0/c29 0 2024-03-11T19:25:04.089 INFO:tasks.workunit.client.0.smithi090.stdout:3/104: write d2/db/d15/f18 [788580,126976] 0 2024-03-11T19:25:04.203 INFO:tasks.workunit.client.0.smithi090.stdout:3/105: stat d2/f9 0 2024-03-11T19:25:04.256 INFO:tasks.workunit.client.0.smithi090.stdout:4/129: getdents d4/d27/d28 0 2024-03-11T19:25:04.343 INFO:tasks.workunit.client.0.smithi090.stdout:1/137: dwrite d5/da/dd/f2c [0,4194304] 0 2024-03-11T19:25:04.488 INFO:tasks.workunit.client.0.smithi090.stdout:5/118: creat d1/d4/f20 x:0 0 0 2024-03-11T19:25:04.513 INFO:tasks.workunit.client.0.smithi090.stdout:5/119: chown d1/d4/c5 257340187 1 2024-03-11T19:25:04.632 INFO:tasks.workunit.client.0.smithi090.stdout:0/130: truncate d2/d11/f18 1559001 0 2024-03-11T19:25:04.743 INFO:tasks.workunit.client.0.smithi090.stdout:7/122: chown d3/f4 154133052 1 2024-03-11T19:25:05.177 INFO:tasks.workunit.client.0.smithi090.stdout:1/138: symlink d5/da/d1c/l2f 0 2024-03-11T19:25:05.198 INFO:tasks.workunit.client.0.smithi090.stdout:6/133: link d4/cc d4/d24/c2f 0 2024-03-11T19:25:05.223 INFO:tasks.workunit.client.0.smithi090.stdout:9/115: mknod d1/dd/df/c26 0 2024-03-11T19:25:05.275 INFO:tasks.workunit.client.0.smithi090.stdout:0/131: read d2/f3 [1086530,98025] 0 2024-03-11T19:25:05.406 INFO:tasks.workunit.client.0.smithi090.stdout:5/120: dread d1/d4/f6 [0,4194304] 0 2024-03-11T19:25:05.661 INFO:tasks.workunit.client.0.smithi090.stdout:2/128: write d0/de/d12/f1a [74236,86665] 0 2024-03-11T19:25:06.216 INFO:tasks.workunit.client.0.smithi090.stdout:1/139: creat d5/da/d1c/f30 x:0 0 0 2024-03-11T19:25:06.378 INFO:tasks.workunit.client.0.smithi090.stdout:5/121: creat d1/d4/d19/f21 x:0 0 0 2024-03-11T19:25:06.396 INFO:tasks.workunit.client.0.smithi090.stdout:4/130: write d4/d18/f1e [2162344,58419] 0 2024-03-11T19:25:06.502 INFO:tasks.workunit.client.0.smithi090.stdout:7/123: write d3/f4 [2624928,710] 0 2024-03-11T19:25:06.592 INFO:tasks.workunit.client.0.smithi090.stdout:8/128: getdents d0/d3 0 2024-03-11T19:25:06.598 INFO:tasks.workunit.client.0.smithi090.stdout:7/124: stat d3/dc/d13/f21 0 2024-03-11T19:25:07.286 INFO:tasks.workunit.client.0.smithi090.stdout:3/106: getdents d2/db 0 2024-03-11T19:25:07.488 INFO:tasks.workunit.client.0.smithi090.stdout:4/131: truncate d4/d18/f2a 280598 0 2024-03-11T19:25:08.012 INFO:tasks.workunit.client.0.smithi090.stdout:6/134: write d4/f11 [758330,30536] 0 2024-03-11T19:25:08.348 INFO:tasks.workunit.client.0.smithi090.stdout:4/132: creat d4/d27/f2b x:0 0 0 2024-03-11T19:25:08.487 INFO:tasks.workunit.client.0.smithi090.stdout:2/129: dwrite d0/f2 [0,4194304] 0 2024-03-11T19:25:08.651 INFO:tasks.workunit.client.0.smithi090.stdout:0/132: truncate d2/d11/f17 440100 0 2024-03-11T19:25:08.759 INFO:tasks.workunit.client.0.smithi090.stdout:9/116: write d1/dd/df/f10 [958006,65873] 0 2024-03-11T19:25:08.875 INFO:tasks.workunit.client.0.smithi090.stdout:5/122: creat d1/f22 x:0 0 0 2024-03-11T19:25:08.980 INFO:tasks.workunit.client.0.smithi090.stdout:8/129: link d0/d3/d26/f23 d0/f2a 0 2024-03-11T19:25:09.109 INFO:tasks.workunit.client.0.smithi090.stdout:5/123: write d1/d4/d19/f21 [595827,63] 0 2024-03-11T19:25:09.190 INFO:tasks.workunit.client.0.smithi090.stdout:9/117: dwrite d1/dd/df/f10 [0,4194304] 0 2024-03-11T19:25:09.299 INFO:tasks.workunit.client.0.smithi090.stdout:4/133: rename d4/d18/l20 to d4/d27/d28/l2c 0 2024-03-11T19:25:09.310 INFO:tasks.workunit.client.0.smithi090.stdout:1/140: mkdir d5/d31 0 2024-03-11T19:25:09.375 INFO:tasks.workunit.client.0.smithi090.stdout:6/135: symlink d4/d13/l30 0 2024-03-11T19:25:09.414 INFO:tasks.workunit.client.0.smithi090.stdout:0/133: chown d2/cc 1999 1 2024-03-11T19:25:09.708 INFO:tasks.workunit.client.0.smithi090.stdout:7/125: dwrite d3/f15 [0,4194304] 0 2024-03-11T19:25:09.862 INFO:tasks.workunit.client.0.smithi090.stdout:8/130: rename d0/d3/d26/f1b to d0/f2b 0 2024-03-11T19:25:09.906 INFO:tasks.workunit.client.0.smithi090.stdout:1/141: dread - d5/da/d23/f2b zero size 2024-03-11T19:25:10.297 INFO:tasks.workunit.client.0.smithi090.stdout:3/107: dwrite d2/db/d15/f17 [0,4194304] 0 2024-03-11T19:25:10.689 INFO:tasks.workunit.client.0.smithi090.stdout:9/118: mknod d1/dd/df/c27 0 2024-03-11T19:25:10.754 INFO:tasks.workunit.client.0.smithi090.stdout:7/126: dread d3/dc/d1e/ff [0,4194304] 0 2024-03-11T19:25:10.758 INFO:tasks.workunit.client.0.smithi090.stdout:2/130: write d0/f7 [8517668,61818] 0 2024-03-11T19:25:11.074 INFO:tasks.workunit.client.0.smithi090.stdout:1/142: mknod d5/da/d23/c32 0 2024-03-11T19:25:11.226 INFO:tasks.workunit.client.0.smithi090.stdout:6/136: creat d4/d14/d25/f31 x:0 0 0 2024-03-11T19:25:11.242 INFO:tasks.workunit.client.0.smithi090.stdout:2/131: dread d0/f7 [0,4194304] 0 2024-03-11T19:25:11.284 INFO:tasks.workunit.client.0.smithi090.stdout:2/132: stat d0/c15 0 2024-03-11T19:25:11.374 INFO:tasks.workunit.client.0.smithi090.stdout:5/124: write d1/f1e [2945906,18964] 0 2024-03-11T19:25:11.434 INFO:tasks.workunit.client.0.smithi090.stdout:4/134: write d4/f5 [874011,77990] 0 2024-03-11T19:25:11.694 INFO:tasks.workunit.client.0.smithi090.stdout:5/125: dwrite d1/d4/d19/f21 [0,4194304] 0 2024-03-11T19:25:12.290 INFO:tasks.workunit.client.0.smithi090.stdout:0/134: dwrite d2/f3 [0,4194304] 0 2024-03-11T19:25:12.518 INFO:tasks.workunit.client.0.smithi090.stdout:6/137: mknod d4/d24/c32 0 2024-03-11T19:25:12.587 INFO:tasks.workunit.client.0.smithi090.stdout:4/135: mknod d4/c2d 0 2024-03-11T19:25:12.665 INFO:tasks.workunit.client.0.smithi090.stdout:5/126: symlink d1/d4/d19/l23 0 2024-03-11T19:25:12.730 INFO:tasks.workunit.client.0.smithi090.stdout:1/143: dread d5/da/dd/f2c [0,4194304] 0 2024-03-11T19:25:12.866 INFO:tasks.workunit.client.0.smithi090.stdout:1/144: dread - d5/da/d23/f2b zero size 2024-03-11T19:25:12.891 INFO:tasks.workunit.client.0.smithi090.stdout:4/136: dread d4/f8 [0,4194304] 0 2024-03-11T19:25:13.005 INFO:tasks.workunit.client.0.smithi090.stdout:6/138: sync 2024-03-11T19:25:13.029 INFO:tasks.workunit.client.0.smithi090.stdout:6/139: chown d4/d14/f21 16483 1 2024-03-11T19:25:13.148 INFO:tasks.workunit.client.0.smithi090.stdout:8/131: rename d0/f2b to d0/d3/f2c 0 2024-03-11T19:25:13.541 INFO:tasks.workunit.client.0.smithi090.stdout:5/127: symlink d1/d17/l24 0 2024-03-11T19:25:13.567 INFO:tasks.workunit.client.0.smithi090.stdout:5/128: chown d1/d4/d19/f21 0 1 2024-03-11T19:25:13.635 INFO:tasks.workunit.client.0.smithi090.stdout:1/145: rmdir d5/da/d1c 39 2024-03-11T19:25:13.784 INFO:tasks.workunit.client.0.smithi090.stdout:6/140: truncate d4/d14/f2a 847889 0 2024-03-11T19:25:13.813 INFO:tasks.workunit.client.0.smithi090.stdout:6/141: stat d4/d13/d22/f27 0 2024-03-11T19:25:13.908 INFO:tasks.workunit.client.0.smithi090.stdout:4/137: rename d4/d18/f1e to d4/f2e 0 2024-03-11T19:25:14.010 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:25:14.013 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:25:14.078 INFO:tasks.workunit.client.0.smithi090.stdout:8/132: creat d0/f2d x:0 0 0 2024-03-11T19:25:14.154 INFO:tasks.workunit.client.0.smithi090.stdout:9/119: write d1/dd/f1e [4349094,109016] 0 2024-03-11T19:25:14.309 INFO:tasks.workunit.client.0.smithi090.stdout:8/133: dread - d0/f2d zero size 2024-03-11T19:25:14.511 INFO:tasks.workunit.client.0.smithi090.stdout:4/138: truncate d4/f24 1011520 0 2024-03-11T19:25:14.739 INFO:tasks.workunit.client.0.smithi090.stdout:3/108: dwrite d2/f8 [4194304,4194304] 0 2024-03-11T19:25:15.061 INFO:tasks.workunit.client.0.smithi090.stdout:7/127: link d3/dc/d17/f1b d3/dc/d1e/d2a/f2b 0 2024-03-11T19:25:15.071 INFO:tasks.workunit.client.0.smithi090.stdout:4/139: write d4/f5 [3380964,43978] 0 2024-03-11T19:25:15.294 INFO:tasks.workunit.client.0.smithi090.stdout:9/120: symlink d1/dd/d18/l28 0 2024-03-11T19:25:15.527 INFO:tasks.workunit.client.0.smithi090.stdout:2/133: dwrite d0/d10/f1c [0,4194304] 0 2024-03-11T19:25:15.770 INFO:tasks.workunit.client.0.smithi090.stdout:1/146: dread d5/fc [0,4194304] 0 2024-03-11T19:25:15.947 INFO:tasks.workunit.client.0.smithi090.stdout:4/140: fsync d4/f12 0 2024-03-11T19:25:16.019 INFO:tasks.workunit.client.0.smithi090.stdout:6/142: dread d4/ff [0,4194304] 0 2024-03-11T19:25:16.058 INFO:tasks.workunit.client.0.smithi090.stdout:9/121: chown d1/dd/df/f10 254155 1 2024-03-11T19:25:16.366 INFO:tasks.workunit.client.0.smithi090.stdout:3/109: rename d2/de/l21 to d2/d1d/l23 0 2024-03-11T19:25:16.474 INFO:tasks.workunit.client.0.smithi090.stdout:7/128: dread d3/dc/d1e/ff [0,4194304] 0 2024-03-11T19:25:16.499 INFO:tasks.workunit.client.0.smithi090.stdout:0/135: dwrite d2/f5 [0,4194304] 0 2024-03-11T19:25:16.542 INFO:tasks.workunit.client.0.smithi090.stdout:8/134: truncate d0/f2a 22492 0 2024-03-11T19:25:16.773 INFO:tasks.workunit.client.0.smithi090.stdout:5/129: dwrite d1/d4/f6 [0,4194304] 0 2024-03-11T19:25:16.801 INFO:tasks.workunit.client.0.smithi090.stdout:7/129: sync 2024-03-11T19:25:16.872 INFO:tasks.workunit.client.0.smithi090.stdout:7/130: chown d3/dc 3 1 2024-03-11T19:25:16.874 INFO:tasks.workunit.client.0.smithi090.stdout:5/130: readlink d1/d4/d19/l23 0 2024-03-11T19:25:16.987 INFO:tasks.workunit.client.0.smithi090.stdout:6/143: mkdir d4/d24/d33 0 2024-03-11T19:25:17.396 INFO:tasks.workunit.client.0.smithi090.stdout:3/110: creat d2/db/f24 x:0 0 0 2024-03-11T19:25:17.412 INFO:tasks.workunit.client.0.smithi090.stdout:0/136: symlink d2/l25 0 2024-03-11T19:25:17.462 INFO:tasks.workunit.client.0.smithi090.stdout:2/134: dwrite d0/f1b [0,4194304] 0 2024-03-11T19:25:17.476 INFO:tasks.workunit.client.0.smithi090.stdout:4/141: symlink d4/l2f 0 2024-03-11T19:25:17.689 INFO:tasks.workunit.client.0.smithi090.stdout:8/135: creat d0/d3/d26/f2e x:0 0 0 2024-03-11T19:25:17.982 INFO:tasks.workunit.client.0.smithi090.stdout:1/147: rename d5/da/d1c/l27 to d5/da/d23/l33 0 2024-03-11T19:25:18.023 INFO:tasks.workunit.client.0.smithi090.stdout:8/136: dwrite d0/d3/d26/f2e [0,4194304] 0 2024-03-11T19:25:18.119 INFO:tasks.workunit.client.0.smithi090.stdout:9/122: getdents d1/dd/d18/d25 0 2024-03-11T19:25:18.165 INFO:tasks.workunit.client.0.smithi090.stdout:6/144: read d4/d14/f23 [212071,101975] 0 2024-03-11T19:25:18.199 INFO:tasks.workunit.client.0.smithi090.stdout:0/137: creat d2/d11/d15/d1f/f26 x:0 0 0 2024-03-11T19:25:18.206 INFO:tasks.workunit.client.0.smithi090.stdout:4/142: creat d4/d27/d28/f30 x:0 0 0 2024-03-11T19:25:18.283 INFO:tasks.workunit.client.0.smithi090.stdout:4/143: chown d4/ff 1695499 1 2024-03-11T19:25:18.304 INFO:tasks.workunit.client.0.smithi090.stdout:4/144: chown d4/d18/c29 0 1 2024-03-11T19:25:18.323 INFO:tasks.workunit.client.0.smithi090.stdout:7/131: rename d3/f15 to d3/dc/d17/f2c 0 2024-03-11T19:25:18.523 INFO:tasks.workunit.client.0.smithi090.stdout:9/123: read f0 [2461628,101227] 0 2024-03-11T19:25:18.525 INFO:tasks.workunit.client.0.smithi090.stdout:8/137: chown d0/f2a 93 1 2024-03-11T19:25:18.611 INFO:tasks.workunit.client.0.smithi090.stdout:8/138: read d0/d3/d26/dc/f12 [84,68015] 0 2024-03-11T19:25:18.998 INFO:tasks.workunit.client.0.smithi090.stdout:0/138: dread d2/fe [0,4194304] 0 2024-03-11T19:25:19.101 INFO:tasks.workunit.client.0.smithi090.stdout:5/131: truncate d1/d4/fc 2093113 0 2024-03-11T19:25:19.354 INFO:tasks.workunit.client.0.smithi090.stdout:6/145: rename d4/d24/d33 to d4/d13/d22/d34 0 2024-03-11T19:25:19.443 INFO:tasks.workunit.client.0.smithi090.stdout:3/111: write d2/f9 [1773575,38400] 0 2024-03-11T19:25:19.502 INFO:tasks.workunit.client.0.smithi090.stdout:7/132: dread d3/dc/fe [0,4194304] 0 2024-03-11T19:25:19.547 INFO:tasks.workunit.client.0.smithi090.stdout:8/139: creat d0/d3/d26/f2f x:0 0 0 2024-03-11T19:25:19.562 INFO:tasks.workunit.client.0.smithi090.stdout:2/135: write d0/f7 [619653,37444] 0 2024-03-11T19:25:19.607 INFO:tasks.workunit.client.0.smithi090.stdout:3/112: write d2/f8 [6457368,83865] 0 2024-03-11T19:25:19.720 INFO:tasks.workunit.client.0.smithi090.stdout:8/140: read - d0/d3/d26/f2f zero size 2024-03-11T19:25:20.512 INFO:tasks.workunit.client.0.smithi090.stdout:1/148: dwrite d5/da/dd/f1f [0,4194304] 0 2024-03-11T19:25:20.614 INFO:tasks.workunit.client.0.smithi090.stdout:1/149: stat l3 0 2024-03-11T19:25:21.090 INFO:tasks.workunit.client.0.smithi090.stdout:2/136: mkdir d0/de/d1e 0 2024-03-11T19:25:21.103 INFO:tasks.workunit.client.0.smithi090.stdout:5/132: write d1/d4/f20 [504035,82225] 0 2024-03-11T19:25:21.254 INFO:tasks.workunit.client.0.smithi090.stdout:2/137: write d0/f1b [404705,110657] 0 2024-03-11T19:25:21.369 INFO:tasks.workunit.client.0.smithi090.stdout:4/145: write d4/d18/f2a [895994,81050] 0 2024-03-11T19:25:21.392 INFO:tasks.workunit.client.0.smithi090.stdout:3/113: dread d2/db/d15/f17 [0,4194304] 0 2024-03-11T19:25:21.655 INFO:tasks.workunit.client.0.smithi090.stdout:7/133: creat d3/dc/f2d x:0 0 0 2024-03-11T19:25:21.782 INFO:tasks.workunit.client.0.smithi090.stdout:9/124: rename d1/f6 to d1/dd/d19/f29 0 2024-03-11T19:25:21.937 INFO:tasks.workunit.client.0.smithi090.stdout:6/146: mknod d4/d13/c35 0 2024-03-11T19:25:21.947 INFO:tasks.workunit.client.0.smithi090.stdout:0/139: link d2/d11/d15/f19 d2/f27 0 2024-03-11T19:25:21.984 INFO:tasks.workunit.client.0.smithi090.stdout:6/147: chown d4/c5 134145108 1 2024-03-11T19:25:21.986 INFO:tasks.workunit.client.0.smithi090.stdout:0/140: chown d2/d11/c12 709 1 2024-03-11T19:25:22.168 INFO:tasks.workunit.client.0.smithi090.stdout:2/138: read d0/f2 [5961694,68932] 0 2024-03-11T19:25:22.275 INFO:tasks.workunit.client.0.smithi090.stdout:8/141: truncate d0/d3/f2c 104560 0 2024-03-11T19:25:22.396 INFO:tasks.workunit.client.0.smithi090.stdout:4/146: mknod d4/d27/d28/c31 0 2024-03-11T19:25:22.518 INFO:tasks.workunit.client.0.smithi090.stdout:3/114: truncate d2/f19 258008 0 2024-03-11T19:25:22.685 INFO:tasks.workunit.client.0.smithi090.stdout:7/134: truncate d3/f4 2454490 0 2024-03-11T19:25:23.031 INFO:tasks.workunit.client.0.smithi090.stdout:9/125: dread d1/dd/df/f16 [0,4194304] 0 2024-03-11T19:25:23.087 INFO:tasks.workunit.client.0.smithi090.stdout:0/141: dread - d2/d11/d15/f19 zero size 2024-03-11T19:25:23.274 INFO:tasks.workunit.client.0.smithi090.stdout:9/126: dread d1/f3 [0,4194304] 0 2024-03-11T19:25:23.286 INFO:tasks.workunit.client.0.smithi090.stdout:8/142: rename d0/d3/cf to d0/d3/c30 0 2024-03-11T19:25:23.392 INFO:tasks.workunit.client.0.smithi090.stdout:8/143: dread d0/d3/d26/f23 [0,4194304] 0 2024-03-11T19:25:23.435 INFO:tasks.workunit.client.0.smithi090.stdout:8/144: dread d0/f17 [0,4194304] 0 2024-03-11T19:25:23.564 INFO:tasks.workunit.client.0.smithi090.stdout:4/147: symlink d4/d27/d28/l32 0 2024-03-11T19:25:23.684 INFO:tasks.workunit.client.0.smithi090.stdout:3/115: fdatasync d2/d1d/fc 0 2024-03-11T19:25:23.702 INFO:tasks.workunit.client.0.smithi090.stdout:7/135: dread - d3/dc/d13/f24 zero size 2024-03-11T19:25:23.845 INFO:tasks.workunit.client.0.smithi090.stdout:1/150: write d5/da/d23/f2a [1269290,126713] 0 2024-03-11T19:25:23.966 INFO:tasks.workunit.client.0.smithi090.stdout:0/142: creat d2/d11/d15/f28 x:0 0 0 2024-03-11T19:25:24.531 INFO:tasks.workunit.client.0.smithi090.stdout:8/145: truncate d0/d3/d26/f23 59905 0 2024-03-11T19:25:24.621 INFO:tasks.workunit.client.0.smithi090.stdout:7/136: symlink d3/dc/d1e/l2e 0 2024-03-11T19:25:24.638 INFO:tasks.workunit.client.0.smithi090.stdout:1/151: truncate d5/da/dd/f14 7201480 0 2024-03-11T19:25:24.949 INFO:tasks.workunit.client.0.smithi090.stdout:0/143: rename d2/d11/d15/d1f to d2/d11/d15/d29 0 2024-03-11T19:25:25.131 INFO:tasks.workunit.client.0.smithi090.stdout:9/127: dread d1/dd/d19/f29 [0,4194304] 0 2024-03-11T19:25:25.151 INFO:tasks.workunit.client.0.smithi090.stdout:8/146: creat d0/f31 x:0 0 0 2024-03-11T19:25:25.255 INFO:tasks.workunit.client.0.smithi090.stdout:2/139: getdents d0/de 0 2024-03-11T19:25:25.391 INFO:tasks.workunit.client.0.smithi090.stdout:6/148: dwrite d4/d14/f23 [0,4194304] 0 2024-03-11T19:25:25.558 INFO:tasks.workunit.client.0.smithi090.stdout:7/137: symlink d3/dc/d13/l2f 0 2024-03-11T19:25:25.765 INFO:tasks.workunit.client.0.smithi090.stdout:4/148: write d4/f5 [2017494,85966] 0 2024-03-11T19:25:26.026 INFO:tasks.workunit.client.0.smithi090.stdout:3/116: write d2/db/f1c [487819,122813] 0 2024-03-11T19:25:26.195 INFO:tasks.workunit.client.0.smithi090.stdout:3/117: dread d2/db/d15/f17 [0,4194304] 0 2024-03-11T19:25:26.411 INFO:tasks.workunit.client.0.smithi090.stdout:0/144: rmdir d2/d11 39 2024-03-11T19:25:26.474 INFO:tasks.workunit.client.0.smithi090.stdout:7/138: truncate d3/dc/d1e/f11 5114792 0 2024-03-11T19:25:26.904 INFO:tasks.workunit.client.0.smithi090.stdout:4/149: mknod d4/d27/d28/c33 0 2024-03-11T19:25:26.973 INFO:tasks.workunit.client.0.smithi090.stdout:5/133: write d1/d4/fc [937104,109439] 0 2024-03-11T19:25:27.224 INFO:tasks.workunit.client.0.smithi090.stdout:4/150: sync 2024-03-11T19:25:27.625 INFO:tasks.workunit.client.0.smithi090.stdout:7/139: truncate d3/dc/d13/f20 1598344 0 2024-03-11T19:25:27.652 INFO:tasks.workunit.client.0.smithi090.stdout:9/128: write d1/f9 [2231635,18900] 0 2024-03-11T19:25:27.761 INFO:tasks.workunit.client.0.smithi090.stdout:2/140: write d0/f18 [556320,104079] 0 2024-03-11T19:25:27.792 INFO:tasks.workunit.client.0.smithi090.stdout:2/141: readlink d0/l8 0 2024-03-11T19:25:28.306 INFO:tasks.workunit.client.0.smithi090.stdout:0/145: write d2/f8 [1443943,45403] 0 2024-03-11T19:25:28.439 INFO:tasks.workunit.client.0.smithi090.stdout:5/134: mknod d1/d4/d19/c25 0 2024-03-11T19:25:28.538 INFO:tasks.workunit.client.0.smithi090.stdout:4/151: dread - d4/d18/f25 zero size 2024-03-11T19:25:28.632 INFO:tasks.workunit.client.0.smithi090.stdout:8/147: dwrite d0/f2a [0,4194304] 0 2024-03-11T19:25:28.814 INFO:tasks.workunit.client.0.smithi090.stdout:2/142: rmdir d0 39 2024-03-11T19:25:28.885 INFO:tasks.workunit.client.0.smithi090.stdout:1/152: rename d5/da/f29 to d5/da/d23/f34 0 2024-03-11T19:25:28.895 INFO:tasks.workunit.client.0.smithi090.stdout:7/140: chown d3/dc/d17/f2c 3 1 2024-03-11T19:25:28.967 INFO:tasks.workunit.client.0.smithi090.stdout:6/149: dwrite d4/d14/f2a [0,4194304] 0 2024-03-11T19:25:29.323 INFO:tasks.workunit.client.0.smithi090.stdout:4/152: creat d4/d27/f34 x:0 0 0 2024-03-11T19:25:29.677 INFO:tasks.workunit.client.0.smithi090.stdout:9/129: rename d1/dd/d19 to d1/dd/df/d1c/d21/d2a 0 2024-03-11T19:25:29.810 INFO:tasks.workunit.client.0.smithi090.stdout:1/153: mknod d5/da/dd/c35 0 2024-03-11T19:25:29.840 INFO:tasks.workunit.client.0.smithi090.stdout:3/118: getdents d2/db 0 2024-03-11T19:25:29.895 INFO:tasks.workunit.client.0.smithi090.stdout:3/119: chown d2/f12 12041901 1 2024-03-11T19:25:30.035 INFO:tasks.workunit.client.0.smithi090.stdout:8/148: write d0/f19 [5032685,9585] 0 2024-03-11T19:25:30.489 INFO:tasks.workunit.client.0.smithi090.stdout:4/153: stat d4/c14 0 2024-03-11T19:25:30.698 INFO:tasks.workunit.client.0.smithi090.stdout:0/146: symlink d2/d11/d15/d29/l2a 0 2024-03-11T19:25:31.011 INFO:tasks.workunit.client.0.smithi090.stdout:6/150: dwrite d4/f11 [0,4194304] 0 2024-03-11T19:25:31.316 INFO:tasks.workunit.client.0.smithi090.stdout:6/151: dwrite d4/f11 [4194304,4194304] 0 2024-03-11T19:25:31.539 INFO:tasks.workunit.client.0.smithi090.stdout:5/135: rmdir d1/d17/d1f 0 2024-03-11T19:25:31.874 INFO:tasks.workunit.client.0.smithi090.stdout:8/149: chown d0/d3/c30 12 1 2024-03-11T19:25:32.486 INFO:tasks.workunit.client.0.smithi090.stdout:7/141: rename d3/dc/d1e/d2a/f2b to d3/dc/f30 0 2024-03-11T19:25:32.877 INFO:tasks.workunit.client.0.smithi090.stdout:6/152: creat d4/d13/d22/d34/f36 x:0 0 0 2024-03-11T19:25:33.286 INFO:tasks.workunit.client.0.smithi090.stdout:8/150: creat d0/f32 x:0 0 0 2024-03-11T19:25:33.466 INFO:tasks.workunit.client.0.smithi090.stdout:8/151: dread d0/d3/d26/f23 [0,4194304] 0 2024-03-11T19:25:33.668 INFO:tasks.workunit.client.0.smithi090.stdout:2/143: rename d0/d10 to d0/de/d1f 0 2024-03-11T19:25:33.722 INFO:tasks.workunit.client.0.smithi090.stdout:9/130: dwrite d1/dd/f13 [0,4194304] 0 2024-03-11T19:25:33.849 INFO:tasks.workunit.client.0.smithi090.stdout:3/120: write d2/db/d15/f18 [1082444,38636] 0 2024-03-11T19:25:34.079 INFO:tasks.workunit.client.0.smithi090.stdout:8/152: unlink d0/d3/c24 0 2024-03-11T19:25:34.242 INFO:tasks.workunit.client.0.smithi090.stdout:4/154: dwrite d4/f2e [0,4194304] 0 2024-03-11T19:25:34.365 INFO:tasks.workunit.client.0.smithi090.stdout:0/147: write d2/d11/d15/f23 [139775,121145] 0 2024-03-11T19:25:34.403 INFO:tasks.workunit.client.0.smithi090.stdout:7/142: fsync d3/dc/d1e/f11 0 2024-03-11T19:25:34.518 INFO:tasks.workunit.client.0.smithi090.stdout:1/154: getdents d5/da 0 2024-03-11T19:25:34.779 INFO:tasks.workunit.client.0.smithi090.stdout:9/131: rename f0 to d1/d1d/f2b 0 2024-03-11T19:25:34.800 INFO:tasks.workunit.client.0.smithi090.stdout:7/143: rename d3/dc/d1e to d3/dc/d1e/d2a/d31 22 2024-03-11T19:25:35.236 INFO:tasks.workunit.client.0.smithi090.stdout:5/136: dwrite d1/d4/f6 [4194304,4194304] 0 2024-03-11T19:25:35.343 INFO:tasks.workunit.client.0.smithi090.stdout:4/155: creat d4/d27/f35 x:0 0 0 2024-03-11T19:25:35.357 INFO:tasks.workunit.client.0.smithi090.stdout:0/148: truncate d2/f5 2028785 0 2024-03-11T19:25:35.827 INFO:tasks.workunit.client.0.smithi090.stdout:9/132: unlink d1/c2 0 2024-03-11T19:25:35.957 INFO:tasks.workunit.client.0.smithi090.stdout:7/144: mkdir d3/dc/d17/d32 0 2024-03-11T19:25:35.961 INFO:tasks.workunit.client.0.smithi090.stdout:6/153: dwrite d4/fe [0,4194304] 0 2024-03-11T19:25:36.137 INFO:tasks.workunit.client.0.smithi090.stdout:7/145: chown d3/f7 564697009 1 2024-03-11T19:25:36.192 INFO:tasks.workunit.client.0.smithi090.stdout:8/153: truncate d0/d3/f8 7680873 0 2024-03-11T19:25:36.552 INFO:tasks.workunit.client.0.smithi090.stdout:0/149: dread - d2/f1b zero size 2024-03-11T19:25:36.795 INFO:tasks.workunit.client.0.smithi090.stdout:5/137: dread d1/d4/f20 [0,4194304] 0 2024-03-11T19:25:36.929 INFO:tasks.workunit.client.0.smithi090.stdout:2/144: dwrite d0/de/d12/f1d [0,4194304] 0 2024-03-11T19:25:37.056 INFO:tasks.workunit.client.0.smithi090.stdout:3/121: link d2/f12 d2/d1d/f25 0 2024-03-11T19:25:37.222 INFO:tasks.workunit.client.0.smithi090.stdout:2/145: sync 2024-03-11T19:25:37.282 INFO:tasks.workunit.client.0.smithi090.stdout:7/146: truncate d3/dc/d13/f18 598503 0 2024-03-11T19:25:37.348 INFO:tasks.workunit.client.0.smithi090.stdout:8/154: symlink d0/d3/d26/l33 0 2024-03-11T19:25:37.581 INFO:tasks.workunit.client.0.smithi090.stdout:1/155: dwrite d5/da/dd/f1a [8388608,4194304] 0 2024-03-11T19:25:37.716 INFO:tasks.workunit.client.0.smithi090.stdout:0/150: creat d2/d11/f2b x:0 0 0 2024-03-11T19:25:37.906 INFO:tasks.workunit.client.0.smithi090.stdout:9/133: symlink d1/dd/d18/d25/l2c 0 2024-03-11T19:25:37.947 INFO:tasks.workunit.client.0.smithi090.stdout:9/134: write d1/dd/df/f16 [937447,127104] 0 2024-03-11T19:25:38.425 INFO:tasks.workunit.client.0.smithi090.stdout:2/146: rename d0/de/d1f/l14 to d0/de/d1e/l20 0 2024-03-11T19:25:38.493 INFO:tasks.workunit.client.0.smithi090.stdout:4/156: link d4/c22 d4/d18/c36 0 2024-03-11T19:25:38.703 INFO:tasks.workunit.client.0.smithi090.stdout:8/155: rmdir d0/d3/d26/dc 39 2024-03-11T19:25:38.715 INFO:tasks.workunit.client.0.smithi090.stdout:0/151: unlink d2/l1e 0 2024-03-11T19:25:39.141 INFO:tasks.workunit.client.0.smithi090.stdout:9/135: rmdir d1/dd 39 2024-03-11T19:25:39.457 INFO:tasks.workunit.client.0.smithi090.stdout:3/122: symlink d2/db/d15/d1a/l26 0 2024-03-11T19:25:39.463 INFO:tasks.workunit.client.0.smithi090.stdout:6/154: write d4/fb [5140014,130702] 0 2024-03-11T19:25:39.851 INFO:tasks.workunit.client.0.smithi090.stdout:0/152: rmdir d2/d11/d15 39 2024-03-11T19:25:39.902 INFO:tasks.workunit.client.0.smithi090.stdout:0/153: truncate d2/d11/f2b 679022 0 2024-03-11T19:25:39.918 INFO:tasks.workunit.client.0.smithi090.stdout:8/156: truncate d0/d3/d26/f2e 1828666 0 2024-03-11T19:25:41.045 INFO:tasks.workunit.client.0.smithi090.stdout:7/147: creat d3/dc/f33 x:0 0 0 2024-03-11T19:25:41.080 INFO:tasks.workunit.client.0.smithi090.stdout:6/155: write d4/d13/d22/d34/f36 [676670,79648] 0 2024-03-11T19:25:41.218 INFO:tasks.workunit.client.0.smithi090.stdout:3/123: truncate d2/db/f24 983990 0 2024-03-11T19:25:41.312 INFO:tasks.workunit.client.0.smithi090.stdout:4/157: rmdir d4/d18 39 2024-03-11T19:25:41.328 INFO:tasks.workunit.client.0.smithi090.stdout:4/158: chown d4/d27 185507361 1 2024-03-11T19:25:41.394 INFO:tasks.workunit.client.0.smithi090.stdout:3/124: sync 2024-03-11T19:25:42.134 INFO:tasks.workunit.client.0.smithi090.stdout:2/147: dwrite d0/fc [0,4194304] 0 2024-03-11T19:25:42.374 INFO:tasks.workunit.client.0.smithi090.stdout:1/156: write d5/da/d23/f34 [554079,71962] 0 2024-03-11T19:25:42.673 INFO:tasks.workunit.client.0.smithi090.stdout:7/148: dread d3/dc/f30 [0,4194304] 0 2024-03-11T19:25:42.735 INFO:tasks.workunit.client.0.smithi090.stdout:6/156: mkdir d4/d13/d22/d34/d37 0 2024-03-11T19:25:42.956 INFO:tasks.workunit.client.0.smithi090.stdout:3/125: creat d2/d1d/f27 x:0 0 0 2024-03-11T19:25:43.003 INFO:tasks.workunit.client.0.smithi090.stdout:0/154: symlink d2/d11/d15/d29/l2c 0 2024-03-11T19:25:43.474 INFO:tasks.workunit.client.0.smithi090.stdout:1/157: truncate d5/da/f19 875363 0 2024-03-11T19:25:44.055 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:25:44.058 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:25:44.108 INFO:tasks.workunit.client.0.smithi090.stdout:4/159: creat d4/d18/f37 x:0 0 0 2024-03-11T19:25:44.152 INFO:tasks.workunit.client.0.smithi090.stdout:3/126: truncate d2/f1b 3308381 0 2024-03-11T19:25:44.430 INFO:tasks.workunit.client.0.smithi090.stdout:1/158: rmdir d5/da/d1c 39 2024-03-11T19:25:44.756 INFO:tasks.workunit.client.0.smithi090.stdout:9/136: dwrite d1/dd/df/f10 [0,4194304] 0 2024-03-11T19:25:44.854 INFO:tasks.workunit.client.0.smithi090.stdout:4/160: mkdir d4/d27/d28/d38 0 2024-03-11T19:25:45.285 INFO:tasks.workunit.client.0.smithi090.stdout:3/127: dread d2/f6 [0,4194304] 0 2024-03-11T19:25:45.375 INFO:tasks.workunit.client.0.smithi090.stdout:3/128: readlink d2/d1d/l22 0 2024-03-11T19:25:45.466 INFO:tasks.workunit.client.0.smithi090.stdout:0/155: symlink d2/d11/d15/l2d 0 2024-03-11T19:25:45.677 INFO:tasks.workunit.client.0.smithi090.stdout:5/138: rename d1/d4 to d1/d26 0 2024-03-11T19:25:45.696 INFO:tasks.workunit.client.0.smithi090.stdout:4/161: dread d4/f12 [4194304,4194304] 0 2024-03-11T19:25:45.860 INFO:tasks.workunit.client.0.smithi090.stdout:1/159: mkdir d5/d31/d36 0 2024-03-11T19:25:46.004 INFO:tasks.workunit.client.0.smithi090.stdout:3/129: truncate d2/f6 44597 0 2024-03-11T19:25:46.244 INFO:tasks.workunit.client.0.smithi090.stdout:0/156: write d2/d11/f2b [1453590,4436] 0 2024-03-11T19:25:46.278 INFO:tasks.workunit.client.0.smithi090.stdout:4/162: fdatasync d4/d18/f25 0 2024-03-11T19:25:46.344 INFO:tasks.workunit.client.0.smithi090.stdout:5/139: dread d1/d26/fc [0,4194304] 0 2024-03-11T19:25:46.468 INFO:tasks.workunit.client.0.smithi090.stdout:9/137: creat d1/dd/df/f2d x:0 0 0 2024-03-11T19:25:46.659 INFO:tasks.workunit.client.0.smithi090.stdout:2/148: dwrite d0/fd [0,4194304] 0 2024-03-11T19:25:46.789 INFO:tasks.workunit.client.0.smithi090.stdout:3/130: rename d2/f3 to d2/db/d15/d1a/f28 0 2024-03-11T19:25:46.935 INFO:tasks.workunit.client.0.smithi090.stdout:4/163: mknod d4/d27/c39 0 2024-03-11T19:25:46.993 INFO:tasks.workunit.client.0.smithi090.stdout:6/157: write d4/f10 [328764,110653] 0 2024-03-11T19:25:47.018 INFO:tasks.workunit.client.0.smithi090.stdout:2/149: sync 2024-03-11T19:25:47.027 INFO:tasks.workunit.client.0.smithi090.stdout:0/157: fsync d2/d11/d15/f28 0 2024-03-11T19:25:47.135 INFO:tasks.workunit.client.0.smithi090.stdout:2/150: fdatasync d0/fc 0 2024-03-11T19:25:47.192 INFO:tasks.workunit.client.0.smithi090.stdout:7/149: dwrite d3/dc/d1e/f12 [4194304,4194304] 0 2024-03-11T19:25:47.209 INFO:tasks.workunit.client.0.smithi090.stdout:3/131: sync 2024-03-11T19:25:47.283 INFO:tasks.workunit.client.0.smithi090.stdout:9/138: rename d1/c5 to d1/dd/d18/c2e 0 2024-03-11T19:25:47.363 INFO:tasks.workunit.client.0.smithi090.stdout:2/151: write d0/fc [5224858,79902] 0 2024-03-11T19:25:47.642 INFO:tasks.workunit.client.0.smithi090.stdout:4/164: creat d4/d27/f3a x:0 0 0 2024-03-11T19:25:47.760 INFO:tasks.workunit.client.0.smithi090.stdout:6/158: dread - d4/d14/d25/f31 zero size 2024-03-11T19:25:47.908 INFO:tasks.workunit.client.0.smithi090.stdout:0/158: mkdir d2/d11/d15/d29/d2e 0 2024-03-11T19:25:47.991 INFO:tasks.workunit.client.0.smithi090.stdout:8/157: dwrite d0/d3/d26/dc/f28 [0,4194304] 0 2024-03-11T19:25:48.049 INFO:tasks.workunit.client.0.smithi090.stdout:0/159: readlink d2/l25 0 2024-03-11T19:25:48.243 INFO:tasks.workunit.client.0.smithi090.stdout:7/150: truncate d3/dc/d13/f24 266155 0 2024-03-11T19:25:48.377 INFO:tasks.workunit.client.0.smithi090.stdout:4/165: creat d4/d27/f3b x:0 0 0 2024-03-11T19:25:48.439 INFO:tasks.workunit.client.0.smithi090.stdout:7/151: dread d3/dc/d1e/f11 [4194304,4194304] 0 2024-03-11T19:25:48.753 INFO:tasks.workunit.client.0.smithi090.stdout:8/158: dread - d0/d3/d26/f2f zero size 2024-03-11T19:25:48.852 INFO:tasks.workunit.client.0.smithi090.stdout:0/160: dread d2/d11/d15/f23 [0,4194304] 0 2024-03-11T19:25:49.134 INFO:tasks.workunit.client.0.smithi090.stdout:1/160: getdents d5/da/d1c 0 2024-03-11T19:25:49.287 INFO:tasks.workunit.client.0.smithi090.stdout:4/166: creat d4/d18/f3c x:0 0 0 2024-03-11T19:25:49.329 INFO:tasks.workunit.client.0.smithi090.stdout:7/152: mkdir d3/dc/d17/d34 0 2024-03-11T19:25:49.459 INFO:tasks.workunit.client.0.smithi090.stdout:7/153: dread d3/dc/d1e/ff [0,4194304] 0 2024-03-11T19:25:49.565 INFO:tasks.workunit.client.0.smithi090.stdout:8/159: mknod d0/c34 0 2024-03-11T19:25:49.721 INFO:tasks.workunit.client.0.smithi090.stdout:8/160: sync 2024-03-11T19:25:49.992 INFO:tasks.workunit.client.0.smithi090.stdout:4/167: mknod d4/d18/c3d 0 2024-03-11T19:25:50.098 INFO:tasks.workunit.client.0.smithi090.stdout:0/161: dread d2/f14 [0,4194304] 0 2024-03-11T19:25:50.231 INFO:tasks.workunit.client.0.smithi090.stdout:6/159: link d4/d13/l28 d4/d13/d22/d34/l38 0 2024-03-11T19:25:50.340 INFO:tasks.workunit.client.0.smithi090.stdout:8/161: dread - d0/f31 zero size 2024-03-11T19:25:50.393 INFO:tasks.workunit.client.0.smithi090.stdout:6/160: chown d4/ff 5553 1 2024-03-11T19:25:50.487 INFO:tasks.workunit.client.0.smithi090.stdout:5/140: dwrite d1/d26/f20 [0,4194304] 0 2024-03-11T19:25:50.626 INFO:tasks.workunit.client.0.smithi090.stdout:5/141: chown d1/d26/f20 54 1 2024-03-11T19:25:50.925 INFO:tasks.workunit.client.0.smithi090.stdout:4/168: mknod d4/d27/d28/c3e 0 2024-03-11T19:25:51.382 INFO:tasks.workunit.client.0.smithi090.stdout:9/139: write d1/dd/f11 [3812526,94249] 0 2024-03-11T19:25:51.453 INFO:tasks.workunit.client.0.smithi090.stdout:0/162: dread d2/fe [0,4194304] 0 2024-03-11T19:25:51.458 INFO:tasks.workunit.client.0.smithi090.stdout:8/162: read - d0/f32 zero size 2024-03-11T19:25:51.567 INFO:tasks.workunit.client.0.smithi090.stdout:1/161: truncate d5/da/dd/f14 7175566 0 2024-03-11T19:25:51.636 INFO:tasks.workunit.client.0.smithi090.stdout:5/142: mkdir d1/d26/d19/d27 0 2024-03-11T19:25:51.775 INFO:tasks.workunit.client.0.smithi090.stdout:5/143: dread - d1/f16 zero size 2024-03-11T19:25:51.892 INFO:tasks.workunit.client.0.smithi090.stdout:2/152: dwrite d0/f4 [4194304,4194304] 0 2024-03-11T19:25:51.909 INFO:tasks.workunit.client.0.smithi090.stdout:4/169: fsync d4/f2e 0 2024-03-11T19:25:52.143 INFO:tasks.workunit.client.0.smithi090.stdout:6/161: symlink d4/d13/l39 0 2024-03-11T19:25:52.563 INFO:tasks.workunit.client.0.smithi090.stdout:8/163: fsync d0/d3/d26/f23 0 2024-03-11T19:25:52.621 INFO:tasks.workunit.client.0.smithi090.stdout:8/164: chown d0/d3/d26/f2f 6 1 2024-03-11T19:25:52.643 INFO:tasks.workunit.client.0.smithi090.stdout:8/165: read - d0/f32 zero size 2024-03-11T19:25:52.734 INFO:tasks.workunit.client.0.smithi090.stdout:7/154: getdents d3 0 2024-03-11T19:25:53.249 INFO:tasks.workunit.client.0.smithi090.stdout:2/153: mkdir d0/d21 0 2024-03-11T19:25:53.261 INFO:tasks.workunit.client.0.smithi090.stdout:2/154: stat d0/l9 0 2024-03-11T19:25:53.310 INFO:tasks.workunit.client.0.smithi090.stdout:4/170: creat d4/d27/f3f x:0 0 0 2024-03-11T19:25:53.341 INFO:tasks.workunit.client.0.smithi090.stdout:4/171: chown d4/d27/c39 5391 1 2024-03-11T19:25:53.469 INFO:tasks.workunit.client.0.smithi090.stdout:6/162: fdatasync d4/f11 0 2024-03-11T19:25:53.827 INFO:tasks.workunit.client.0.smithi090.stdout:3/132: rename f0 to d2/f29 0 2024-03-11T19:25:54.772 INFO:tasks.workunit.client.0.smithi090.stdout:6/163: fsync d4/d13/d22/f27 0 2024-03-11T19:25:55.145 INFO:tasks.workunit.client.0.smithi090.stdout:3/133: dread d2/f9 [0,4194304] 0 2024-03-11T19:25:55.411 INFO:tasks.workunit.client.0.smithi090.stdout:6/164: unlink d4/d13/d22/f27 0 2024-03-11T19:25:55.561 INFO:tasks.workunit.client.0.smithi090.stdout:5/144: link d1/c8 d1/c28 0 2024-03-11T19:25:55.787 INFO:tasks.workunit.client.0.smithi090.stdout:4/172: link d4/d18/c29 d4/d27/c40 0 2024-03-11T19:25:55.835 INFO:tasks.workunit.client.0.smithi090.stdout:1/162: dwrite d5/f15 [0,4194304] 0 2024-03-11T19:25:55.852 INFO:tasks.workunit.client.0.smithi090.stdout:9/140: dwrite d1/f17 [0,4194304] 0 2024-03-11T19:25:56.183 INFO:tasks.workunit.client.0.smithi090.stdout:6/165: mkdir d4/d13/d22/d3a 0 2024-03-11T19:25:56.238 INFO:tasks.workunit.client.0.smithi090.stdout:5/145: dread - d1/f22 zero size 2024-03-11T19:25:56.273 INFO:tasks.workunit.client.0.smithi090.stdout:9/141: write d1/dd/f11 [4994726,6420] 0 2024-03-11T19:25:56.369 INFO:tasks.workunit.client.0.smithi090.stdout:4/173: dread d4/f2e [0,4194304] 0 2024-03-11T19:25:56.504 INFO:tasks.workunit.client.0.smithi090.stdout:0/163: unlink d2/d11/c12 0 2024-03-11T19:25:56.662 INFO:tasks.workunit.client.0.smithi090.stdout:1/163: read d5/da/d1c/f21 [197137,23670] 0 2024-03-11T19:25:56.771 INFO:tasks.workunit.client.0.smithi090.stdout:7/155: write d3/dc/f33 [1011284,59842] 0 2024-03-11T19:25:57.130 INFO:tasks.workunit.client.0.smithi090.stdout:9/142: symlink d1/d1d/l2f 0 2024-03-11T19:25:57.462 INFO:tasks.workunit.client.0.smithi090.stdout:2/155: write d0/f3 [2582660,120008] 0 2024-03-11T19:25:57.492 INFO:tasks.workunit.client.0.smithi090.stdout:8/166: unlink d0/d3/l21 0 2024-03-11T19:25:57.516 INFO:tasks.workunit.client.0.smithi090.stdout:0/164: mkdir d2/d11/d15/d29/d2f 0 2024-03-11T19:25:57.660 INFO:tasks.workunit.client.0.smithi090.stdout:7/156: chown d3/dc/d13/f20 244185 1 2024-03-11T19:25:58.061 INFO:tasks.workunit.client.0.smithi090.stdout:9/143: chown d1/d1d/f2b 80 1 2024-03-11T19:25:58.227 INFO:tasks.workunit.client.0.smithi090.stdout:4/174: chown d4/d18/c29 4060458 1 2024-03-11T19:25:58.263 INFO:tasks.workunit.client.0.smithi090.stdout:4/175: readlink d4/d18/l1f 0 2024-03-11T19:25:58.283 INFO:tasks.workunit.client.0.smithi090.stdout:4/176: chown d4/c19 0 1 2024-03-11T19:25:58.572 INFO:tasks.workunit.client.0.smithi090.stdout:7/157: fsync d3/f9 0 2024-03-11T19:25:58.583 INFO:tasks.workunit.client.0.smithi090.stdout:3/134: write d2/d1d/fc [971413,100004] 0 2024-03-11T19:25:58.666 INFO:tasks.workunit.client.0.smithi090.stdout:3/135: chown d2/db/d15/f1e 244 1 2024-03-11T19:25:58.858 INFO:tasks.workunit.client.0.smithi090.stdout:8/167: symlink d0/d3/d26/l35 0 2024-03-11T19:25:58.862 INFO:tasks.workunit.client.0.smithi090.stdout:9/144: fdatasync d1/dd/f1e 0 2024-03-11T19:25:58.972 INFO:tasks.workunit.client.0.smithi090.stdout:8/168: dread - d0/d3/d26/f2f zero size 2024-03-11T19:25:59.036 INFO:tasks.workunit.client.0.smithi090.stdout:7/158: truncate d3/f22 45085 0 2024-03-11T19:25:59.204 INFO:tasks.workunit.client.0.smithi090.stdout:1/164: getdents d5 0 2024-03-11T19:25:59.311 INFO:tasks.workunit.client.0.smithi090.stdout:2/156: link d0/f18 d0/de/d1e/f22 0 2024-03-11T19:25:59.528 INFO:tasks.workunit.client.0.smithi090.stdout:9/145: fdatasync d1/dd/df/f10 0 2024-03-11T19:25:59.545 INFO:tasks.workunit.client.0.smithi090.stdout:9/146: stat d1 0 2024-03-11T19:25:59.716 INFO:tasks.workunit.client.0.smithi090.stdout:7/159: fdatasync d3/dc/d13/f20 0 2024-03-11T19:25:59.806 INFO:tasks.workunit.client.0.smithi090.stdout:5/146: write d1/f16 [900886,105407] 0 2024-03-11T19:26:00.032 INFO:tasks.workunit.client.0.smithi090.stdout:1/165: creat d5/f37 x:0 0 0 2024-03-11T19:26:00.046 INFO:tasks.workunit.client.0.smithi090.stdout:5/147: sync 2024-03-11T19:26:00.258 INFO:tasks.workunit.client.0.smithi090.stdout:1/166: chown d5/da/d1c/l2f 405 1 2024-03-11T19:26:00.287 INFO:tasks.workunit.client.0.smithi090.stdout:6/166: dwrite d4/fe [0,4194304] 0 2024-03-11T19:26:00.860 INFO:tasks.workunit.client.0.smithi090.stdout:5/148: fdatasync d1/f22 0 2024-03-11T19:26:01.136 INFO:tasks.workunit.client.0.smithi090.stdout:0/165: write d2/d11/d15/d29/f26 [740849,17813] 0 2024-03-11T19:26:01.335 INFO:tasks.workunit.client.0.smithi090.stdout:4/177: write d4/d18/f2a [148836,99677] 0 2024-03-11T19:26:01.400 INFO:tasks.workunit.client.0.smithi090.stdout:1/167: mknod d5/d31/c38 0 2024-03-11T19:26:01.971 INFO:tasks.workunit.client.0.smithi090.stdout:2/157: link d0/de/d12/l19 d0/d21/l23 0 2024-03-11T19:26:02.185 INFO:tasks.workunit.client.0.smithi090.stdout:3/136: symlink d2/l2a 0 2024-03-11T19:26:02.234 INFO:tasks.workunit.client.0.smithi090.stdout:8/169: write d0/f32 [798713,2139] 0 2024-03-11T19:26:02.296 INFO:tasks.workunit.client.0.smithi090.stdout:8/170: chown d0/d3/d26 40188 1 2024-03-11T19:26:02.350 INFO:tasks.workunit.client.0.smithi090.stdout:8/171: dread d0/d3/d26/dc/f12 [0,4194304] 0 2024-03-11T19:26:02.810 INFO:tasks.workunit.client.0.smithi090.stdout:5/149: symlink d1/l29 0 2024-03-11T19:26:02.851 INFO:tasks.workunit.client.0.smithi090.stdout:9/147: write d1/dd/df/f2d [707800,17854] 0 2024-03-11T19:26:02.910 INFO:tasks.workunit.client.0.smithi090.stdout:2/158: creat d0/de/d12/f24 x:0 0 0 2024-03-11T19:26:03.038 INFO:tasks.workunit.client.0.smithi090.stdout:0/166: symlink d2/d11/d15/d29/d2e/l30 0 2024-03-11T19:26:03.197 INFO:tasks.workunit.client.0.smithi090.stdout:3/137: creat d2/db/f2b x:0 0 0 2024-03-11T19:26:03.358 INFO:tasks.workunit.client.0.smithi090.stdout:3/138: fdatasync d2/db/f2b 0 2024-03-11T19:26:03.436 INFO:tasks.workunit.client.0.smithi090.stdout:0/167: dwrite d2/d11/d15/d29/f26 [0,4194304] 0 2024-03-11T19:26:03.605 INFO:tasks.workunit.client.0.smithi090.stdout:1/168: fsync d5/da/f19 0 2024-03-11T19:26:03.928 INFO:tasks.workunit.client.0.smithi090.stdout:2/159: readlink d0/de/d1e/l20 0 2024-03-11T19:26:03.928 INFO:tasks.workunit.client.0.smithi090.stdout:5/150: dread d1/d26/f20 [0,4194304] 0 2024-03-11T19:26:03.978 INFO:tasks.workunit.client.0.smithi090.stdout:5/151: fdatasync d1/f16 0 2024-03-11T19:26:04.308 INFO:tasks.workunit.client.0.smithi090.stdout:7/160: write d3/dc/d13/f18 [1508314,4322] 0 2024-03-11T19:26:04.363 INFO:tasks.workunit.client.0.smithi090.stdout:3/139: read d2/db/d15/f1e [180644,6316] 0 2024-03-11T19:26:04.564 INFO:tasks.workunit.client.0.smithi090.stdout:0/168: mknod d2/d11/d15/d29/d2e/c31 0 2024-03-11T19:26:04.607 INFO:tasks.workunit.client.0.smithi090.stdout:6/167: dwrite d4/d14/d25/f31 [0,4194304] 0 2024-03-11T19:26:04.610 INFO:tasks.workunit.client.0.smithi090.stdout:3/140: read - d2/db/f2b zero size 2024-03-11T19:26:04.757 INFO:tasks.workunit.client.0.smithi090.stdout:2/160: mkdir d0/de/d1e/d25 0 2024-03-11T19:26:04.770 INFO:tasks.workunit.client.0.smithi090.stdout:9/148: truncate d1/f17 601925 0 2024-03-11T19:26:04.872 INFO:tasks.workunit.client.0.smithi090.stdout:2/161: chown d0/f6 10908 1 2024-03-11T19:26:05.035 INFO:tasks.workunit.client.0.smithi090.stdout:5/152: fsync d1/d26/f11 0 2024-03-11T19:26:05.042 INFO:tasks.workunit.client.0.smithi090.stdout:8/172: rename d0/d3/c18 to d0/d3/d26/c36 0 2024-03-11T19:26:05.051 INFO:tasks.workunit.client.0.smithi090.stdout:4/178: write d4/d27/f3a [1003257,40658] 0 2024-03-11T19:26:05.712 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:51:25.471 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:26:05.717 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:51:25.476 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:26:05.782 INFO:tasks.workunit.client.0.smithi090.stdout:9/149: creat d1/dd/f30 x:0 0 0 2024-03-11T19:26:05.811 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:51:25.570 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:26:05.830 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:51:25.526 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:26:05.844 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:51:25.531 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:26:05.844 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:51:25.540 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:26:05.847 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:51:25.535 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:26:05.944 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:51:25.632 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:26:05.957 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:51:25.652 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:26:06.064 INFO:tasks.workunit.client.0.smithi090.stdout:2/162: symlink d0/de/l26 0 2024-03-11T19:26:06.358 INFO:tasks.workunit.client.0.smithi090.stdout:8/173: creat d0/d3/f37 x:0 0 0 2024-03-11T19:26:06.564 INFO:tasks.workunit.client.0.smithi090.stdout:8/174: sync 2024-03-11T19:26:06.692 INFO:tasks.workunit.client.0.smithi090.stdout:8/175: dread d0/f32 [0,4194304] 0 2024-03-11T19:26:07.081 INFO:tasks.workunit.client.0.smithi090.stdout:2/163: dread d0/de/d1f/f1c [0,4194304] 0 2024-03-11T19:26:07.616 INFO:tasks.workunit.client.0.smithi090.stdout:0/169: rmdir d2/d11/d15/d29/d2f 0 2024-03-11T19:26:07.890 INFO:tasks.workunit.client.0.smithi090.stdout:1/169: link d5/da/d23/f2d d5/da/d1c/f39 0 2024-03-11T19:26:07.980 INFO:tasks.workunit.client.0.smithi090.stdout:2/164: truncate d0/f3 981499 0 2024-03-11T19:26:08.490 INFO:tasks.workunit.client.0.smithi090.stdout:6/168: dwrite d4/d13/f1b [0,4194304] 0 2024-03-11T19:26:08.508 INFO:tasks.workunit.client.0.smithi090.stdout:7/161: dwrite d3/dc/d17/f1b [0,4194304] 0 2024-03-11T19:26:08.650 INFO:tasks.workunit.client.0.smithi090.stdout:0/170: creat d2/d11/d15/d29/d2e/f32 x:0 0 0 2024-03-11T19:26:08.831 INFO:tasks.workunit.client.0.smithi090.stdout:3/141: link d2/f6 d2/f2c 0 2024-03-11T19:26:08.841 INFO:tasks.workunit.client.0.smithi090.stdout:4/179: write d4/d27/f35 [980994,20514] 0 2024-03-11T19:26:08.893 INFO:tasks.workunit.client.0.smithi090.stdout:0/171: readlink d2/d11/l16 0 2024-03-11T19:26:09.083 INFO:tasks.workunit.client.0.smithi090.stdout:9/150: write d1/dd/f11 [5172853,72764] 0 2024-03-11T19:26:09.298 INFO:tasks.workunit.client.0.smithi090.stdout:5/153: dwrite d1/d26/f6 [0,4194304] 0 2024-03-11T19:26:09.519 INFO:tasks.workunit.client.0.smithi090.stdout:7/162: creat d3/dc/d13/f35 x:0 0 0 2024-03-11T19:26:09.534 INFO:tasks.workunit.client.0.smithi090.stdout:4/180: creat d4/d18/f41 x:0 0 0 2024-03-11T19:26:09.634 INFO:tasks.workunit.client.0.smithi090.stdout:6/169: dread d4/d14/f23 [0,4194304] 0 2024-03-11T19:26:09.672 INFO:tasks.workunit.client.0.smithi090.stdout:4/181: sync 2024-03-11T19:26:09.683 INFO:tasks.workunit.client.0.smithi090.stdout:3/142: rename d2/de/l11 to d2/d1d/l2d 0 2024-03-11T19:26:09.902 INFO:tasks.workunit.client.0.smithi090.stdout:2/165: write d0/f4 [6704166,65920] 0 2024-03-11T19:26:10.155 INFO:tasks.workunit.client.0.smithi090.stdout:9/151: creat d1/dd/d18/f31 x:0 0 0 2024-03-11T19:26:10.174 INFO:tasks.workunit.client.0.smithi090.stdout:8/176: dwrite d0/d3/d26/f2e [0,4194304] 0 2024-03-11T19:26:10.310 INFO:tasks.workunit.client.0.smithi090.stdout:5/154: creat d1/d17/f2a x:0 0 0 2024-03-11T19:26:10.351 INFO:tasks.workunit.client.0.smithi090.stdout:8/177: fdatasync d0/d3/f37 0 2024-03-11T19:26:10.514 INFO:tasks.workunit.client.0.smithi090.stdout:1/170: write d5/da/dd/f14 [2890905,96956] 0 2024-03-11T19:26:10.819 INFO:tasks.workunit.client.0.smithi090.stdout:0/172: link d2/d11/d15/d29/d2e/f32 d2/d11/d15/f33 0 2024-03-11T19:26:10.872 INFO:tasks.workunit.client.0.smithi090.stdout:0/173: rename d2/d11/d15/d29/d2e to d2/d11/d15/d29/d2e/d34 22 2024-03-11T19:26:10.897 INFO:tasks.workunit.client.0.smithi090.stdout:0/174: dread - d2/d11/d15/f33 zero size 2024-03-11T19:26:10.976 INFO:tasks.workunit.client.0.smithi090.stdout:9/152: mkdir d1/dd/d18/d32 0 2024-03-11T19:26:10.990 INFO:tasks.workunit.client.0.smithi090.stdout:5/155: creat d1/d26/f2b x:0 0 0 2024-03-11T19:26:11.115 INFO:tasks.workunit.client.0.smithi090.stdout:9/153: sync 2024-03-11T19:26:11.867 INFO:tasks.workunit.client.0.smithi090.stdout:2/166: write d0/f6 [240928,68845] 0 2024-03-11T19:26:12.137 INFO:tasks.workunit.client.0.smithi090.stdout:2/167: dwrite d0/f4 [0,4194304] 0 2024-03-11T19:26:12.578 INFO:tasks.workunit.client.0.smithi090.stdout:8/178: mkdir d0/d3/d26/d38 0 2024-03-11T19:26:13.197 INFO:tasks.workunit.client.0.smithi090.stdout:4/182: link d4/c14 d4/d27/d28/d38/c42 0 2024-03-11T19:26:13.399 INFO:tasks.workunit.client.0.smithi090.stdout:3/143: creat d2/f2e x:0 0 0 2024-03-11T19:26:13.550 INFO:tasks.workunit.client.0.smithi090.stdout:6/170: creat d4/d13/f3b x:0 0 0 2024-03-11T19:26:13.657 INFO:tasks.workunit.client.0.smithi090.stdout:6/171: stat d4/d14 0 2024-03-11T19:26:13.738 INFO:tasks.workunit.client.0.smithi090.stdout:7/163: dwrite d3/f22 [0,4194304] 0 2024-03-11T19:26:13.866 INFO:tasks.workunit.client.0.smithi090.stdout:6/172: dread - d4/d13/f1f zero size 2024-03-11T19:26:14.041 INFO:tasks.workunit.client.0.smithi090.stdout:6/173: sync 2024-03-11T19:26:14.226 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:26:14.229 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:26:14.249 INFO:tasks.workunit.client.0.smithi090.stdout:0/175: write d2/d11/f1a [372983,1700] 0 2024-03-11T19:26:14.256 INFO:tasks.workunit.client.0.smithi090.stdout:4/183: dread - d4/d27/f34 zero size 2024-03-11T19:26:14.367 INFO:tasks.workunit.client.0.smithi090.stdout:0/176: dread d2/d11/d15/f23 [0,4194304] 0 2024-03-11T19:26:14.526 INFO:tasks.workunit.client.0.smithi090.stdout:5/156: creat d1/f2c x:0 0 0 2024-03-11T19:26:14.729 INFO:tasks.workunit.client.0.smithi090.stdout:9/154: creat d1/dd/df/f33 x:0 0 0 2024-03-11T19:26:15.004 INFO:tasks.workunit.client.0.smithi090.stdout:1/171: getdents d5/d31 0 2024-03-11T19:26:15.070 INFO:tasks.workunit.client.0.smithi090.stdout:3/144: truncate d2/f2c 991655 0 2024-03-11T19:26:15.208 INFO:tasks.workunit.client.0.smithi090.stdout:6/174: dread d4/fb [4194304,4194304] 0 2024-03-11T19:26:15.219 INFO:tasks.workunit.client.0.smithi090.stdout:3/145: chown d2/d1d/l7 11232 1 2024-03-11T19:26:15.313 INFO:tasks.workunit.client.0.smithi090.stdout:0/177: rename d2/l25 to d2/d11/d15/d29/d2e/l35 0 2024-03-11T19:26:15.635 INFO:tasks.workunit.client.0.smithi090.stdout:8/179: dwrite d0/f17 [0,4194304] 0 2024-03-11T19:26:15.933 INFO:tasks.workunit.client.0.smithi090.stdout:5/157: creat d1/d26/f2d x:0 0 0 2024-03-11T19:26:15.995 INFO:tasks.workunit.client.0.smithi090.stdout:2/168: dwrite d0/de/d1e/f22 [0,4194304] 0 2024-03-11T19:26:16.452 INFO:tasks.workunit.client.0.smithi090.stdout:7/164: dwrite d3/dc/d13/f18 [0,4194304] 0 2024-03-11T19:26:16.605 INFO:tasks.workunit.client.0.smithi090.stdout:6/175: creat d4/d13/d22/d34/f3c x:0 0 0 2024-03-11T19:26:16.708 INFO:tasks.workunit.client.0.smithi090.stdout:9/155: rename d1/dd/df/f16 to d1/dd/d18/d32/f34 0 2024-03-11T19:26:16.731 INFO:tasks.workunit.client.0.smithi090.stdout:9/156: chown d1/dd/f30 20429 1 2024-03-11T19:26:17.009 INFO:tasks.workunit.client.0.smithi090.stdout:0/178: mkdir d2/d11/d15/d36 0 2024-03-11T19:26:17.034 INFO:tasks.workunit.client.0.smithi090.stdout:0/179: chown d2/f3 1363 1 2024-03-11T19:26:17.684 INFO:tasks.workunit.client.0.smithi090.stdout:8/180: fsync d0/f2a 0 2024-03-11T19:26:17.881 INFO:tasks.workunit.client.0.smithi090.stdout:4/184: link d4/c19 d4/d27/d28/c43 0 2024-03-11T19:26:17.913 INFO:tasks.workunit.client.0.smithi090.stdout:1/172: write d5/da/d1c/f21 [1688241,90389] 0 2024-03-11T19:26:18.046 INFO:tasks.workunit.client.0.smithi090.stdout:2/169: mknod d0/de/d12/c27 0 2024-03-11T19:26:18.559 INFO:tasks.workunit.client.0.smithi090.stdout:3/146: truncate d2/db/f24 819182 0 2024-03-11T19:26:18.579 INFO:tasks.workunit.client.0.smithi090.stdout:3/147: readlink d2/d1d/l7 0 2024-03-11T19:26:18.818 INFO:tasks.workunit.client.0.smithi090.stdout:6/176: fsync d4/fb 0 2024-03-11T19:26:18.964 INFO:tasks.workunit.client.0.smithi090.stdout:9/157: fdatasync d1/f9 0 2024-03-11T19:26:19.590 INFO:tasks.workunit.client.0.smithi090.stdout:4/185: mkdir d4/d27/d44 0 2024-03-11T19:26:19.721 INFO:tasks.workunit.client.0.smithi090.stdout:1/173: creat d5/da/d23/f3a x:0 0 0 2024-03-11T19:26:19.774 INFO:tasks.workunit.client.0.smithi090.stdout:2/170: rmdir d0/de/d1e 39 2024-03-11T19:26:20.030 INFO:tasks.workunit.client.0.smithi090.stdout:8/181: dwrite d0/d3/d26/f23 [0,4194304] 0 2024-03-11T19:26:20.168 INFO:tasks.workunit.client.0.smithi090.stdout:7/165: symlink d3/dc/d17/d32/l36 0 2024-03-11T19:26:20.394 INFO:tasks.workunit.client.0.smithi090.stdout:5/158: dwrite d1/d26/d19/f21 [0,4194304] 0 2024-03-11T19:26:20.575 INFO:tasks.workunit.client.0.smithi090.stdout:6/177: dread d4/fe [0,4194304] 0 2024-03-11T19:26:21.054 INFO:tasks.workunit.client.0.smithi090.stdout:3/148: mkdir d2/d2f 0 2024-03-11T19:26:21.294 INFO:tasks.workunit.client.0.smithi090.stdout:2/171: dread d0/de/d12/f1d [0,4194304] 0 2024-03-11T19:26:21.499 INFO:tasks.workunit.client.0.smithi090.stdout:3/149: dread d2/db/d15/f1e [0,4194304] 0 2024-03-11T19:26:21.709 INFO:tasks.workunit.client.0.smithi090.stdout:6/178: read d4/d13/f1b [928568,119294] 0 2024-03-11T19:26:21.722 INFO:tasks.workunit.client.0.smithi090.stdout:0/180: dwrite d2/f1c [0,4194304] 0 2024-03-11T19:26:21.896 INFO:tasks.workunit.client.0.smithi090.stdout:0/181: read - d2/f1b zero size 2024-03-11T19:26:22.075 INFO:tasks.workunit.client.0.smithi090.stdout:6/179: sync 2024-03-11T19:26:22.318 INFO:tasks.workunit.client.0.smithi090.stdout:9/158: link d1/dd/d18/l23 d1/d1d/l35 0 2024-03-11T19:26:22.921 INFO:tasks.workunit.client.0.smithi090.stdout:4/186: dwrite d4/d27/f34 [0,4194304] 0 2024-03-11T19:26:23.234 INFO:tasks.workunit.client.0.smithi090.stdout:6/180: unlink d4/f10 0 2024-03-11T19:26:23.399 INFO:tasks.workunit.client.0.smithi090.stdout:6/181: chown d4/d13/d22/c29 3482 1 2024-03-11T19:26:23.475 INFO:tasks.workunit.client.0.smithi090.stdout:5/159: link d1/c9 d1/d26/d19/d27/c2e 0 2024-03-11T19:26:23.493 INFO:tasks.workunit.client.0.smithi090.stdout:8/182: write d0/f20 [917173,54315] 0 2024-03-11T19:26:23.676 INFO:tasks.workunit.client.0.smithi090.stdout:1/174: dwrite d5/da/d23/f2b [0,4194304] 0 2024-03-11T19:26:23.842 INFO:tasks.workunit.client.0.smithi090.stdout:4/187: symlink d4/d27/d28/l45 0 2024-03-11T19:26:24.069 INFO:tasks.workunit.client.0.smithi090.stdout:7/166: dwrite d3/f8 [0,4194304] 0 2024-03-11T19:26:24.385 INFO:tasks.workunit.client.0.smithi090.stdout:6/182: readlink d4/d13/l28 0 2024-03-11T19:26:24.453 INFO:tasks.workunit.client.0.smithi090.stdout:1/175: dread d5/da/dd/f2c [0,4194304] 0 2024-03-11T19:26:24.466 INFO:tasks.workunit.client.0.smithi090.stdout:6/183: chown d4/d13/l28 69468 1 2024-03-11T19:26:24.520 INFO:tasks.workunit.client.0.smithi090.stdout:6/184: stat d4/d13/d22/d34/d37 0 2024-03-11T19:26:24.612 INFO:tasks.workunit.client.0.smithi090.stdout:5/160: fsync d1/d26/f20 0 2024-03-11T19:26:24.619 INFO:tasks.workunit.client.0.smithi090.stdout:2/172: write d0/f1b [4677018,94600] 0 2024-03-11T19:26:24.958 INFO:tasks.workunit.client.0.smithi090.stdout:1/176: fsync d5/da/d1c/f30 0 2024-03-11T19:26:25.144 INFO:tasks.workunit.client.0.smithi090.stdout:0/182: getdents d2/d11/d15/d29/d2e 0 2024-03-11T19:26:25.169 INFO:tasks.workunit.client.0.smithi090.stdout:8/183: dread d0/d3/d26/dc/fd [0,4194304] 0 2024-03-11T19:26:25.287 INFO:tasks.workunit.client.0.smithi090.stdout:6/185: dread d4/d14/f2a [0,4194304] 0 2024-03-11T19:26:25.315 INFO:tasks.workunit.client.0.smithi090.stdout:6/186: chown d4/d13/d22/d3a 27013 1 2024-03-11T19:26:25.437 INFO:tasks.workunit.client.0.smithi090.stdout:2/173: fdatasync d0/f4 0 2024-03-11T19:26:25.548 INFO:tasks.workunit.client.0.smithi090.stdout:5/161: fsync d1/d26/f20 0 2024-03-11T19:26:25.935 INFO:tasks.workunit.client.0.smithi090.stdout:9/159: dwrite d1/d1d/f2b [0,4194304] 0 2024-03-11T19:26:26.043 INFO:tasks.workunit.client.0.smithi090.stdout:1/177: mkdir d5/da/d1c/d3b 0 2024-03-11T19:26:26.095 INFO:tasks.workunit.client.0.smithi090.stdout:9/160: read d1/f3 [3410719,19793] 0 2024-03-11T19:26:26.317 INFO:tasks.workunit.client.0.smithi090.stdout:4/188: creat d4/f46 x:0 0 0 2024-03-11T19:26:26.321 INFO:tasks.workunit.client.0.smithi090.stdout:6/187: dread d4/d14/d25/f31 [0,4194304] 0 2024-03-11T19:26:26.383 INFO:tasks.workunit.client.0.smithi090.stdout:4/189: dread - d4/d18/f25 zero size 2024-03-11T19:26:27.164 INFO:tasks.workunit.client.0.smithi090.stdout:7/167: rmdir d3/dc/d13 39 2024-03-11T19:26:27.182 INFO:tasks.workunit.client.0.smithi090.stdout:7/168: chown d3/ca 433669204 1 2024-03-11T19:26:27.220 INFO:tasks.workunit.client.0.smithi090.stdout:8/184: mknod d0/d3/d26/c39 0 2024-03-11T19:26:27.403 INFO:tasks.workunit.client.0.smithi090.stdout:9/161: mknod d1/d1d/c36 0 2024-03-11T19:26:27.766 INFO:tasks.workunit.client.0.smithi090.stdout:0/183: creat d2/f37 x:0 0 0 2024-03-11T19:26:27.959 INFO:tasks.workunit.client.0.smithi090.stdout:7/169: unlink d3/dc/d17/f2c 0 2024-03-11T19:26:28.024 INFO:tasks.workunit.client.0.smithi090.stdout:8/185: rmdir d0/d3/d26/dc 39 2024-03-11T19:26:28.204 INFO:tasks.workunit.client.0.smithi090.stdout:9/162: truncate d1/dd/d18/f31 728630 0 2024-03-11T19:26:28.393 INFO:tasks.workunit.client.0.smithi090.stdout:0/184: mknod d2/d11/d15/d29/d2e/c38 0 2024-03-11T19:26:29.350 INFO:tasks.workunit.client.0.smithi090.stdout:0/185: dread d2/d11/f18 [0,4194304] 0 2024-03-11T19:26:29.352 INFO:tasks.workunit.client.0.smithi090.stdout:8/186: fsync d0/d3/d26/dc/f12 0 2024-03-11T19:26:29.387 INFO:tasks.workunit.client.0.smithi090.stdout:5/162: write d1/d17/f2a [24446,45414] 0 2024-03-11T19:26:29.495 INFO:tasks.workunit.client.0.smithi090.stdout:6/188: getdents d4/d14/d25 0 2024-03-11T19:26:29.635 INFO:tasks.workunit.client.0.smithi090.stdout:3/150: write d2/db/f24 [562219,123683] 0 2024-03-11T19:26:29.687 INFO:tasks.workunit.client.0.smithi090.stdout:2/174: dwrite d0/de/d1f/f1c [4194304,4194304] 0 2024-03-11T19:26:30.440 INFO:tasks.workunit.client.0.smithi090.stdout:6/189: truncate d4/d14/f23 3405937 0 2024-03-11T19:26:31.131 INFO:tasks.workunit.client.0.smithi090.stdout:5/163: creat d1/f2f x:0 0 0 2024-03-11T19:26:31.159 INFO:tasks.workunit.client.0.smithi090.stdout:7/170: dwrite d3/dc/d17/f1b [4194304,4194304] 0 2024-03-11T19:26:31.378 INFO:tasks.workunit.client.0.smithi090.stdout:5/164: dread d1/d26/fc [0,4194304] 0 2024-03-11T19:26:31.758 INFO:tasks.workunit.client.0.smithi090.stdout:7/171: fsync d3/f27 0 2024-03-11T19:26:31.791 INFO:tasks.workunit.client.0.smithi090.stdout:9/163: dwrite d1/dd/df/f2d [0,4194304] 0 2024-03-11T19:26:31.926 INFO:tasks.workunit.client.0.smithi090.stdout:6/190: symlink d4/l3d 0 2024-03-11T19:26:32.190 INFO:tasks.workunit.client.0.smithi090.stdout:1/178: mkdir d5/d3c 0 2024-03-11T19:26:32.287 INFO:tasks.workunit.client.0.smithi090.stdout:3/151: write d2/db/f1c [816715,59564] 0 2024-03-11T19:26:32.539 INFO:tasks.workunit.client.0.smithi090.stdout:2/175: write d0/de/d12/f24 [803320,27795] 0 2024-03-11T19:26:32.576 INFO:tasks.workunit.client.0.smithi090.stdout:3/152: sync 2024-03-11T19:26:32.595 INFO:tasks.workunit.client.0.smithi090.stdout:4/190: creat d4/f47 x:0 0 0 2024-03-11T19:26:32.665 INFO:tasks.workunit.client.0.smithi090.stdout:0/186: dwrite d2/f1b [0,4194304] 0 2024-03-11T19:26:32.899 INFO:tasks.workunit.client.0.smithi090.stdout:0/187: dread d2/f14 [0,4194304] 0 2024-03-11T19:26:32.923 INFO:tasks.workunit.client.0.smithi090.stdout:9/164: rmdir d1/dd/df/d1c 39 2024-03-11T19:26:32.934 INFO:tasks.workunit.client.0.smithi090.stdout:8/187: write d0/d3/f8 [838056,77858] 0 2024-03-11T19:26:32.992 INFO:tasks.workunit.client.0.smithi090.stdout:8/188: readlink d0/d3/d26/l1e 0 2024-03-11T19:26:33.026 INFO:tasks.workunit.client.0.smithi090.stdout:9/165: sync 2024-03-11T19:26:33.160 INFO:tasks.workunit.client.0.smithi090.stdout:9/166: sync 2024-03-11T19:26:33.235 INFO:tasks.workunit.client.0.smithi090.stdout:1/179: rename d5/c7 to d5/d31/d36/c3d 0 2024-03-11T19:26:33.387 INFO:tasks.workunit.client.0.smithi090.stdout:2/176: fsync d0/f2 0 2024-03-11T19:26:33.489 INFO:tasks.workunit.client.0.smithi090.stdout:4/191: fsync d4/f8 0 2024-03-11T19:26:34.021 INFO:tasks.workunit.client.0.smithi090.stdout:0/188: truncate d2/d11/d15/d29/f20 98739 0 2024-03-11T19:26:34.023 INFO:tasks.workunit.client.0.smithi090.stdout:8/189: fsync d0/d3/d26/f23 0 2024-03-11T19:26:34.079 INFO:tasks.workunit.client.0.smithi090.stdout:0/189: dread - d2/d11/d15/f33 zero size 2024-03-11T19:26:34.111 INFO:tasks.workunit.client.0.smithi090.stdout:0/190: readlink d2/d11/l13 0 2024-03-11T19:26:34.327 INFO:tasks.workunit.client.0.smithi090.stdout:7/172: fdatasync d3/dc/d13/f24 0 2024-03-11T19:26:34.344 INFO:tasks.workunit.client.0.smithi090.stdout:5/165: write d1/d26/d19/f21 [4459996,2512] 0 2024-03-11T19:26:34.560 INFO:tasks.workunit.client.0.smithi090.stdout:4/192: mknod d4/d27/d28/c48 0 2024-03-11T19:26:34.577 INFO:tasks.workunit.client.0.smithi090.stdout:4/193: chown d4/c21 748322523 1 2024-03-11T19:26:34.682 INFO:tasks.workunit.client.0.smithi090.stdout:3/153: creat d2/d2f/f30 x:0 0 0 2024-03-11T19:26:35.212 INFO:tasks.workunit.client.0.smithi090.stdout:6/191: dwrite d4/d13/f1b [0,4194304] 0 2024-03-11T19:26:35.404 INFO:tasks.workunit.client.0.smithi090.stdout:5/166: creat d1/d17/f30 x:0 0 0 2024-03-11T19:26:35.440 INFO:tasks.workunit.client.0.smithi090.stdout:7/173: dread d3/f22 [0,4194304] 0 2024-03-11T19:26:35.492 INFO:tasks.workunit.client.0.smithi090.stdout:7/174: readlink d3/dc/d13/l2f 0 2024-03-11T19:26:35.898 INFO:tasks.workunit.client.0.smithi090.stdout:0/191: mknod d2/c39 0 2024-03-11T19:26:36.394 INFO:tasks.workunit.client.0.smithi090.stdout:2/177: getdents d0/de/d1e/d25 0 2024-03-11T19:26:36.511 INFO:tasks.workunit.client.0.smithi090.stdout:5/167: mknod d1/d17/c31 0 2024-03-11T19:26:36.638 INFO:tasks.workunit.client.0.smithi090.stdout:7/175: creat d3/dc/d1e/f37 x:0 0 0 2024-03-11T19:26:37.039 INFO:tasks.workunit.client.0.smithi090.stdout:9/167: link d1/f17 d1/dd/d18/d25/f37 0 2024-03-11T19:26:37.311 INFO:tasks.workunit.client.0.smithi090.stdout:1/180: getdents d5/da/d23 0 2024-03-11T19:26:37.438 INFO:tasks.workunit.client.0.smithi090.stdout:8/190: dwrite d0/f31 [0,4194304] 0 2024-03-11T19:26:37.599 INFO:tasks.workunit.client.0.smithi090.stdout:7/176: mknod d3/dc/d17/d32/c38 0 2024-03-11T19:26:37.671 INFO:tasks.workunit.client.0.smithi090.stdout:8/191: read d0/f17 [1212135,107846] 0 2024-03-11T19:26:37.793 INFO:tasks.workunit.client.0.smithi090.stdout:7/177: chown d3/f7 1554887058 1 2024-03-11T19:26:37.955 INFO:tasks.workunit.client.0.smithi090.stdout:4/194: dwrite d4/d27/d28/f30 [0,4194304] 0 2024-03-11T19:26:38.172 INFO:tasks.workunit.client.0.smithi090.stdout:4/195: chown d4/d27/f3b 34 1 2024-03-11T19:26:38.425 INFO:tasks.workunit.client.0.smithi090.stdout:1/181: mknod d5/d31/c3e 0 2024-03-11T19:26:38.492 INFO:tasks.workunit.client.0.smithi090.stdout:6/192: dwrite d4/d14/f21 [0,4194304] 0 2024-03-11T19:26:38.785 INFO:tasks.workunit.client.0.smithi090.stdout:3/154: link d2/db/d15/d1a/f28 d2/d2f/f31 0 2024-03-11T19:26:38.979 INFO:tasks.workunit.client.0.smithi090.stdout:8/192: fsync d0/d3/d26/f2f 0 2024-03-11T19:26:39.055 INFO:tasks.workunit.client.0.smithi090.stdout:2/178: write d0/f7 [2055742,109752] 0 2024-03-11T19:26:39.540 INFO:tasks.workunit.client.0.smithi090.stdout:7/178: truncate d3/dc/d13/f25 797988 0 2024-03-11T19:26:39.961 INFO:tasks.workunit.client.0.smithi090.stdout:1/182: fdatasync d5/da/dd/f1f 0 2024-03-11T19:26:40.055 INFO:tasks.workunit.client.0.smithi090.stdout:6/193: symlink d4/d14/d25/l3e 0 2024-03-11T19:26:40.211 INFO:tasks.workunit.client.0.smithi090.stdout:5/168: creat d1/f32 x:0 0 0 2024-03-11T19:26:40.262 INFO:tasks.workunit.client.0.smithi090.stdout:3/155: creat d2/d2f/f32 x:0 0 0 2024-03-11T19:26:40.286 INFO:tasks.workunit.client.0.smithi090.stdout:3/156: stat d2/d1d/l7 0 2024-03-11T19:26:40.521 INFO:tasks.workunit.client.0.smithi090.stdout:3/157: sync 2024-03-11T19:26:40.640 INFO:tasks.workunit.client.0.smithi090.stdout:2/179: creat d0/de/d1e/f28 x:0 0 0 2024-03-11T19:26:40.797 INFO:tasks.workunit.client.0.smithi090.stdout:8/193: dread d0/f19 [0,4194304] 0 2024-03-11T19:26:40.833 INFO:tasks.workunit.client.0.smithi090.stdout:8/194: chown d0/d3/d26/f2f 377 1 2024-03-11T19:26:40.878 INFO:tasks.workunit.client.0.smithi090.stdout:7/179: symlink d3/dc/d1e/l39 0 2024-03-11T19:26:41.015 INFO:tasks.workunit.client.0.smithi090.stdout:0/192: getdents d2/d11/d15/d29/d2e 0 2024-03-11T19:26:41.084 INFO:tasks.workunit.client.0.smithi090.stdout:4/196: symlink d4/l49 0 2024-03-11T19:26:41.390 INFO:tasks.workunit.client.0.smithi090.stdout:6/194: truncate d4/d14/f2a 1894031 0 2024-03-11T19:26:41.706 INFO:tasks.workunit.client.0.smithi090.stdout:3/158: fdatasync d2/db/f2b 0 2024-03-11T19:26:42.307 INFO:tasks.workunit.client.0.smithi090.stdout:9/168: dwrite d1/f17 [0,4194304] 0 2024-03-11T19:26:42.806 INFO:tasks.workunit.client.0.smithi090.stdout:6/195: creat d4/d24/f3f x:0 0 0 2024-03-11T19:26:43.057 INFO:tasks.workunit.client.0.smithi090.stdout:3/159: creat d2/db/d15/d1a/f33 x:0 0 0 2024-03-11T19:26:43.093 INFO:tasks.workunit.client.0.smithi090.stdout:3/160: chown d2/db/d15/d1a/l26 55 1 2024-03-11T19:26:43.654 INFO:tasks.workunit.client.0.smithi090.stdout:1/183: write d5/da/d23/f34 [921318,79807] 0 2024-03-11T19:26:43.944 INFO:tasks.workunit.client.0.smithi090.stdout:9/169: creat d1/dd/d18/f38 x:0 0 0 2024-03-11T19:26:44.021 INFO:tasks.workunit.client.0.smithi090.stdout:9/170: truncate d1/dd/d18/f38 566582 0 2024-03-11T19:26:44.295 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:26:44.299 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:26:44.464 INFO:tasks.workunit.client.0.smithi090.stdout:2/180: dwrite d0/fd [4194304,4194304] 0 2024-03-11T19:26:44.576 INFO:tasks.workunit.client.0.smithi090.stdout:0/193: write d2/d11/d15/d29/f26 [4922022,57916] 0 2024-03-11T19:26:44.576 INFO:tasks.workunit.client.0.smithi090.stdout:8/195: rmdir d0/d3/d26/d38 0 2024-03-11T19:26:44.612 INFO:tasks.workunit.client.0.smithi090.stdout:7/180: creat d3/dc/f3a x:0 0 0 2024-03-11T19:26:44.666 INFO:tasks.workunit.client.0.smithi090.stdout:1/184: creat d5/d31/f3f x:0 0 0 2024-03-11T19:26:44.701 INFO:tasks.workunit.client.0.smithi090.stdout:0/194: readlink d2/d11/d15/l2d 0 2024-03-11T19:26:44.765 INFO:tasks.workunit.client.0.smithi090.stdout:9/171: mknod d1/dd/d18/d32/c39 0 2024-03-11T19:26:45.480 INFO:tasks.workunit.client.0.smithi090.stdout:4/197: getdents d4/d27/d28/d38 0 2024-03-11T19:26:45.584 INFO:tasks.workunit.client.0.smithi090.stdout:1/185: stat d5/da/l2e 0 2024-03-11T19:26:45.714 INFO:tasks.workunit.client.0.smithi090.stdout:8/196: creat d0/d3/d26/dc/f3a x:0 0 0 2024-03-11T19:26:45.886 INFO:tasks.workunit.client.0.smithi090.stdout:8/197: dread d0/d3/d26/dc/f12 [0,4194304] 0 2024-03-11T19:26:46.064 INFO:tasks.workunit.client.0.smithi090.stdout:8/198: dread d0/f17 [0,4194304] 0 2024-03-11T19:26:46.371 INFO:tasks.workunit.client.0.smithi090.stdout:4/198: fsync d4/d27/f35 0 2024-03-11T19:26:46.393 INFO:tasks.workunit.client.0.smithi090.stdout:4/199: readlink d4/l49 0 2024-03-11T19:26:46.762 INFO:tasks.workunit.client.0.smithi090.stdout:3/161: dwrite d2/db/d15/f17 [4194304,4194304] 0 2024-03-11T19:26:46.867 INFO:tasks.workunit.client.0.smithi090.stdout:9/172: truncate d1/dd/d18/f31 1083301 0 2024-03-11T19:26:47.697 INFO:tasks.workunit.client.0.smithi090.stdout:8/199: fsync d0/d3/f2c 0 2024-03-11T19:26:48.151 INFO:tasks.workunit.client.0.smithi090.stdout:0/195: write d2/f27 [276545,72870] 0 2024-03-11T19:26:48.169 INFO:tasks.workunit.client.0.smithi090.stdout:1/186: getdents d5/d31/d36 0 2024-03-11T19:26:48.191 INFO:tasks.workunit.client.0.smithi090.stdout:2/181: dwrite d0/f1b [0,4194304] 0 2024-03-11T19:26:48.435 INFO:tasks.workunit.client.0.smithi090.stdout:4/200: creat d4/f4a x:0 0 0 2024-03-11T19:26:48.465 INFO:tasks.workunit.client.0.smithi090.stdout:7/181: dwrite d3/dc/f2d [0,4194304] 0 2024-03-11T19:26:48.469 INFO:tasks.workunit.client.0.smithi090.stdout:8/200: creat d0/d3/d26/dc/f3b x:0 0 0 2024-03-11T19:26:48.589 INFO:tasks.workunit.client.0.smithi090.stdout:5/169: symlink d1/l33 0 2024-03-11T19:26:48.637 INFO:tasks.workunit.client.0.smithi090.stdout:1/187: dread - d5/da/d23/f3a zero size 2024-03-11T19:26:48.780 INFO:tasks.workunit.client.0.smithi090.stdout:6/196: read d4/d14/f2a [736889,116480] 0 2024-03-11T19:26:49.402 INFO:tasks.workunit.client.0.smithi090.stdout:2/182: truncate d0/de/d12/f1d 2250677 0 2024-03-11T19:26:49.878 INFO:tasks.workunit.client.0.smithi090.stdout:6/197: truncate d4/d24/f26 3575087 0 2024-03-11T19:26:49.891 INFO:tasks.workunit.client.0.smithi090.stdout:0/196: mkdir d2/d11/d3a 0 2024-03-11T19:26:49.962 INFO:tasks.workunit.client.0.smithi090.stdout:5/170: mknod d1/c34 0 2024-03-11T19:26:50.004 INFO:tasks.workunit.client.0.smithi090.stdout:9/173: write d1/dd/f13 [717819,38587] 0 2024-03-11T19:26:50.592 INFO:tasks.workunit.client.0.smithi090.stdout:0/197: symlink d2/d11/d15/l3b 0 2024-03-11T19:26:50.718 INFO:tasks.workunit.client.0.smithi090.stdout:6/198: mkdir d4/d13/d22/d34/d40 0 2024-03-11T19:26:50.796 INFO:tasks.workunit.client.0.smithi090.stdout:6/199: dread d4/d14/f2a [0,4194304] 0 2024-03-11T19:26:50.802 INFO:tasks.workunit.client.0.smithi090.stdout:9/174: unlink d1/dd/l14 0 2024-03-11T19:26:51.797 INFO:tasks.workunit.client.0.smithi090.stdout:7/182: rmdir d3/dc/d13 39 2024-03-11T19:26:51.833 INFO:tasks.workunit.client.0.smithi090.stdout:1/188: dwrite d5/fc [4194304,4194304] 0 2024-03-11T19:26:52.063 INFO:tasks.workunit.client.0.smithi090.stdout:8/201: rename d0/d3/c30 to d0/d3/d26/c3c 0 2024-03-11T19:26:52.080 INFO:tasks.workunit.client.0.smithi090.stdout:2/183: write d0/fc [5454411,20806] 0 2024-03-11T19:26:52.166 INFO:tasks.workunit.client.0.smithi090.stdout:2/184: write d0/f1b [3754499,120630] 0 2024-03-11T19:26:52.792 INFO:tasks.workunit.client.0.smithi090.stdout:3/162: creat d2/f34 x:0 0 0 2024-03-11T19:26:52.990 INFO:tasks.workunit.client.0.smithi090.stdout:4/201: rename d4/d27/f3f to d4/d27/d28/d38/f4b 0 2024-03-11T19:26:53.059 INFO:tasks.workunit.client.0.smithi090.stdout:8/202: dread - d0/f2d zero size 2024-03-11T19:26:53.261 INFO:tasks.workunit.client.0.smithi090.stdout:2/185: truncate d0/de/f17 206889 0 2024-03-11T19:26:53.780 INFO:tasks.workunit.client.0.smithi090.stdout:4/202: chown d4/c1a 123 1 2024-03-11T19:26:53.806 INFO:tasks.workunit.client.0.smithi090.stdout:5/171: rename d1/f22 to d1/d26/f35 0 2024-03-11T19:26:53.900 INFO:tasks.workunit.client.0.smithi090.stdout:9/175: dwrite d1/f9 [0,4194304] 0 2024-03-11T19:26:54.065 INFO:tasks.workunit.client.0.smithi090.stdout:6/200: write d4/d24/f26 [2237746,81088] 0 2024-03-11T19:26:54.120 INFO:tasks.workunit.client.0.smithi090.stdout:6/201: stat d4/d14/d25/l3e 0 2024-03-11T19:26:54.312 INFO:tasks.workunit.client.0.smithi090.stdout:3/163: creat d2/d1f/f35 x:0 0 0 2024-03-11T19:26:54.646 INFO:tasks.workunit.client.0.smithi090.stdout:7/183: write d3/dc/d1e/ff [2011211,46062] 0 2024-03-11T19:26:54.906 INFO:tasks.workunit.client.0.smithi090.stdout:8/203: write d0/d3/d26/dc/f28 [3859770,124933] 0 2024-03-11T19:26:55.160 INFO:tasks.workunit.client.0.smithi090.stdout:1/189: dwrite d5/da/d1c/f39 [0,4194304] 0 2024-03-11T19:26:55.260 INFO:tasks.workunit.client.0.smithi090.stdout:4/203: unlink d4/d18/c3d 0 2024-03-11T19:26:55.261 INFO:tasks.workunit.client.0.smithi090.stdout:0/198: rename c1 to d2/d11/d15/d29/d2e/c3c 0 2024-03-11T19:26:55.521 INFO:tasks.workunit.client.0.smithi090.stdout:9/176: unlink d1/dd/df/f2d 0 2024-03-11T19:26:55.534 INFO:tasks.workunit.client.0.smithi090.stdout:6/202: fsync d4/fe 0 2024-03-11T19:26:55.719 INFO:tasks.workunit.client.0.smithi090.stdout:3/164: creat d2/d1f/f36 x:0 0 0 2024-03-11T19:26:55.724 INFO:tasks.workunit.client.0.smithi090.stdout:2/186: fsync d0/de/f17 0 2024-03-11T19:26:56.634 INFO:tasks.workunit.client.0.smithi090.stdout:4/204: fdatasync d4/d27/f34 0 2024-03-11T19:26:56.906 INFO:tasks.workunit.client.0.smithi090.stdout:5/172: dwrite d1/d17/f2a [0,4194304] 0 2024-03-11T19:26:57.023 INFO:tasks.workunit.client.0.smithi090.stdout:5/173: read - d1/d26/f11 zero size 2024-03-11T19:26:57.127 INFO:tasks.workunit.client.0.smithi090.stdout:6/203: read d4/d14/f21 [2152927,39128] 0 2024-03-11T19:26:57.264 INFO:tasks.workunit.client.0.smithi090.stdout:9/177: mkdir d1/dd/d18/d3a 0 2024-03-11T19:26:57.771 INFO:tasks.workunit.client.0.smithi090.stdout:3/165: creat d2/db/d15/f37 x:0 0 0 2024-03-11T19:26:57.792 INFO:tasks.workunit.client.0.smithi090.stdout:7/184: write d3/f7 [385038,115790] 0 2024-03-11T19:26:57.904 INFO:tasks.workunit.client.0.smithi090.stdout:8/204: write d0/d3/f37 [587247,108081] 0 2024-03-11T19:26:57.974 INFO:tasks.workunit.client.0.smithi090.stdout:0/199: write d2/f27 [1023446,91942] 0 2024-03-11T19:26:58.557 INFO:tasks.workunit.client.0.smithi090.stdout:6/204: unlink d4/d14/f2a 0 2024-03-11T19:26:58.605 INFO:tasks.workunit.client.0.smithi090.stdout:1/190: dwrite d5/da/f19 [0,4194304] 0 2024-03-11T19:26:58.752 INFO:tasks.workunit.client.0.smithi090.stdout:9/178: symlink d1/dd/d18/d25/l3b 0 2024-03-11T19:26:58.785 INFO:tasks.workunit.client.0.smithi090.stdout:4/205: dread d4/f5 [0,4194304] 0 2024-03-11T19:26:58.821 INFO:tasks.workunit.client.0.smithi090.stdout:1/191: chown c4 13 1 2024-03-11T19:26:59.152 INFO:tasks.workunit.client.0.smithi090.stdout:3/166: mknod d2/de/c38 0 2024-03-11T19:26:59.164 INFO:tasks.workunit.client.0.smithi090.stdout:7/185: rename d3/f9 to d3/f3b 0 2024-03-11T19:26:59.305 INFO:tasks.workunit.client.0.smithi090.stdout:5/174: mknod d1/d26/d19/d27/c36 0 2024-03-11T19:26:59.329 INFO:tasks.workunit.client.0.smithi090.stdout:6/205: creat d4/d14/d25/f41 x:0 0 0 2024-03-11T19:26:59.404 INFO:tasks.workunit.client.0.smithi090.stdout:6/206: chown d4/d13/f1f 6 1 2024-03-11T19:26:59.692 INFO:tasks.workunit.client.0.smithi090.stdout:1/192: dread d5/fc [4194304,4194304] 0 2024-03-11T19:26:59.858 INFO:tasks.workunit.client.0.smithi090.stdout:8/205: rename d0/d3/d26/c3c to d0/d3/d26/dc/c3d 0 2024-03-11T19:26:59.888 INFO:tasks.workunit.client.0.smithi090.stdout:8/206: chown d0/d3/d26/dc/fd 334295 1 2024-03-11T19:27:00.090 INFO:tasks.workunit.client.0.smithi090.stdout:3/167: symlink d2/db/d15/l39 0 2024-03-11T19:27:00.443 INFO:tasks.workunit.client.0.smithi090.stdout:6/207: mkdir d4/d13/d22/d42 0 2024-03-11T19:27:01.050 INFO:tasks.workunit.client.0.smithi090.stdout:8/207: mkdir d0/d3/d3e 0 2024-03-11T19:27:01.236 INFO:tasks.workunit.client.0.smithi090.stdout:3/168: creat d2/db/d15/d1a/f3a x:0 0 0 2024-03-11T19:27:01.377 INFO:tasks.workunit.client.0.smithi090.stdout:2/187: write d0/f3 [645912,77297] 0 2024-03-11T19:27:01.432 INFO:tasks.workunit.client.0.smithi090.stdout:2/188: chown d0/de/cf 3730128 1 2024-03-11T19:27:01.754 INFO:tasks.workunit.client.0.smithi090.stdout:0/200: getdents d2/d11 0 2024-03-11T19:27:01.848 INFO:tasks.workunit.client.0.smithi090.stdout:8/208: symlink d0/d3/d26/dc/l3f 0 2024-03-11T19:27:02.120 INFO:tasks.workunit.client.0.smithi090.stdout:2/189: mkdir d0/de/d1e/d29 0 2024-03-11T19:27:02.135 INFO:tasks.workunit.client.0.smithi090.stdout:9/179: write d1/f3 [5019646,116050] 0 2024-03-11T19:27:02.628 INFO:tasks.workunit.client.0.smithi090.stdout:7/186: dwrite d3/f4 [0,4194304] 0 2024-03-11T19:27:02.733 INFO:tasks.workunit.client.0.smithi090.stdout:0/201: mkdir d2/d11/d15/d29/d3d 0 2024-03-11T19:27:02.786 INFO:tasks.workunit.client.0.smithi090.stdout:8/209: dread - d0/d3/d26/dc/f3a zero size 2024-03-11T19:27:02.821 INFO:tasks.workunit.client.0.smithi090.stdout:0/202: chown d2/d11/d15/d29/d2e/c38 112071 1 2024-03-11T19:27:03.010 INFO:tasks.workunit.client.0.smithi090.stdout:5/175: write d1/d17/f30 [710445,87120] 0 2024-03-11T19:27:03.125 INFO:tasks.workunit.client.0.smithi090.stdout:4/206: dwrite d4/f24 [0,4194304] 0 2024-03-11T19:27:03.240 INFO:tasks.workunit.client.0.smithi090.stdout:5/176: read d1/d26/f20 [639606,96034] 0 2024-03-11T19:27:03.737 INFO:tasks.workunit.client.0.smithi090.stdout:1/193: dwrite d5/da/dd/f1f [0,4194304] 0 2024-03-11T19:27:03.844 INFO:tasks.workunit.client.0.smithi090.stdout:0/203: fsync d2/d11/f24 0 2024-03-11T19:27:03.882 INFO:tasks.workunit.client.0.smithi090.stdout:3/169: write d2/db/d15/f1e [1723633,130232] 0 2024-03-11T19:27:04.006 INFO:tasks.workunit.client.0.smithi090.stdout:6/208: dwrite d4/fe [4194304,4194304] 0 2024-03-11T19:27:04.162 INFO:tasks.workunit.client.0.smithi090.stdout:8/210: mkdir d0/d40 0 2024-03-11T19:27:04.320 INFO:tasks.workunit.client.0.smithi090.stdout:4/207: rename d4/d27/c39 to d4/d27/d28/c4c 0 2024-03-11T19:27:04.444 INFO:tasks.workunit.client.0.smithi090.stdout:4/208: chown d4/f24 1232 1 2024-03-11T19:27:04.605 INFO:tasks.workunit.client.0.smithi090.stdout:9/180: dwrite d1/dd/d18/f31 [0,4194304] 0 2024-03-11T19:27:04.697 INFO:tasks.workunit.client.0.smithi090.stdout:7/187: creat d3/dc/f3c x:0 0 0 2024-03-11T19:27:04.723 INFO:tasks.workunit.client.0.smithi090.stdout:2/190: getdents d0/de/d1f 0 2024-03-11T19:27:04.827 INFO:tasks.workunit.client.0.smithi090.stdout:3/170: creat d2/d1f/f3b x:0 0 0 2024-03-11T19:27:04.908 INFO:tasks.workunit.client.0.smithi090.stdout:5/177: write d1/f2f [920470,42724] 0 2024-03-11T19:27:05.029 INFO:tasks.workunit.client.0.smithi090.stdout:6/209: fsync d4/ff 0 2024-03-11T19:27:05.117 INFO:tasks.workunit.client.0.smithi090.stdout:3/171: chown d2/db/f2b 3707109 1 2024-03-11T19:27:05.185 INFO:tasks.workunit.client.0.smithi090.stdout:1/194: dread d5/f15 [0,4194304] 0 2024-03-11T19:27:05.186 INFO:tasks.workunit.client.0.smithi090.stdout:8/211: stat d0/d3/d26/dc/c3d 0 2024-03-11T19:27:05.365 INFO:tasks.workunit.client.0.smithi090.stdout:4/209: creat d4/d27/f4d x:0 0 0 2024-03-11T19:27:05.689 INFO:tasks.workunit.client.0.smithi090.stdout:4/210: dread d4/fa [0,4194304] 0 2024-03-11T19:27:05.724 INFO:tasks.workunit.client.0.smithi090.stdout:4/211: stat d4/d27/d28/f30 0 2024-03-11T19:27:05.896 INFO:tasks.workunit.client.0.smithi090.stdout:0/204: getdents d2/d11/d15/d36 0 2024-03-11T19:27:06.173 INFO:tasks.workunit.client.0.smithi090.stdout:5/178: fsync d1/f32 0 2024-03-11T19:27:06.207 INFO:tasks.workunit.client.0.smithi090.stdout:5/179: chown d1/d17/f2a 26781619 1 2024-03-11T19:27:06.472 INFO:tasks.workunit.client.0.smithi090.stdout:6/210: rename d4/d14/d25/f31 to d4/d24/f43 0 2024-03-11T19:27:06.514 INFO:tasks.workunit.client.0.smithi090.stdout:6/211: readlink d4/l8 0 2024-03-11T19:27:06.755 INFO:tasks.workunit.client.0.smithi090.stdout:2/191: creat d0/d21/f2a x:0 0 0 2024-03-11T19:27:06.788 INFO:tasks.workunit.client.0.smithi090.stdout:4/212: dread - d4/d18/f3c zero size 2024-03-11T19:27:06.829 INFO:tasks.workunit.client.0.smithi090.stdout:7/188: mknod d3/dc/d17/d34/c3d 0 2024-03-11T19:27:07.004 INFO:tasks.workunit.client.0.smithi090.stdout:0/205: truncate d2/f8 2459887 0 2024-03-11T19:27:07.170 INFO:tasks.workunit.client.0.smithi090.stdout:5/180: creat d1/d26/d19/d27/f37 x:0 0 0 2024-03-11T19:27:07.212 INFO:tasks.workunit.client.0.smithi090.stdout:5/181: chown d1/d26/c18 72 1 2024-03-11T19:27:07.481 INFO:tasks.workunit.client.0.smithi090.stdout:9/181: write d1/dd/f30 [461565,113474] 0 2024-03-11T19:27:07.535 INFO:tasks.workunit.client.0.smithi090.stdout:9/182: chown d1/dd/d18/d32/f34 242201 1 2024-03-11T19:27:08.177 INFO:tasks.workunit.client.0.smithi090.stdout:8/212: write d0/d3/d26/f2e [4669194,36613] 0 2024-03-11T19:27:08.267 INFO:tasks.workunit.client.0.smithi090.stdout:3/172: dwrite d2/d1d/f27 [0,4194304] 0 2024-03-11T19:27:08.401 INFO:tasks.workunit.client.0.smithi090.stdout:5/182: dread - d1/f2c zero size 2024-03-11T19:27:08.447 INFO:tasks.workunit.client.0.smithi090.stdout:3/173: stat d2/f34 0 2024-03-11T19:27:08.453 INFO:tasks.workunit.client.0.smithi090.stdout:8/213: dread - d0/d3/d26/dc/f3a zero size 2024-03-11T19:27:08.551 INFO:tasks.workunit.client.0.smithi090.stdout:9/183: creat d1/dd/f3c x:0 0 0 2024-03-11T19:27:08.583 INFO:tasks.workunit.client.0.smithi090.stdout:3/174: chown d2/db/d15/d1a/l26 4160430 1 2024-03-11T19:27:08.737 INFO:tasks.workunit.client.0.smithi090.stdout:8/214: dread d0/d3/d26/f2e [0,4194304] 0 2024-03-11T19:27:08.922 INFO:tasks.workunit.client.0.smithi090.stdout:8/215: dread d0/f17 [0,4194304] 0 2024-03-11T19:27:08.955 INFO:tasks.workunit.client.0.smithi090.stdout:6/212: write d4/d24/f26 [4024937,60868] 0 2024-03-11T19:27:09.003 INFO:tasks.workunit.client.0.smithi090.stdout:6/213: chown d4/d14/f21 47826851 1 2024-03-11T19:27:09.181 INFO:tasks.workunit.client.0.smithi090.stdout:0/206: symlink d2/l3e 0 2024-03-11T19:27:09.197 INFO:tasks.workunit.client.0.smithi090.stdout:7/189: rename d3/dc/d1e/f37 to d3/dc/d13/f3e 0 2024-03-11T19:27:09.387 INFO:tasks.workunit.client.0.smithi090.stdout:1/195: getdents d5/da/dd 0 2024-03-11T19:27:09.518 INFO:tasks.workunit.client.0.smithi090.stdout:3/175: stat d2/f1b 0 2024-03-11T19:27:09.651 INFO:tasks.workunit.client.0.smithi090.stdout:9/184: chown d1/dd/d18/l23 254217 1 2024-03-11T19:27:09.778 INFO:tasks.workunit.client.0.smithi090.stdout:2/192: dwrite d0/f4 [4194304,4194304] 0 2024-03-11T19:27:09.939 INFO:tasks.workunit.client.0.smithi090.stdout:4/213: write d4/d27/d28/f30 [4710432,89831] 0 2024-03-11T19:27:10.018 INFO:tasks.workunit.client.0.smithi090.stdout:8/216: fdatasync d0/f17 0 2024-03-11T19:27:10.180 INFO:tasks.workunit.client.0.smithi090.stdout:6/214: dread d4/d24/f43 [0,4194304] 0 2024-03-11T19:27:10.816 INFO:tasks.workunit.client.0.smithi090.stdout:3/176: dread d2/f8 [0,4194304] 0 2024-03-11T19:27:10.949 INFO:tasks.workunit.client.0.smithi090.stdout:9/185: unlink d1/cb 0 2024-03-11T19:27:10.987 INFO:tasks.workunit.client.0.smithi090.stdout:9/186: dread - d1/dd/f3c zero size 2024-03-11T19:27:11.467 INFO:tasks.workunit.client.0.smithi090.stdout:2/193: mkdir d0/de/d1f/d2b 0 2024-03-11T19:27:11.856 INFO:tasks.workunit.client.0.smithi090.stdout:6/215: mkdir d4/d14/d44 0 2024-03-11T19:27:12.211 INFO:tasks.workunit.client.0.smithi090.stdout:6/216: dread d4/d24/f26 [0,4194304] 0 2024-03-11T19:27:12.399 INFO:tasks.workunit.client.0.smithi090.stdout:3/177: mkdir d2/db/d15/d1a/d3c 0 2024-03-11T19:27:12.555 INFO:tasks.workunit.client.0.smithi090.stdout:3/178: chown d2/d1d 3604 1 2024-03-11T19:27:12.573 INFO:tasks.workunit.client.0.smithi090.stdout:2/194: symlink d0/l2c 0 2024-03-11T19:27:12.720 INFO:tasks.workunit.client.0.smithi090.stdout:0/207: dwrite d2/d11/d15/d29/d2e/f32 [0,4194304] 0 2024-03-11T19:27:12.985 INFO:tasks.workunit.client.0.smithi090.stdout:1/196: dwrite d5/f1b [4194304,4194304] 0 2024-03-11T19:27:13.296 INFO:tasks.workunit.client.0.smithi090.stdout:7/190: link d3/f22 d3/dc/d17/d32/f3f 0 2024-03-11T19:27:13.349 INFO:tasks.workunit.client.0.smithi090.stdout:3/179: creat d2/d1d/f3d x:0 0 0 2024-03-11T19:27:13.402 INFO:tasks.workunit.client.0.smithi090.stdout:4/214: write d4/f5 [335845,103160] 0 2024-03-11T19:27:13.848 INFO:tasks.workunit.client.0.smithi090.stdout:8/217: dwrite d0/f32 [0,4194304] 0 2024-03-11T19:27:14.059 INFO:tasks.workunit.client.0.smithi090.stdout:6/217: mknod d4/d14/d44/c45 0 2024-03-11T19:27:14.169 INFO:tasks.workunit.client.0.smithi090.stdout:4/215: dread d4/f2e [0,4194304] 0 2024-03-11T19:27:14.375 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:27:14.378 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:27:14.413 INFO:tasks.workunit.client.0.smithi090.stdout:7/191: creat d3/dc/d13/f40 x:0 0 0 2024-03-11T19:27:14.445 INFO:tasks.workunit.client.0.smithi090.stdout:6/218: sync 2024-03-11T19:27:14.472 INFO:tasks.workunit.client.0.smithi090.stdout:8/218: dread d0/d3/d26/f23 [0,4194304] 0 2024-03-11T19:27:14.957 INFO:tasks.workunit.client.0.smithi090.stdout:9/187: dwrite d1/dd/df/d1c/d21/d2a/f29 [4194304,4194304] 0 2024-03-11T19:27:15.246 INFO:tasks.workunit.client.0.smithi090.stdout:2/195: dwrite d0/fb [0,4194304] 0 2024-03-11T19:27:15.640 INFO:tasks.workunit.client.0.smithi090.stdout:0/208: dwrite d2/f27 [0,4194304] 0 2024-03-11T19:27:15.667 INFO:tasks.workunit.client.0.smithi090.stdout:1/197: write d5/da/d1c/f30 [1029641,21351] 0 2024-03-11T19:27:15.941 INFO:tasks.workunit.client.0.smithi090.stdout:9/188: dread d1/f3 [0,4194304] 0 2024-03-11T19:27:15.976 INFO:tasks.workunit.client.0.smithi090.stdout:2/196: symlink d0/de/d1f/l2d 0 2024-03-11T19:27:16.251 INFO:tasks.workunit.client.0.smithi090.stdout:0/209: mkdir d2/d11/d3f 0 2024-03-11T19:27:16.674 INFO:tasks.workunit.client.0.smithi090.stdout:9/189: dread d1/dd/f13 [0,4194304] 0 2024-03-11T19:27:16.864 INFO:tasks.workunit.client.0.smithi090.stdout:9/190: chown d1/dd/f11 227464 1 2024-03-11T19:27:16.951 INFO:tasks.workunit.client.0.smithi090.stdout:6/219: creat d4/f46 x:0 0 0 2024-03-11T19:27:16.979 INFO:tasks.workunit.client.0.smithi090.stdout:5/183: rename d1/d26/f35 to d1/f38 0 2024-03-11T19:27:17.013 INFO:tasks.workunit.client.0.smithi090.stdout:4/216: dwrite d4/d18/f41 [0,4194304] 0 2024-03-11T19:27:17.178 INFO:tasks.workunit.client.0.smithi090.stdout:1/198: write d5/fc [4822316,116855] 0 2024-03-11T19:27:17.399 INFO:tasks.workunit.client.0.smithi090.stdout:3/180: link d2/d1d/l22 d2/l3e 0 2024-03-11T19:27:17.605 INFO:tasks.workunit.client.0.smithi090.stdout:0/210: chown d2/d11/d15/d36 1409 1 2024-03-11T19:27:17.646 INFO:tasks.workunit.client.0.smithi090.stdout:6/220: chown d4/d13/l20 1336184 1 2024-03-11T19:27:17.917 INFO:tasks.workunit.client.0.smithi090.stdout:9/191: dread d1/dd/d18/f38 [0,4194304] 0 2024-03-11T19:27:17.958 INFO:tasks.workunit.client.0.smithi090.stdout:4/217: dread - d4/d27/f3b zero size 2024-03-11T19:27:18.122 INFO:tasks.workunit.client.0.smithi090.stdout:7/192: dwrite d3/dc/d13/f21 [0,4194304] 0 2024-03-11T19:27:18.151 INFO:tasks.workunit.client.0.smithi090.stdout:3/181: rmdir d2/de 39 2024-03-11T19:27:18.306 INFO:tasks.workunit.client.0.smithi090.stdout:8/219: dwrite d0/f20 [0,4194304] 0 2024-03-11T19:27:18.473 INFO:tasks.workunit.client.0.smithi090.stdout:2/197: dwrite d0/d21/f2a [0,4194304] 0 2024-03-11T19:27:18.727 INFO:tasks.workunit.client.0.smithi090.stdout:5/184: dwrite d1/d17/f2a [0,4194304] 0 2024-03-11T19:27:18.787 INFO:tasks.workunit.client.0.smithi090.stdout:1/199: dread d5/da/d23/f34 [0,4194304] 0 2024-03-11T19:27:18.976 INFO:tasks.workunit.client.0.smithi090.stdout:3/182: chown d2/de/c20 313 1 2024-03-11T19:27:19.139 INFO:tasks.workunit.client.0.smithi090.stdout:8/220: truncate d0/f31 836694 0 2024-03-11T19:27:19.293 INFO:tasks.workunit.client.0.smithi090.stdout:0/211: creat d2/d11/d3f/f40 x:0 0 0 2024-03-11T19:27:19.443 INFO:tasks.workunit.client.0.smithi090.stdout:0/212: dread d2/d11/d15/f19 [0,4194304] 0 2024-03-11T19:27:19.474 INFO:tasks.workunit.client.0.smithi090.stdout:0/213: stat d2/d11/d15/d29 0 2024-03-11T19:27:19.516 INFO:tasks.workunit.client.0.smithi090.stdout:5/185: mkdir d1/d17/d39 0 2024-03-11T19:27:19.889 INFO:tasks.workunit.client.0.smithi090.stdout:6/221: mkdir d4/d13/d22/d34/d37/d47 0 2024-03-11T19:27:20.068 INFO:tasks.workunit.client.0.smithi090.stdout:8/221: symlink d0/d3/d26/dc/l41 0 2024-03-11T19:27:20.070 INFO:tasks.workunit.client.0.smithi090.stdout:1/200: dread d5/da/d1c/f39 [0,4194304] 0 2024-03-11T19:27:20.179 INFO:tasks.workunit.client.0.smithi090.stdout:0/214: dwrite d2/d11/d15/f19 [0,4194304] 0 2024-03-11T19:27:20.201 INFO:tasks.workunit.client.0.smithi090.stdout:7/193: write d3/f3b [591933,101660] 0 2024-03-11T19:27:20.395 INFO:tasks.workunit.client.0.smithi090.stdout:4/218: dwrite d4/d27/f2b [0,4194304] 0 2024-03-11T19:27:20.534 INFO:tasks.workunit.client.0.smithi090.stdout:2/198: creat d0/de/d1e/d29/f2e x:0 0 0 2024-03-11T19:27:20.565 INFO:tasks.workunit.client.0.smithi090.stdout:1/201: read d5/da/d1c/f39 [2974487,89145] 0 2024-03-11T19:27:20.593 INFO:tasks.workunit.client.0.smithi090.stdout:4/219: dread - d4/d18/f25 zero size 2024-03-11T19:27:20.653 INFO:tasks.workunit.client.0.smithi090.stdout:2/199: chown d0/c15 603 1 2024-03-11T19:27:20.736 INFO:tasks.workunit.client.0.smithi090.stdout:5/186: unlink d1/f16 0 2024-03-11T19:27:20.750 INFO:tasks.workunit.client.0.smithi090.stdout:5/187: readlink d1/l15 0 2024-03-11T19:27:20.976 INFO:tasks.workunit.client.0.smithi090.stdout:2/200: sync 2024-03-11T19:27:21.436 INFO:tasks.workunit.client.0.smithi090.stdout:6/222: truncate d4/d14/f21 3938579 0 2024-03-11T19:27:21.548 INFO:tasks.workunit.client.0.smithi090.stdout:7/194: symlink d3/l41 0 2024-03-11T19:27:21.913 INFO:tasks.workunit.client.0.smithi090.stdout:9/192: getdents d1 0 2024-03-11T19:27:21.931 INFO:tasks.workunit.client.0.smithi090.stdout:0/215: unlink d2/d11/d15/d29/f26 0 2024-03-11T19:27:21.950 INFO:tasks.workunit.client.0.smithi090.stdout:4/220: truncate d4/f2e 2932983 0 2024-03-11T19:27:21.955 INFO:tasks.workunit.client.0.smithi090.stdout:5/188: rmdir d1 39 2024-03-11T19:27:21.988 INFO:tasks.workunit.client.0.smithi090.stdout:8/222: mknod d0/d3/d26/c42 0 2024-03-11T19:27:22.077 INFO:tasks.workunit.client.0.smithi090.stdout:9/193: stat d1/dd/df/d1c/d21 0 2024-03-11T19:27:22.196 INFO:tasks.workunit.client.0.smithi090.stdout:6/223: rmdir d4/d13/d22 39 2024-03-11T19:27:22.307 INFO:tasks.workunit.client.0.smithi090.stdout:3/183: getdents d2/db/d15 0 2024-03-11T19:27:22.648 INFO:tasks.workunit.client.0.smithi090.stdout:0/216: unlink d2/d11/d15/f33 0 2024-03-11T19:27:23.085 INFO:tasks.workunit.client.0.smithi090.stdout:9/194: creat d1/dd/df/d1c/d21/d2a/f3d x:0 0 0 2024-03-11T19:27:23.193 INFO:tasks.workunit.client.0.smithi090.stdout:1/202: write d5/f15 [4824607,16963] 0 2024-03-11T19:27:23.784 INFO:tasks.workunit.client.0.smithi090.stdout:8/223: rename d0/d3/d26 to d0/d43 0 2024-03-11T19:27:23.880 INFO:tasks.workunit.client.0.smithi090.stdout:5/189: symlink d1/d26/l3a 0 2024-03-11T19:27:24.127 INFO:tasks.workunit.client.0.smithi090.stdout:7/195: dwrite d3/dc/d13/f18 [0,4194304] 0 2024-03-11T19:27:24.265 INFO:tasks.workunit.client.0.smithi090.stdout:9/195: fsync d1/dd/d18/f31 0 2024-03-11T19:27:24.618 INFO:tasks.workunit.client.0.smithi090.stdout:9/196: dread d1/dd/df/f10 [0,4194304] 0 2024-03-11T19:27:24.829 INFO:tasks.workunit.client.0.smithi090.stdout:2/201: getdents d0/de/d1f 0 2024-03-11T19:27:24.874 INFO:tasks.workunit.client.0.smithi090.stdout:2/202: read d0/de/f16 [1070424,64238] 0 2024-03-11T19:27:25.003 INFO:tasks.workunit.client.0.smithi090.stdout:4/221: rename d4/d27/f2b to d4/d27/d44/f4e 0 2024-03-11T19:27:25.169 INFO:tasks.workunit.client.0.smithi090.stdout:5/190: mkdir d1/d26/d19/d27/d3b 0 2024-03-11T19:27:25.183 INFO:tasks.workunit.client.0.smithi090.stdout:5/191: readlink d1/l12 0 2024-03-11T19:27:25.327 INFO:tasks.workunit.client.0.smithi090.stdout:5/192: dread d1/d26/f20 [0,4194304] 0 2024-03-11T19:27:25.507 INFO:tasks.workunit.client.0.smithi090.stdout:6/224: mkdir d4/d13/d22/d42/d48 0 2024-03-11T19:27:25.793 INFO:tasks.workunit.client.0.smithi090.stdout:7/196: fsync d3/dc/f33 0 2024-03-11T19:27:26.146 INFO:tasks.workunit.client.0.smithi090.stdout:9/197: fdatasync d1/dd/df/f33 0 2024-03-11T19:27:26.297 INFO:tasks.workunit.client.0.smithi090.stdout:2/203: chown d0/de/d12/f1d 322262958 1 2024-03-11T19:27:26.430 INFO:tasks.workunit.client.0.smithi090.stdout:1/203: dwrite d5/da/dd/f2c [0,4194304] 0 2024-03-11T19:27:26.630 INFO:tasks.workunit.client.0.smithi090.stdout:4/222: mknod d4/d27/d28/d38/c4f 0 2024-03-11T19:27:26.784 INFO:tasks.workunit.client.0.smithi090.stdout:0/217: creat d2/d11/f41 x:0 0 0 2024-03-11T19:27:27.007 INFO:tasks.workunit.client.0.smithi090.stdout:5/193: creat d1/d26/f3c x:0 0 0 2024-03-11T19:27:27.780 INFO:tasks.workunit.client.0.smithi090.stdout:3/184: rename d2/f19 to d2/d1f/f3f 0 2024-03-11T19:27:28.410 INFO:tasks.workunit.client.0.smithi090.stdout:4/223: creat d4/d27/d28/f50 x:0 0 0 2024-03-11T19:27:28.448 INFO:tasks.workunit.client.0.smithi090.stdout:0/218: dread - d2/d11/f24 zero size 2024-03-11T19:27:28.955 INFO:tasks.workunit.client.0.smithi090.stdout:8/224: getdents d0/d43/dc 0 2024-03-11T19:27:29.122 INFO:tasks.workunit.client.0.smithi090.stdout:7/197: rename d3/dc/f2d to d3/dc/d13/f42 0 2024-03-11T19:27:29.143 INFO:tasks.workunit.client.0.smithi090.stdout:7/198: chown d3/f8 54386351 1 2024-03-11T19:27:29.540 INFO:tasks.workunit.client.0.smithi090.stdout:9/198: dwrite d1/f9 [0,4194304] 0 2024-03-11T19:27:29.704 INFO:tasks.workunit.client.0.smithi090.stdout:1/204: mkdir d5/da/d1c/d3b/d40 0 2024-03-11T19:27:29.708 INFO:tasks.workunit.client.0.smithi090.stdout:9/199: dread - d1/dd/df/f33 zero size 2024-03-11T19:27:30.038 INFO:tasks.workunit.client.0.smithi090.stdout:5/194: mknod d1/c3d 0 2024-03-11T19:27:30.072 INFO:tasks.workunit.client.0.smithi090.stdout:6/225: link c2 d4/d13/d22/d34/d37/c49 0 2024-03-11T19:27:30.213 INFO:tasks.workunit.client.0.smithi090.stdout:8/225: creat d0/d43/dc/f44 x:0 0 0 2024-03-11T19:27:30.342 INFO:tasks.workunit.client.0.smithi090.stdout:7/199: rmdir d3/dc/d17 39 2024-03-11T19:27:30.367 INFO:tasks.workunit.client.0.smithi090.stdout:7/200: readlink d3/dc/d13/l14 0 2024-03-11T19:27:30.380 INFO:tasks.workunit.client.0.smithi090.stdout:7/201: stat d3/f7 0 2024-03-11T19:27:30.406 INFO:tasks.workunit.client.0.smithi090.stdout:7/202: chown d3/dc/d13/f35 5 1 2024-03-11T19:27:30.422 INFO:tasks.workunit.client.0.smithi090.stdout:7/203: readlink d3/dc/l1a 0 2024-03-11T19:27:30.433 INFO:tasks.workunit.client.0.smithi090.stdout:7/204: stat d3/ca 0 2024-03-11T19:27:30.472 INFO:tasks.workunit.client.0.smithi090.stdout:2/204: link d0/de/d1e/l20 d0/d21/l2f 0 2024-03-11T19:27:31.019 INFO:tasks.workunit.client.0.smithi090.stdout:1/205: dread d5/da/d1c/f30 [0,4194304] 0 2024-03-11T19:27:31.069 INFO:tasks.workunit.client.0.smithi090.stdout:4/224: mkdir d4/d51 0 2024-03-11T19:27:31.831 INFO:tasks.workunit.client.0.smithi090.stdout:2/205: truncate d0/fc 3928249 0 2024-03-11T19:27:32.124 INFO:tasks.workunit.client.0.smithi090.stdout:1/206: unlink d5/da/f19 0 2024-03-11T19:27:32.180 INFO:tasks.workunit.client.0.smithi090.stdout:1/207: write d5/da/dd/f1f [1913826,73446] 0 2024-03-11T19:27:32.261 INFO:tasks.workunit.client.0.smithi090.stdout:1/208: dread d5/da/d1c/f28 [0,4194304] 0 2024-03-11T19:27:32.328 INFO:tasks.workunit.client.0.smithi090.stdout:0/219: creat d2/d11/d3a/f42 x:0 0 0 2024-03-11T19:27:32.673 INFO:tasks.workunit.client.0.smithi090.stdout:4/225: mkdir d4/d51/d52 0 2024-03-11T19:27:33.049 INFO:tasks.workunit.client.0.smithi090.stdout:6/226: write d4/d13/f1b [3135134,73057] 0 2024-03-11T19:27:33.247 INFO:tasks.workunit.client.0.smithi090.stdout:3/185: dwrite d2/d1f/f3f [0,4194304] 0 2024-03-11T19:27:33.588 INFO:tasks.workunit.client.0.smithi090.stdout:9/200: dwrite d1/f3 [0,4194304] 0 2024-03-11T19:27:33.653 INFO:tasks.workunit.client.0.smithi090.stdout:6/227: read - d4/d13/f1f zero size 2024-03-11T19:27:33.846 INFO:tasks.workunit.client.0.smithi090.stdout:1/209: dread d5/da/d1c/f21 [0,4194304] 0 2024-03-11T19:27:33.884 INFO:tasks.workunit.client.0.smithi090.stdout:0/220: write d2/f14 [4978660,101398] 0 2024-03-11T19:27:33.905 INFO:tasks.workunit.client.0.smithi090.stdout:2/206: dread d0/f6 [0,4194304] 0 2024-03-11T19:27:33.917 INFO:tasks.workunit.client.0.smithi090.stdout:7/205: write d3/dc/d13/f35 [830761,33451] 0 2024-03-11T19:27:33.932 INFO:tasks.workunit.client.0.smithi090.stdout:4/226: write d4/d18/f2a [753143,31056] 0 2024-03-11T19:27:34.172 INFO:tasks.workunit.client.0.smithi090.stdout:6/228: readlink d4/d24/l2e 0 2024-03-11T19:27:34.203 INFO:tasks.workunit.client.0.smithi090.stdout:2/207: readlink d0/l8 0 2024-03-11T19:27:34.231 INFO:tasks.workunit.client.0.smithi090.stdout:5/195: dwrite d1/d26/f6 [4194304,4194304] 0 2024-03-11T19:27:34.419 INFO:tasks.workunit.client.0.smithi090.stdout:3/186: mknod d2/d1d/c40 0 2024-03-11T19:27:34.516 INFO:tasks.workunit.client.0.smithi090.stdout:7/206: dread - d3/dc/f3c zero size 2024-03-11T19:27:34.542 INFO:tasks.workunit.client.0.smithi090.stdout:8/226: dwrite d0/f2a [0,4194304] 0 2024-03-11T19:27:34.651 INFO:tasks.workunit.client.0.smithi090.stdout:1/210: symlink d5/da/d1c/d3b/l41 0 2024-03-11T19:27:34.731 INFO:tasks.workunit.client.0.smithi090.stdout:7/207: chown d3/dc/f3a 88763783 1 2024-03-11T19:27:34.736 INFO:tasks.workunit.client.0.smithi090.stdout:9/201: mknod d1/dd/df/c3e 0 2024-03-11T19:27:34.758 INFO:tasks.workunit.client.0.smithi090.stdout:5/196: creat d1/d26/f3e x:0 0 0 2024-03-11T19:27:34.782 INFO:tasks.workunit.client.0.smithi090.stdout:8/227: chown d0/d3/l25 487181958 1 2024-03-11T19:27:34.917 INFO:tasks.workunit.client.0.smithi090.stdout:4/227: write d4/d27/d28/f30 [5221325,113111] 0 2024-03-11T19:27:34.930 INFO:tasks.workunit.client.0.smithi090.stdout:3/187: fdatasync d2/d1f/f3b 0 2024-03-11T19:27:34.946 INFO:tasks.workunit.client.0.smithi090.stdout:0/221: dwrite d2/f14 [0,4194304] 0 2024-03-11T19:27:35.158 INFO:tasks.workunit.client.0.smithi090.stdout:6/229: write d4/d24/f3f [66738,56148] 0 2024-03-11T19:27:35.341 INFO:tasks.workunit.client.0.smithi090.stdout:7/208: chown d3/dc/d17/d32/c38 821114989 1 2024-03-11T19:27:35.419 INFO:tasks.workunit.client.0.smithi090.stdout:9/202: write d1/dd/f3c [221195,57803] 0 2024-03-11T19:27:35.569 INFO:tasks.workunit.client.0.smithi090.stdout:5/197: truncate d1/d26/f2d 469310 0 2024-03-11T19:27:35.683 INFO:tasks.workunit.client.0.smithi090.stdout:8/228: dread d0/f2a [0,4194304] 0 2024-03-11T19:27:35.777 INFO:tasks.workunit.client.0.smithi090.stdout:5/198: stat d1/l15 0 2024-03-11T19:27:35.895 INFO:tasks.workunit.client.0.smithi090.stdout:2/208: dwrite d0/fd [4194304,4194304] 0 2024-03-11T19:27:36.005 INFO:tasks.workunit.client.0.smithi090.stdout:9/203: creat d1/dd/f3f x:0 0 0 2024-03-11T19:27:36.261 INFO:tasks.workunit.client.0.smithi090.stdout:8/229: rmdir d0/d43 39 2024-03-11T19:27:36.358 INFO:tasks.workunit.client.0.smithi090.stdout:3/188: dread d2/f2c [0,4194304] 0 2024-03-11T19:27:36.390 INFO:tasks.workunit.client.0.smithi090.stdout:9/204: sync 2024-03-11T19:27:36.420 INFO:tasks.workunit.client.0.smithi090.stdout:6/230: rename d4/d24/l2e to d4/d13/l4a 0 2024-03-11T19:27:36.457 INFO:tasks.workunit.client.0.smithi090.stdout:1/211: dwrite d5/da/d23/f2b [4194304,4194304] 0 2024-03-11T19:27:36.787 INFO:tasks.workunit.client.0.smithi090.stdout:8/230: chown d0/f31 190253 1 2024-03-11T19:27:36.811 INFO:tasks.workunit.client.0.smithi090.stdout:4/228: dwrite d4/d27/d44/f4e [4194304,4194304] 0 2024-03-11T19:27:36.992 INFO:tasks.workunit.client.0.smithi090.stdout:9/205: rename d1 to d1/dd/d18/d32/d40 22 2024-03-11T19:27:37.029 INFO:tasks.workunit.client.0.smithi090.stdout:3/189: mknod d2/de/c41 0 2024-03-11T19:27:37.116 INFO:tasks.workunit.client.0.smithi090.stdout:7/209: dwrite d3/dc/d13/f21 [0,4194304] 0 2024-03-11T19:27:37.152 INFO:tasks.workunit.client.0.smithi090.stdout:1/212: read d5/f15 [3846463,52068] 0 2024-03-11T19:27:37.292 INFO:tasks.workunit.client.0.smithi090.stdout:6/231: symlink d4/d13/d22/d3a/l4b 0 2024-03-11T19:27:37.319 INFO:tasks.workunit.client.0.smithi090.stdout:5/199: truncate d1/d26/f6 3418603 0 2024-03-11T19:27:37.544 INFO:tasks.workunit.client.0.smithi090.stdout:4/229: write d4/d27/d28/f30 [112935,15226] 0 2024-03-11T19:27:37.552 INFO:tasks.workunit.client.0.smithi090.stdout:3/190: mknod d2/d1d/c42 0 2024-03-11T19:27:37.605 INFO:tasks.workunit.client.0.smithi090.stdout:2/209: dwrite d0/f1b [0,4194304] 0 2024-03-11T19:27:37.754 INFO:tasks.workunit.client.0.smithi090.stdout:0/222: dwrite d2/d11/d15/d29/d2e/f32 [0,4194304] 0 2024-03-11T19:27:37.962 INFO:tasks.workunit.client.0.smithi090.stdout:1/213: write d5/da/d1c/f39 [2944826,65471] 0 2024-03-11T19:27:38.036 INFO:tasks.workunit.client.0.smithi090.stdout:6/232: read d4/d14/f23 [898924,94752] 0 2024-03-11T19:27:38.166 INFO:tasks.workunit.client.0.smithi090.stdout:2/210: chown d0/de/d1f/l2d 41127 1 2024-03-11T19:27:38.284 INFO:tasks.workunit.client.0.smithi090.stdout:4/230: mknod d4/d51/c53 0 2024-03-11T19:27:38.400 INFO:tasks.workunit.client.0.smithi090.stdout:5/200: unlink d1/c28 0 2024-03-11T19:27:38.403 INFO:tasks.workunit.client.0.smithi090.stdout:3/191: rmdir d2/db/d15 39 2024-03-11T19:27:38.467 INFO:tasks.workunit.client.0.smithi090.stdout:8/231: dwrite d0/f2d [0,4194304] 0 2024-03-11T19:27:38.692 INFO:tasks.workunit.client.0.smithi090.stdout:8/232: sync 2024-03-11T19:27:38.709 INFO:tasks.workunit.client.0.smithi090.stdout:7/210: dread d3/dc/d17/d32/f3f [0,4194304] 0 2024-03-11T19:27:39.118 INFO:tasks.workunit.client.0.smithi090.stdout:9/206: truncate d1/dd/d18/d32/f34 4309430 0 2024-03-11T19:27:39.152 INFO:tasks.workunit.client.0.smithi090.stdout:6/233: truncate d4/ff 3591195 0 2024-03-11T19:27:39.152 INFO:tasks.workunit.client.0.smithi090.stdout:1/214: readlink d5/da/lf 0 2024-03-11T19:27:39.162 INFO:tasks.workunit.client.0.smithi090.stdout:4/231: symlink d4/d18/l54 0 2024-03-11T19:27:39.234 INFO:tasks.workunit.client.0.smithi090.stdout:9/207: read - d1/dd/f3f zero size 2024-03-11T19:27:39.235 INFO:tasks.workunit.client.0.smithi090.stdout:4/232: readlink d4/d27/d28/l32 0 2024-03-11T19:27:39.252 INFO:tasks.workunit.client.0.smithi090.stdout:1/215: readlink d5/da/l2e 0 2024-03-11T19:27:39.433 INFO:tasks.workunit.client.0.smithi090.stdout:2/211: rename d0/de/d1e to d0/d21/d30 0 2024-03-11T19:27:39.463 INFO:tasks.workunit.client.0.smithi090.stdout:4/233: rename d4 to d4/d51/d55 22 2024-03-11T19:27:39.516 INFO:tasks.workunit.client.0.smithi090.stdout:8/233: chown d0/d43/dc/f3b 52998329 1 2024-03-11T19:27:39.548 INFO:tasks.workunit.client.0.smithi090.stdout:7/211: mknod d3/dc/d17/d34/c43 0 2024-03-11T19:27:39.581 INFO:tasks.workunit.client.0.smithi090.stdout:8/234: chown d0/d43/dc/c3d 1842515 1 2024-03-11T19:27:39.774 INFO:tasks.workunit.client.0.smithi090.stdout:6/234: rmdir d4/d13/d22/d42 39 2024-03-11T19:27:39.797 INFO:tasks.workunit.client.0.smithi090.stdout:5/201: truncate d1/d26/f6 3941064 0 2024-03-11T19:27:39.830 INFO:tasks.workunit.client.0.smithi090.stdout:2/212: creat d0/d21/d30/f31 x:0 0 0 2024-03-11T19:27:39.911 INFO:tasks.workunit.client.0.smithi090.stdout:7/212: mknod d3/dc/d13/c44 0 2024-03-11T19:27:39.925 INFO:tasks.workunit.client.0.smithi090.stdout:8/235: mknod d0/d3/c45 0 2024-03-11T19:27:40.571 INFO:tasks.workunit.client.0.smithi090.stdout:5/202: creat d1/d26/f3f x:0 0 0 2024-03-11T19:27:40.746 INFO:tasks.workunit.client.0.smithi090.stdout:2/213: creat d0/de/d12/f32 x:0 0 0 2024-03-11T19:27:41.041 INFO:tasks.workunit.client.0.smithi090.stdout:0/223: dwrite d2/d11/f18 [0,4194304] 0 2024-03-11T19:27:41.056 INFO:tasks.workunit.client.0.smithi090.stdout:7/213: mkdir d3/dc/d17/d32/d45 0 2024-03-11T19:27:41.399 INFO:tasks.workunit.client.0.smithi090.stdout:9/208: link d1/l20 d1/d1d/l41 0 2024-03-11T19:27:41.491 INFO:tasks.workunit.client.0.smithi090.stdout:3/192: link d2/db/d15/d1a/l26 d2/db/d15/d1a/d3c/l43 0 2024-03-11T19:27:41.566 INFO:tasks.workunit.client.0.smithi090.stdout:4/234: link d4/l13 d4/d51/l56 0 2024-03-11T19:27:41.743 INFO:tasks.workunit.client.0.smithi090.stdout:1/216: write d5/f37 [688554,18743] 0 2024-03-11T19:27:41.923 INFO:tasks.workunit.client.0.smithi090.stdout:2/214: rmdir d0/de 39 2024-03-11T19:27:41.945 INFO:tasks.workunit.client.0.smithi090.stdout:8/236: mknod d0/d3/d3e/c46 0 2024-03-11T19:27:42.137 INFO:tasks.workunit.client.0.smithi090.stdout:8/237: dread d0/d43/dc/f12 [0,4194304] 0 2024-03-11T19:27:42.758 INFO:tasks.workunit.client.0.smithi090.stdout:9/209: mknod d1/dd/df/d1c/c42 0 2024-03-11T19:27:43.021 INFO:tasks.workunit.client.0.smithi090.stdout:6/235: rename d4/d14/d25/l3e to d4/d13/d22/l4c 0 2024-03-11T19:27:43.037 INFO:tasks.workunit.client.0.smithi090.stdout:5/203: rename d1/d17 to d1/d17/d39/d40 22 2024-03-11T19:27:43.202 INFO:tasks.workunit.client.0.smithi090.stdout:1/217: truncate d5/d31/f3f 964857 0 2024-03-11T19:27:43.771 INFO:tasks.workunit.client.0.smithi090.stdout:8/238: rmdir d0/d3 39 2024-03-11T19:27:43.792 INFO:tasks.workunit.client.0.smithi090.stdout:8/239: chown d0/d43/f23 1 1 2024-03-11T19:27:44.259 INFO:tasks.workunit.client.0.smithi090.stdout:5/204: fsync d1/d26/f2b 0 2024-03-11T19:27:44.570 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:27:44.573 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:27:44.595 INFO:tasks.workunit.client.0.smithi090.stdout:0/224: write d2/d11/d3f/f40 [559912,24966] 0 2024-03-11T19:27:44.778 INFO:tasks.workunit.client.0.smithi090.stdout:7/214: dwrite d3/dc/d13/f24 [0,4194304] 0 2024-03-11T19:27:44.899 INFO:tasks.workunit.client.0.smithi090.stdout:7/215: chown d3/dc/d1e/l39 0 1 2024-03-11T19:27:45.418 INFO:tasks.workunit.client.0.smithi090.stdout:8/240: chown d0/d43/c36 3233632 1 2024-03-11T19:27:45.458 INFO:tasks.workunit.client.0.smithi090.stdout:4/235: dwrite d4/f4a [0,4194304] 0 2024-03-11T19:27:45.576 INFO:tasks.workunit.client.0.smithi090.stdout:9/210: symlink d1/dd/d18/d3a/l43 0 2024-03-11T19:27:45.630 INFO:tasks.workunit.client.0.smithi090.stdout:4/236: chown d4/f46 12602 1 2024-03-11T19:27:46.073 INFO:tasks.workunit.client.0.smithi090.stdout:0/225: mkdir d2/d11/d15/d29/d43 0 2024-03-11T19:27:46.253 INFO:tasks.workunit.client.0.smithi090.stdout:7/216: truncate d3/f22 3473108 0 2024-03-11T19:27:46.713 INFO:tasks.workunit.client.0.smithi090.stdout:8/241: rmdir d0 39 2024-03-11T19:27:46.915 INFO:tasks.workunit.client.0.smithi090.stdout:3/193: rename d2/de/c41 to d2/db/c44 0 2024-03-11T19:27:47.209 INFO:tasks.workunit.client.0.smithi090.stdout:0/226: rmdir d2/d11/d15/d29/d2e 39 2024-03-11T19:27:47.643 INFO:tasks.workunit.client.0.smithi090.stdout:1/218: dwrite d5/da/dd/f1f [0,4194304] 0 2024-03-11T19:27:47.790 INFO:tasks.workunit.client.0.smithi090.stdout:4/237: creat d4/d27/f57 x:0 0 0 2024-03-11T19:27:47.866 INFO:tasks.workunit.client.0.smithi090.stdout:2/215: dwrite d0/de/d12/f1d [0,4194304] 0 2024-03-11T19:27:48.067 INFO:tasks.workunit.client.0.smithi090.stdout:4/238: chown d4/d18/f37 1378 1 2024-03-11T19:27:48.280 INFO:tasks.workunit.client.0.smithi090.stdout:5/205: unlink d1/c3d 0 2024-03-11T19:27:48.499 INFO:tasks.workunit.client.0.smithi090.stdout:6/236: mknod d4/c4d 0 2024-03-11T19:27:48.509 INFO:tasks.workunit.client.0.smithi090.stdout:9/211: write d1/dd/df/d1c/d21/d2a/f29 [5445017,84121] 0 2024-03-11T19:27:48.574 INFO:tasks.workunit.client.0.smithi090.stdout:6/237: chown d4/d14/l19 116 1 2024-03-11T19:27:48.893 INFO:tasks.workunit.client.0.smithi090.stdout:7/217: dwrite d3/f27 [0,4194304] 0 2024-03-11T19:27:49.033 INFO:tasks.workunit.client.0.smithi090.stdout:0/227: creat d2/d11/d15/d29/d43/f44 x:0 0 0 2024-03-11T19:27:49.348 INFO:tasks.workunit.client.0.smithi090.stdout:8/242: read d0/f2a [3851840,97019] 0 2024-03-11T19:27:49.446 INFO:tasks.workunit.client.0.smithi090.stdout:3/194: truncate d2/d2f/f31 1006152 0 2024-03-11T19:27:49.466 INFO:tasks.workunit.client.0.smithi090.stdout:3/195: stat d2/d1f/f36 0 2024-03-11T19:27:49.589 INFO:tasks.workunit.client.0.smithi090.stdout:0/228: sync 2024-03-11T19:27:49.995 INFO:tasks.workunit.client.0.smithi090.stdout:6/238: fdatasync d4/f46 0 2024-03-11T19:27:50.067 INFO:tasks.workunit.client.0.smithi090.stdout:5/206: dread d1/f1e [0,4194304] 0 2024-03-11T19:27:50.118 INFO:tasks.workunit.client.0.smithi090.stdout:6/239: sync 2024-03-11T19:27:50.131 INFO:tasks.workunit.client.0.smithi090.stdout:6/240: chown d4/l8 15650 1 2024-03-11T19:27:50.317 INFO:tasks.workunit.client.0.smithi090.stdout:7/218: creat d3/dc/d1e/f46 x:0 0 0 2024-03-11T19:27:50.765 INFO:tasks.workunit.client.0.smithi090.stdout:1/219: link d5/da/d1c/f39 d5/da/d1c/d3b/f42 0 2024-03-11T19:27:50.792 INFO:tasks.workunit.client.0.smithi090.stdout:1/220: chown d5/da/l2e 1470153 1 2024-03-11T19:27:50.956 INFO:tasks.workunit.client.0.smithi090.stdout:9/212: link d1/dd/d18/d25/l3b d1/dd/d18/d25/l44 0 2024-03-11T19:27:51.170 INFO:tasks.workunit.client.0.smithi090.stdout:0/229: symlink d2/d11/d3f/l45 0 2024-03-11T19:27:51.428 INFO:tasks.workunit.client.0.smithi090.stdout:2/216: dwrite d0/d21/f2a [0,4194304] 0 2024-03-11T19:27:51.533 INFO:tasks.workunit.client.0.smithi090.stdout:7/219: read d3/dc/d13/f24 [998255,350] 0 2024-03-11T19:27:51.693 INFO:tasks.workunit.client.0.smithi090.stdout:9/213: creat d1/dd/d18/d32/f45 x:0 0 0 2024-03-11T19:27:51.747 INFO:tasks.workunit.client.0.smithi090.stdout:0/230: truncate f0 5235848 0 2024-03-11T19:27:51.889 INFO:tasks.workunit.client.0.smithi090.stdout:8/243: creat d0/f47 x:0 0 0 2024-03-11T19:27:51.895 INFO:tasks.workunit.client.0.smithi090.stdout:3/196: rmdir d2/db/d15/d1a 39 2024-03-11T19:27:51.982 INFO:tasks.workunit.client.0.smithi090.stdout:4/239: dwrite d4/f12 [4194304,4194304] 0 2024-03-11T19:27:52.350 INFO:tasks.workunit.client.0.smithi090.stdout:1/221: creat d5/da/d1c/d3b/d40/f43 x:0 0 0 2024-03-11T19:27:52.384 INFO:tasks.workunit.client.0.smithi090.stdout:9/214: mknod d1/dd/d18/d25/c46 0 2024-03-11T19:27:52.386 INFO:tasks.workunit.client.0.smithi090.stdout:2/217: fdatasync d0/f2 0 2024-03-11T19:27:52.451 INFO:tasks.workunit.client.0.smithi090.stdout:6/241: write d4/d13/f1f [382362,40127] 0 2024-03-11T19:27:52.850 INFO:tasks.workunit.client.0.smithi090.stdout:5/207: dwrite d1/f38 [0,4194304] 0 2024-03-11T19:27:52.982 INFO:tasks.workunit.client.0.smithi090.stdout:0/231: creat d2/d11/d15/d36/f46 x:0 0 0 2024-03-11T19:27:53.097 INFO:tasks.workunit.client.0.smithi090.stdout:9/215: mkdir d1/d47 0 2024-03-11T19:27:53.304 INFO:tasks.workunit.client.0.smithi090.stdout:3/197: mkdir d2/d45 0 2024-03-11T19:27:53.452 INFO:tasks.workunit.client.0.smithi090.stdout:2/218: mknod d0/de/d12/c33 0 2024-03-11T19:27:53.647 INFO:tasks.workunit.client.0.smithi090.stdout:6/242: readlink d4/d13/l4a 0 2024-03-11T19:27:53.735 INFO:tasks.workunit.client.0.smithi090.stdout:7/220: write d3/dc/d1e/f12 [5809660,113620] 0 2024-03-11T19:27:53.914 INFO:tasks.workunit.client.0.smithi090.stdout:8/244: mknod d0/d40/c48 0 2024-03-11T19:27:54.136 INFO:tasks.workunit.client.0.smithi090.stdout:5/208: creat d1/d17/f41 x:0 0 0 2024-03-11T19:27:54.343 INFO:tasks.workunit.client.0.smithi090.stdout:1/222: write d5/f37 [1453896,7456] 0 2024-03-11T19:27:54.366 INFO:tasks.workunit.client.0.smithi090.stdout:4/240: dwrite d4/d18/f25 [0,4194304] 0 2024-03-11T19:27:54.769 INFO:tasks.workunit.client.0.smithi090.stdout:3/198: truncate d2/db/f1c 1137210 0 2024-03-11T19:27:54.784 INFO:tasks.workunit.client.0.smithi090.stdout:1/223: dread d5/da/dd/f1a [8388608,4194304] 0 2024-03-11T19:27:55.079 INFO:tasks.workunit.client.0.smithi090.stdout:6/243: unlink d4/d13/d22/c29 0 2024-03-11T19:27:55.237 INFO:tasks.workunit.client.0.smithi090.stdout:7/221: mkdir d3/d47 0 2024-03-11T19:27:55.492 INFO:tasks.workunit.client.0.smithi090.stdout:5/209: creat d1/d26/d19/f42 x:0 0 0 2024-03-11T19:27:56.188 INFO:tasks.workunit.client.0.smithi090.stdout:3/199: fsync d2/f29 0 2024-03-11T19:27:56.428 INFO:tasks.workunit.client.0.smithi090.stdout:2/219: mknod d0/c34 0 2024-03-11T19:27:56.943 INFO:tasks.workunit.client.0.smithi090.stdout:5/210: creat d1/d26/f43 x:0 0 0 2024-03-11T19:27:56.989 INFO:tasks.workunit.client.0.smithi090.stdout:5/211: write d1/f38 [2779295,63148] 0 2024-03-11T19:27:57.167 INFO:tasks.workunit.client.0.smithi090.stdout:0/232: rename d2/d11/d15/f23 to d2/d11/f47 0 2024-03-11T19:27:57.483 INFO:tasks.workunit.client.0.smithi090.stdout:9/216: dread d1/dd/f13 [0,4194304] 0 2024-03-11T19:27:57.512 INFO:tasks.workunit.client.0.smithi090.stdout:8/245: write d0/d43/f2f [1022347,94213] 0 2024-03-11T19:27:57.585 INFO:tasks.workunit.client.0.smithi090.stdout:9/217: chown d1/dd/d18/d25/c46 0 1 2024-03-11T19:27:57.733 INFO:tasks.workunit.client.0.smithi090.stdout:9/218: dread d1/dd/d18/d32/f34 [0,4194304] 0 2024-03-11T19:27:57.818 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:53:17.576 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:27:57.856 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:53:17.614 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:27:57.858 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:53:17.546 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:27:57.862 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:53:17.558 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:27:57.875 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:53:17.563 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:27:57.886 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:53:17.582 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:27:58.018 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:53:17.706 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:27:58.020 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:53:17.716 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:27:58.072 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:53:17.831 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:27:58.087 INFO:tasks.workunit.client.0.smithi090.stdout:5/212: creat d1/d26/d19/d27/f44 x:0 0 0 2024-03-11T19:27:58.296 INFO:tasks.workunit.client.0.smithi090.stdout:4/241: dwrite d4/f24 [0,4194304] 0 2024-03-11T19:27:58.407 INFO:tasks.workunit.client.0.smithi090.stdout:1/224: write d5/da/dd/f1a [2513671,28775] 0 2024-03-11T19:27:58.710 INFO:tasks.workunit.client.0.smithi090.stdout:3/200: rmdir d2/d2f 39 2024-03-11T19:27:59.080 INFO:tasks.workunit.client.0.smithi090.stdout:7/222: dwrite d3/dc/fe [0,4194304] 0 2024-03-11T19:27:59.221 INFO:tasks.workunit.client.0.smithi090.stdout:2/220: write d0/f7 [9532877,16429] 0 2024-03-11T19:27:59.238 INFO:tasks.workunit.client.0.smithi090.stdout:6/244: write d4/ff [3620330,74384] 0 2024-03-11T19:27:59.280 INFO:tasks.workunit.client.0.smithi090.stdout:9/219: unlink d1/dd/df/c26 0 2024-03-11T19:28:00.158 INFO:tasks.workunit.client.0.smithi090.stdout:8/246: symlink d0/d3/l49 0 2024-03-11T19:28:00.291 INFO:tasks.workunit.client.0.smithi090.stdout:7/223: chown d3/dc/d17/d32/f3f 0 1 2024-03-11T19:28:00.363 INFO:tasks.workunit.client.0.smithi090.stdout:2/221: creat d0/d21/d30/f35 x:0 0 0 2024-03-11T19:28:00.437 INFO:tasks.workunit.client.0.smithi090.stdout:7/224: dread d3/f22 [0,4194304] 0 2024-03-11T19:28:01.501 INFO:tasks.workunit.client.0.smithi090.stdout:0/233: dwrite d2/f8 [0,4194304] 0 2024-03-11T19:28:02.356 INFO:tasks.workunit.client.0.smithi090.stdout:3/201: dwrite d2/ff [0,4194304] 0 2024-03-11T19:28:02.518 INFO:tasks.workunit.client.0.smithi090.stdout:6/245: write d4/d13/d22/d34/f36 [1614413,46924] 0 2024-03-11T19:28:02.778 INFO:tasks.workunit.client.0.smithi090.stdout:0/234: symlink d2/d11/d15/d29/l48 0 2024-03-11T19:28:02.834 INFO:tasks.workunit.client.0.smithi090.stdout:1/225: write d5/da/d23/f2d [5114794,78800] 0 2024-03-11T19:28:02.839 INFO:tasks.workunit.client.0.smithi090.stdout:4/242: symlink d4/l58 0 2024-03-11T19:28:02.868 INFO:tasks.workunit.client.0.smithi090.stdout:5/213: write d1/d26/f6 [2064426,129598] 0 2024-03-11T19:28:02.957 INFO:tasks.workunit.client.0.smithi090.stdout:2/222: write d0/d21/d30/f31 [540134,39048] 0 2024-03-11T19:28:03.171 INFO:tasks.workunit.client.0.smithi090.stdout:9/220: getdents d1/dd 0 2024-03-11T19:28:03.335 INFO:tasks.workunit.client.0.smithi090.stdout:2/223: sync 2024-03-11T19:28:03.447 INFO:tasks.workunit.client.0.smithi090.stdout:7/225: dwrite d3/dc/d1e/f26 [0,4194304] 0 2024-03-11T19:28:03.595 INFO:tasks.workunit.client.0.smithi090.stdout:8/247: dwrite d0/d3/f8 [0,4194304] 0 2024-03-11T19:28:03.609 INFO:tasks.workunit.client.0.smithi090.stdout:0/235: dread d2/d11/f17 [0,4194304] 0 2024-03-11T19:28:03.706 INFO:tasks.workunit.client.0.smithi090.stdout:4/243: symlink d4/l59 0 2024-03-11T19:28:03.875 INFO:tasks.workunit.client.0.smithi090.stdout:3/202: chown d2/db/d15/d1a/d3c/l43 179747641 1 2024-03-11T19:28:03.985 INFO:tasks.workunit.client.0.smithi090.stdout:5/214: mknod d1/d17/d39/c45 0 2024-03-11T19:28:03.992 INFO:tasks.workunit.client.0.smithi090.stdout:6/246: mkdir d4/d13/d22/d42/d48/d4e 0 2024-03-11T19:28:04.018 INFO:tasks.workunit.client.0.smithi090.stdout:7/226: dread d3/dc/d1e/f26 [0,4194304] 0 2024-03-11T19:28:04.089 INFO:tasks.workunit.client.0.smithi090.stdout:7/227: stat d3/ca 0 2024-03-11T19:28:04.102 INFO:tasks.workunit.client.0.smithi090.stdout:9/221: read d1/d1d/f2b [4345062,128289] 0 2024-03-11T19:28:04.212 INFO:tasks.workunit.client.0.smithi090.stdout:8/248: creat d0/d43/dc/f4a x:0 0 0 2024-03-11T19:28:04.285 INFO:tasks.workunit.client.0.smithi090.stdout:9/222: dread d1/dd/f11 [4194304,4194304] 0 2024-03-11T19:28:04.390 INFO:tasks.workunit.client.0.smithi090.stdout:4/244: symlink d4/d51/l5a 0 2024-03-11T19:28:04.780 INFO:tasks.workunit.client.0.smithi090.stdout:3/203: truncate d2/f8 173146 0 2024-03-11T19:28:04.936 INFO:tasks.workunit.client.0.smithi090.stdout:6/247: mknod d4/d13/d22/d3a/c4f 0 2024-03-11T19:28:04.974 INFO:tasks.workunit.client.0.smithi090.stdout:6/248: chown d4/d13/d22/d42/d48/d4e 1 1 2024-03-11T19:28:05.187 INFO:tasks.workunit.client.0.smithi090.stdout:7/228: mkdir d3/dc/d13/d48 0 2024-03-11T19:28:05.205 INFO:tasks.workunit.client.0.smithi090.stdout:7/229: chown d3/f27 461440618 1 2024-03-11T19:28:05.384 INFO:tasks.workunit.client.0.smithi090.stdout:9/223: mkdir d1/dd/d18/d3a/d48 0 2024-03-11T19:28:05.452 INFO:tasks.workunit.client.0.smithi090.stdout:4/245: mknod d4/d27/c5b 0 2024-03-11T19:28:05.628 INFO:tasks.workunit.client.0.smithi090.stdout:4/246: sync 2024-03-11T19:28:05.823 INFO:tasks.workunit.client.0.smithi090.stdout:8/249: mknod d0/d43/c4b 0 2024-03-11T19:28:05.885 INFO:tasks.workunit.client.0.smithi090.stdout:8/250: read - d0/d43/dc/f3b zero size 2024-03-11T19:28:06.105 INFO:tasks.workunit.client.0.smithi090.stdout:0/236: link d2/d11/d15/c21 d2/d11/d15/d29/d3d/c49 0 2024-03-11T19:28:06.109 INFO:tasks.workunit.client.0.smithi090.stdout:9/224: readlink d1/d1d/l41 0 2024-03-11T19:28:06.152 INFO:tasks.workunit.client.0.smithi090.stdout:0/237: dread - d2/d11/f24 zero size 2024-03-11T19:28:06.302 INFO:tasks.workunit.client.0.smithi090.stdout:4/247: truncate d4/f4a 1418478 0 2024-03-11T19:28:06.462 INFO:tasks.workunit.client.0.smithi090.stdout:2/224: getdents d0/d21/d30/d29 0 2024-03-11T19:28:06.493 INFO:tasks.workunit.client.0.smithi090.stdout:2/225: chown d0/de/d12/c33 161054341 1 2024-03-11T19:28:06.640 INFO:tasks.workunit.client.0.smithi090.stdout:1/226: creat d5/da/f44 x:0 0 0 2024-03-11T19:28:06.817 INFO:tasks.workunit.client.0.smithi090.stdout:9/225: fsync d1/dd/f13 0 2024-03-11T19:28:07.161 INFO:tasks.workunit.client.0.smithi090.stdout:4/248: creat d4/d27/d44/f5c x:0 0 0 2024-03-11T19:28:07.183 INFO:tasks.workunit.client.0.smithi090.stdout:3/204: creat d2/f46 x:0 0 0 2024-03-11T19:28:07.344 INFO:tasks.workunit.client.0.smithi090.stdout:5/215: rename d1/d26/d19/d27/c2e to d1/c46 0 2024-03-11T19:28:07.364 INFO:tasks.workunit.client.0.smithi090.stdout:5/216: readlink d1/d26/lb 0 2024-03-11T19:28:07.381 INFO:tasks.workunit.client.0.smithi090.stdout:5/217: chown d1/l33 4254 1 2024-03-11T19:28:07.410 INFO:tasks.workunit.client.0.smithi090.stdout:5/218: dread - d1/d26/f11 zero size 2024-03-11T19:28:07.434 INFO:tasks.workunit.client.0.smithi090.stdout:5/219: read - d1/f2c zero size 2024-03-11T19:28:07.456 INFO:tasks.workunit.client.0.smithi090.stdout:2/226: mknod d0/de/d12/c36 0 2024-03-11T19:28:07.739 INFO:tasks.workunit.client.0.smithi090.stdout:1/227: truncate d5/da/dd/f14 1198008 0 2024-03-11T19:28:07.922 INFO:tasks.workunit.client.0.smithi090.stdout:6/249: write d4/fb [675680,53308] 0 2024-03-11T19:28:08.104 INFO:tasks.workunit.client.0.smithi090.stdout:4/249: rmdir d4/d51 39 2024-03-11T19:28:08.281 INFO:tasks.workunit.client.0.smithi090.stdout:3/205: mkdir d2/d1d/d47 0 2024-03-11T19:28:08.442 INFO:tasks.workunit.client.0.smithi090.stdout:2/227: readlink d0/d21/d30/l20 0 2024-03-11T19:28:08.566 INFO:tasks.workunit.client.0.smithi090.stdout:9/226: dread d1/dd/d18/d25/f37 [0,4194304] 0 2024-03-11T19:28:08.687 INFO:tasks.workunit.client.0.smithi090.stdout:9/227: chown d1/d1d 101157 1 2024-03-11T19:28:08.697 INFO:tasks.workunit.client.0.smithi090.stdout:7/230: dwrite d3/dc/d13/f24 [4194304,4194304] 0 2024-03-11T19:28:09.032 INFO:tasks.workunit.client.0.smithi090.stdout:0/238: creat d2/d11/d15/d29/f4a x:0 0 0 2024-03-11T19:28:09.409 INFO:tasks.workunit.client.0.smithi090.stdout:8/251: dwrite d0/d43/dc/f3b [0,4194304] 0 2024-03-11T19:28:09.538 INFO:tasks.workunit.client.0.smithi090.stdout:1/228: dread d5/da/d23/f2b [0,4194304] 0 2024-03-11T19:28:09.637 INFO:tasks.workunit.client.0.smithi090.stdout:4/250: fdatasync d4/d27/f4d 0 2024-03-11T19:28:09.669 INFO:tasks.workunit.client.0.smithi090.stdout:3/206: dread - d2/db/d15/d1a/f3a zero size 2024-03-11T19:28:10.390 INFO:tasks.workunit.client.0.smithi090.stdout:8/252: mkdir d0/d3/d4c 0 2024-03-11T19:28:10.667 INFO:tasks.workunit.client.0.smithi090.stdout:5/220: link d1/d26/lb d1/d26/l47 0 2024-03-11T19:28:11.287 INFO:tasks.workunit.client.0.smithi090.stdout:0/239: mknod d2/d11/d15/d29/c4b 0 2024-03-11T19:28:11.518 INFO:tasks.workunit.client.0.smithi090.stdout:6/250: dwrite d4/f11 [0,4194304] 0 2024-03-11T19:28:11.628 INFO:tasks.workunit.client.0.smithi090.stdout:6/251: readlink d4/d13/l4a 0 2024-03-11T19:28:11.738 INFO:tasks.workunit.client.0.smithi090.stdout:1/229: mkdir d5/d3c/d45 0 2024-03-11T19:28:11.822 INFO:tasks.workunit.client.0.smithi090.stdout:0/240: mknod d2/d11/d3f/c4c 0 2024-03-11T19:28:11.993 INFO:tasks.workunit.client.0.smithi090.stdout:8/253: symlink d0/d3/d4c/l4d 0 2024-03-11T19:28:12.035 INFO:tasks.workunit.client.0.smithi090.stdout:6/252: read - d4/d13/d22/d34/f3c zero size 2024-03-11T19:28:12.182 INFO:tasks.workunit.client.0.smithi090.stdout:1/230: fdatasync d5/fc 0 2024-03-11T19:28:12.395 INFO:tasks.workunit.client.0.smithi090.stdout:0/241: creat d2/d11/d15/d29/d43/f4d x:0 0 0 2024-03-11T19:28:12.855 INFO:tasks.workunit.client.0.smithi090.stdout:9/228: dwrite d1/dd/f13 [0,4194304] 0 2024-03-11T19:28:13.039 INFO:tasks.workunit.client.0.smithi090.stdout:1/231: mknod d5/d31/c46 0 2024-03-11T19:28:13.072 INFO:tasks.workunit.client.0.smithi090.stdout:0/242: mknod d2/d11/d15/d29/d43/c4e 0 2024-03-11T19:28:13.158 INFO:tasks.workunit.client.0.smithi090.stdout:9/229: dread - d1/dd/f3f zero size 2024-03-11T19:28:13.230 INFO:tasks.workunit.client.0.smithi090.stdout:2/228: rmdir d0/d21 39 2024-03-11T19:28:13.238 INFO:tasks.workunit.client.0.smithi090.stdout:4/251: write d4/d27/f3a [1096467,14784] 0 2024-03-11T19:28:13.248 INFO:tasks.workunit.client.0.smithi090.stdout:7/231: dwrite d3/dc/f3a [0,4194304] 0 2024-03-11T19:28:13.282 INFO:tasks.workunit.client.0.smithi090.stdout:3/207: write d2/db/f1c [119810,117042] 0 2024-03-11T19:28:13.289 INFO:tasks.workunit.client.0.smithi090.stdout:5/221: write d1/d26/f11 [727767,118125] 0 2024-03-11T19:28:13.642 INFO:tasks.workunit.client.0.smithi090.stdout:0/243: symlink d2/d11/l4f 0 2024-03-11T19:28:13.937 INFO:tasks.workunit.client.0.smithi090.stdout:8/254: dwrite d0/f20 [0,4194304] 0 2024-03-11T19:28:14.104 INFO:tasks.workunit.client.0.smithi090.stdout:6/253: dwrite d4/ff [0,4194304] 0 2024-03-11T19:28:14.312 INFO:tasks.workunit.client.0.smithi090.stdout:9/230: dread d1/dd/f11 [0,4194304] 0 2024-03-11T19:28:14.313 INFO:tasks.workunit.client.0.smithi090.stdout:7/232: dread d3/dc/f19 [0,4194304] 0 2024-03-11T19:28:14.383 INFO:tasks.workunit.client.0.smithi090.stdout:9/231: dread - d1/dd/f3f zero size 2024-03-11T19:28:14.428 INFO:tasks.workunit.client.0.smithi090.stdout:4/252: symlink d4/l5d 0 2024-03-11T19:28:14.448 INFO:tasks.workunit.client.0.smithi090.stdout:5/222: creat d1/d26/d19/d27/f48 x:0 0 0 2024-03-11T19:28:14.716 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:28:14.719 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:28:14.722 INFO:tasks.workunit.client.0.smithi090.stdout:9/232: sync 2024-03-11T19:28:14.853 INFO:tasks.workunit.client.0.smithi090.stdout:1/232: rename d5/da/d1c/f21 to d5/da/f47 0 2024-03-11T19:28:14.928 INFO:tasks.workunit.client.0.smithi090.stdout:3/208: creat d2/db/d15/f48 x:0 0 0 2024-03-11T19:28:14.977 INFO:tasks.workunit.client.0.smithi090.stdout:1/233: dread d5/da/d1c/f30 [0,4194304] 0 2024-03-11T19:28:15.128 INFO:tasks.workunit.client.0.smithi090.stdout:8/255: creat d0/d3/d3e/f4e x:0 0 0 2024-03-11T19:28:15.299 INFO:tasks.workunit.client.0.smithi090.stdout:4/253: mknod d4/d18/c5e 0 2024-03-11T19:28:15.380 INFO:tasks.workunit.client.0.smithi090.stdout:8/256: read d0/f31 [458398,19605] 0 2024-03-11T19:28:15.570 INFO:tasks.workunit.client.0.smithi090.stdout:3/209: dread d2/db/d15/f17 [0,4194304] 0 2024-03-11T19:28:15.625 INFO:tasks.workunit.client.0.smithi090.stdout:0/244: symlink d2/d11/d15/d29/d3d/l50 0 2024-03-11T19:28:15.758 INFO:tasks.workunit.client.0.smithi090.stdout:2/229: rename d0/l11 to d0/d21/d30/d25/l37 0 2024-03-11T19:28:15.974 INFO:tasks.workunit.client.0.smithi090.stdout:1/234: symlink d5/l48 0 2024-03-11T19:28:16.292 INFO:tasks.workunit.client.0.smithi090.stdout:8/257: fdatasync d0/d43/f2e 0 2024-03-11T19:28:16.424 INFO:tasks.workunit.client.0.smithi090.stdout:0/245: truncate d2/f14 1199574 0 2024-03-11T19:28:16.451 INFO:tasks.workunit.client.0.smithi090.stdout:4/254: dread d4/f5 [0,4194304] 0 2024-03-11T19:28:16.459 INFO:tasks.workunit.client.0.smithi090.stdout:5/223: write d1/d26/f3f [156792,39294] 0 2024-03-11T19:28:16.576 INFO:tasks.workunit.client.0.smithi090.stdout:9/233: mkdir d1/d49 0 2024-03-11T19:28:16.659 INFO:tasks.workunit.client.0.smithi090.stdout:6/254: rename d4/l8 to d4/d13/d22/d34/d37/l50 0 2024-03-11T19:28:16.670 INFO:tasks.workunit.client.0.smithi090.stdout:4/255: dread - d4/d18/f3c zero size 2024-03-11T19:28:16.803 INFO:tasks.workunit.client.0.smithi090.stdout:7/233: dwrite d3/dc/d1e/f26 [0,4194304] 0 2024-03-11T19:28:17.078 INFO:tasks.workunit.client.0.smithi090.stdout:3/210: mkdir d2/db/d15/d49 0 2024-03-11T19:28:17.173 INFO:tasks.workunit.client.0.smithi090.stdout:8/258: symlink d0/d43/dc/l4f 0 2024-03-11T19:28:17.258 INFO:tasks.workunit.client.0.smithi090.stdout:0/246: fsync d2/d11/f17 0 2024-03-11T19:28:17.273 INFO:tasks.workunit.client.0.smithi090.stdout:5/224: dread d1/d26/fc [0,4194304] 0 2024-03-11T19:28:17.282 INFO:tasks.workunit.client.0.smithi090.stdout:1/235: dread d5/da/d23/f2a [0,4194304] 0 2024-03-11T19:28:17.410 INFO:tasks.workunit.client.0.smithi090.stdout:8/259: chown d0/d3/d4c 4 1 2024-03-11T19:28:17.415 INFO:tasks.workunit.client.0.smithi090.stdout:3/211: chown d2/d1d/l7 0 1 2024-03-11T19:28:17.583 INFO:tasks.workunit.client.0.smithi090.stdout:1/236: stat d5/da/dd/l1e 0 2024-03-11T19:28:17.611 INFO:tasks.workunit.client.0.smithi090.stdout:3/212: readlink d2/de/l10 0 2024-03-11T19:28:17.622 INFO:tasks.workunit.client.0.smithi090.stdout:4/256: fsync d4/ff 0 2024-03-11T19:28:17.632 INFO:tasks.workunit.client.0.smithi090.stdout:7/234: mkdir d3/d49 0 2024-03-11T19:28:17.654 INFO:tasks.workunit.client.0.smithi090.stdout:8/260: chown d0/d43/c42 0 1 2024-03-11T19:28:17.694 INFO:tasks.workunit.client.0.smithi090.stdout:2/230: dread d0/d21/d30/f22 [0,4194304] 0 2024-03-11T19:28:17.773 INFO:tasks.workunit.client.0.smithi090.stdout:7/235: chown d3/l6 238868368 1 2024-03-11T19:28:18.083 INFO:tasks.workunit.client.0.smithi090.stdout:9/234: fdatasync d1/d1d/f2b 0 2024-03-11T19:28:18.435 INFO:tasks.workunit.client.0.smithi090.stdout:6/255: dwrite d4/d24/f3f [0,4194304] 0 2024-03-11T19:28:18.559 INFO:tasks.workunit.client.0.smithi090.stdout:6/256: chown d4/d13/l20 38593 1 2024-03-11T19:28:18.636 INFO:tasks.workunit.client.0.smithi090.stdout:4/257: mknod d4/d27/d28/c5f 0 2024-03-11T19:28:18.924 INFO:tasks.workunit.client.0.smithi090.stdout:4/258: dread d4/f5 [0,4194304] 0 2024-03-11T19:28:19.178 INFO:tasks.workunit.client.0.smithi090.stdout:7/236: dread d3/dc/f30 [0,4194304] 0 2024-03-11T19:28:19.186 INFO:tasks.workunit.client.0.smithi090.stdout:3/213: getdents d2/d45 0 2024-03-11T19:28:19.242 INFO:tasks.workunit.client.0.smithi090.stdout:8/261: write d0/d43/dc/f12 [1888378,47980] 0 2024-03-11T19:28:19.490 INFO:tasks.workunit.client.0.smithi090.stdout:1/237: dwrite d5/da/d1c/f39 [0,4194304] 0 2024-03-11T19:28:19.897 INFO:tasks.workunit.client.0.smithi090.stdout:5/225: rename d1/d26/f2b to d1/f49 0 2024-03-11T19:28:20.448 INFO:tasks.workunit.client.0.smithi090.stdout:6/257: dwrite d4/fe [8388608,4194304] 0 2024-03-11T19:28:20.613 INFO:tasks.workunit.client.0.smithi090.stdout:6/258: chown d4/d13/d22/d34/d40 114 1 2024-03-11T19:28:20.646 INFO:tasks.workunit.client.0.smithi090.stdout:2/231: rmdir d0/de/d1f/d2b 0 2024-03-11T19:28:20.662 INFO:tasks.workunit.client.0.smithi090.stdout:6/259: chown d4/d14/d44/c45 70633566 1 2024-03-11T19:28:20.723 INFO:tasks.workunit.client.0.smithi090.stdout:7/237: read d3/dc/d13/f35 [295422,85072] 0 2024-03-11T19:28:20.968 INFO:tasks.workunit.client.0.smithi090.stdout:0/247: rename d2/d11/d15/d29/d2e/c38 to d2/d11/d3f/c51 0 2024-03-11T19:28:21.768 INFO:tasks.workunit.client.0.smithi090.stdout:7/238: creat d3/dc/d17/d32/f4a x:0 0 0 2024-03-11T19:28:21.878 INFO:tasks.workunit.client.0.smithi090.stdout:9/235: creat d1/dd/df/f4a x:0 0 0 2024-03-11T19:28:21.914 INFO:tasks.workunit.client.0.smithi090.stdout:1/238: getdents d5/da/d23 0 2024-03-11T19:28:22.036 INFO:tasks.workunit.client.0.smithi090.stdout:8/262: rename d0/d3/f8 to d0/d43/dc/f50 0 2024-03-11T19:28:22.061 INFO:tasks.workunit.client.0.smithi090.stdout:4/259: write d4/f8 [2628130,17536] 0 2024-03-11T19:28:22.103 INFO:tasks.workunit.client.0.smithi090.stdout:0/248: unlink d2/fe 0 2024-03-11T19:28:22.289 INFO:tasks.workunit.client.0.smithi090.stdout:2/232: symlink d0/de/l38 0 2024-03-11T19:28:22.631 INFO:tasks.workunit.client.0.smithi090.stdout:3/214: dwrite d2/ff [4194304,4194304] 0 2024-03-11T19:28:22.760 INFO:tasks.workunit.client.0.smithi090.stdout:1/239: creat d5/d31/d36/f49 x:0 0 0 2024-03-11T19:28:23.099 INFO:tasks.workunit.client.0.smithi090.stdout:6/260: write d4/d13/f1b [2128135,39649] 0 2024-03-11T19:28:23.242 INFO:tasks.workunit.client.0.smithi090.stdout:6/261: readlink d4/d13/l39 0 2024-03-11T19:28:23.358 INFO:tasks.workunit.client.0.smithi090.stdout:8/263: mknod d0/d3/c51 0 2024-03-11T19:28:23.391 INFO:tasks.workunit.client.0.smithi090.stdout:0/249: creat d2/d11/d15/d29/d2e/f52 x:0 0 0 2024-03-11T19:28:23.427 INFO:tasks.workunit.client.0.smithi090.stdout:7/239: dwrite d3/dc/d1e/ff [0,4194304] 0 2024-03-11T19:28:23.549 INFO:tasks.workunit.client.0.smithi090.stdout:4/260: truncate d4/d18/f41 1372605 0 2024-03-11T19:28:23.584 INFO:tasks.workunit.client.0.smithi090.stdout:2/233: unlink d0/f6 0 2024-03-11T19:28:23.870 INFO:tasks.workunit.client.0.smithi090.stdout:1/240: truncate d5/f15 5149521 0 2024-03-11T19:28:24.033 INFO:tasks.workunit.client.0.smithi090.stdout:0/250: read - d2/d11/f41 zero size 2024-03-11T19:28:24.366 INFO:tasks.workunit.client.0.smithi090.stdout:2/234: creat d0/d21/d30/f39 x:0 0 0 2024-03-11T19:28:24.823 INFO:tasks.workunit.client.0.smithi090.stdout:1/241: unlink d5/da/d1c/d3b/f42 0 2024-03-11T19:28:24.893 INFO:tasks.workunit.client.0.smithi090.stdout:9/236: write d1/f17 [145210,77893] 0 2024-03-11T19:28:24.944 INFO:tasks.workunit.client.0.smithi090.stdout:9/237: stat d1/d1d/l35 0 2024-03-11T19:28:25.010 INFO:tasks.workunit.client.0.smithi090.stdout:0/251: creat d2/d11/d3a/f53 x:0 0 0 2024-03-11T19:28:25.250 INFO:tasks.workunit.client.0.smithi090.stdout:6/262: write d4/f46 [352914,21622] 0 2024-03-11T19:28:25.909 INFO:tasks.workunit.client.0.smithi090.stdout:5/226: rename d1/d26/f2d to d1/f4a 0 2024-03-11T19:28:25.929 INFO:tasks.workunit.client.0.smithi090.stdout:9/238: mkdir d1/dd/d18/d25/d4b 0 2024-03-11T19:28:26.085 INFO:tasks.workunit.client.0.smithi090.stdout:7/240: dwrite d3/dc/d1e/f26 [0,4194304] 0 2024-03-11T19:28:26.945 INFO:tasks.workunit.client.0.smithi090.stdout:3/215: rename d2/f2e to d2/d45/f4a 0 2024-03-11T19:28:26.961 INFO:tasks.workunit.client.0.smithi090.stdout:5/227: truncate d1/d26/f43 182907 0 2024-03-11T19:28:27.059 INFO:tasks.workunit.client.0.smithi090.stdout:3/216: chown d2/db/f24 14678 1 2024-03-11T19:28:27.133 INFO:tasks.workunit.client.0.smithi090.stdout:4/261: dread d4/d18/f41 [0,4194304] 0 2024-03-11T19:28:27.215 INFO:tasks.workunit.client.0.smithi090.stdout:9/239: symlink d1/dd/d18/d25/d4b/l4c 0 2024-03-11T19:28:27.477 INFO:tasks.workunit.client.0.smithi090.stdout:8/264: rename d0/d3/c51 to d0/d3/d3e/c52 0 2024-03-11T19:28:27.747 INFO:tasks.workunit.client.0.smithi090.stdout:3/217: chown d2/l3e 0 1 2024-03-11T19:28:27.766 INFO:tasks.workunit.client.0.smithi090.stdout:4/262: dread - d4/d27/d28/f50 zero size 2024-03-11T19:28:27.901 INFO:tasks.workunit.client.0.smithi090.stdout:1/242: dwrite d5/f37 [0,4194304] 0 2024-03-11T19:28:28.304 INFO:tasks.workunit.client.0.smithi090.stdout:6/263: write d4/d13/d22/d34/f36 [694516,130204] 0 2024-03-11T19:28:28.347 INFO:tasks.workunit.client.0.smithi090.stdout:8/265: dread - d0/d3/d3e/f4e zero size 2024-03-11T19:28:28.394 INFO:tasks.workunit.client.0.smithi090.stdout:8/266: chown d0/d3/d3e 508546 1 2024-03-11T19:28:28.616 INFO:tasks.workunit.client.0.smithi090.stdout:3/218: symlink d2/db/d15/d1a/l4b 0 2024-03-11T19:28:29.111 INFO:tasks.workunit.client.0.smithi090.stdout:7/241: getdents d3/dc/d17/d32 0 2024-03-11T19:28:29.256 INFO:tasks.workunit.client.0.smithi090.stdout:2/235: creat d0/f3a x:0 0 0 2024-03-11T19:28:29.790 INFO:tasks.workunit.client.0.smithi090.stdout:6/264: mknod d4/d13/d22/d3a/c51 0 2024-03-11T19:28:30.091 INFO:tasks.workunit.client.0.smithi090.stdout:9/240: creat d1/dd/df/f4d x:0 0 0 2024-03-11T19:28:30.200 INFO:tasks.workunit.client.0.smithi090.stdout:5/228: dwrite d1/d26/f20 [0,4194304] 0 2024-03-11T19:28:30.343 INFO:tasks.workunit.client.0.smithi090.stdout:0/252: link d2/d11/d3f/c51 d2/d11/c54 0 2024-03-11T19:28:30.356 INFO:tasks.workunit.client.0.smithi090.stdout:7/242: unlink d3/dc/f19 0 2024-03-11T19:28:30.845 INFO:tasks.workunit.client.0.smithi090.stdout:6/265: rmdir d4/d13/d22/d3a 39 2024-03-11T19:28:31.067 INFO:tasks.workunit.client.0.smithi090.stdout:4/263: creat d4/f60 x:0 0 0 2024-03-11T19:28:31.182 INFO:tasks.workunit.client.0.smithi090.stdout:9/241: rmdir d1 39 2024-03-11T19:28:31.276 INFO:tasks.workunit.client.0.smithi090.stdout:1/243: creat d5/da/f4a x:0 0 0 2024-03-11T19:28:31.783 INFO:tasks.workunit.client.0.smithi090.stdout:0/253: read d2/f10 [359433,100878] 0 2024-03-11T19:28:31.807 INFO:tasks.workunit.client.0.smithi090.stdout:8/267: write d0/d43/f23 [2473407,63318] 0 2024-03-11T19:28:31.899 INFO:tasks.workunit.client.0.smithi090.stdout:0/254: stat d2/d11/d15/d29/d43/c4e 0 2024-03-11T19:28:31.902 INFO:tasks.workunit.client.0.smithi090.stdout:8/268: chown d0/d3/c45 1791961 1 2024-03-11T19:28:32.212 INFO:tasks.workunit.client.0.smithi090.stdout:8/269: dread - d0/d43/dc/f44 zero size 2024-03-11T19:28:32.484 INFO:tasks.workunit.client.0.smithi090.stdout:2/236: write d0/de/d12/f32 [643437,81956] 0 2024-03-11T19:28:32.535 INFO:tasks.workunit.client.0.smithi090.stdout:7/243: creat d3/f4b x:0 0 0 2024-03-11T19:28:32.537 INFO:tasks.workunit.client.0.smithi090.stdout:0/255: sync 2024-03-11T19:28:32.611 INFO:tasks.workunit.client.0.smithi090.stdout:4/264: mknod d4/d51/c61 0 2024-03-11T19:28:32.620 INFO:tasks.workunit.client.0.smithi090.stdout:8/270: sync 2024-03-11T19:28:32.925 INFO:tasks.workunit.client.0.smithi090.stdout:6/266: link d4/c4d d4/d13/d22/d42/d48/c52 0 2024-03-11T19:28:33.183 INFO:tasks.workunit.client.0.smithi090.stdout:7/244: creat d3/dc/d13/f4c x:0 0 0 2024-03-11T19:28:33.343 INFO:tasks.workunit.client.0.smithi090.stdout:7/245: dread d3/f22 [0,4194304] 0 2024-03-11T19:28:33.443 INFO:tasks.workunit.client.0.smithi090.stdout:9/242: symlink d1/dd/df/l4e 0 2024-03-11T19:28:33.451 INFO:tasks.workunit.client.0.smithi090.stdout:5/229: write d1/d26/d19/d27/f48 [43122,100607] 0 2024-03-11T19:28:33.455 INFO:tasks.workunit.client.0.smithi090.stdout:7/246: sync 2024-03-11T19:28:33.480 INFO:tasks.workunit.client.0.smithi090.stdout:6/267: creat d4/d14/d44/f53 x:0 0 0 2024-03-11T19:28:33.590 INFO:tasks.workunit.client.0.smithi090.stdout:0/256: truncate d2/f5 1442434 0 2024-03-11T19:28:33.654 INFO:tasks.workunit.client.0.smithi090.stdout:8/271: mkdir d0/d53 0 2024-03-11T19:28:33.800 INFO:tasks.workunit.client.0.smithi090.stdout:3/219: rmdir d2/db 39 2024-03-11T19:28:33.879 INFO:tasks.workunit.client.0.smithi090.stdout:1/244: dwrite d5/da/d23/f3a [0,4194304] 0 2024-03-11T19:28:34.031 INFO:tasks.workunit.client.0.smithi090.stdout:6/268: chown c2 2020421123 1 2024-03-11T19:28:34.337 INFO:tasks.workunit.client.0.smithi090.stdout:5/230: unlink d1/l33 0 2024-03-11T19:28:34.580 INFO:tasks.workunit.client.0.smithi090.stdout:7/247: mkdir d3/dc/d17/d34/d4d 0 2024-03-11T19:28:34.623 INFO:tasks.workunit.client.0.smithi090.stdout:4/265: write d4/fa [1884393,51440] 0 2024-03-11T19:28:34.687 INFO:tasks.workunit.client.0.smithi090.stdout:4/266: chown d4/l58 309654 1 2024-03-11T19:28:34.714 INFO:tasks.workunit.client.0.smithi090.stdout:0/257: fsync d2/f3 0 2024-03-11T19:28:34.857 INFO:tasks.workunit.client.0.smithi090.stdout:0/258: sync 2024-03-11T19:28:35.306 INFO:tasks.workunit.client.0.smithi090.stdout:1/245: mkdir d5/da/d1c/d3b/d4b 0 2024-03-11T19:28:35.618 INFO:tasks.workunit.client.0.smithi090.stdout:5/231: creat d1/d26/f4b x:0 0 0 2024-03-11T19:28:35.640 INFO:tasks.workunit.client.0.smithi090.stdout:5/232: stat d1/d17/d39 0 2024-03-11T19:28:35.673 INFO:tasks.workunit.client.0.smithi090.stdout:2/237: getdents d0/d21/d30/d25 0 2024-03-11T19:28:35.941 INFO:tasks.workunit.client.0.smithi090.stdout:7/248: truncate d3/dc/d1e/f12 5594787 0 2024-03-11T19:28:35.965 INFO:tasks.workunit.client.0.smithi090.stdout:7/249: stat d3/dc/d13/f18 0 2024-03-11T19:28:36.048 INFO:tasks.workunit.client.0.smithi090.stdout:4/267: mknod d4/d51/c62 0 2024-03-11T19:28:36.065 INFO:tasks.workunit.client.0.smithi090.stdout:4/268: readlink d4/d18/l54 0 2024-03-11T19:28:36.393 INFO:tasks.workunit.client.0.smithi090.stdout:3/220: fsync d2/d2f/f32 0 2024-03-11T19:28:36.666 INFO:tasks.workunit.client.0.smithi090.stdout:1/246: mkdir d5/d31/d4c 0 2024-03-11T19:28:36.946 INFO:tasks.workunit.client.0.smithi090.stdout:8/272: write d0/d3/f2c [1026310,38203] 0 2024-03-11T19:28:37.122 INFO:tasks.workunit.client.0.smithi090.stdout:2/238: fdatasync d0/d21/f2a 0 2024-03-11T19:28:37.479 INFO:tasks.workunit.client.0.smithi090.stdout:4/269: dread d4/d18/f25 [0,4194304] 0 2024-03-11T19:28:38.027 INFO:tasks.workunit.client.0.smithi090.stdout:8/273: fdatasync d0/d3/d3e/f4e 0 2024-03-11T19:28:38.832 INFO:tasks.workunit.client.0.smithi090.stdout:9/243: creat d1/f4f x:0 0 0 2024-03-11T19:28:38.846 INFO:tasks.workunit.client.0.smithi090.stdout:3/221: dread d2/db/d15/f1e [0,4194304] 0 2024-03-11T19:28:38.989 INFO:tasks.workunit.client.0.smithi090.stdout:6/269: getdents d4/d13/d22 0 2024-03-11T19:28:39.026 INFO:tasks.workunit.client.0.smithi090.stdout:7/250: write d3/dc/d13/f20 [731788,26364] 0 2024-03-11T19:28:39.028 INFO:tasks.workunit.client.0.smithi090.stdout:5/233: creat d1/f4c x:0 0 0 2024-03-11T19:28:39.388 INFO:tasks.workunit.client.0.smithi090.stdout:8/274: symlink d0/d43/dc/l54 0 2024-03-11T19:28:39.404 INFO:tasks.workunit.client.0.smithi090.stdout:8/275: chown d0/c29 64046 1 2024-03-11T19:28:39.773 INFO:tasks.workunit.client.0.smithi090.stdout:0/259: rename d2/d11/d15/d29/d2e/c3c to d2/d11/d15/d29/c55 0 2024-03-11T19:28:39.812 INFO:tasks.workunit.client.0.smithi090.stdout:0/260: dread - d2/d11/d15/f28 zero size 2024-03-11T19:28:40.189 INFO:tasks.workunit.client.0.smithi090.stdout:4/270: write d4/f24 [4542222,61898] 0 2024-03-11T19:28:40.438 INFO:tasks.workunit.client.0.smithi090.stdout:3/222: symlink d2/d45/l4c 0 2024-03-11T19:28:40.634 INFO:tasks.workunit.client.0.smithi090.stdout:3/223: sync 2024-03-11T19:28:40.718 INFO:tasks.workunit.client.0.smithi090.stdout:5/234: mkdir d1/d26/d4d 0 2024-03-11T19:28:40.956 INFO:tasks.workunit.client.0.smithi090.stdout:6/270: dread d4/fe [0,4194304] 0 2024-03-11T19:28:41.299 INFO:tasks.workunit.client.0.smithi090.stdout:1/247: getdents d5/d3c 0 2024-03-11T19:28:41.309 INFO:tasks.workunit.client.0.smithi090.stdout:4/271: dread - d4/f46 zero size 2024-03-11T19:28:41.627 INFO:tasks.workunit.client.0.smithi090.stdout:9/244: fsync d1/f3 0 2024-03-11T19:28:41.644 INFO:tasks.workunit.client.0.smithi090.stdout:7/251: creat d3/dc/d29/f4e x:0 0 0 2024-03-11T19:28:41.685 INFO:tasks.workunit.client.0.smithi090.stdout:9/245: fdatasync d1/f4f 0 2024-03-11T19:28:41.768 INFO:tasks.workunit.client.0.smithi090.stdout:9/246: readlink d1/dd/d18/d25/d4b/l4c 0 2024-03-11T19:28:41.837 INFO:tasks.workunit.client.0.smithi090.stdout:2/239: getdents d0/de/d1f 0 2024-03-11T19:28:42.159 INFO:tasks.workunit.client.0.smithi090.stdout:0/261: mknod d2/c56 0 2024-03-11T19:28:42.425 INFO:tasks.workunit.client.0.smithi090.stdout:4/272: unlink d4/d27/d28/f50 0 2024-03-11T19:28:42.723 INFO:tasks.workunit.client.0.smithi090.stdout:7/252: creat d3/dc/d1e/f4f x:0 0 0 2024-03-11T19:28:42.997 INFO:tasks.workunit.client.0.smithi090.stdout:2/240: mknod d0/d21/c3b 0 2024-03-11T19:28:43.085 INFO:tasks.workunit.client.0.smithi090.stdout:8/276: dwrite d0/f20 [0,4194304] 0 2024-03-11T19:28:43.249 INFO:tasks.workunit.client.0.smithi090.stdout:0/262: fdatasync d2/f27 0 2024-03-11T19:28:43.720 INFO:tasks.workunit.client.0.smithi090.stdout:2/241: symlink d0/de/d1f/l3c 0 2024-03-11T19:28:43.796 INFO:tasks.workunit.client.0.smithi090.stdout:8/277: fdatasync d0/d43/dc/f3a 0 2024-03-11T19:28:43.927 INFO:tasks.workunit.client.0.smithi090.stdout:3/224: link d2/db/d15/d1a/l4b d2/d45/l4d 0 2024-03-11T19:28:43.961 INFO:tasks.workunit.client.0.smithi090.stdout:0/263: creat d2/d11/d15/d36/f57 x:0 0 0 2024-03-11T19:28:43.967 INFO:tasks.workunit.client.0.smithi090.stdout:6/271: write d4/d13/f1f [926089,113313] 0 2024-03-11T19:28:44.052 INFO:tasks.workunit.client.0.smithi090.stdout:7/253: getdents d3/d47 0 2024-03-11T19:28:44.142 INFO:tasks.workunit.client.0.smithi090.stdout:4/273: dread d4/d27/d44/f4e [4194304,4194304] 0 2024-03-11T19:28:44.190 INFO:tasks.workunit.client.0.smithi090.stdout:2/242: chown d0/de/d12/c36 714 1 2024-03-11T19:28:44.254 INFO:tasks.workunit.client.0.smithi090.stdout:0/264: readlink d2/d11/d15/l2d 0 2024-03-11T19:28:44.325 INFO:tasks.workunit.client.0.smithi090.stdout:4/274: sync 2024-03-11T19:28:44.397 INFO:tasks.workunit.client.0.smithi090.stdout:5/235: dwrite d1/f2f [0,4194304] 0 2024-03-11T19:28:44.505 INFO:tasks.workunit.client.0.smithi090.stdout:8/278: creat d0/d40/f55 x:0 0 0 2024-03-11T19:28:44.528 INFO:tasks.workunit.client.0.smithi090.stdout:0/265: readlink d2/d11/d15/d29/l48 0 2024-03-11T19:28:44.605 INFO:tasks.workunit.client.0.smithi090.stdout:7/254: dread d3/dc/d17/f1b [0,4194304] 0 2024-03-11T19:28:44.747 INFO:tasks.workunit.client.0.smithi090.stdout:6/272: read - d4/d13/f3b zero size 2024-03-11T19:28:44.773 INFO:tasks.workunit.client.0.smithi090.stdout:0/266: stat d2/d11/d15/d29/d43/f4d 0 2024-03-11T19:28:44.852 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:28:44.855 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:28:44.988 INFO:tasks.workunit.client.0.smithi090.stdout:5/236: dread d1/d26/fc [0,4194304] 0 2024-03-11T19:28:45.184 INFO:tasks.workunit.client.0.smithi090.stdout:2/243: rename d0/d21/d30/f35 to d0/d21/f3d 0 2024-03-11T19:28:45.204 INFO:tasks.workunit.client.0.smithi090.stdout:8/279: unlink d0/d3/d4c/l4d 0 2024-03-11T19:28:45.222 INFO:tasks.workunit.client.0.smithi090.stdout:9/247: dwrite d1/dd/f13 [4194304,4194304] 0 2024-03-11T19:28:45.247 INFO:tasks.workunit.client.0.smithi090.stdout:3/225: truncate d2/d45/f4a 869057 0 2024-03-11T19:28:45.429 INFO:tasks.workunit.client.0.smithi090.stdout:1/248: dwrite d5/da/d1c/f39 [0,4194304] 0 2024-03-11T19:28:45.654 INFO:tasks.workunit.client.0.smithi090.stdout:7/255: read d3/dc/d1e/f11 [5232988,128260] 0 2024-03-11T19:28:45.717 INFO:tasks.workunit.client.0.smithi090.stdout:3/226: dread - d2/db/d15/f37 zero size 2024-03-11T19:28:45.717 INFO:tasks.workunit.client.0.smithi090.stdout:0/267: mkdir d2/d11/d15/d29/d43/d58 0 2024-03-11T19:28:45.848 INFO:tasks.workunit.client.0.smithi090.stdout:5/237: dwrite d1/f2f [0,4194304] 0 2024-03-11T19:28:46.075 INFO:tasks.workunit.client.0.smithi090.stdout:4/275: dwrite d4/f5 [0,4194304] 0 2024-03-11T19:28:46.300 INFO:tasks.workunit.client.0.smithi090.stdout:3/227: mknod d2/de/c4e 0 2024-03-11T19:28:46.519 INFO:tasks.workunit.client.0.smithi090.stdout:6/273: getdents d4/d13 0 2024-03-11T19:28:46.524 INFO:tasks.workunit.client.0.smithi090.stdout:5/238: read - d1/d26/f3c zero size 2024-03-11T19:28:46.635 INFO:tasks.workunit.client.0.smithi090.stdout:2/244: creat d0/f3e x:0 0 0 2024-03-11T19:28:46.678 INFO:tasks.workunit.client.0.smithi090.stdout:0/268: symlink d2/d11/l59 0 2024-03-11T19:28:46.879 INFO:tasks.workunit.client.0.smithi090.stdout:4/276: write d4/f24 [4487799,93011] 0 2024-03-11T19:28:47.146 INFO:tasks.workunit.client.0.smithi090.stdout:5/239: stat d1/f4a 0 2024-03-11T19:28:47.174 INFO:tasks.workunit.client.0.smithi090.stdout:3/228: read d2/f6 [247386,114821] 0 2024-03-11T19:28:47.518 INFO:tasks.workunit.client.0.smithi090.stdout:8/280: dwrite d0/f2d [4194304,4194304] 0 2024-03-11T19:28:47.545 INFO:tasks.workunit.client.0.smithi090.stdout:7/256: dread d3/dc/d1e/f12 [0,4194304] 0 2024-03-11T19:28:47.775 INFO:tasks.workunit.client.0.smithi090.stdout:9/248: dwrite d1/f3 [4194304,4194304] 0 2024-03-11T19:28:47.791 INFO:tasks.workunit.client.0.smithi090.stdout:1/249: dread d5/f1b [0,4194304] 0 2024-03-11T19:28:47.792 INFO:tasks.workunit.client.0.smithi090.stdout:2/245: symlink d0/de/d1f/l3f 0 2024-03-11T19:28:48.053 INFO:tasks.workunit.client.0.smithi090.stdout:4/277: write d4/d27/f35 [1722209,54981] 0 2024-03-11T19:28:48.157 INFO:tasks.workunit.client.0.smithi090.stdout:4/278: chown d4/d27/d28/l32 13628 1 2024-03-11T19:28:48.205 INFO:tasks.workunit.client.0.smithi090.stdout:6/274: creat d4/d13/d22/d3a/f54 x:0 0 0 2024-03-11T19:28:48.222 INFO:tasks.workunit.client.0.smithi090.stdout:5/240: symlink d1/d17/l4e 0 2024-03-11T19:28:48.265 INFO:tasks.workunit.client.0.smithi090.stdout:6/275: dread - d4/d13/d22/d34/f3c zero size 2024-03-11T19:28:48.386 INFO:tasks.workunit.client.0.smithi090.stdout:3/229: write d2/f2c [4000730,42582] 0 2024-03-11T19:28:48.445 INFO:tasks.workunit.client.0.smithi090.stdout:9/249: rename d1/dd/df/f33 to d1/dd/df/d1c/d21/d2a/f50 0 2024-03-11T19:28:48.473 INFO:tasks.workunit.client.0.smithi090.stdout:2/246: read d0/f7 [7028515,44546] 0 2024-03-11T19:28:48.505 INFO:tasks.workunit.client.0.smithi090.stdout:1/250: truncate d5/da/d23/f2a 1890689 0 2024-03-11T19:28:48.569 INFO:tasks.workunit.client.0.smithi090.stdout:8/281: dread d0/d3/f37 [0,4194304] 0 2024-03-11T19:28:48.589 INFO:tasks.workunit.client.0.smithi090.stdout:6/276: symlink d4/d13/d22/d42/l55 0 2024-03-11T19:28:48.622 INFO:tasks.workunit.client.0.smithi090.stdout:8/282: chown d0/d43/dc/fd 78185821 1 2024-03-11T19:28:48.660 INFO:tasks.workunit.client.0.smithi090.stdout:8/283: write d0/d40/f55 [26645,1498] 0 2024-03-11T19:28:48.686 INFO:tasks.workunit.client.0.smithi090.stdout:0/269: write d2/d11/d15/d29/f20 [285116,129283] 0 2024-03-11T19:28:48.803 INFO:tasks.workunit.client.0.smithi090.stdout:4/279: rename d4/f47 to d4/d27/d28/f63 0 2024-03-11T19:28:48.872 INFO:tasks.workunit.client.0.smithi090.stdout:3/230: dread d2/db/f1c [0,4194304] 0 2024-03-11T19:28:48.928 INFO:tasks.workunit.client.0.smithi090.stdout:6/277: truncate d4/d13/f1b 3034043 0 2024-03-11T19:28:49.129 INFO:tasks.workunit.client.0.smithi090.stdout:6/278: chown d4/d13/l4a 31166 1 2024-03-11T19:28:49.154 INFO:tasks.workunit.client.0.smithi090.stdout:0/270: mknod d2/d11/d3f/c5a 0 2024-03-11T19:28:49.200 INFO:tasks.workunit.client.0.smithi090.stdout:1/251: creat d5/d3c/d45/f4d x:0 0 0 2024-03-11T19:28:49.248 INFO:tasks.workunit.client.0.smithi090.stdout:9/250: creat d1/dd/d18/d3a/d48/f51 x:0 0 0 2024-03-11T19:28:49.416 INFO:tasks.workunit.client.0.smithi090.stdout:7/257: dwrite d3/dc/d17/d32/f4a [0,4194304] 0 2024-03-11T19:28:49.607 INFO:tasks.workunit.client.0.smithi090.stdout:4/280: mkdir d4/d27/d28/d38/d64 0 2024-03-11T19:28:49.958 INFO:tasks.workunit.client.0.smithi090.stdout:8/284: link d0/d3/c45 d0/d40/c56 0 2024-03-11T19:28:49.983 INFO:tasks.workunit.client.0.smithi090.stdout:8/285: chown d0/d40/f55 0 1 2024-03-11T19:28:50.314 INFO:tasks.workunit.client.0.smithi090.stdout:1/252: mkdir d5/d4e 0 2024-03-11T19:28:50.345 INFO:tasks.workunit.client.0.smithi090.stdout:1/253: chown d5/d31/d36/c3d 6752 1 2024-03-11T19:28:50.722 INFO:tasks.workunit.client.0.smithi090.stdout:7/258: truncate d3/dc/d1e/f26 2199179 0 2024-03-11T19:28:51.324 INFO:tasks.workunit.client.0.smithi090.stdout:8/286: rmdir d0/d40 39 2024-03-11T19:28:51.450 INFO:tasks.workunit.client.0.smithi090.stdout:5/241: rename d1/d26/f11 to d1/f4f 0 2024-03-11T19:28:51.793 INFO:tasks.workunit.client.0.smithi090.stdout:7/259: creat d3/dc/d17/d34/f50 x:0 0 0 2024-03-11T19:28:52.267 INFO:tasks.workunit.client.0.smithi090.stdout:8/287: dread d0/d43/dc/f3b [0,4194304] 0 2024-03-11T19:28:52.320 INFO:tasks.workunit.client.0.smithi090.stdout:2/247: rename d0/d21/d30/f28 to d0/de/d1f/f40 0 2024-03-11T19:28:52.479 INFO:tasks.workunit.client.0.smithi090.stdout:6/279: write d4/d14/f23 [4199572,110111] 0 2024-03-11T19:28:52.502 INFO:tasks.workunit.client.0.smithi090.stdout:0/271: creat d2/d11/d15/d29/f5b x:0 0 0 2024-03-11T19:28:52.551 INFO:tasks.workunit.client.0.smithi090.stdout:3/231: dwrite d2/f12 [0,4194304] 0 2024-03-11T19:28:52.653 INFO:tasks.workunit.client.0.smithi090.stdout:6/280: readlink d4/l3d 0 2024-03-11T19:28:52.675 INFO:tasks.workunit.client.0.smithi090.stdout:9/251: write d1/dd/df/f10 [2571054,39538] 0 2024-03-11T19:28:52.710 INFO:tasks.workunit.client.0.smithi090.stdout:3/232: chown d2/d1d 582 1 2024-03-11T19:28:53.078 INFO:tasks.workunit.client.0.smithi090.stdout:2/248: creat d0/d21/d30/d29/f41 x:0 0 0 2024-03-11T19:28:53.092 INFO:tasks.workunit.client.0.smithi090.stdout:8/288: dread - d0/d43/dc/f4a zero size 2024-03-11T19:28:53.213 INFO:tasks.workunit.client.0.smithi090.stdout:2/249: readlink d0/l8 0 2024-03-11T19:28:53.339 INFO:tasks.workunit.client.0.smithi090.stdout:4/281: dwrite d4/fa [0,4194304] 0 2024-03-11T19:28:53.448 INFO:tasks.workunit.client.0.smithi090.stdout:1/254: truncate d5/da/d23/f2d 883590 0 2024-03-11T19:28:53.471 INFO:tasks.workunit.client.0.smithi090.stdout:9/252: creat d1/dd/d18/f52 x:0 0 0 2024-03-11T19:28:53.597 INFO:tasks.workunit.client.0.smithi090.stdout:6/281: chown d4/d13/d22/d34/d37/l50 644 1 2024-03-11T19:28:53.861 INFO:tasks.workunit.client.0.smithi090.stdout:0/272: rename d2/d11/d15/d29/d43/f44 to d2/d11/d3f/f5c 0 2024-03-11T19:28:53.866 INFO:tasks.workunit.client.0.smithi090.stdout:7/260: write d3/f27 [270410,38159] 0 2024-03-11T19:28:54.223 INFO:tasks.workunit.client.0.smithi090.stdout:5/242: creat d1/f50 x:0 0 0 2024-03-11T19:28:54.239 INFO:tasks.workunit.client.0.smithi090.stdout:5/243: readlink d1/l29 0 2024-03-11T19:28:54.288 INFO:tasks.workunit.client.0.smithi090.stdout:8/289: stat d0/d3/c45 0 2024-03-11T19:28:54.380 INFO:tasks.workunit.client.0.smithi090.stdout:2/250: readlink d0/de/d12/l19 0 2024-03-11T19:28:54.650 INFO:tasks.workunit.client.0.smithi090.stdout:1/255: fsync d5/da/d23/f2b 0 2024-03-11T19:28:54.788 INFO:tasks.workunit.client.0.smithi090.stdout:4/282: creat d4/d27/d44/f65 x:0 0 0 2024-03-11T19:28:54.807 INFO:tasks.workunit.client.0.smithi090.stdout:4/283: readlink d4/l2f 0 2024-03-11T19:28:55.621 INFO:tasks.workunit.client.0.smithi090.stdout:3/233: dwrite d2/db/d15/d1a/f33 [0,4194304] 0 2024-03-11T19:28:56.228 INFO:tasks.workunit.client.0.smithi090.stdout:7/261: creat d3/dc/d17/d32/f51 x:0 0 0 2024-03-11T19:28:56.338 INFO:tasks.workunit.client.0.smithi090.stdout:5/244: fdatasync d1/f1e 0 2024-03-11T19:28:56.777 INFO:tasks.workunit.client.0.smithi090.stdout:9/253: write d1/dd/df/d1c/d21/d2a/f3d [524693,76489] 0 2024-03-11T19:28:56.813 INFO:tasks.workunit.client.0.smithi090.stdout:9/254: readlink d1/dd/d18/l24 0 2024-03-11T19:28:57.236 INFO:tasks.workunit.client.0.smithi090.stdout:6/282: dwrite d4/d13/f3b [0,4194304] 0 2024-03-11T19:28:57.262 INFO:tasks.workunit.client.0.smithi090.stdout:0/273: write d2/d11/d3f/f40 [134483,62730] 0 2024-03-11T19:28:57.895 INFO:tasks.workunit.client.0.smithi090.stdout:7/262: dread d3/dc/d13/f42 [0,4194304] 0 2024-03-11T19:28:59.069 INFO:tasks.workunit.client.0.smithi090.stdout:7/263: mkdir d3/dc/d17/d34/d52 0 2024-03-11T19:28:59.108 INFO:tasks.workunit.client.0.smithi090.stdout:8/290: link d0/d43/c39 d0/d3/d3e/c57 0 2024-03-11T19:28:59.239 INFO:tasks.workunit.client.0.smithi090.stdout:6/283: dread d4/fb [0,4194304] 0 2024-03-11T19:28:59.323 INFO:tasks.workunit.client.0.smithi090.stdout:7/264: readlink d3/dc/l1a 0 2024-03-11T19:28:59.338 INFO:tasks.workunit.client.0.smithi090.stdout:8/291: stat d0/d43/dc/f12 0 2024-03-11T19:28:59.528 INFO:tasks.workunit.client.0.smithi090.stdout:2/251: link d0/f3e d0/d21/f42 0 2024-03-11T19:29:00.147 INFO:tasks.workunit.client.0.smithi090.stdout:3/234: dwrite d2/db/d15/f17 [0,4194304] 0 2024-03-11T19:29:00.259 INFO:tasks.workunit.client.0.smithi090.stdout:4/284: rmdir d4/d51 39 2024-03-11T19:29:00.286 INFO:tasks.workunit.client.0.smithi090.stdout:0/274: mknod d2/d11/d15/d29/c5d 0 2024-03-11T19:29:00.298 INFO:tasks.workunit.client.0.smithi090.stdout:5/245: write d1/d26/d19/d27/f37 [472417,129485] 0 2024-03-11T19:29:00.367 INFO:tasks.workunit.client.0.smithi090.stdout:5/246: chown d1/f4a 183795 1 2024-03-11T19:29:00.587 INFO:tasks.workunit.client.0.smithi090.stdout:6/284: chown d4/c16 213083 1 2024-03-11T19:29:00.718 INFO:tasks.workunit.client.0.smithi090.stdout:7/265: fdatasync d3/dc/d17/d32/f4a 0 2024-03-11T19:29:01.062 INFO:tasks.workunit.client.0.smithi090.stdout:5/247: rmdir d1/d17/d39 39 2024-03-11T19:29:01.062 INFO:tasks.workunit.client.0.smithi090.stdout:4/285: mkdir d4/d27/d44/d66 0 2024-03-11T19:29:01.179 INFO:tasks.workunit.client.0.smithi090.stdout:6/285: chown d4/d13/f1b 5288 1 2024-03-11T19:29:01.772 INFO:tasks.workunit.client.0.smithi090.stdout:6/286: sync 2024-03-11T19:29:01.805 INFO:tasks.workunit.client.0.smithi090.stdout:9/255: dwrite d1/dd/df/f4a [0,4194304] 0 2024-03-11T19:29:01.829 INFO:tasks.workunit.client.0.smithi090.stdout:3/235: mkdir d2/d4f 0 2024-03-11T19:29:01.948 INFO:tasks.workunit.client.0.smithi090.stdout:1/256: dwrite d5/da/f47 [0,4194304] 0 2024-03-11T19:29:02.088 INFO:tasks.workunit.client.0.smithi090.stdout:2/252: truncate d0/f3e 386844 0 2024-03-11T19:29:02.127 INFO:tasks.workunit.client.0.smithi090.stdout:4/286: write d4/d27/d44/f65 [562153,95131] 0 2024-03-11T19:29:02.206 INFO:tasks.workunit.client.0.smithi090.stdout:3/236: chown d2/db/d15/d1a/d3c/l43 0 1 2024-03-11T19:29:02.338 INFO:tasks.workunit.client.0.smithi090.stdout:8/292: rename d0/d43/dc/f44 to d0/f58 0 2024-03-11T19:29:02.562 INFO:tasks.workunit.client.0.smithi090.stdout:1/257: creat d5/f4f x:0 0 0 2024-03-11T19:29:02.877 INFO:tasks.workunit.client.0.smithi090.stdout:2/253: mknod d0/d21/c43 0 2024-03-11T19:29:03.087 INFO:tasks.workunit.client.0.smithi090.stdout:3/237: fdatasync d2/f1b 0 2024-03-11T19:29:03.111 INFO:tasks.workunit.client.0.smithi090.stdout:7/266: write d3/dc/d13/f42 [1995666,60027] 0 2024-03-11T19:29:03.427 INFO:tasks.workunit.client.0.smithi090.stdout:0/275: getdents d2/d11/d3f 0 2024-03-11T19:29:03.677 INFO:tasks.workunit.client.0.smithi090.stdout:1/258: creat d5/da/d1c/d3b/d40/f50 x:0 0 0 2024-03-11T19:29:03.832 INFO:tasks.workunit.client.0.smithi090.stdout:2/254: dread d0/de/d12/f24 [0,4194304] 0 2024-03-11T19:29:04.080 INFO:tasks.workunit.client.0.smithi090.stdout:4/287: mknod d4/d27/d28/d38/d64/c67 0 2024-03-11T19:29:04.507 INFO:tasks.workunit.client.0.smithi090.stdout:7/267: creat d3/dc/d13/f53 x:0 0 0 2024-03-11T19:29:04.719 INFO:tasks.workunit.client.0.smithi090.stdout:8/293: rename d0/d40 to d0/d53/d59 0 2024-03-11T19:29:04.735 INFO:tasks.workunit.client.0.smithi090.stdout:9/256: symlink d1/d47/l53 0 2024-03-11T19:29:05.115 INFO:tasks.workunit.client.0.smithi090.stdout:5/248: getdents d1/d26/d19/d27 0 2024-03-11T19:29:05.341 INFO:tasks.workunit.client.0.smithi090.stdout:2/255: fdatasync d0/d21/d30/f39 0 2024-03-11T19:29:05.463 INFO:tasks.workunit.client.0.smithi090.stdout:6/287: link d4/d24/f3f d4/d13/d22/d3a/f56 0 2024-03-11T19:29:06.089 INFO:tasks.workunit.client.0.smithi090.stdout:9/257: rename d1/dd/d18/d3a/d48/f51 to d1/dd/df/d1c/d21/f54 0 2024-03-11T19:29:06.188 INFO:tasks.workunit.client.0.smithi090.stdout:5/249: fsync d1/f2f 0 2024-03-11T19:29:06.506 INFO:tasks.workunit.client.0.smithi090.stdout:8/294: dread d0/f2a [0,4194304] 0 2024-03-11T19:29:06.690 INFO:tasks.workunit.client.0.smithi090.stdout:2/256: rmdir d0/d21/d30/d25 39 2024-03-11T19:29:06.803 INFO:tasks.workunit.client.0.smithi090.stdout:6/288: symlink d4/d13/d22/d34/d37/l57 0 2024-03-11T19:29:07.142 INFO:tasks.workunit.client.0.smithi090.stdout:3/238: dwrite d2/f34 [0,4194304] 0 2024-03-11T19:29:07.321 INFO:tasks.workunit.client.0.smithi090.stdout:3/239: chown d2/f1b 83565635 1 2024-03-11T19:29:07.750 INFO:tasks.workunit.client.0.smithi090.stdout:9/258: dread d1/d1d/f2b [0,4194304] 0 2024-03-11T19:29:07.934 INFO:tasks.workunit.client.0.smithi090.stdout:0/276: dwrite d2/d11/d15/d29/f20 [0,4194304] 0 2024-03-11T19:29:08.142 INFO:tasks.workunit.client.0.smithi090.stdout:2/257: dread d0/f3 [0,4194304] 0 2024-03-11T19:29:08.175 INFO:tasks.workunit.client.0.smithi090.stdout:2/258: stat d0/c34 0 2024-03-11T19:29:08.260 INFO:tasks.workunit.client.0.smithi090.stdout:7/268: rmdir d3/d47 0 2024-03-11T19:29:08.462 INFO:tasks.workunit.client.0.smithi090.stdout:3/240: symlink d2/d2f/l50 0 2024-03-11T19:29:08.495 INFO:tasks.workunit.client.0.smithi090.stdout:7/269: sync 2024-03-11T19:29:08.685 INFO:tasks.workunit.client.0.smithi090.stdout:3/241: readlink d2/de/l10 0 2024-03-11T19:29:08.827 INFO:tasks.workunit.client.0.smithi090.stdout:4/288: dwrite d4/d27/f57 [0,4194304] 0 2024-03-11T19:29:08.944 INFO:tasks.workunit.client.0.smithi090.stdout:1/259: dwrite d5/f15 [4194304,4194304] 0 2024-03-11T19:29:09.021 INFO:tasks.workunit.client.0.smithi090.stdout:9/259: truncate d1/dd/f13 226857 0 2024-03-11T19:29:09.271 INFO:tasks.workunit.client.0.smithi090.stdout:8/295: creat d0/f5a x:0 0 0 2024-03-11T19:29:09.341 INFO:tasks.workunit.client.0.smithi090.stdout:9/260: stat d1/dd/df/d1c/d21/d2a/f3d 0 2024-03-11T19:29:09.528 INFO:tasks.workunit.client.0.smithi090.stdout:5/250: dwrite d1/d26/f4b [0,4194304] 0 2024-03-11T19:29:09.650 INFO:tasks.workunit.client.0.smithi090.stdout:0/277: creat d2/d11/d15/d29/d43/d58/f5e x:0 0 0 2024-03-11T19:29:09.766 INFO:tasks.workunit.client.0.smithi090.stdout:4/289: unlink d4/f5 0 2024-03-11T19:29:09.854 INFO:tasks.workunit.client.0.smithi090.stdout:6/289: dwrite d4/d13/d22/d34/f36 [0,4194304] 0 2024-03-11T19:29:10.001 INFO:tasks.workunit.client.0.smithi090.stdout:6/290: stat d4/d13/l4a 0 2024-03-11T19:29:10.760 INFO:tasks.workunit.client.0.smithi090.stdout:7/270: symlink d3/dc/d17/d34/d4d/l54 0 2024-03-11T19:29:10.782 INFO:tasks.workunit.client.0.smithi090.stdout:8/296: fdatasync d0/d43/dc/f3a 0 2024-03-11T19:29:11.022 INFO:tasks.workunit.client.0.smithi090.stdout:3/242: truncate d2/f8 1040238 0 2024-03-11T19:29:11.225 INFO:tasks.workunit.client.0.smithi090.stdout:5/251: dread d1/d26/f20 [0,4194304] 0 2024-03-11T19:29:11.377 INFO:tasks.workunit.client.0.smithi090.stdout:2/259: write d0/f3e [1312362,103046] 0 2024-03-11T19:29:11.379 INFO:tasks.workunit.client.0.smithi090.stdout:4/290: creat d4/d27/d28/f68 x:0 0 0 2024-03-11T19:29:11.418 INFO:tasks.workunit.client.0.smithi090.stdout:6/291: mkdir d4/d13/d58 0 2024-03-11T19:29:11.624 INFO:tasks.workunit.client.0.smithi090.stdout:0/278: truncate d2/d11/d15/d29/d2e/f52 652403 0 2024-03-11T19:29:11.679 INFO:tasks.workunit.client.0.smithi090.stdout:0/279: rename d2 to d2/d11/d15/d29/d2e/d5f 22 2024-03-11T19:29:12.686 INFO:tasks.workunit.client.0.smithi090.stdout:1/260: link d5/d31/c3e d5/da/d1c/d3b/d40/c51 0 2024-03-11T19:29:12.958 INFO:tasks.workunit.client.0.smithi090.stdout:8/297: symlink d0/d43/l5b 0 2024-03-11T19:29:13.254 INFO:tasks.workunit.client.0.smithi090.stdout:5/252: mkdir d1/d17/d39/d51 0 2024-03-11T19:29:13.540 INFO:tasks.workunit.client.0.smithi090.stdout:6/292: truncate d4/d13/d22/d3a/f56 17697 0 2024-03-11T19:29:13.742 INFO:tasks.workunit.client.0.smithi090.stdout:2/260: rename d0/de/d12/c33 to d0/d21/d30/d25/c44 0 2024-03-11T19:29:13.808 INFO:tasks.workunit.client.0.smithi090.stdout:1/261: mknod d5/da/d1c/d3b/d40/c52 0 2024-03-11T19:29:14.320 INFO:tasks.workunit.client.0.smithi090.stdout:8/298: dwrite d0/d43/dc/f3a [0,4194304] 0 2024-03-11T19:29:14.478 INFO:tasks.workunit.client.0.smithi090.stdout:6/293: creat d4/d13/d22/d34/f59 x:0 0 0 2024-03-11T19:29:14.837 INFO:tasks.workunit.client.0.smithi090.stdout:6/294: dwrite d4/d13/d22/d34/f59 [0,4194304] 0 2024-03-11T19:29:14.907 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:29:14.910 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:29:14.931 INFO:tasks.workunit.client.0.smithi090.stdout:1/262: symlink d5/d3c/d45/l53 0 2024-03-11T19:29:15.005 INFO:tasks.workunit.client.0.smithi090.stdout:3/243: getdents d2/d1f 0 2024-03-11T19:29:15.203 INFO:tasks.workunit.client.0.smithi090.stdout:7/271: write d3/dc/d13/f35 [1356992,96369] 0 2024-03-11T19:29:15.512 INFO:tasks.workunit.client.0.smithi090.stdout:8/299: mknod d0/d3/c5c 0 2024-03-11T19:29:15.789 INFO:tasks.workunit.client.0.smithi090.stdout:9/261: dwrite d1/dd/df/d1c/d21/f54 [0,4194304] 0 2024-03-11T19:29:15.891 INFO:tasks.workunit.client.0.smithi090.stdout:1/263: dread - d5/f4f zero size 2024-03-11T19:29:15.946 INFO:tasks.workunit.client.0.smithi090.stdout:3/244: mknod d2/d45/c51 0 2024-03-11T19:29:16.040 INFO:tasks.workunit.client.0.smithi090.stdout:4/291: write d4/f12 [951178,117166] 0 2024-03-11T19:29:16.376 INFO:tasks.workunit.client.0.smithi090.stdout:0/280: write d2/d11/d3f/f5c [698093,126627] 0 2024-03-11T19:29:16.483 INFO:tasks.workunit.client.0.smithi090.stdout:6/295: symlink d4/d13/d22/d34/d40/l5a 0 2024-03-11T19:29:16.539 INFO:tasks.workunit.client.0.smithi090.stdout:6/296: read d4/fb [3595980,76385] 0 2024-03-11T19:29:16.865 INFO:tasks.workunit.client.0.smithi090.stdout:1/264: rename d5/da/d1c/d3b/d4b to d5/d4e/d54 0 2024-03-11T19:29:17.142 INFO:tasks.workunit.client.0.smithi090.stdout:0/281: rmdir d2/d11/d15/d29 39 2024-03-11T19:29:17.416 INFO:tasks.workunit.client.0.smithi090.stdout:5/253: write d1/d26/f20 [2257296,96753] 0 2024-03-11T19:29:17.689 INFO:tasks.workunit.client.0.smithi090.stdout:6/297: rename d4/d13/l28 to d4/d13/d22/d42/d48/d4e/l5b 0 2024-03-11T19:29:17.724 INFO:tasks.workunit.client.0.smithi090.stdout:1/265: truncate d5/f1b 9142924 0 2024-03-11T19:29:17.776 INFO:tasks.workunit.client.0.smithi090.stdout:6/298: chown d4/d13/d22/d34/d37/l57 0 1 2024-03-11T19:29:17.983 INFO:tasks.workunit.client.0.smithi090.stdout:5/254: creat d1/d26/f52 x:0 0 0 2024-03-11T19:29:18.040 INFO:tasks.workunit.client.0.smithi090.stdout:5/255: stat d1/c10 0 2024-03-11T19:29:18.180 INFO:tasks.workunit.client.0.smithi090.stdout:5/256: chown d1/d26/d19/f21 59 1 2024-03-11T19:29:18.236 INFO:tasks.workunit.client.0.smithi090.stdout:2/261: dwrite d0/f7 [4194304,4194304] 0 2024-03-11T19:29:18.673 INFO:tasks.workunit.client.0.smithi090.stdout:2/262: sync 2024-03-11T19:29:18.885 INFO:tasks.workunit.client.0.smithi090.stdout:2/263: readlink d0/d21/d30/d25/l37 0 2024-03-11T19:29:19.036 INFO:tasks.workunit.client.0.smithi090.stdout:7/272: dwrite d3/dc/d1e/f46 [0,4194304] 0 2024-03-11T19:29:19.250 INFO:tasks.workunit.client.0.smithi090.stdout:3/245: write d2/ff [6140231,42317] 0 2024-03-11T19:29:19.468 INFO:tasks.workunit.client.0.smithi090.stdout:2/264: chown d0/de/d12/l19 124820631 1 2024-03-11T19:29:19.605 INFO:tasks.workunit.client.0.smithi090.stdout:0/282: write d2/f8 [4651670,96780] 0 2024-03-11T19:29:19.635 INFO:tasks.workunit.client.0.smithi090.stdout:6/299: write d4/d14/f23 [3572015,105788] 0 2024-03-11T19:29:19.754 INFO:tasks.workunit.client.0.smithi090.stdout:8/300: dwrite d0/f47 [0,4194304] 0 2024-03-11T19:29:19.934 INFO:tasks.workunit.client.0.smithi090.stdout:7/273: mkdir d3/d55 0 2024-03-11T19:29:19.974 INFO:tasks.workunit.client.0.smithi090.stdout:3/246: fdatasync d2/db/f1c 0 2024-03-11T19:29:20.324 INFO:tasks.workunit.client.0.smithi090.stdout:9/262: dwrite d1/dd/df/d1c/d21/f54 [4194304,4194304] 0 2024-03-11T19:29:20.473 INFO:tasks.workunit.client.0.smithi090.stdout:4/292: dwrite d4/fa [4194304,4194304] 0 2024-03-11T19:29:20.480 INFO:tasks.workunit.client.0.smithi090.stdout:2/265: mknod d0/de/d12/c45 0 2024-03-11T19:29:20.735 INFO:tasks.workunit.client.0.smithi090.stdout:5/257: dwrite d1/f50 [0,4194304] 0 2024-03-11T19:29:20.884 INFO:tasks.workunit.client.0.smithi090.stdout:4/293: read - d4/d27/f3b zero size 2024-03-11T19:29:21.020 INFO:tasks.workunit.client.0.smithi090.stdout:6/300: mknod d4/d14/c5c 0 2024-03-11T19:29:21.244 INFO:tasks.workunit.client.0.smithi090.stdout:8/301: creat d0/d53/f5d x:0 0 0 2024-03-11T19:29:21.244 INFO:tasks.workunit.client.0.smithi090.stdout:1/266: link d5/da/dd/l17 d5/l55 0 2024-03-11T19:29:21.590 INFO:tasks.workunit.client.0.smithi090.stdout:8/302: dread d0/d43/dc/f28 [0,4194304] 0 2024-03-11T19:29:21.757 INFO:tasks.workunit.client.0.smithi090.stdout:7/274: creat d3/dc/d17/f56 x:0 0 0 2024-03-11T19:29:21.817 INFO:tasks.workunit.client.0.smithi090.stdout:0/283: write d2/f3 [4361282,92433] 0 2024-03-11T19:29:21.982 INFO:tasks.workunit.client.0.smithi090.stdout:9/263: unlink d1/dd/d18/d25/f37 0 2024-03-11T19:29:22.011 INFO:tasks.workunit.client.0.smithi090.stdout:3/247: dread d2/d1d/fc [0,4194304] 0 2024-03-11T19:29:22.061 INFO:tasks.workunit.client.0.smithi090.stdout:3/248: chown d2/db/f2b 231 1 2024-03-11T19:29:22.174 INFO:tasks.workunit.client.0.smithi090.stdout:2/266: fsync d0/d21/d30/f39 0 2024-03-11T19:29:22.296 INFO:tasks.workunit.client.0.smithi090.stdout:4/294: mkdir d4/d27/d28/d69 0 2024-03-11T19:29:22.581 INFO:tasks.workunit.client.0.smithi090.stdout:1/267: truncate d5/f15 3024716 0 2024-03-11T19:29:22.918 INFO:tasks.workunit.client.0.smithi090.stdout:8/303: unlink d0/f32 0 2024-03-11T19:29:23.332 INFO:tasks.workunit.client.0.smithi090.stdout:5/258: mkdir d1/d17/d39/d51/d53 0 2024-03-11T19:29:23.557 INFO:tasks.workunit.client.0.smithi090.stdout:2/267: mkdir d0/de/d12/d46 0 2024-03-11T19:29:24.360 INFO:tasks.workunit.client.0.smithi090.stdout:9/264: rename d1/l8 to d1/dd/df/d1c/l55 0 2024-03-11T19:29:24.476 INFO:tasks.workunit.client.0.smithi090.stdout:3/249: creat d2/db/d15/d1a/d3c/f52 x:0 0 0 2024-03-11T19:29:24.722 INFO:tasks.workunit.client.0.smithi090.stdout:5/259: read - d1/d26/d19/f42 zero size 2024-03-11T19:29:25.378 INFO:tasks.workunit.client.0.smithi090.stdout:6/301: write d4/d14/f21 [1376053,14307] 0 2024-03-11T19:29:25.429 INFO:tasks.workunit.client.0.smithi090.stdout:1/268: write d5/da/f4a [528764,77417] 0 2024-03-11T19:29:25.677 INFO:tasks.workunit.client.0.smithi090.stdout:0/284: creat d2/d11/d15/f60 x:0 0 0 2024-03-11T19:29:25.987 INFO:tasks.workunit.client.0.smithi090.stdout:0/285: sync 2024-03-11T19:29:26.243 INFO:tasks.workunit.client.0.smithi090.stdout:7/275: dwrite d3/dc/d13/f25 [0,4194304] 0 2024-03-11T19:29:26.368 INFO:tasks.workunit.client.0.smithi090.stdout:8/304: dwrite d0/d53/d59/f55 [0,4194304] 0 2024-03-11T19:29:26.446 INFO:tasks.workunit.client.0.smithi090.stdout:4/295: mkdir d4/d51/d52/d6a 0 2024-03-11T19:29:26.507 INFO:tasks.workunit.client.0.smithi090.stdout:5/260: creat d1/d26/f54 x:0 0 0 2024-03-11T19:29:26.543 INFO:tasks.workunit.client.0.smithi090.stdout:4/296: chown d4/d27/d28/l32 2131677 1 2024-03-11T19:29:26.583 INFO:tasks.workunit.client.0.smithi090.stdout:4/297: read - d4/d18/f3c zero size 2024-03-11T19:29:26.615 INFO:tasks.workunit.client.0.smithi090.stdout:1/269: fdatasync d5/da/d1c/f30 0 2024-03-11T19:29:27.240 INFO:tasks.workunit.client.0.smithi090.stdout:3/250: mknod d2/d4f/c53 0 2024-03-11T19:29:27.257 INFO:tasks.workunit.client.0.smithi090.stdout:3/251: readlink d2/de/l10 0 2024-03-11T19:29:27.838 INFO:tasks.workunit.client.0.smithi090.stdout:9/265: dwrite d1/dd/f11 [4194304,4194304] 0 2024-03-11T19:29:27.979 INFO:tasks.workunit.client.0.smithi090.stdout:8/305: fsync d0/d43/dc/f3b 0 2024-03-11T19:29:28.032 INFO:tasks.workunit.client.0.smithi090.stdout:6/302: mknod d4/d13/d22/d34/d37/d47/c5d 0 2024-03-11T19:29:28.063 INFO:tasks.workunit.client.0.smithi090.stdout:7/276: truncate d3/dc/d17/d32/f3f 4211605 0 2024-03-11T19:29:28.109 INFO:tasks.workunit.client.0.smithi090.stdout:6/303: stat d4/d24/f43 0 2024-03-11T19:29:28.196 INFO:tasks.workunit.client.0.smithi090.stdout:8/306: dread d0/d43/dc/f3b [0,4194304] 0 2024-03-11T19:29:28.303 INFO:tasks.workunit.client.0.smithi090.stdout:4/298: dread d4/d27/f35 [0,4194304] 0 2024-03-11T19:29:28.325 INFO:tasks.workunit.client.0.smithi090.stdout:4/299: read - d4/d27/f4d zero size 2024-03-11T19:29:28.449 INFO:tasks.workunit.client.0.smithi090.stdout:3/252: truncate d2/d2f/f30 478531 0 2024-03-11T19:29:28.815 INFO:tasks.workunit.client.0.smithi090.stdout:2/268: link d0/d21/d30/f22 d0/d21/d30/f47 0 2024-03-11T19:29:29.129 INFO:tasks.workunit.client.0.smithi090.stdout:5/261: creat d1/d26/d4d/f55 x:0 0 0 2024-03-11T19:29:29.219 INFO:tasks.workunit.client.0.smithi090.stdout:9/266: rename d1/dd/d18/d25/l2c to d1/dd/d18/d32/l56 0 2024-03-11T19:29:29.494 INFO:tasks.workunit.client.0.smithi090.stdout:6/304: mknod d4/d13/d22/d42/d48/c5e 0 2024-03-11T19:29:29.678 INFO:tasks.workunit.client.0.smithi090.stdout:4/300: creat d4/d27/d28/d38/f6b x:0 0 0 2024-03-11T19:29:29.878 INFO:tasks.workunit.client.0.smithi090.stdout:2/269: fdatasync d0/de/d1f/f1c 0 2024-03-11T19:29:29.908 INFO:tasks.workunit.client.0.smithi090.stdout:2/270: chown d0/f3 43315 1 2024-03-11T19:29:30.019 INFO:tasks.workunit.client.0.smithi090.stdout:5/262: symlink d1/d26/d19/l56 0 2024-03-11T19:29:30.078 INFO:tasks.workunit.client.0.smithi090.stdout:5/263: chown d1/d26/d19/l23 3 1 2024-03-11T19:29:30.125 INFO:tasks.workunit.client.0.smithi090.stdout:0/286: write d2/d11/f41 [483165,63648] 0 2024-03-11T19:29:30.314 INFO:tasks.workunit.client.0.smithi090.stdout:0/287: sync 2024-03-11T19:29:30.439 INFO:tasks.workunit.client.0.smithi090.stdout:4/301: dread d4/d27/f57 [0,4194304] 0 2024-03-11T19:29:30.461 INFO:tasks.workunit.client.0.smithi090.stdout:4/302: readlink d4/d18/l54 0 2024-03-11T19:29:30.472 INFO:tasks.workunit.client.0.smithi090.stdout:4/303: readlink d4/l58 0 2024-03-11T19:29:31.020 INFO:tasks.workunit.client.0.smithi090.stdout:5/264: symlink d1/d26/d19/l57 0 2024-03-11T19:29:32.290 INFO:tasks.workunit.client.0.smithi090.stdout:0/288: rmdir d2/d11/d15/d29/d43 39 2024-03-11T19:29:32.626 INFO:tasks.workunit.client.0.smithi090.stdout:8/307: write d0/f31 [143774,33520] 0 2024-03-11T19:29:32.694 INFO:tasks.workunit.client.0.smithi090.stdout:5/265: mknod d1/d17/d39/d51/d53/c58 0 2024-03-11T19:29:32.794 INFO:tasks.workunit.client.0.smithi090.stdout:1/270: dwrite d5/f1b [4194304,4194304] 0 2024-03-11T19:29:33.017 INFO:tasks.workunit.client.0.smithi090.stdout:6/305: getdents d4/d14 0 2024-03-11T19:29:33.041 INFO:tasks.workunit.client.0.smithi090.stdout:7/277: dwrite d3/dc/d17/d32/f51 [0,4194304] 0 2024-03-11T19:29:33.236 INFO:tasks.workunit.client.0.smithi090.stdout:0/289: fsync d2/d11/d15/d29/f20 0 2024-03-11T19:29:33.301 INFO:tasks.workunit.client.0.smithi090.stdout:5/266: fdatasync d1/d26/fc 0 2024-03-11T19:29:33.847 INFO:tasks.workunit.client.0.smithi090.stdout:9/267: dwrite d1/f3 [8388608,4194304] 0 2024-03-11T19:29:33.901 INFO:tasks.workunit.client.0.smithi090.stdout:0/290: write d2/d11/d3f/f40 [1253410,8474] 0 2024-03-11T19:29:34.089 INFO:tasks.workunit.client.0.smithi090.stdout:3/253: dwrite d2/db/d15/d1a/f28 [0,4194304] 0 2024-03-11T19:29:34.336 INFO:tasks.workunit.client.0.smithi090.stdout:7/278: creat d3/dc/d1e/f57 x:0 0 0 2024-03-11T19:29:34.455 INFO:tasks.workunit.client.0.smithi090.stdout:4/304: dwrite d4/ff [0,4194304] 0 2024-03-11T19:29:34.693 INFO:tasks.workunit.client.0.smithi090.stdout:9/268: chown d1/l20 279314747 1 2024-03-11T19:29:34.699 INFO:tasks.workunit.client.0.smithi090.stdout:2/271: dwrite d0/d21/f3d [0,4194304] 0 2024-03-11T19:29:34.899 INFO:tasks.workunit.client.0.smithi090.stdout:9/269: chown d1/d1d/c1f 56436 1 2024-03-11T19:29:34.923 INFO:tasks.workunit.client.0.smithi090.stdout:4/305: dread d4/d27/f35 [0,4194304] 0 2024-03-11T19:29:35.007 INFO:tasks.workunit.client.0.smithi090.stdout:8/308: link d0/d43/f23 d0/d3/d4c/f5e 0 2024-03-11T19:29:35.110 INFO:tasks.workunit.client.0.smithi090.stdout:7/279: symlink d3/dc/d17/d34/d4d/l58 0 2024-03-11T19:29:35.190 INFO:tasks.workunit.client.0.smithi090.stdout:2/272: truncate d0/f3e 1504792 0 2024-03-11T19:29:35.341 INFO:tasks.workunit.client.0.smithi090.stdout:8/309: chown d0/d43/dc/fd 2640231 1 2024-03-11T19:29:35.530 INFO:tasks.workunit.client.0.smithi090.stdout:1/271: dwrite d5/fc [8388608,4194304] 0 2024-03-11T19:29:35.726 INFO:tasks.workunit.client.0.smithi090.stdout:6/306: dwrite d4/d14/f23 [4194304,4194304] 0 2024-03-11T19:29:36.003 INFO:tasks.workunit.client.0.smithi090.stdout:4/306: fdatasync d4/d27/d44/f5c 0 2024-03-11T19:29:36.010 INFO:tasks.workunit.client.0.smithi090.stdout:5/267: write d1/d26/f43 [535741,41402] 0 2024-03-11T19:29:36.146 INFO:tasks.workunit.client.0.smithi090.stdout:9/270: mkdir d1/dd/d18/d3a/d57 0 2024-03-11T19:29:36.532 INFO:tasks.workunit.client.0.smithi090.stdout:0/291: dwrite d2/d11/d15/d29/d43/f4d [0,4194304] 0 2024-03-11T19:29:36.758 INFO:tasks.workunit.client.0.smithi090.stdout:1/272: symlink d5/da/d1c/d3b/d40/l56 0 2024-03-11T19:29:36.785 INFO:tasks.workunit.client.0.smithi090.stdout:1/273: readlink d5/da/dd/l17 0 2024-03-11T19:29:36.926 INFO:tasks.workunit.client.0.smithi090.stdout:6/307: read d4/f11 [1830017,89110] 0 2024-03-11T19:29:37.697 INFO:tasks.workunit.client.0.smithi090.stdout:7/280: creat d3/f59 x:0 0 0 2024-03-11T19:29:37.720 INFO:tasks.workunit.client.0.smithi090.stdout:4/307: dread - d4/d27/d28/d38/f6b zero size 2024-03-11T19:29:38.070 INFO:tasks.workunit.client.0.smithi090.stdout:3/254: getdents d2/db 0 2024-03-11T19:29:38.109 INFO:tasks.workunit.client.0.smithi090.stdout:8/310: dwrite d0/d53/f5d [0,4194304] 0 2024-03-11T19:29:38.168 INFO:tasks.workunit.client.0.smithi090.stdout:2/273: creat d0/de/d12/d46/f48 x:0 0 0 2024-03-11T19:29:38.348 INFO:tasks.workunit.client.0.smithi090.stdout:0/292: read d2/d11/d15/d29/d2e/f32 [450124,109722] 0 2024-03-11T19:29:38.427 INFO:tasks.workunit.client.0.smithi090.stdout:6/308: creat d4/d24/f5f x:0 0 0 2024-03-11T19:29:38.664 INFO:tasks.workunit.client.0.smithi090.stdout:5/268: dwrite d1/f38 [0,4194304] 0 2024-03-11T19:29:39.199 INFO:tasks.workunit.client.0.smithi090.stdout:1/274: mknod d5/d31/d4c/c57 0 2024-03-11T19:29:39.401 INFO:tasks.workunit.client.0.smithi090.stdout:2/274: mkdir d0/de/d12/d46/d49 0 2024-03-11T19:29:39.500 INFO:tasks.workunit.client.0.smithi090.stdout:4/308: write d4/d27/f34 [4085530,39659] 0 2024-03-11T19:29:39.512 INFO:tasks.workunit.client.0.smithi090.stdout:8/311: truncate d0/f5a 744706 0 2024-03-11T19:29:39.548 INFO:tasks.workunit.client.0.smithi090.stdout:7/281: write d3/dc/d1e/f46 [217070,59736] 0 2024-03-11T19:29:39.707 INFO:tasks.workunit.client.0.smithi090.stdout:1/275: dread d5/fc [0,4194304] 0 2024-03-11T19:29:39.712 INFO:tasks.workunit.client.0.smithi090.stdout:9/271: write d1/dd/d18/d32/f34 [1030574,54307] 0 2024-03-11T19:29:39.888 INFO:tasks.workunit.client.0.smithi090.stdout:5/269: rmdir d1/d17/d39/d51/d53 39 2024-03-11T19:29:40.080 INFO:tasks.workunit.client.0.smithi090.stdout:8/312: dread d0/d53/f5d [0,4194304] 0 2024-03-11T19:29:40.265 INFO:tasks.workunit.client.0.smithi090.stdout:5/270: sync 2024-03-11T19:29:40.281 INFO:tasks.workunit.client.0.smithi090.stdout:3/255: write d2/d1d/f25 [896506,19583] 0 2024-03-11T19:29:40.465 INFO:tasks.workunit.client.0.smithi090.stdout:0/293: write d2/d11/d3f/f5c [1130642,4126] 0 2024-03-11T19:29:40.781 INFO:tasks.workunit.client.0.smithi090.stdout:2/275: dread d0/d21/d30/f47 [0,4194304] 0 2024-03-11T19:29:40.917 INFO:tasks.workunit.client.0.smithi090.stdout:2/276: sync 2024-03-11T19:29:41.332 INFO:tasks.workunit.client.0.smithi090.stdout:7/282: creat d3/dc/d13/f5a x:0 0 0 2024-03-11T19:29:41.486 INFO:tasks.workunit.client.0.smithi090.stdout:4/309: rename d4/d18/l54 to d4/d27/l6c 0 2024-03-11T19:29:41.665 INFO:tasks.workunit.client.0.smithi090.stdout:3/256: symlink d2/d2f/l54 0 2024-03-11T19:29:41.676 INFO:tasks.workunit.client.0.smithi090.stdout:5/271: mknod d1/d17/d39/c59 0 2024-03-11T19:29:41.703 INFO:tasks.workunit.client.0.smithi090.stdout:7/283: dread d3/dc/d1e/f11 [0,4194304] 0 2024-03-11T19:29:41.840 INFO:tasks.workunit.client.0.smithi090.stdout:8/313: dread d0/f31 [0,4194304] 0 2024-03-11T19:29:41.940 INFO:tasks.workunit.client.0.smithi090.stdout:2/277: mknod d0/d21/d30/d25/c4a 0 2024-03-11T19:29:41.993 INFO:tasks.workunit.client.0.smithi090.stdout:2/278: dread d0/f3 [0,4194304] 0 2024-03-11T19:29:42.061 INFO:tasks.workunit.client.0.smithi090.stdout:2/279: truncate d0/de/d12/d46/f48 153460 0 2024-03-11T19:29:42.094 INFO:tasks.workunit.client.0.smithi090.stdout:2/280: dread - d0/d21/d30/d29/f2e zero size 2024-03-11T19:29:42.260 INFO:tasks.workunit.client.0.smithi090.stdout:6/309: getdents d4/d24 0 2024-03-11T19:29:42.390 INFO:tasks.workunit.client.0.smithi090.stdout:4/310: rename d4/l59 to d4/d51/l6d 0 2024-03-11T19:29:42.872 INFO:tasks.workunit.client.0.smithi090.stdout:8/314: mknod d0/d3/d3e/c5f 0 2024-03-11T19:29:42.943 INFO:tasks.workunit.client.0.smithi090.stdout:9/272: creat d1/f58 x:0 0 0 2024-03-11T19:29:43.062 INFO:tasks.workunit.client.0.smithi090.stdout:1/276: link d5/da/d23/l33 d5/d3c/d45/l58 0 2024-03-11T19:29:43.635 INFO:tasks.workunit.client.0.smithi090.stdout:0/294: write d2/d11/d15/f60 [917551,114774] 0 2024-03-11T19:29:43.645 INFO:tasks.workunit.client.0.smithi090.stdout:5/272: creat d1/d26/d19/d27/d3b/f5a x:0 0 0 2024-03-11T19:29:43.703 INFO:tasks.workunit.client.0.smithi090.stdout:5/273: readlink d1/d26/l3a 0 2024-03-11T19:29:43.754 INFO:tasks.workunit.client.0.smithi090.stdout:7/284: creat d3/dc/d17/d32/d45/f5b x:0 0 0 2024-03-11T19:29:43.773 INFO:tasks.workunit.client.0.smithi090.stdout:5/274: chown d1/d17/c1c 67 1 2024-03-11T19:29:43.798 INFO:tasks.workunit.client.0.smithi090.stdout:5/275: stat d1/d26/f3c 0 2024-03-11T19:29:43.955 INFO:tasks.workunit.client.0.smithi090.stdout:7/285: dread d3/dc/f33 [0,4194304] 0 2024-03-11T19:29:44.066 INFO:tasks.workunit.client.0.smithi090.stdout:1/277: creat d5/d31/d4c/f59 x:0 0 0 2024-03-11T19:29:44.270 INFO:tasks.workunit.client.0.smithi090.stdout:6/310: mknod d4/d13/d58/c60 0 2024-03-11T19:29:44.467 INFO:tasks.workunit.client.0.smithi090.stdout:0/295: symlink d2/d11/d15/d36/l61 0 2024-03-11T19:29:44.507 INFO:tasks.workunit.client.0.smithi090.stdout:3/257: write d2/d1d/fc [3033734,98970] 0 2024-03-11T19:29:44.655 INFO:tasks.workunit.client.0.smithi090.stdout:5/276: read d1/d26/d19/f21 [3430970,45151] 0 2024-03-11T19:29:44.811 INFO:tasks.workunit.client.0.smithi090.stdout:8/315: fsync d0/f2a 0 2024-03-11T19:29:45.019 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:29:45.022 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:29:45.106 INFO:tasks.workunit.client.0.smithi090.stdout:4/311: dwrite d4/d27/d28/d38/f6b [0,4194304] 0 2024-03-11T19:29:45.178 INFO:tasks.workunit.client.0.smithi090.stdout:7/286: rmdir d3/dc/d17 39 2024-03-11T19:29:45.185 INFO:tasks.workunit.client.0.smithi090.stdout:1/278: mknod d5/da/dd/c5a 0 2024-03-11T19:29:45.242 INFO:tasks.workunit.client.0.smithi090.stdout:9/273: write d1/dd/f1e [515515,38097] 0 2024-03-11T19:29:45.277 INFO:tasks.workunit.client.0.smithi090.stdout:9/274: chown d1/dd/f30 827662604 1 2024-03-11T19:29:45.378 INFO:tasks.workunit.client.0.smithi090.stdout:3/258: mkdir d2/d45/d55 0 2024-03-11T19:29:45.464 INFO:tasks.workunit.client.0.smithi090.stdout:0/296: truncate d2/d11/d15/d29/d2e/f32 4491522 0 2024-03-11T19:29:45.811 INFO:tasks.workunit.client.0.smithi090.stdout:2/281: getdents d0/d21 0 2024-03-11T19:29:46.439 INFO:tasks.workunit.client.0.smithi090.stdout:1/279: mknod d5/d31/c5b 0 2024-03-11T19:29:46.752 INFO:tasks.workunit.client.0.smithi090.stdout:6/311: dwrite d4/fb [4194304,4194304] 0 2024-03-11T19:29:46.882 INFO:tasks.workunit.client.0.smithi090.stdout:6/312: truncate d4/d14/f21 4393238 0 2024-03-11T19:29:47.134 INFO:tasks.workunit.client.0.smithi090.stdout:5/277: dwrite d1/d17/f2a [0,4194304] 0 2024-03-11T19:29:47.332 INFO:tasks.workunit.client.0.smithi090.stdout:3/259: truncate d2/f2c 8139326 0 2024-03-11T19:29:47.344 INFO:tasks.workunit.client.0.smithi090.stdout:0/297: fsync d2/f37 0 2024-03-11T19:29:47.550 INFO:tasks.workunit.client.0.smithi090.stdout:3/260: rename d2/d45 to d2/d45/d56 22 2024-03-11T19:29:47.697 INFO:tasks.workunit.client.0.smithi090.stdout:2/282: chown d0/fc 3257084 1 2024-03-11T19:29:47.893 INFO:tasks.workunit.client.0.smithi090.stdout:5/278: dread d1/d26/d19/f21 [0,4194304] 0 2024-03-11T19:29:48.020 INFO:tasks.workunit.client.0.smithi090.stdout:9/275: symlink d1/dd/df/l59 0 2024-03-11T19:29:48.144 INFO:tasks.workunit.client.0.smithi090.stdout:6/313: mkdir d4/d13/d22/d42/d48/d4e/d61 0 2024-03-11T19:29:48.244 INFO:tasks.workunit.client.0.smithi090.stdout:8/316: dwrite d0/d3/f37 [0,4194304] 0 2024-03-11T19:29:48.488 INFO:tasks.workunit.client.0.smithi090.stdout:1/280: dread d5/da/f4a [0,4194304] 0 2024-03-11T19:29:48.625 INFO:tasks.workunit.client.0.smithi090.stdout:4/312: dwrite d4/d18/f2a [0,4194304] 0 2024-03-11T19:29:48.993 INFO:tasks.workunit.client.0.smithi090.stdout:5/279: creat d1/d26/d19/d27/f5b x:0 0 0 2024-03-11T19:29:49.372 INFO:tasks.workunit.client.0.smithi090.stdout:9/276: dread - d1/dd/d18/d32/f45 zero size 2024-03-11T19:29:49.481 INFO:tasks.workunit.client.0.smithi090.stdout:3/261: rename d2/d1d/l2d to d2/d1d/l57 0 2024-03-11T19:29:49.572 INFO:tasks.workunit.client.0.smithi090.stdout:1/281: read d5/da/dd/f1a [4418823,95934] 0 2024-03-11T19:29:49.673 INFO:tasks.workunit.client.0.smithi090.stdout:0/298: truncate d2/d11/f1a 347142 0 2024-03-11T19:29:49.790 INFO:tasks.workunit.client.0.smithi090.stdout:8/317: mkdir d0/d60 0 2024-03-11T19:29:49.810 INFO:tasks.workunit.client.0.smithi090.stdout:0/299: chown d2/d11/f17 1 1 2024-03-11T19:29:49.878 INFO:tasks.workunit.client.0.smithi090.stdout:4/313: dread d4/f2e [0,4194304] 0 2024-03-11T19:29:50.013 INFO:tasks.workunit.client.0.smithi090.stdout:5/280: truncate d1/d17/f30 865820 0 2024-03-11T19:29:50.549 INFO:tasks.workunit.client.0.smithi090.stdout:3/262: creat d2/de/f58 x:0 0 0 2024-03-11T19:29:50.846 INFO:tasks.workunit.client.0.smithi090.stdout:7/287: dwrite d3/dc/d17/d32/f3f [0,4194304] 0 2024-03-11T19:29:51.125 INFO:tasks.workunit.client.0.smithi090.stdout:4/314: creat d4/d18/f6e x:0 0 0 2024-03-11T19:29:51.330 INFO:tasks.workunit.client.0.smithi090.stdout:2/283: dwrite d0/d21/d30/d29/f2e [0,4194304] 0 2024-03-11T19:29:51.451 INFO:tasks.workunit.client.0.smithi090.stdout:5/281: symlink d1/d26/d4d/l5c 0 2024-03-11T19:29:51.504 INFO:tasks.workunit.client.0.smithi090.stdout:6/314: write d4/f46 [1102121,40776] 0 2024-03-11T19:29:52.043 INFO:tasks.workunit.client.0.smithi090.stdout:7/288: fdatasync d3/dc/d13/f53 0 2024-03-11T19:29:52.467 INFO:tasks.workunit.client.0.smithi090.stdout:2/284: rmdir d0/de 39 2024-03-11T19:29:52.594 INFO:tasks.workunit.client.0.smithi090.stdout:5/282: truncate d1/f32 968968 0 2024-03-11T19:29:52.827 INFO:tasks.workunit.client.0.smithi090.stdout:0/300: dwrite d2/f8 [0,4194304] 0 2024-03-11T19:29:52.944 INFO:tasks.workunit.client.0.smithi090.stdout:8/318: write d0/d3/d3e/f4e [539966,47998] 0 2024-03-11T19:29:52.981 INFO:tasks.workunit.client.0.smithi090.stdout:0/301: chown d2/f37 3304373 1 2024-03-11T19:29:53.309 INFO:tasks.workunit.client.0.smithi090.stdout:0/302: dread d2/f10 [4194304,4194304] 0 2024-03-11T19:29:53.547 INFO:tasks.workunit.client.0.smithi090.stdout:5/283: mknod d1/d17/d39/c5d 0 2024-03-11T19:29:54.088 INFO:tasks.workunit.client.0.smithi090.stdout:5/284: dread - d1/d26/f52 zero size 2024-03-11T19:29:54.467 INFO:tasks.workunit.client.0.smithi090.stdout:0/303: truncate d2/d11/f2b 1064758 0 2024-03-11T19:29:54.507 INFO:tasks.workunit.client.0.smithi090.stdout:4/315: write d4/d27/f4d [271566,129543] 0 2024-03-11T19:29:54.510 INFO:tasks.workunit.client.0.smithi090.stdout:8/319: creat d0/d43/f61 x:0 0 0 2024-03-11T19:29:55.093 INFO:tasks.workunit.client.0.smithi090.stdout:5/285: truncate d1/d26/f4b 279517 0 2024-03-11T19:29:55.162 INFO:tasks.workunit.client.0.smithi090.stdout:9/277: rename d1/la to d1/d49/l5a 0 2024-03-11T19:29:55.230 INFO:tasks.workunit.client.0.smithi090.stdout:6/315: creat d4/f62 x:0 0 0 2024-03-11T19:29:55.293 INFO:tasks.workunit.client.0.smithi090.stdout:2/285: write d0/de/f17 [911803,83490] 0 2024-03-11T19:29:55.339 INFO:tasks.workunit.client.0.smithi090.stdout:5/286: chown d1/f50 6674 1 2024-03-11T19:29:55.395 INFO:tasks.workunit.client.0.smithi090.stdout:4/316: mkdir d4/d18/d6f 0 2024-03-11T19:29:55.541 INFO:tasks.workunit.client.0.smithi090.stdout:9/278: stat d1/dd/d18/d32 0 2024-03-11T19:29:55.676 INFO:tasks.workunit.client.0.smithi090.stdout:3/263: dwrite d2/f8 [0,4194304] 0 2024-03-11T19:29:55.690 INFO:tasks.workunit.client.0.smithi090.stdout:8/320: write d0/f31 [218521,130527] 0 2024-03-11T19:29:55.861 INFO:tasks.workunit.client.0.smithi090.stdout:1/282: rename d5/da/d1c/l26 to d5/d31/l5c 0 2024-03-11T19:29:56.344 INFO:tasks.workunit.client.0.smithi090.stdout:6/316: creat d4/d13/d22/d34/d37/d47/f63 x:0 0 0 2024-03-11T19:29:56.362 INFO:tasks.workunit.client.0.smithi090.stdout:8/321: symlink d0/d53/d59/l62 0 2024-03-11T19:29:56.480 INFO:tasks.workunit.client.0.smithi090.stdout:8/322: chown d0/d43/l1e 167648489 1 2024-03-11T19:29:56.539 INFO:tasks.workunit.client.0.smithi090.stdout:9/279: dread d1/dd/d18/d32/f34 [0,4194304] 0 2024-03-11T19:29:56.648 INFO:tasks.workunit.client.0.smithi090.stdout:7/289: rename d3/dc/d17/d34/d4d/l54 to d3/dc/d17/d32/d45/l5c 0 2024-03-11T19:29:56.723 INFO:tasks.workunit.client.0.smithi090.stdout:9/280: dread d1/dd/d18/f38 [0,4194304] 0 2024-03-11T19:29:57.254 INFO:tasks.workunit.client.0.smithi090.stdout:2/286: creat d0/d21/d30/f4b x:0 0 0 2024-03-11T19:29:57.285 INFO:tasks.workunit.client.0.smithi090.stdout:5/287: write d1/d17/f2a [4159171,65722] 0 2024-03-11T19:29:57.462 INFO:tasks.workunit.client.0.smithi090.stdout:6/317: mkdir d4/d13/d64 0 2024-03-11T19:29:57.485 INFO:tasks.workunit.client.0.smithi090.stdout:4/317: write d4/d27/d44/f4e [8390278,62505] 0 2024-03-11T19:29:57.622 INFO:tasks.workunit.client.0.smithi090.stdout:4/318: dread d4/d27/f4d [0,4194304] 0 2024-03-11T19:29:57.762 INFO:tasks.workunit.client.0.smithi090.stdout:2/287: sync 2024-03-11T19:29:58.041 INFO:tasks.workunit.client.0.smithi090.stdout:1/283: write d5/da/d1c/d3b/d40/f43 [18578,63811] 0 2024-03-11T19:29:58.120 INFO:tasks.workunit.client.0.smithi090.stdout:8/323: read d0/d43/dc/f12 [2462126,115227] 0 2024-03-11T19:29:58.293 INFO:tasks.workunit.client.0.smithi090.stdout:0/304: rename d2/d11/d15/d29/f20 to d2/d11/d3f/f62 0 2024-03-11T19:29:58.982 INFO:tasks.workunit.client.0.smithi090.stdout:4/319: mknod d4/d27/d28/d38/d64/c70 0 2024-03-11T19:29:58.997 INFO:tasks.workunit.client.0.smithi090.stdout:6/318: dwrite d4/f46 [0,4194304] 0 2024-03-11T19:29:59.155 INFO:tasks.workunit.client.0.smithi090.stdout:6/319: chown d4/d13/d22/d34 512780710 1 2024-03-11T19:29:59.242 INFO:tasks.workunit.client.0.smithi090.stdout:7/290: rename d3/dc/d13 to d3/dc/d17/d34/d52/d5d 0 2024-03-11T19:29:59.388 INFO:tasks.workunit.client.0.smithi090.stdout:0/305: symlink d2/d11/d15/d29/d2e/l63 0 2024-03-11T19:29:59.415 INFO:tasks.workunit.client.0.smithi090.stdout:3/264: getdents d2/d2f 0 2024-03-11T19:29:59.457 INFO:tasks.workunit.client.0.smithi090.stdout:0/306: chown d2/d11/d3f/c4c 44 1 2024-03-11T19:29:59.472 INFO:tasks.workunit.client.0.smithi090.stdout:3/265: read d2/db/d15/f18 [1637405,52287] 0 2024-03-11T19:29:59.493 INFO:tasks.workunit.client.0.smithi090.stdout:9/281: creat d1/dd/d18/d3a/d57/f5b x:0 0 0 2024-03-11T19:29:59.527 INFO:tasks.workunit.client.0.smithi090.stdout:3/266: stat d2/f1b 0 2024-03-11T19:29:59.550 INFO:tasks.workunit.client.0.smithi090.stdout:3/267: readlink d2/de/l10 0 2024-03-11T19:29:59.771 INFO:tasks.workunit.client.0.smithi090.stdout:8/324: getdents d0/d60 0 2024-03-11T19:30:00.073 INFO:tasks.workunit.client.0.smithi090.stdout:6/320: creat d4/d13/d22/d3a/f65 x:0 0 0 2024-03-11T19:30:00.314 INFO:tasks.workunit.client.0.smithi090.stdout:0/307: mkdir d2/d11/d3a/d64 0 2024-03-11T19:30:00.456 INFO:tasks.workunit.client.0.smithi090.stdout:9/282: mkdir d1/dd/df/d1c/d21/d5c 0 2024-03-11T19:30:00.591 INFO:tasks.workunit.client.0.smithi090.stdout:3/268: mkdir d2/d4f/d59 0 2024-03-11T19:30:00.614 INFO:tasks.workunit.client.0.smithi090.stdout:5/288: creat d1/d17/d39/d51/f5e x:0 0 0 2024-03-11T19:30:00.909 INFO:tasks.workunit.client.0.smithi090.stdout:4/320: symlink d4/d27/d28/d69/l71 0 2024-03-11T19:30:00.937 INFO:tasks.workunit.client.0.smithi090.stdout:4/321: chown d4/d27/d28/c48 1446056 1 2024-03-11T19:30:01.189 INFO:tasks.workunit.client.0.smithi090.stdout:8/325: dread d0/d43/dc/f3a [0,4194304] 0 2024-03-11T19:30:01.231 INFO:tasks.workunit.client.0.smithi090.stdout:8/326: write d0/f31 [6275,2987] 0 2024-03-11T19:30:01.570 INFO:tasks.workunit.client.0.smithi090.stdout:6/321: truncate d4/d13/d22/d34/f59 3228766 0 2024-03-11T19:30:01.767 INFO:tasks.workunit.client.0.smithi090.stdout:0/308: creat d2/d11/d15/d29/d43/d58/f65 x:0 0 0 2024-03-11T19:30:01.818 INFO:tasks.workunit.client.0.smithi090.stdout:0/309: read - d2/d11/d15/d29/d43/d58/f65 zero size 2024-03-11T19:30:02.255 INFO:tasks.workunit.client.0.smithi090.stdout:7/291: dwrite d3/dc/d17/d34/d52/d5d/f53 [0,4194304] 0 2024-03-11T19:30:02.597 INFO:tasks.workunit.client.0.smithi090.stdout:2/288: getdents d0/de/d12 0 2024-03-11T19:30:02.758 INFO:tasks.workunit.client.0.smithi090.stdout:8/327: symlink d0/d3/d4c/l63 0 2024-03-11T19:30:03.038 INFO:tasks.workunit.client.0.smithi090.stdout:6/322: truncate d4/ff 3906956 0 2024-03-11T19:30:03.052 INFO:tasks.workunit.client.0.smithi090.stdout:1/284: rename d5/da/c22 to d5/c5d 0 2024-03-11T19:30:03.994 INFO:tasks.workunit.client.0.smithi090.stdout:5/289: write d1/f4c [36620,28753] 0 2024-03-11T19:30:04.076 INFO:tasks.workunit.client.0.smithi090.stdout:9/283: dwrite d1/dd/f3c [0,4194304] 0 2024-03-11T19:30:04.266 INFO:tasks.workunit.client.0.smithi090.stdout:2/289: mkdir d0/de/d12/d46/d4c 0 2024-03-11T19:30:04.365 INFO:tasks.workunit.client.0.smithi090.stdout:8/328: truncate d0/d43/dc/f12 4395427 0 2024-03-11T19:30:04.414 INFO:tasks.workunit.client.0.smithi090.stdout:1/285: rmdir d5/da/d23 39 2024-03-11T19:30:04.460 INFO:tasks.workunit.client.0.smithi090.stdout:1/286: read d5/da/dd/f1a [9932299,11073] 0 2024-03-11T19:30:04.550 INFO:tasks.workunit.client.0.smithi090.stdout:6/323: symlink d4/d13/d22/d42/l66 0 2024-03-11T19:30:04.793 INFO:tasks.workunit.client.0.smithi090.stdout:3/269: creat d2/f5a x:0 0 0 2024-03-11T19:30:04.841 INFO:tasks.workunit.client.0.smithi090.stdout:3/270: read d2/db/f1c [326442,11323] 0 2024-03-11T19:30:05.449 INFO:tasks.workunit.client.0.smithi090.stdout:5/290: fsync d1/d26/f3c 0 2024-03-11T19:30:05.665 INFO:tasks.workunit.client.0.smithi090.stdout:0/310: write d2/d11/d15/d36/f57 [660459,126317] 0 2024-03-11T19:30:05.899 INFO:tasks.workunit.client.0.smithi090.stdout:6/324: mkdir d4/d13/d22/d3a/d67 0 2024-03-11T19:30:05.932 INFO:tasks.workunit.client.0.smithi090.stdout:9/284: dread d1/dd/df/d1c/d21/d2a/f29 [4194304,4194304] 0 2024-03-11T19:30:05.978 INFO:tasks.workunit.client.0.smithi090.stdout:9/285: readlink d1/d1d/l2f 0 2024-03-11T19:30:06.214 INFO:tasks.workunit.client.0.smithi090.stdout:3/271: mknod d2/de/c5b 0 2024-03-11T19:30:06.230 INFO:tasks.workunit.client.0.smithi090.stdout:4/322: truncate d4/d27/d44/f4e 3312527 0 2024-03-11T19:30:06.435 INFO:tasks.workunit.client.0.smithi090.stdout:7/292: link d3/dc/d1e/f12 d3/dc/d1e/d2a/f5e 0 2024-03-11T19:30:06.454 INFO:tasks.workunit.client.0.smithi090.stdout:2/290: mknod d0/de/d12/d46/d4c/c4d 0 2024-03-11T19:30:06.641 INFO:tasks.workunit.client.0.smithi090.stdout:5/291: chown d1/d26/l47 627161 1 2024-03-11T19:30:06.889 INFO:tasks.workunit.client.0.smithi090.stdout:8/329: unlink d0/d3/d3e/c57 0 2024-03-11T19:30:07.238 INFO:tasks.workunit.client.0.smithi090.stdout:0/311: symlink d2/d11/d15/d29/d43/l66 0 2024-03-11T19:30:07.383 INFO:tasks.workunit.client.0.smithi090.stdout:6/325: creat d4/d13/d22/d34/f68 x:0 0 0 2024-03-11T19:30:07.409 INFO:tasks.workunit.client.0.smithi090.stdout:6/326: rename d4 to d4/d24/d69 22 2024-03-11T19:30:07.542 INFO:tasks.workunit.client.0.smithi090.stdout:9/286: creat d1/dd/d18/d3a/d57/f5d x:0 0 0 2024-03-11T19:30:07.570 INFO:tasks.workunit.client.0.smithi090.stdout:9/287: chown d1/dd/df/d1c 0 1 2024-03-11T19:30:07.597 INFO:tasks.workunit.client.0.smithi090.stdout:4/323: creat d4/d51/f72 x:0 0 0 2024-03-11T19:30:07.950 INFO:tasks.workunit.client.0.smithi090.stdout:5/292: truncate d1/f2f 362398 0 2024-03-11T19:30:08.073 INFO:tasks.workunit.client.0.smithi090.stdout:3/272: dread d2/d1d/f27 [0,4194304] 0 2024-03-11T19:30:08.135 INFO:tasks.workunit.client.0.smithi090.stdout:0/312: truncate d2/d11/d15/d29/f5b 665234 0 2024-03-11T19:30:08.308 INFO:tasks.workunit.client.0.smithi090.stdout:6/327: creat d4/d14/f6a x:0 0 0 2024-03-11T19:30:08.487 INFO:tasks.workunit.client.0.smithi090.stdout:0/313: dread d2/f10 [0,4194304] 0 2024-03-11T19:30:09.760 INFO:tasks.workunit.client.0.smithi090.stdout:4/324: symlink d4/d18/d6f/l73 0 2024-03-11T19:30:09.905 INFO:tasks.workunit.client.0.smithi090.stdout:2/291: creat d0/f4e x:0 0 0 2024-03-11T19:30:09.920 INFO:tasks.workunit.client.0.smithi090.stdout:1/287: write d5/da/d23/f2a [2696344,97762] 0 2024-03-11T19:30:10.148 INFO:tasks.workunit.client.0.smithi090.stdout:4/325: mkdir d4/d27/d44/d74 0 2024-03-11T19:30:10.695 INFO:tasks.workunit.client.0.smithi090.stdout:1/288: creat d5/d4e/f5e x:0 0 0 2024-03-11T19:30:10.708 INFO:tasks.workunit.client.0.smithi090.stdout:8/330: write d0/d43/f2e [1174031,90912] 0 2024-03-11T19:30:10.811 INFO:tasks.workunit.client.0.smithi090.stdout:4/326: truncate d4/d18/f6e 356434 0 2024-03-11T19:30:10.913 INFO:tasks.workunit.client.0.smithi090.stdout:2/292: truncate d0/d21/f42 1382916 0 2024-03-11T19:30:10.991 INFO:tasks.workunit.client.0.smithi090.stdout:6/328: link d4/d13/d22/d42/d48/d4e/l5b d4/d13/d22/d3a/d67/l6b 0 2024-03-11T19:30:11.027 INFO:tasks.workunit.client.0.smithi090.stdout:6/329: read d4/f11 [3777477,38268] 0 2024-03-11T19:30:11.518 INFO:tasks.workunit.client.0.smithi090.stdout:1/289: mknod d5/d31/c5f 0 2024-03-11T19:30:11.565 INFO:tasks.workunit.client.0.smithi090.stdout:2/293: mknod d0/de/d1f/c4f 0 2024-03-11T19:30:11.820 INFO:tasks.workunit.client.0.smithi090.stdout:9/288: dwrite d1/d1d/f2b [0,4194304] 0 2024-03-11T19:30:11.952 INFO:tasks.workunit.client.0.smithi090.stdout:4/327: unlink d4/d27/d28/l2c 0 2024-03-11T19:30:12.322 INFO:tasks.workunit.client.0.smithi090.stdout:5/293: dwrite d1/d26/f3f [0,4194304] 0 2024-03-11T19:30:12.493 INFO:tasks.workunit.client.0.smithi090.stdout:8/331: write d0/d3/f2c [645204,25984] 0 2024-03-11T19:30:12.675 INFO:tasks.workunit.client.0.smithi090.stdout:2/294: mknod d0/de/d12/d46/d49/c50 0 2024-03-11T19:30:12.718 INFO:tasks.workunit.client.0.smithi090.stdout:5/294: dread - d1/d26/f52 zero size 2024-03-11T19:30:12.794 INFO:tasks.workunit.client.0.smithi090.stdout:8/332: stat d0/d43/dc/l1f 0 2024-03-11T19:30:13.036 INFO:tasks.workunit.client.0.smithi090.stdout:5/295: read - d1/f2c zero size 2024-03-11T19:30:13.072 INFO:tasks.workunit.client.0.smithi090.stdout:3/273: dwrite d2/db/f24 [0,4194304] 0 2024-03-11T19:30:13.092 INFO:tasks.workunit.client.0.smithi090.stdout:2/295: chown d0/f7 7987 1 2024-03-11T19:30:13.211 INFO:tasks.workunit.client.0.smithi090.stdout:0/314: dwrite d2/d11/d3a/f42 [0,4194304] 0 2024-03-11T19:30:13.323 INFO:tasks.workunit.client.0.smithi090.stdout:4/328: fdatasync d4/d18/f25 0 2024-03-11T19:30:13.451 INFO:tasks.workunit.client.0.smithi090.stdout:8/333: chown d0/f17 208650016 1 2024-03-11T19:30:13.516 INFO:tasks.workunit.client.0.smithi090.stdout:7/293: mknod d3/c5f 0 2024-03-11T19:30:13.735 INFO:tasks.workunit.client.0.smithi090.stdout:3/274: fsync d2/db/f1c 0 2024-03-11T19:30:13.742 INFO:tasks.workunit.client.0.smithi090.stdout:6/330: dwrite d4/d13/d22/d34/d37/d47/f63 [0,4194304] 0 2024-03-11T19:30:13.901 INFO:tasks.workunit.client.0.smithi090.stdout:5/296: mknod d1/d26/d19/c5f 0 2024-03-11T19:30:13.916 INFO:tasks.workunit.client.0.smithi090.stdout:3/275: dread - d2/d1f/f35 zero size 2024-03-11T19:30:13.959 INFO:tasks.workunit.client.0.smithi090.stdout:3/276: chown d2/d2f/l50 4 1 2024-03-11T19:30:14.289 INFO:tasks.workunit.client.0.smithi090.stdout:8/334: fsync d0/f20 0 2024-03-11T19:30:14.292 INFO:tasks.workunit.client.0.smithi090.stdout:4/329: mkdir d4/d27/d28/d38/d75 0 2024-03-11T19:30:14.449 INFO:tasks.workunit.client.0.smithi090.stdout:7/294: rmdir d3/dc/d1e 39 2024-03-11T19:30:14.714 INFO:tasks.workunit.client.0.smithi090.stdout:2/296: symlink d0/de/l51 0 2024-03-11T19:30:15.123 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:30:15.126 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:30:15.127 INFO:tasks.workunit.client.0.smithi090.stdout:1/290: write d5/da/dd/f14 [1097104,87800] 0 2024-03-11T19:30:15.563 INFO:tasks.workunit.client.0.smithi090.stdout:9/289: dwrite d1/dd/d18/f31 [0,4194304] 0 2024-03-11T19:30:16.348 INFO:tasks.workunit.client.0.smithi090.stdout:9/290: readlink d1/dd/df/d1c/l55 0 2024-03-11T19:30:16.755 INFO:tasks.workunit.client.0.smithi090.stdout:0/315: getdents d2/d11/d15/d29/d43/d58 0 2024-03-11T19:30:17.240 INFO:tasks.workunit.client.0.smithi090.stdout:0/316: dread d2/d11/d3f/f5c [0,4194304] 0 2024-03-11T19:30:17.541 INFO:tasks.workunit.client.0.smithi090.stdout:3/277: write d2/f34 [2825273,123781] 0 2024-03-11T19:30:17.552 INFO:tasks.workunit.client.0.smithi090.stdout:5/297: write d1/d26/f43 [456906,47095] 0 2024-03-11T19:30:17.610 INFO:tasks.workunit.client.0.smithi090.stdout:8/335: write d0/d43/f23 [844341,17617] 0 2024-03-11T19:30:17.714 INFO:tasks.workunit.client.0.smithi090.stdout:7/295: write d3/dc/f3c [615786,123825] 0 2024-03-11T19:30:17.827 INFO:tasks.workunit.client.0.smithi090.stdout:9/291: write d1/dd/d18/d32/f34 [884509,47179] 0 2024-03-11T19:30:17.867 INFO:tasks.workunit.client.0.smithi090.stdout:0/317: stat d2/d11/c54 0 2024-03-11T19:30:17.922 INFO:tasks.workunit.client.0.smithi090.stdout:2/297: fdatasync d0/f3e 0 2024-03-11T19:30:18.438 INFO:tasks.workunit.client.0.smithi090.stdout:7/296: fsync d3/dc/d29/f4e 0 2024-03-11T19:30:18.594 INFO:tasks.workunit.client.0.smithi090.stdout:6/331: dwrite d4/d13/f3b [0,4194304] 0 2024-03-11T19:30:18.750 INFO:tasks.workunit.client.0.smithi090.stdout:5/298: creat d1/f60 x:0 0 0 2024-03-11T19:30:18.784 INFO:tasks.workunit.client.0.smithi090.stdout:4/330: dwrite d4/d18/f6e [0,4194304] 0 2024-03-11T19:30:18.938 INFO:tasks.workunit.client.0.smithi090.stdout:1/291: dwrite d5/f4f [0,4194304] 0 2024-03-11T19:30:19.124 INFO:tasks.workunit.client.0.smithi090.stdout:5/299: chown d1/d17 1874 1 2024-03-11T19:30:19.178 INFO:tasks.workunit.client.0.smithi090.stdout:6/332: sync 2024-03-11T19:30:19.209 INFO:tasks.workunit.client.0.smithi090.stdout:0/318: write d2/d11/d15/d29/f5b [463359,57558] 0 2024-03-11T19:30:19.356 INFO:tasks.workunit.client.0.smithi090.stdout:5/300: chown d1/d26/d19/l1d 3 1 2024-03-11T19:30:19.415 INFO:tasks.workunit.client.0.smithi090.stdout:9/292: dwrite d1/d1d/f2b [0,4194304] 0 2024-03-11T19:30:19.554 INFO:tasks.workunit.client.0.smithi090.stdout:3/278: getdents d2/db/d15 0 2024-03-11T19:30:19.563 INFO:tasks.workunit.client.0.smithi090.stdout:2/298: creat d0/de/f52 x:0 0 0 2024-03-11T19:30:19.568 INFO:tasks.workunit.client.0.smithi090.stdout:7/297: mknod d3/dc/d17/c60 0 2024-03-11T19:30:19.675 INFO:tasks.workunit.client.0.smithi090.stdout:4/331: truncate d4/fa 5119044 0 2024-03-11T19:30:19.719 INFO:tasks.workunit.client.0.smithi090.stdout:8/336: link d0/f17 d0/d43/f64 0 2024-03-11T19:30:19.771 INFO:tasks.workunit.client.0.smithi090.stdout:1/292: rename d5/d31/f3f to d5/f60 0 2024-03-11T19:30:19.793 INFO:tasks.workunit.client.0.smithi090.stdout:9/293: dread - d1/f4f zero size 2024-03-11T19:30:20.037 INFO:tasks.workunit.client.0.smithi090.stdout:5/301: dread d1/f1e [0,4194304] 0 2024-03-11T19:30:20.065 INFO:tasks.workunit.client.0.smithi090.stdout:9/294: write d1/dd/d18/d32/f34 [424098,50046] 0 2024-03-11T19:30:20.297 INFO:tasks.workunit.client.0.smithi090.stdout:2/299: rmdir d0/de/d12/d46/d4c 39 2024-03-11T19:30:20.327 INFO:tasks.workunit.client.0.smithi090.stdout:3/279: fsync d2/db/d15/f48 0 2024-03-11T19:30:20.362 INFO:tasks.workunit.client.0.smithi090.stdout:8/337: unlink d0/d53/d59/f55 0 2024-03-11T19:30:20.418 INFO:tasks.workunit.client.0.smithi090.stdout:2/300: sync 2024-03-11T19:30:20.435 INFO:tasks.workunit.client.0.smithi090.stdout:1/293: fdatasync d5/d31/d4c/f59 0 2024-03-11T19:30:20.502 INFO:tasks.workunit.client.0.smithi090.stdout:5/302: truncate d1/d26/fc 1785945 0 2024-03-11T19:30:20.752 INFO:tasks.workunit.client.0.smithi090.stdout:2/301: readlink d0/de/d1f/l3c 0 2024-03-11T19:30:20.774 INFO:tasks.workunit.client.0.smithi090.stdout:7/298: symlink d3/dc/l61 0 2024-03-11T19:30:20.943 INFO:tasks.workunit.client.0.smithi090.stdout:0/319: dwrite d2/d11/d15/d29/f4a [0,4194304] 0 2024-03-11T19:30:21.123 INFO:tasks.workunit.client.0.smithi090.stdout:6/333: dwrite d4/f46 [0,4194304] 0 2024-03-11T19:30:21.348 INFO:tasks.workunit.client.0.smithi090.stdout:1/294: symlink d5/d4e/l61 0 2024-03-11T19:30:21.355 INFO:tasks.workunit.client.0.smithi090.stdout:5/303: rmdir d1/d26/d19/d27/d3b 39 2024-03-11T19:30:21.427 INFO:tasks.workunit.client.0.smithi090.stdout:0/320: dread - d2/f37 zero size 2024-03-11T19:30:21.456 INFO:tasks.workunit.client.0.smithi090.stdout:7/299: readlink d3/dc/d17/d34/d4d/l58 0 2024-03-11T19:30:21.551 INFO:tasks.workunit.client.0.smithi090.stdout:0/321: sync 2024-03-11T19:30:21.648 INFO:tasks.workunit.client.0.smithi090.stdout:5/304: chown d1/d17/f2a 28014952 1 2024-03-11T19:30:21.650 INFO:tasks.workunit.client.0.smithi090.stdout:2/302: unlink d0/d21/d30/f4b 0 2024-03-11T19:30:21.680 INFO:tasks.workunit.client.0.smithi090.stdout:4/332: dwrite d4/d27/d44/f5c [0,4194304] 0 2024-03-11T19:30:21.996 INFO:tasks.workunit.client.0.smithi090.stdout:3/280: dwrite d2/f5a [0,4194304] 0 2024-03-11T19:30:21.996 INFO:tasks.workunit.client.0.smithi090.stdout:5/305: dread - d1/d26/d19/d27/f44 zero size 2024-03-11T19:30:22.138 INFO:tasks.workunit.client.0.smithi090.stdout:8/338: creat d0/f65 x:0 0 0 2024-03-11T19:30:23.147 INFO:tasks.workunit.client.0.smithi090.stdout:6/334: truncate d4/d24/f5f 266483 0 2024-03-11T19:30:23.254 INFO:tasks.workunit.client.0.smithi090.stdout:6/335: chown d4/d13/d22/d34 678292 1 2024-03-11T19:30:23.324 INFO:tasks.workunit.client.0.smithi090.stdout:9/295: getdents d1/dd/d18/d3a/d57 0 2024-03-11T19:30:23.469 INFO:tasks.workunit.client.0.smithi090.stdout:3/281: mknod d2/d4f/c5c 0 2024-03-11T19:30:23.634 INFO:tasks.workunit.client.0.smithi090.stdout:8/339: dread d0/d3/f37 [0,4194304] 0 2024-03-11T19:30:23.740 INFO:tasks.workunit.client.0.smithi090.stdout:5/306: rename d1/d26/d4d/l5c to d1/d26/d4d/l61 0 2024-03-11T19:30:23.767 INFO:tasks.workunit.client.0.smithi090.stdout:5/307: chown d1/d26/d19/l57 1704131 1 2024-03-11T19:30:24.130 INFO:tasks.workunit.client.0.smithi090.stdout:1/295: write d5/da/f4a [875941,87428] 0 2024-03-11T19:30:24.215 INFO:tasks.workunit.client.0.smithi090.stdout:2/303: creat d0/de/d12/d46/d4c/f53 x:0 0 0 2024-03-11T19:30:24.656 INFO:tasks.workunit.client.0.smithi090.stdout:9/296: rmdir d1/dd/d18/d3a 39 2024-03-11T19:30:24.672 INFO:tasks.workunit.client.0.smithi090.stdout:9/297: chown d1/d1d 509 1 2024-03-11T19:30:24.950 INFO:tasks.workunit.client.0.smithi090.stdout:4/333: write d4/d27/d28/f30 [2458259,31789] 0 2024-03-11T19:30:25.067 INFO:tasks.workunit.client.0.smithi090.stdout:5/308: dread d1/d26/f43 [0,4194304] 0 2024-03-11T19:30:25.113 INFO:tasks.workunit.client.0.smithi090.stdout:5/309: chown d1/d26 0 1 2024-03-11T19:30:25.607 INFO:tasks.workunit.client.0.smithi090.stdout:6/336: truncate d4/d13/d22/d34/f59 1270583 0 2024-03-11T19:30:26.172 INFO:tasks.workunit.client.0.smithi090.stdout:8/340: write d0/d43/f61 [606667,39665] 0 2024-03-11T19:30:26.308 INFO:tasks.workunit.client.0.smithi090.stdout:7/300: dwrite d3/dc/d1e/ff [4194304,4194304] 0 2024-03-11T19:30:26.547 INFO:tasks.workunit.client.0.smithi090.stdout:0/322: dwrite d2/d11/f1a [0,4194304] 0 2024-03-11T19:30:26.910 INFO:tasks.workunit.client.0.smithi090.stdout:4/334: creat d4/d27/d44/d66/f76 x:0 0 0 2024-03-11T19:30:26.937 INFO:tasks.workunit.client.0.smithi090.stdout:9/298: write d1/f58 [59433,101180] 0 2024-03-11T19:30:27.021 INFO:tasks.workunit.client.0.smithi090.stdout:6/337: dread d4/d13/f1b [0,4194304] 0 2024-03-11T19:30:27.107 INFO:tasks.workunit.client.0.smithi090.stdout:3/282: dwrite d2/d2f/f31 [4194304,4194304] 0 2024-03-11T19:30:27.382 INFO:tasks.workunit.client.0.smithi090.stdout:4/335: chown d4/d27/d28/d38/d64/c67 424 1 2024-03-11T19:30:27.540 INFO:tasks.workunit.client.0.smithi090.stdout:1/296: dwrite d5/f15 [0,4194304] 0 2024-03-11T19:30:27.700 INFO:tasks.workunit.client.0.smithi090.stdout:1/297: chown d5/cb 360 1 2024-03-11T19:30:28.037 INFO:tasks.workunit.client.0.smithi090.stdout:8/341: dread d0/d43/f2e [0,4194304] 0 2024-03-11T19:30:28.236 INFO:tasks.workunit.client.0.smithi090.stdout:6/338: dread - d4/f62 zero size 2024-03-11T19:30:28.315 INFO:tasks.workunit.client.0.smithi090.stdout:1/298: dread d5/da/dd/f1a [8388608,4194304] 0 2024-03-11T19:30:28.632 INFO:tasks.workunit.client.0.smithi090.stdout:9/299: dread d1/dd/f1e [0,4194304] 0 2024-03-11T19:30:28.951 INFO:tasks.workunit.client.0.smithi090.stdout:3/283: creat d2/d4f/f5d x:0 0 0 2024-03-11T19:30:28.963 INFO:tasks.workunit.client.0.smithi090.stdout:4/336: creat d4/d18/f77 x:0 0 0 2024-03-11T19:30:29.012 INFO:tasks.workunit.client.0.smithi090.stdout:2/304: link d0/de/l51 d0/l54 0 2024-03-11T19:30:29.039 INFO:tasks.workunit.client.0.smithi090.stdout:2/305: chown d0/de/d1f/l3f 3386824 1 2024-03-11T19:30:29.098 INFO:tasks.workunit.client.0.smithi090.stdout:2/306: dread d0/de/d12/f1a [0,4194304] 0 2024-03-11T19:30:29.211 INFO:tasks.workunit.client.0.smithi090.stdout:3/284: sync 2024-03-11T19:30:29.237 INFO:tasks.workunit.client.0.smithi090.stdout:3/285: dread - d2/d2f/f32 zero size 2024-03-11T19:30:29.351 INFO:tasks.workunit.client.0.smithi090.stdout:5/310: link d1/d26/d19/l1d d1/d26/d19/d27/l62 0 2024-03-11T19:30:29.417 INFO:tasks.workunit.client.0.smithi090.stdout:5/311: chown d1/d26/c18 22857713 1 2024-03-11T19:30:29.979 INFO:tasks.workunit.client.0.smithi090.stdout:3/286: symlink d2/db/d15/d1a/l5e 0 2024-03-11T19:30:30.022 INFO:tasks.workunit.client.0.smithi090.stdout:3/287: stat d2/db/d15/d1a/l26 0 2024-03-11T19:30:30.049 INFO:tasks.workunit.client.0.smithi090.stdout:3/288: dread - d2/d1f/f36 zero size 2024-03-11T19:30:30.777 INFO:tasks.workunit.client.0.smithi090.stdout:9/300: symlink d1/dd/df/l5e 0 2024-03-11T19:30:30.830 INFO:tasks.workunit.client.0.smithi090.stdout:9/301: dread d1/f58 [0,4194304] 0 2024-03-11T19:30:31.060 INFO:tasks.workunit.client.0.smithi090.stdout:7/301: symlink d3/dc/l62 0 2024-03-11T19:30:31.583 INFO:tasks.workunit.client.0.smithi090.stdout:6/339: write d4/d13/d22/d34/d37/d47/f63 [1479281,94091] 0 2024-03-11T19:30:31.645 INFO:tasks.workunit.client.0.smithi090.stdout:0/323: rename d2/d11/d15/l3b to d2/d11/d15/d29/d43/d58/l67 0 2024-03-11T19:30:31.702 INFO:tasks.workunit.client.0.smithi090.stdout:1/299: write d5/f60 [1281250,34000] 0 2024-03-11T19:30:31.714 INFO:tasks.workunit.client.0.smithi090.stdout:2/307: write d0/d21/f3d [3848257,7498] 0 2024-03-11T19:30:31.920 INFO:tasks.workunit.client.0.smithi090.stdout:9/302: mknod d1/dd/d18/d3a/c5f 0 2024-03-11T19:30:31.938 INFO:tasks.workunit.client.0.smithi090.stdout:5/312: write d1/d26/d19/d27/f48 [647936,45753] 0 2024-03-11T19:30:32.122 INFO:tasks.workunit.client.0.smithi090.stdout:4/337: dwrite d4/d18/f41 [0,4194304] 0 2024-03-11T19:30:32.456 INFO:tasks.workunit.client.0.smithi090.stdout:8/342: rename d0/f20 to d0/d3/d3e/f66 0 2024-03-11T19:30:32.727 INFO:tasks.workunit.client.0.smithi090.stdout:7/302: write d3/f3b [1476337,44594] 0 2024-03-11T19:30:33.383 INFO:tasks.workunit.client.0.smithi090.stdout:6/340: mknod d4/d13/d22/d42/d48/d4e/c6c 0 2024-03-11T19:30:33.394 INFO:tasks.workunit.client.0.smithi090.stdout:2/308: creat d0/de/d1f/f55 x:0 0 0 2024-03-11T19:30:33.451 INFO:tasks.workunit.client.0.smithi090.stdout:2/309: stat d0/fd 0 2024-03-11T19:30:33.471 INFO:tasks.workunit.client.0.smithi090.stdout:6/341: chown d4/d24/c32 29916404 1 2024-03-11T19:30:33.532 INFO:tasks.workunit.client.0.smithi090.stdout:4/338: symlink d4/d27/d28/l78 0 2024-03-11T19:30:33.674 INFO:tasks.workunit.client.0.smithi090.stdout:9/303: truncate d1/dd/d18/f38 7391 0 2024-03-11T19:30:33.698 INFO:tasks.workunit.client.0.smithi090.stdout:9/304: chown d1/dd/df/d1c 72839047 1 2024-03-11T19:30:34.674 INFO:tasks.workunit.client.0.smithi090.stdout:5/313: mkdir d1/d26/d19/d27/d3b/d63 0 2024-03-11T19:30:35.801 INFO:tasks.workunit.client.0.smithi090.stdout:8/343: mknod d0/d60/c67 0 2024-03-11T19:30:35.825 INFO:tasks.workunit.client.0.smithi090.stdout:0/324: creat d2/d11/d15/d29/f68 x:0 0 0 2024-03-11T19:30:35.828 INFO:tasks.workunit.client.0.smithi090.stdout:1/300: link d5/c16 d5/d31/d36/c62 0 2024-03-11T19:30:36.038 INFO:tasks.workunit.client.0.smithi090.stdout:1/301: sync 2024-03-11T19:30:36.049 INFO:tasks.workunit.client.0.smithi090.stdout:6/342: unlink d4/d13/d22/d34/l38 0 2024-03-11T19:30:36.119 INFO:tasks.workunit.client.0.smithi090.stdout:3/289: rename d2/db/f2b to d2/f5f 0 2024-03-11T19:30:36.212 INFO:tasks.workunit.client.0.smithi090.stdout:3/290: sync 2024-03-11T19:30:36.245 INFO:tasks.workunit.client.0.smithi090.stdout:6/343: stat d4/c5 0 2024-03-11T19:30:36.307 INFO:tasks.workunit.client.0.smithi090.stdout:8/344: creat d0/d3/f68 x:0 0 0 2024-03-11T19:30:36.343 INFO:tasks.workunit.client.0.smithi090.stdout:5/314: write d1/d17/f2a [3668233,1613] 0 2024-03-11T19:30:36.375 INFO:tasks.workunit.client.0.smithi090.stdout:7/303: write d3/dc/d1e/d2a/f5e [2743956,83265] 0 2024-03-11T19:30:36.476 INFO:tasks.workunit.client.0.smithi090.stdout:4/339: dread d4/ff [0,4194304] 0 2024-03-11T19:30:36.532 INFO:tasks.workunit.client.0.smithi090.stdout:0/325: creat d2/d11/d3a/f69 x:0 0 0 2024-03-11T19:30:36.615 INFO:tasks.workunit.client.0.smithi090.stdout:2/310: dwrite d0/f4 [0,4194304] 0 2024-03-11T19:30:37.194 INFO:tasks.workunit.client.0.smithi090.stdout:7/304: creat d3/dc/d1e/f63 x:0 0 0 2024-03-11T19:30:37.243 INFO:tasks.workunit.client.0.smithi090.stdout:3/291: rename d2/d2f/l50 to d2/db/l60 0 2024-03-11T19:30:37.255 INFO:tasks.workunit.client.0.smithi090.stdout:9/305: link d1/f9 d1/dd/d18/d25/f60 0 2024-03-11T19:30:37.304 INFO:tasks.workunit.client.0.smithi090.stdout:4/340: mkdir d4/d27/d28/d38/d79 0 2024-03-11T19:30:37.352 INFO:tasks.workunit.client.0.smithi090.stdout:2/311: symlink d0/de/d12/l56 0 2024-03-11T19:30:37.420 INFO:tasks.workunit.client.0.smithi090.stdout:1/302: dwrite d5/da/dd/f14 [0,4194304] 0 2024-03-11T19:30:37.435 INFO:tasks.workunit.client.0.smithi090.stdout:0/326: read d2/d11/d15/d29/d2e/f52 [272752,106866] 0 2024-03-11T19:30:37.630 INFO:tasks.workunit.client.0.smithi090.stdout:6/344: dread d4/d13/d22/d3a/f56 [0,4194304] 0 2024-03-11T19:30:37.721 INFO:tasks.workunit.client.0.smithi090.stdout:7/305: chown d3/dc/d17/d34/d52/d5d/f21 51319 1 2024-03-11T19:30:37.972 INFO:tasks.workunit.client.0.smithi090.stdout:4/341: fdatasync d4/f46 0 2024-03-11T19:30:38.282 INFO:tasks.workunit.client.0.smithi090.stdout:9/306: creat d1/dd/f61 x:0 0 0 2024-03-11T19:30:38.288 INFO:tasks.workunit.client.0.smithi090.stdout:3/292: mknod d2/d45/c61 0 2024-03-11T19:30:38.519 INFO:tasks.workunit.client.0.smithi090.stdout:8/345: dwrite d0/d43/dc/f3a [0,4194304] 0 2024-03-11T19:30:39.604 INFO:tasks.workunit.client.0.smithi090.stdout:4/342: mkdir d4/d27/d28/d7a 0 2024-03-11T19:30:39.729 INFO:tasks.workunit.client.0.smithi090.stdout:3/293: unlink d2/db/d15/f18 0 2024-03-11T19:30:39.905 INFO:tasks.workunit.client.0.smithi090.stdout:3/294: sync 2024-03-11T19:30:40.113 INFO:tasks.workunit.client.0.smithi090.stdout:8/346: creat d0/d3/d4c/f69 x:0 0 0 2024-03-11T19:30:40.127 INFO:tasks.workunit.client.0.smithi090.stdout:5/315: dwrite d1/d26/fc [0,4194304] 0 2024-03-11T19:30:40.279 INFO:tasks.workunit.client.0.smithi090.stdout:1/303: symlink d5/da/l63 0 2024-03-11T19:30:40.304 INFO:tasks.workunit.client.0.smithi090.stdout:9/307: dread d1/f17 [0,4194304] 0 2024-03-11T19:30:40.321 INFO:tasks.workunit.client.0.smithi090.stdout:7/306: creat d3/d55/f64 x:0 0 0 2024-03-11T19:30:40.472 INFO:tasks.workunit.client.0.smithi090.stdout:6/345: creat d4/d13/d22/d34/f6d x:0 0 0 2024-03-11T19:30:40.479 INFO:tasks.workunit.client.0.smithi090.stdout:4/343: dread - d4/d27/d44/d66/f76 zero size 2024-03-11T19:30:40.529 INFO:tasks.workunit.client.0.smithi090.stdout:4/344: chown d4/d18/f2a 2 1 2024-03-11T19:30:40.625 INFO:tasks.workunit.client.0.smithi090.stdout:3/295: fsync d2/de/f58 0 2024-03-11T19:30:40.741 INFO:tasks.workunit.client.0.smithi090.stdout:4/345: dread d4/f8 [0,4194304] 0 2024-03-11T19:30:40.776 INFO:tasks.workunit.client.0.smithi090.stdout:5/316: chown d1/c8 271230 1 2024-03-11T19:30:40.915 INFO:tasks.workunit.client.0.smithi090.stdout:8/347: rename d0/d43/c22 to d0/d3/d3e/c6a 0 2024-03-11T19:30:41.077 INFO:tasks.workunit.client.0.smithi090.stdout:2/312: write d0/d21/d30/f39 [379402,37840] 0 2024-03-11T19:30:41.174 INFO:tasks.workunit.client.0.smithi090.stdout:7/307: truncate d3/dc/d17/d32/f4a 3525031 0 2024-03-11T19:30:41.215 INFO:tasks.workunit.client.0.smithi090.stdout:2/313: sync 2024-03-11T19:30:41.242 INFO:tasks.workunit.client.0.smithi090.stdout:7/308: write d3/dc/d1e/d2a/f5e [4334206,117715] 0 2024-03-11T19:30:41.613 INFO:tasks.workunit.client.0.smithi090.stdout:0/327: dwrite d2/f27 [4194304,4194304] 0 2024-03-11T19:30:41.773 INFO:tasks.workunit.client.0.smithi090.stdout:3/296: rename d2/f34 to d2/db/f62 0 2024-03-11T19:30:41.798 INFO:tasks.workunit.client.0.smithi090.stdout:8/348: fdatasync d0/d3/d4c/f5e 0 2024-03-11T19:30:41.952 INFO:tasks.workunit.client.0.smithi090.stdout:0/328: dread d2/f27 [4194304,4194304] 0 2024-03-11T19:30:42.797 INFO:tasks.workunit.client.0.smithi090.stdout:4/346: fsync d4/d27/d28/f63 0 2024-03-11T19:30:42.804 INFO:tasks.workunit.client.0.smithi090.stdout:5/317: mkdir d1/d17/d39/d51/d53/d64 0 2024-03-11T19:30:43.118 INFO:tasks.workunit.client.0.smithi090.stdout:1/304: write d5/f60 [783148,128519] 0 2024-03-11T19:30:43.298 INFO:tasks.workunit.client.0.smithi090.stdout:9/308: write d1/dd/df/d1c/d21/f54 [933165,56440] 0 2024-03-11T19:30:43.374 INFO:tasks.workunit.client.0.smithi090.stdout:9/309: read d1/dd/df/d1c/d21/f54 [7666285,13691] 0 2024-03-11T19:30:43.422 INFO:tasks.workunit.client.0.smithi090.stdout:9/310: rename d1/dd/d18/d25 to d1/dd/d18/d25/d62 22 2024-03-11T19:30:44.010 INFO:tasks.workunit.client.0.smithi090.stdout:6/346: dwrite d4/d24/f3f [0,4194304] 0 2024-03-11T19:30:44.279 INFO:tasks.workunit.client.0.smithi090.stdout:6/347: dread d4/d13/d22/d3a/f56 [0,4194304] 0 2024-03-11T19:30:44.304 INFO:tasks.workunit.client.0.smithi090.stdout:3/297: readlink d2/d45/l4d 0 2024-03-11T19:30:44.325 INFO:tasks.workunit.client.0.smithi090.stdout:2/314: write d0/de/f52 [557443,112479] 0 2024-03-11T19:30:44.485 INFO:tasks.workunit.client.0.smithi090.stdout:8/349: dread d0/d3/d3e/f4e [0,4194304] 0 2024-03-11T19:30:44.780 INFO:tasks.workunit.client.0.smithi090.stdout:7/309: creat d3/d49/f65 x:0 0 0 2024-03-11T19:30:45.173 INFO:tasks.workunit.client.0.smithi090.stdout:1/305: symlink d5/da/d23/l64 0 2024-03-11T19:30:45.244 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:30:45.247 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:30:45.353 INFO:tasks.workunit.client.0.smithi090.stdout:4/347: dread d4/d27/d28/f30 [0,4194304] 0 2024-03-11T19:30:45.400 INFO:tasks.workunit.client.0.smithi090.stdout:2/315: symlink d0/d21/d30/d25/l57 0 2024-03-11T19:30:45.415 INFO:tasks.workunit.client.0.smithi090.stdout:9/311: dread - d1/dd/df/d1c/d21/d2a/f50 zero size 2024-03-11T19:30:45.703 INFO:tasks.workunit.client.0.smithi090.stdout:0/329: truncate d2/d11/d15/d29/f68 358721 0 2024-03-11T19:30:45.908 INFO:tasks.workunit.client.0.smithi090.stdout:7/310: read - d3/f59 zero size 2024-03-11T19:30:46.015 INFO:tasks.workunit.client.0.smithi090.stdout:5/318: symlink d1/d17/d39/d51/l65 0 2024-03-11T19:30:46.608 INFO:tasks.workunit.client.0.smithi090.stdout:9/312: dread d1/dd/df/d1c/d21/d2a/f3d [0,4194304] 0 2024-03-11T19:30:46.635 INFO:tasks.workunit.client.0.smithi090.stdout:9/313: stat d1/dd/c15 0 2024-03-11T19:30:46.864 INFO:tasks.workunit.client.0.smithi090.stdout:0/330: readlink d2/d11/l13 0 2024-03-11T19:30:47.334 INFO:tasks.workunit.client.0.smithi090.stdout:7/311: creat d3/dc/d1e/d2a/f66 x:0 0 0 2024-03-11T19:30:47.528 INFO:tasks.workunit.client.0.smithi090.stdout:5/319: dread - d1/d17/f41 zero size 2024-03-11T19:30:47.844 INFO:tasks.workunit.client.0.smithi090.stdout:2/316: symlink d0/l58 0 2024-03-11T19:30:48.037 INFO:tasks.workunit.client.0.smithi090.stdout:9/314: dread - d1/dd/d18/f52 zero size 2024-03-11T19:30:48.309 INFO:tasks.workunit.client.0.smithi090.stdout:7/312: fsync d3/dc/d1e/f63 0 2024-03-11T19:30:48.589 INFO:tasks.workunit.client.0.smithi090.stdout:4/348: rename d4/d27/d28/c48 to d4/c7b 0 2024-03-11T19:30:48.704 INFO:tasks.workunit.client.0.smithi090.stdout:3/298: write d2/f8 [2532315,53557] 0 2024-03-11T19:30:48.930 INFO:tasks.workunit.client.0.smithi090.stdout:7/313: mkdir d3/dc/d17/d32/d67 0 2024-03-11T19:30:49.270 INFO:tasks.workunit.client.0.smithi090.stdout:6/348: dwrite d4/d13/d22/d3a/f54 [0,4194304] 0 2024-03-11T19:30:49.397 INFO:tasks.workunit.client.0.smithi090.stdout:4/349: truncate d4/d27/d44/d66/f76 333462 0 2024-03-11T19:30:49.411 INFO:tasks.workunit.client.0.smithi090.stdout:0/331: creat d2/d11/d15/d29/f6a x:0 0 0 2024-03-11T19:30:49.428 INFO:tasks.workunit.client.0.smithi090.stdout:8/350: write d0/d53/f5d [90134,19789] 0 2024-03-11T19:30:49.507 INFO:tasks.workunit.client.0.smithi090.stdout:0/332: chown d2/f10 48 1 2024-03-11T19:30:49.511 INFO:tasks.workunit.client.0.smithi090.stdout:6/349: chown d4/d13/d22/d34/f68 9418 1 2024-03-11T19:30:49.558 INFO:tasks.workunit.client.0.smithi090.stdout:6/350: chown d4/d13/d22/d34/d37/l50 3772736 1 2024-03-11T19:30:49.612 INFO:tasks.workunit.client.0.smithi090.stdout:5/320: creat d1/d17/f66 x:0 0 0 2024-03-11T19:30:49.795 INFO:tasks.workunit.client.0.smithi090.stdout:9/315: creat d1/dd/f63 x:0 0 0 2024-03-11T19:30:49.805 INFO:tasks.workunit.client.0.smithi090.stdout:1/306: write d5/da/d23/f2b [6068203,48908] 0 2024-03-11T19:30:50.303 INFO:tasks.workunit.client.0.smithi090.stdout:7/314: rename d3/dc/f3c to d3/f68 0 2024-03-11T19:30:50.569 INFO:tasks.workunit.client.0.smithi090.stdout:8/351: dread - d0/f58 zero size 2024-03-11T19:30:50.623 INFO:tasks.workunit.client.0.smithi090.stdout:2/317: dwrite d0/fc [0,4194304] 0 2024-03-11T19:30:50.794 INFO:tasks.workunit.client.0.smithi090.stdout:6/351: truncate d4/d14/f23 3840756 0 2024-03-11T19:30:50.954 INFO:tasks.workunit.client.0.smithi090.stdout:5/321: unlink d1/la 0 2024-03-11T19:30:51.013 INFO:tasks.workunit.client.0.smithi090.stdout:6/352: dread d4/d13/d22/d3a/f54 [0,4194304] 0 2024-03-11T19:30:51.376 INFO:tasks.workunit.client.0.smithi090.stdout:9/316: rename d1/d1d/c36 to d1/dd/d18/d3a/c64 0 2024-03-11T19:30:51.422 INFO:tasks.workunit.client.0.smithi090.stdout:7/315: symlink d3/dc/d17/d32/d45/l69 0 2024-03-11T19:30:51.743 INFO:tasks.workunit.client.0.smithi090.stdout:0/333: mkdir d2/d11/d15/d29/d6b 0 2024-03-11T19:30:51.743 INFO:tasks.workunit.client.0.smithi090.stdout:7/316: sync 2024-03-11T19:30:52.141 INFO:tasks.workunit.client.0.smithi090.stdout:5/322: creat d1/d17/d39/d51/d53/f67 x:0 0 0 2024-03-11T19:30:52.165 INFO:tasks.workunit.client.0.smithi090.stdout:5/323: chown d1/ld 204993 1 2024-03-11T19:30:52.400 INFO:tasks.workunit.client.0.smithi090.stdout:3/299: rename d2/db/d15/d1a/f28 to d2/d45/f63 0 2024-03-11T19:30:52.452 INFO:tasks.workunit.client.0.smithi090.stdout:9/317: truncate d1/dd/d18/d3a/d57/f5b 536779 0 2024-03-11T19:30:52.490 INFO:tasks.workunit.client.0.smithi090.stdout:4/350: write d4/d18/f25 [134896,74487] 0 2024-03-11T19:30:53.557 INFO:tasks.workunit.client.0.smithi090.stdout:8/352: dwrite d0/d43/f2f [0,4194304] 0 2024-03-11T19:30:53.681 INFO:tasks.workunit.client.0.smithi090.stdout:8/353: stat d0/d43/dc/l3f 0 2024-03-11T19:30:53.767 INFO:tasks.workunit.client.0.smithi090.stdout:1/307: mkdir d5/d65 0 2024-03-11T19:30:53.781 INFO:tasks.workunit.client.0.smithi090.stdout:6/353: symlink d4/d13/d22/d3a/d67/l6e 0 2024-03-11T19:30:53.908 INFO:tasks.workunit.client.0.smithi090.stdout:4/351: mknod d4/d27/d44/d66/c7c 0 2024-03-11T19:30:53.914 INFO:tasks.workunit.client.0.smithi090.stdout:2/318: write d0/d21/f2a [4156688,18204] 0 2024-03-11T19:30:54.033 INFO:tasks.workunit.client.0.smithi090.stdout:0/334: symlink d2/d11/d15/d29/d6b/l6c 0 2024-03-11T19:30:54.525 INFO:tasks.workunit.client.0.smithi090.stdout:8/354: read d0/d43/dc/f12 [4246583,77311] 0 2024-03-11T19:30:54.770 INFO:tasks.workunit.client.0.smithi090.stdout:7/317: dwrite d3/dc/f30 [0,4194304] 0 2024-03-11T19:30:54.886 INFO:tasks.workunit.client.0.smithi090.stdout:6/354: mkdir d4/d13/d22/d3a/d6f 0 2024-03-11T19:30:54.934 INFO:tasks.workunit.client.0.smithi090.stdout:7/318: stat d3/dc/d1e/d2a/f66 0 2024-03-11T19:30:54.960 INFO:tasks.workunit.client.0.smithi090.stdout:7/319: readlink d3/dc/d17/d32/d45/l69 0 2024-03-11T19:30:54.992 INFO:tasks.workunit.client.0.smithi090.stdout:7/320: readlink d3/dc/d17/d34/d52/d5d/l2f 0 2024-03-11T19:30:55.092 INFO:tasks.workunit.client.0.smithi090.stdout:3/300: mknod d2/d1d/d47/c64 0 2024-03-11T19:30:55.126 INFO:tasks.workunit.client.0.smithi090.stdout:4/352: chown d4/d51/l6d 840949715 1 2024-03-11T19:30:55.463 INFO:tasks.workunit.client.0.smithi090.stdout:5/324: write d1/d26/d19/f42 [717612,67767] 0 2024-03-11T19:30:55.573 INFO:tasks.workunit.client.0.smithi090.stdout:8/355: rmdir d0/d3/d3e 39 2024-03-11T19:30:55.621 INFO:tasks.workunit.client.0.smithi090.stdout:2/319: dread d0/d21/f3d [0,4194304] 0 2024-03-11T19:30:55.861 INFO:tasks.workunit.client.0.smithi090.stdout:6/355: creat d4/d13/d58/f70 x:0 0 0 2024-03-11T19:30:56.121 INFO:tasks.workunit.client.0.smithi090.stdout:9/318: write d1/dd/d18/d3a/d57/f5d [48531,10489] 0 2024-03-11T19:30:56.561 INFO:tasks.workunit.client.0.smithi090.stdout:7/321: creat d3/d55/f6a x:0 0 0 2024-03-11T19:30:56.564 INFO:tasks.workunit.client.0.smithi090.stdout:3/301: rmdir d2/d1d/d47 39 2024-03-11T19:30:56.603 INFO:tasks.workunit.client.0.smithi090.stdout:7/322: stat d3/dc/d29 0 2024-03-11T19:30:56.679 INFO:tasks.workunit.client.0.smithi090.stdout:4/353: fdatasync d4/f46 0 2024-03-11T19:30:57.202 INFO:tasks.workunit.client.0.smithi090.stdout:8/356: creat d0/d53/d59/f6b x:0 0 0 2024-03-11T19:30:57.398 INFO:tasks.workunit.client.0.smithi090.stdout:1/308: write d5/da/f44 [9642,128275] 0 2024-03-11T19:30:57.712 INFO:tasks.workunit.client.0.smithi090.stdout:7/323: creat d3/dc/d29/f6b x:0 0 0 2024-03-11T19:30:58.275 INFO:tasks.workunit.client.0.smithi090.stdout:5/325: rename d1/d26/f4b to d1/d17/d39/f68 0 2024-03-11T19:30:58.608 INFO:tasks.workunit.client.0.smithi090.stdout:0/335: write d2/d11/f18 [1328622,70379] 0 2024-03-11T19:30:58.783 INFO:tasks.workunit.client.0.smithi090.stdout:7/324: symlink d3/dc/d17/l6c 0 2024-03-11T19:30:59.546 INFO:tasks.workunit.client.0.smithi090.stdout:7/325: rmdir d3/dc/d1e/d2a 39 2024-03-11T19:30:59.879 INFO:tasks.workunit.client.0.smithi090.stdout:8/357: creat d0/d3/f6c x:0 0 0 2024-03-11T19:30:59.912 INFO:tasks.workunit.client.0.smithi090.stdout:7/326: rmdir d3/dc/d17/d34/d52/d5d 39 2024-03-11T19:31:00.300 INFO:tasks.workunit.client.0.smithi090.stdout:7/327: dread d3/dc/d1e/f26 [0,4194304] 0 2024-03-11T19:31:00.475 INFO:tasks.workunit.client.0.smithi090.stdout:6/356: write d4/d13/f1b [2237978,91811] 0 2024-03-11T19:31:00.908 INFO:tasks.workunit.client.0.smithi090.stdout:2/320: dwrite d0/f2 [0,4194304] 0 2024-03-11T19:31:01.063 INFO:tasks.workunit.client.0.smithi090.stdout:0/336: rmdir d2/d11/d15 39 2024-03-11T19:31:01.083 INFO:tasks.workunit.client.0.smithi090.stdout:1/309: write d5/da/dd/f2c [3174413,67881] 0 2024-03-11T19:31:01.231 INFO:tasks.workunit.client.0.smithi090.stdout:6/357: rmdir d4/d13/d22/d34/d37/d47 39 2024-03-11T19:31:01.263 INFO:tasks.workunit.client.0.smithi090.stdout:4/354: dwrite d4/d27/d28/f68 [0,4194304] 0 2024-03-11T19:31:01.626 INFO:tasks.workunit.client.0.smithi090.stdout:1/310: fsync d5/da/f44 0 2024-03-11T19:31:01.744 INFO:tasks.workunit.client.0.smithi090.stdout:2/321: mknod d0/d21/c59 0 2024-03-11T19:31:02.497 INFO:tasks.workunit.client.0.smithi090.stdout:4/355: fdatasync d4/d18/f37 0 2024-03-11T19:31:02.659 INFO:tasks.workunit.client.0.smithi090.stdout:0/337: write f0 [1087695,70494] 0 2024-03-11T19:31:02.726 INFO:tasks.workunit.client.0.smithi090.stdout:9/319: dwrite d1/dd/d18/d25/f60 [0,4194304] 0 2024-03-11T19:31:02.837 INFO:tasks.workunit.client.0.smithi090.stdout:5/326: dwrite d1/f32 [0,4194304] 0 2024-03-11T19:31:03.032 INFO:tasks.workunit.client.0.smithi090.stdout:3/302: dwrite d2/db/d15/f1e [0,4194304] 0 2024-03-11T19:31:03.134 INFO:tasks.workunit.client.0.smithi090.stdout:8/358: dwrite d0/f17 [0,4194304] 0 2024-03-11T19:31:03.361 INFO:tasks.workunit.client.0.smithi090.stdout:3/303: sync 2024-03-11T19:31:03.597 INFO:tasks.workunit.client.0.smithi090.stdout:6/358: dread d4/d24/f3f [0,4194304] 0 2024-03-11T19:31:03.668 INFO:tasks.workunit.client.0.smithi090.stdout:9/320: chown d1/dd/df/l5e 3 1 2024-03-11T19:31:03.668 INFO:tasks.workunit.client.0.smithi090.stdout:2/322: dread d0/f18 [0,4194304] 0 2024-03-11T19:31:03.683 INFO:tasks.workunit.client.0.smithi090.stdout:7/328: dwrite d3/dc/d17/d34/d52/d5d/f5a [0,4194304] 0 2024-03-11T19:31:03.775 INFO:tasks.workunit.client.0.smithi090.stdout:2/323: sync 2024-03-11T19:31:03.856 INFO:tasks.workunit.client.0.smithi090.stdout:6/359: stat d4/d13/d22/d42/l55 0 2024-03-11T19:31:04.070 INFO:tasks.workunit.client.0.smithi090.stdout:4/356: mknod d4/d51/c7d 0 2024-03-11T19:31:04.088 INFO:tasks.workunit.client.0.smithi090.stdout:5/327: symlink d1/d26/d4d/l69 0 2024-03-11T19:31:04.186 INFO:tasks.workunit.client.0.smithi090.stdout:4/357: sync 2024-03-11T19:31:04.391 INFO:tasks.workunit.client.0.smithi090.stdout:3/304: symlink d2/db/d15/d1a/d3c/l65 0 2024-03-11T19:31:04.437 INFO:tasks.workunit.client.0.smithi090.stdout:0/338: truncate d2/d11/f47 1173364 0 2024-03-11T19:31:04.462 INFO:tasks.workunit.client.0.smithi090.stdout:2/324: symlink d0/d21/d30/d29/l5a 0 2024-03-11T19:31:04.502 INFO:tasks.workunit.client.0.smithi090.stdout:8/359: dread d0/f47 [0,4194304] 0 2024-03-11T19:31:04.821 INFO:tasks.workunit.client.0.smithi090.stdout:5/328: dread d1/d17/f2a [0,4194304] 0 2024-03-11T19:31:04.848 INFO:tasks.workunit.client.0.smithi090.stdout:1/311: truncate d5/fc 10396712 0 2024-03-11T19:31:04.934 INFO:tasks.workunit.client.0.smithi090.stdout:3/305: mknod d2/de/c66 0 2024-03-11T19:31:05.050 INFO:tasks.workunit.client.0.smithi090.stdout:4/358: truncate d4/d27/d44/f65 479706 0 2024-03-11T19:31:05.128 INFO:tasks.workunit.client.0.smithi090.stdout:9/321: rename d1/dd/f63 to d1/dd/d18/f65 0 2024-03-11T19:31:05.555 INFO:tasks.workunit.client.0.smithi090.stdout:1/312: creat d5/d31/f66 x:0 0 0 2024-03-11T19:31:05.633 INFO:tasks.workunit.client.0.smithi090.stdout:8/360: write d0/f5a [610446,19758] 0 2024-03-11T19:31:05.726 INFO:tasks.workunit.client.0.smithi090.stdout:7/329: dwrite d3/f4b [0,4194304] 0 2024-03-11T19:31:05.885 INFO:tasks.workunit.client.0.smithi090.stdout:6/360: dwrite d4/f62 [0,4194304] 0 2024-03-11T19:31:05.911 INFO:tasks.workunit.client.0.smithi090.stdout:0/339: mkdir d2/d11/d15/d29/d2e/d6d 0 2024-03-11T19:31:06.088 INFO:tasks.workunit.client.0.smithi090.stdout:6/361: chown d4/fe 112244 1 2024-03-11T19:31:06.091 INFO:tasks.workunit.client.0.smithi090.stdout:7/330: read - d3/dc/d1e/f63 zero size 2024-03-11T19:31:06.197 INFO:tasks.workunit.client.0.smithi090.stdout:2/325: getdents d0 0 2024-03-11T19:31:06.209 INFO:tasks.workunit.client.0.smithi090.stdout:9/322: truncate d1/f3 12397433 0 2024-03-11T19:31:06.224 INFO:tasks.workunit.client.0.smithi090.stdout:7/331: chown d3/l23 6840353 1 2024-03-11T19:31:06.449 INFO:tasks.workunit.client.0.smithi090.stdout:3/306: creat d2/f67 x:0 0 0 2024-03-11T19:31:06.483 INFO:tasks.workunit.client.0.smithi090.stdout:8/361: truncate d0/f31 400763 0 2024-03-11T19:31:06.497 INFO:tasks.workunit.client.0.smithi090.stdout:6/362: symlink d4/d24/l71 0 2024-03-11T19:31:06.623 INFO:tasks.workunit.client.0.smithi090.stdout:5/329: creat d1/d17/f6a x:0 0 0 2024-03-11T19:31:06.664 INFO:tasks.workunit.client.0.smithi090.stdout:3/307: chown d2/d1d/l22 276 1 2024-03-11T19:31:06.703 INFO:tasks.workunit.client.0.smithi090.stdout:2/326: unlink d0/de/d12/l13 0 2024-03-11T19:31:06.707 INFO:tasks.workunit.client.0.smithi090.stdout:8/362: chown d0/d43/f64 58707541 1 2024-03-11T19:31:06.829 INFO:tasks.workunit.client.0.smithi090.stdout:4/359: dwrite d4/d27/f35 [0,4194304] 0 2024-03-11T19:31:06.836 INFO:tasks.workunit.client.0.smithi090.stdout:7/332: fdatasync d3/dc/d17/d32/f3f 0 2024-03-11T19:31:07.164 INFO:tasks.workunit.client.0.smithi090.stdout:1/313: write d5/da/d23/f34 [1186781,4611] 0 2024-03-11T19:31:07.249 INFO:tasks.workunit.client.0.smithi090.stdout:0/340: mkdir d2/d11/d6e 0 2024-03-11T19:31:07.523 INFO:tasks.workunit.client.0.smithi090.stdout:5/330: symlink d1/d17/d39/l6b 0 2024-03-11T19:31:07.866 INFO:tasks.workunit.client.0.smithi090.stdout:6/363: truncate d4/d13/f3b 204970 0 2024-03-11T19:31:07.928 INFO:tasks.workunit.client.0.smithi090.stdout:6/364: stat d4/d13/d22/d34/d37/l50 0 2024-03-11T19:31:08.162 INFO:tasks.workunit.client.0.smithi090.stdout:6/365: dwrite d4/f62 [0,4194304] 0 2024-03-11T19:31:08.386 INFO:tasks.workunit.client.0.smithi090.stdout:3/308: mknod d2/db/d15/d1a/d3c/c68 0 2024-03-11T19:31:08.536 INFO:tasks.workunit.client.0.smithi090.stdout:4/360: creat d4/d27/d44/f7e x:0 0 0 2024-03-11T19:31:08.639 INFO:tasks.workunit.client.0.smithi090.stdout:1/314: rename d5/d31/d36/c3d to d5/da/d1c/c67 0 2024-03-11T19:31:09.052 INFO:tasks.workunit.client.0.smithi090.stdout:5/331: read d1/d26/f20 [3742144,19080] 0 2024-03-11T19:31:09.192 INFO:tasks.workunit.client.0.smithi090.stdout:8/363: chown d0/d3/d3e/c52 0 1 2024-03-11T19:31:09.258 INFO:tasks.workunit.client.0.smithi090.stdout:0/341: dread d2/f1b [0,4194304] 0 2024-03-11T19:31:09.345 INFO:tasks.workunit.client.0.smithi090.stdout:9/323: dwrite d1/f3 [0,4194304] 0 2024-03-11T19:31:09.524 INFO:tasks.workunit.client.0.smithi090.stdout:7/333: mknod d3/dc/c6d 0 2024-03-11T19:31:09.772 INFO:tasks.workunit.client.0.smithi090.stdout:9/324: dwrite d1/f3 [8388608,4194304] 0 2024-03-11T19:31:10.212 INFO:tasks.workunit.client.0.smithi090.stdout:2/327: write d0/d21/f2a [3174273,18100] 0 2024-03-11T19:31:10.325 INFO:tasks.workunit.client.0.smithi090.stdout:4/361: chown d4/d51/l56 1301143064 1 2024-03-11T19:31:10.718 INFO:tasks.workunit.client.0.smithi090.stdout:1/315: dread d5/da/d23/f3a [0,4194304] 0 2024-03-11T19:31:10.798 INFO:tasks.workunit.client.0.smithi090.stdout:3/309: mkdir d2/db/d15/d69 0 2024-03-11T19:31:10.856 INFO:tasks.workunit.client.0.smithi090.stdout:0/342: dread d2/d11/f1a [0,4194304] 0 2024-03-11T19:31:11.030 INFO:tasks.workunit.client.0.smithi090.stdout:3/310: dread d2/f9 [0,4194304] 0 2024-03-11T19:31:11.185 INFO:tasks.workunit.client.0.smithi090.stdout:5/332: rename d1/d26/d19/c25 to d1/d17/d39/d51/d53/d64/c6c 0 2024-03-11T19:31:11.280 INFO:tasks.workunit.client.0.smithi090.stdout:8/364: truncate d0/d53/d59/f6b 451695 0 2024-03-11T19:31:11.289 INFO:tasks.workunit.client.0.smithi090.stdout:7/334: unlink d3/dc/d17/d34/f50 0 2024-03-11T19:31:11.445 INFO:tasks.workunit.client.0.smithi090.stdout:9/325: creat d1/d1d/f66 x:0 0 0 2024-03-11T19:31:11.505 INFO:tasks.workunit.client.0.smithi090.stdout:2/328: truncate d0/de/f52 165136 0 2024-03-11T19:31:11.737 INFO:tasks.workunit.client.0.smithi090.stdout:4/362: write d4/d18/f37 [117593,73632] 0 2024-03-11T19:31:11.757 INFO:tasks.workunit.client.0.smithi090.stdout:6/366: link d4/d13/d22/d34/f6d d4/d13/d22/d42/f72 0 2024-03-11T19:31:12.375 INFO:tasks.workunit.client.0.smithi090.stdout:8/365: mknod d0/d3/d4c/c6d 0 2024-03-11T19:31:12.390 INFO:tasks.workunit.client.0.smithi090.stdout:5/333: chown d1/d26/c1a 2120 1 2024-03-11T19:31:12.405 INFO:tasks.workunit.client.0.smithi090.stdout:8/366: read - d0/f58 zero size 2024-03-11T19:31:12.441 INFO:tasks.workunit.client.0.smithi090.stdout:7/335: mknod d3/dc/d17/d34/c6e 0 2024-03-11T19:31:13.168 INFO:tasks.workunit.client.0.smithi090.stdout:6/367: unlink d4/d13/l1d 0 2024-03-11T19:31:13.849 INFO:tasks.workunit.client.0.smithi090.stdout:3/311: mkdir d2/d1d/d47/d6a 0 2024-03-11T19:31:14.049 INFO:tasks.workunit.client.0.smithi090.stdout:5/334: creat d1/d17/d39/d51/d53/d64/f6d x:0 0 0 2024-03-11T19:31:14.228 INFO:tasks.workunit.client.0.smithi090.stdout:7/336: mknod d3/d55/c6f 0 2024-03-11T19:31:14.682 INFO:tasks.workunit.client.0.smithi090.stdout:0/343: write d2/d11/d3f/f40 [1210269,34180] 0 2024-03-11T19:31:14.954 INFO:tasks.workunit.client.0.smithi090.stdout:4/363: creat d4/d51/d52/d6a/f7f x:0 0 0 2024-03-11T19:31:15.363 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:31:15.366 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:31:15.592 INFO:tasks.workunit.client.0.smithi090.stdout:9/326: dwrite d1/dd/f61 [0,4194304] 0 2024-03-11T19:31:15.709 INFO:tasks.workunit.client.0.smithi090.stdout:7/337: fsync d3/dc/d1e/f63 0 2024-03-11T19:31:15.740 INFO:tasks.workunit.client.0.smithi090.stdout:2/329: creat d0/de/d12/d46/f5b x:0 0 0 2024-03-11T19:31:15.862 INFO:tasks.workunit.client.0.smithi090.stdout:0/344: chown d2/f14 3 1 2024-03-11T19:31:15.896 INFO:tasks.workunit.client.0.smithi090.stdout:0/345: chown d2/l3e 460377345 1 2024-03-11T19:31:16.619 INFO:tasks.workunit.client.0.smithi090.stdout:8/367: dwrite d0/d43/f2f [4194304,4194304] 0 2024-03-11T19:31:16.767 INFO:tasks.workunit.client.0.smithi090.stdout:7/338: creat d3/f70 x:0 0 0 2024-03-11T19:31:17.013 INFO:tasks.workunit.client.0.smithi090.stdout:6/368: link d4/d13/d22/d34/d37/c49 d4/d14/d44/c73 0 2024-03-11T19:31:17.077 INFO:tasks.workunit.client.0.smithi090.stdout:0/346: chown d2/d11/d15/c21 134508 1 2024-03-11T19:31:17.241 INFO:tasks.workunit.client.0.smithi090.stdout:7/339: sync 2024-03-11T19:31:17.267 INFO:tasks.workunit.client.0.smithi090.stdout:7/340: chown d3/dc/d17/f56 903097 1 2024-03-11T19:31:17.363 INFO:tasks.workunit.client.0.smithi090.stdout:8/368: truncate d0/f58 30621 0 2024-03-11T19:31:17.434 INFO:tasks.workunit.client.0.smithi090.stdout:7/341: sync 2024-03-11T19:31:17.673 INFO:tasks.workunit.client.0.smithi090.stdout:3/312: write d2/d1d/fc [3581034,14788] 0 2024-03-11T19:31:17.897 INFO:tasks.workunit.client.0.smithi090.stdout:7/342: fdatasync d3/f4 0 2024-03-11T19:31:18.241 INFO:tasks.workunit.client.0.smithi090.stdout:6/369: mkdir d4/d13/d22/d34/d37/d47/d74 0 2024-03-11T19:31:18.271 INFO:tasks.workunit.client.0.smithi090.stdout:5/335: write d1/d26/f20 [5165850,27406] 0 2024-03-11T19:31:18.372 INFO:tasks.workunit.client.0.smithi090.stdout:5/336: dread - d1/d26/f3c zero size 2024-03-11T19:31:19.071 INFO:tasks.workunit.client.0.smithi090.stdout:2/330: link d0/de/d1f/f40 d0/d21/d30/d25/f5c 0 2024-03-11T19:31:19.091 INFO:tasks.workunit.client.0.smithi090.stdout:4/364: write d4/d27/d28/f63 [1021716,107100] 0 2024-03-11T19:31:19.380 INFO:tasks.workunit.client.0.smithi090.stdout:7/343: dread - d3/dc/d1e/d2a/f66 zero size 2024-03-11T19:31:19.400 INFO:tasks.workunit.client.0.smithi090.stdout:6/370: read d4/d13/d22/d34/f59 [890595,30845] 0 2024-03-11T19:31:19.535 INFO:tasks.workunit.client.0.smithi090.stdout:2/331: truncate d0/fd 5144151 0 2024-03-11T19:31:19.986 INFO:tasks.workunit.client.0.smithi090.stdout:5/337: mknod d1/d17/d39/d51/c6e 0 2024-03-11T19:31:20.014 INFO:tasks.workunit.client.0.smithi090.stdout:9/327: write d1/dd/f3f [605522,117194] 0 2024-03-11T19:31:20.037 INFO:tasks.workunit.client.0.smithi090.stdout:1/316: unlink d5/c5d 0 2024-03-11T19:31:20.054 INFO:tasks.workunit.client.0.smithi090.stdout:8/369: write d0/d43/dc/fd [3840103,103168] 0 2024-03-11T19:31:20.130 INFO:tasks.workunit.client.0.smithi090.stdout:6/371: mkdir d4/d13/d22/d42/d75 0 2024-03-11T19:31:20.145 INFO:tasks.workunit.client.0.smithi090.stdout:1/317: readlink d5/l9 0 2024-03-11T19:31:20.336 INFO:tasks.workunit.client.0.smithi090.stdout:4/365: mknod d4/d27/d28/d7a/c80 0 2024-03-11T19:31:20.353 INFO:tasks.workunit.client.0.smithi090.stdout:0/347: write d2/d11/f47 [1285938,53997] 0 2024-03-11T19:31:20.445 INFO:tasks.workunit.client.0.smithi090.stdout:7/344: chown d3/dc/d17/d34/d52/d5d/f3e 43 1 2024-03-11T19:31:21.219 INFO:tasks.workunit.client.0.smithi090.stdout:9/328: rename d1/dd/d18/d3a/d57/f5d to d1/d1d/f67 0 2024-03-11T19:31:21.248 INFO:tasks.workunit.client.0.smithi090.stdout:3/313: write d2/db/d15/f37 [697567,33021] 0 2024-03-11T19:31:21.340 INFO:tasks.workunit.client.0.smithi090.stdout:6/372: truncate d4/d13/d22/d34/f68 761488 0 2024-03-11T19:31:21.396 INFO:tasks.workunit.client.0.smithi090.stdout:3/314: sync 2024-03-11T19:31:21.412 INFO:tasks.workunit.client.0.smithi090.stdout:3/315: chown d2/d1d/d47 793 1 2024-03-11T19:31:21.433 INFO:tasks.workunit.client.0.smithi090.stdout:4/366: mknod d4/d18/c81 0 2024-03-11T19:31:21.516 INFO:tasks.workunit.client.0.smithi090.stdout:0/348: dread d2/d11/f41 [0,4194304] 0 2024-03-11T19:31:21.622 INFO:tasks.workunit.client.0.smithi090.stdout:2/332: mkdir d0/de/d12/d46/d5d 0 2024-03-11T19:31:21.640 INFO:tasks.workunit.client.0.smithi090.stdout:2/333: chown d0/f18 31366867 1 2024-03-11T19:31:22.142 INFO:tasks.workunit.client.0.smithi090.stdout:5/338: mkdir d1/d26/d19/d27/d6f 0 2024-03-11T19:31:22.480 INFO:tasks.workunit.client.0.smithi090.stdout:7/345: rename d3/dc/d17/d34/d52 to d3/dc/d17/d32/d71 0 2024-03-11T19:31:22.896 INFO:tasks.workunit.client.0.smithi090.stdout:9/329: truncate d1/dd/f13 1285757 0 2024-03-11T19:31:22.933 INFO:tasks.workunit.client.0.smithi090.stdout:8/370: write d0/d3/f37 [5068448,51534] 0 2024-03-11T19:31:22.938 INFO:tasks.workunit.client.0.smithi090.stdout:1/318: write d5/da/d23/f3a [1155908,100831] 0 2024-03-11T19:31:22.975 INFO:tasks.workunit.client.0.smithi090.stdout:6/373: symlink d4/d24/l76 0 2024-03-11T19:31:23.193 INFO:tasks.workunit.client.0.smithi090.stdout:4/367: truncate d4/d27/d28/f30 1892462 0 2024-03-11T19:31:23.496 INFO:tasks.workunit.client.0.smithi090.stdout:2/334: dread - d0/de/d1f/f55 zero size 2024-03-11T19:31:23.569 INFO:tasks.workunit.client.0.smithi090.stdout:0/349: dread d2/d11/f18 [0,4194304] 0 2024-03-11T19:31:23.790 INFO:tasks.workunit.client.0.smithi090.stdout:5/339: mknod d1/d17/d39/d51/d53/d64/c70 0 2024-03-11T19:31:24.349 INFO:tasks.workunit.client.0.smithi090.stdout:7/346: dread d3/f3b [0,4194304] 0 2024-03-11T19:31:24.667 INFO:tasks.workunit.client.0.smithi090.stdout:9/330: mknod d1/dd/c68 0 2024-03-11T19:31:24.813 INFO:tasks.workunit.client.0.smithi090.stdout:9/331: sync 2024-03-11T19:31:25.027 INFO:tasks.workunit.client.0.smithi090.stdout:4/368: symlink d4/d18/l82 0 2024-03-11T19:31:25.467 INFO:tasks.workunit.client.0.smithi090.stdout:0/350: symlink d2/d11/d15/d29/d2e/l6f 0 2024-03-11T19:31:25.541 INFO:tasks.workunit.client.0.smithi090.stdout:0/351: truncate d2/d11/d15/f19 8848178 0 2024-03-11T19:31:25.646 INFO:tasks.workunit.client.0.smithi090.stdout:0/352: sync 2024-03-11T19:31:25.971 INFO:tasks.workunit.client.0.smithi090.stdout:6/374: mkdir d4/d13/d22/d42/d75/d77 0 2024-03-11T19:31:26.132 INFO:tasks.workunit.client.0.smithi090.stdout:9/332: truncate d1/f3 6297201 0 2024-03-11T19:31:26.441 INFO:tasks.workunit.client.0.smithi090.stdout:4/369: symlink d4/d51/d52/l83 0 2024-03-11T19:31:26.916 INFO:tasks.workunit.client.0.smithi090.stdout:3/316: getdents d2 0 2024-03-11T19:31:27.592 INFO:tasks.workunit.client.0.smithi090.stdout:6/375: fsync d4/d14/f6a 0 2024-03-11T19:31:27.789 INFO:tasks.workunit.client.0.smithi090.stdout:9/333: unlink d1/dd/f11 0 2024-03-11T19:31:27.947 INFO:tasks.workunit.client.0.smithi090.stdout:5/340: link d1/f60 d1/d26/f71 0 2024-03-11T19:31:27.958 INFO:tasks.workunit.client.0.smithi090.stdout:2/335: dwrite d0/de/d12/f1a [0,4194304] 0 2024-03-11T19:31:28.112 INFO:tasks.workunit.client.0.smithi090.stdout:7/347: link d3/dc/d1e/l2e d3/d55/l72 0 2024-03-11T19:31:28.167 INFO:tasks.workunit.client.0.smithi090.stdout:9/334: chown d1/dd/df/c27 3042255 1 2024-03-11T19:31:28.357 INFO:tasks.workunit.client.0.smithi090.stdout:8/371: getdents d0/d43/dc 0 2024-03-11T19:31:28.981 INFO:tasks.workunit.client.0.smithi090.stdout:6/376: unlink d4/d24/f43 0 2024-03-11T19:31:29.718 INFO:tasks.workunit.client.0.smithi090.stdout:0/353: write d2/d11/d15/d29/d43/d58/f5e [115921,95024] 0 2024-03-11T19:31:29.886 INFO:tasks.workunit.client.0.smithi090.stdout:0/354: dread d2/f1b [0,4194304] 0 2024-03-11T19:31:30.303 INFO:tasks.workunit.client.0.smithi090.stdout:5/341: creat d1/d26/d19/d27/d6f/f72 x:0 0 0 2024-03-11T19:31:30.673 INFO:tasks.workunit.client.0.smithi090.stdout:4/370: dwrite d4/f24 [0,4194304] 0 2024-03-11T19:31:30.838 INFO:tasks.workunit.client.0.smithi090.stdout:6/377: fdatasync d4/d13/d22/d34/f68 0 2024-03-11T19:31:31.189 INFO:tasks.workunit.client.0.smithi090.stdout:5/342: dread d1/f38 [0,4194304] 0 2024-03-11T19:31:31.280 INFO:tasks.workunit.client.0.smithi090.stdout:8/372: link d0/d3/d4c/f5e d0/d53/f6e 0 2024-03-11T19:31:31.380 INFO:tasks.workunit.client.0.smithi090.stdout:5/343: sync 2024-03-11T19:31:31.420 INFO:tasks.workunit.client.0.smithi090.stdout:8/373: stat d0/d43/c42 0 2024-03-11T19:31:31.431 INFO:tasks.workunit.client.0.smithi090.stdout:4/371: chown d4/d27/d28/c4c 226 1 2024-03-11T19:31:31.564 INFO:tasks.workunit.client.0.smithi090.stdout:6/378: unlink d4/d13/d58/f70 0 2024-03-11T19:31:31.568 INFO:tasks.workunit.client.0.smithi090.stdout:8/374: stat d0/d43/f2f 0 2024-03-11T19:31:31.684 INFO:tasks.workunit.client.0.smithi090.stdout:1/319: dwrite d5/da/d23/f34 [0,4194304] 0 2024-03-11T19:31:31.788 INFO:tasks.workunit.client.0.smithi090.stdout:1/320: sync 2024-03-11T19:31:32.295 INFO:tasks.workunit.client.0.smithi090.stdout:8/375: truncate d0/f19 6027051 0 2024-03-11T19:31:32.347 INFO:tasks.workunit.client.0.smithi090.stdout:0/355: getdents d2/d11/d15/d29/d43 0 2024-03-11T19:31:32.358 INFO:tasks.workunit.client.0.smithi090.stdout:3/317: write d2/db/d15/f48 [210757,57159] 0 2024-03-11T19:31:32.546 INFO:tasks.workunit.client.0.smithi090.stdout:6/379: rename d4/d13/d22/d34/f68 to d4/d14/f78 0 2024-03-11T19:31:32.581 INFO:tasks.workunit.client.0.smithi090.stdout:8/376: read d0/f47 [3960580,110345] 0 2024-03-11T19:31:32.697 INFO:tasks.workunit.client.0.smithi090.stdout:0/356: chown d2/d11/d15/f28 14 1 2024-03-11T19:31:32.798 INFO:tasks.workunit.client.0.smithi090.stdout:2/336: dwrite d0/de/d12/d46/d4c/f53 [0,4194304] 0 2024-03-11T19:31:32.883 INFO:tasks.workunit.client.0.smithi090.stdout:5/344: mkdir d1/d17/d39/d51/d73 0 2024-03-11T19:31:32.897 INFO:tasks.workunit.client.0.smithi090.stdout:9/335: creat d1/dd/f69 x:0 0 0 2024-03-11T19:31:32.920 INFO:tasks.workunit.client.0.smithi090.stdout:4/372: creat d4/d27/d44/d74/f84 x:0 0 0 2024-03-11T19:31:33.213 INFO:tasks.workunit.client.0.smithi090.stdout:8/377: creat d0/d3/d4c/f6f x:0 0 0 2024-03-11T19:31:33.433 INFO:tasks.workunit.client.0.smithi090.stdout:0/357: unlink d2/d11/d15/d29/d43/d58/f5e 0 2024-03-11T19:31:33.881 INFO:tasks.workunit.client.0.smithi090.stdout:2/337: truncate d0/de/d12/d46/f48 46776 0 2024-03-11T19:31:33.912 INFO:tasks.workunit.client.0.smithi090.stdout:4/373: dread d4/d27/d44/f4e [0,4194304] 0 2024-03-11T19:31:33.934 INFO:tasks.workunit.client.0.smithi090.stdout:9/336: truncate d1/dd/df/f10 2359566 0 2024-03-11T19:31:34.023 INFO:tasks.workunit.client.0.smithi090.stdout:2/338: sync 2024-03-11T19:31:34.139 INFO:tasks.workunit.client.0.smithi090.stdout:7/348: unlink d3/l10 0 2024-03-11T19:31:34.377 INFO:tasks.workunit.client.0.smithi090.stdout:6/380: rmdir d4/d13/d58 39 2024-03-11T19:31:34.651 INFO:tasks.workunit.client.0.smithi090.stdout:5/345: symlink d1/d17/l74 0 2024-03-11T19:31:34.883 INFO:tasks.workunit.client.0.smithi090.stdout:0/358: dread - d2/d11/d15/d36/f46 zero size 2024-03-11T19:31:34.920 INFO:tasks.workunit.client.0.smithi090.stdout:0/359: chown d2/d11/d3f/f40 338766 1 2024-03-11T19:31:34.975 INFO:tasks.workunit.client.0.smithi090.stdout:4/374: creat d4/d27/d28/d69/f85 x:0 0 0 2024-03-11T19:31:35.046 INFO:tasks.workunit.client.0.smithi090.stdout:4/375: chown d4/d27/d28/d38/d64/c67 913 1 2024-03-11T19:31:35.152 INFO:tasks.workunit.client.0.smithi090.stdout:9/337: unlink d1/dd/d18/l28 0 2024-03-11T19:31:35.377 INFO:tasks.workunit.client.0.smithi090.stdout:2/339: fdatasync d0/de/f17 0 2024-03-11T19:31:35.459 INFO:tasks.workunit.client.0.smithi090.stdout:7/349: fdatasync d3/f8 0 2024-03-11T19:31:35.483 INFO:tasks.workunit.client.0.smithi090.stdout:1/321: dwrite d5/da/d1c/f30 [0,4194304] 0 2024-03-11T19:31:35.636 INFO:tasks.workunit.client.0.smithi090.stdout:6/381: rename d4/d13/d22/d34/d37/d47 to d4/d13/d22/d42/d48/d4e/d61/d79 0 2024-03-11T19:31:35.656 INFO:tasks.workunit.client.0.smithi090.stdout:2/340: chown d0/de/cf 99 1 2024-03-11T19:31:35.709 INFO:tasks.workunit.client.0.smithi090.stdout:2/341: chown d0/d21/c59 3045892 1 2024-03-11T19:31:36.004 INFO:tasks.workunit.client.0.smithi090.stdout:0/360: truncate d2/d11/d15/d36/f57 170564 0 2024-03-11T19:31:36.035 INFO:tasks.workunit.client.0.smithi090.stdout:4/376: creat d4/d27/d44/f86 x:0 0 0 2024-03-11T19:31:36.087 INFO:tasks.workunit.client.0.smithi090.stdout:9/338: mkdir d1/d1d/d6a 0 2024-03-11T19:31:36.278 INFO:tasks.workunit.client.0.smithi090.stdout:8/378: fsync d0/d3/d4c/f6f 0 2024-03-11T19:31:36.314 INFO:tasks.workunit.client.0.smithi090.stdout:9/339: readlink d1/dd/df/l5e 0 2024-03-11T19:31:36.570 INFO:tasks.workunit.client.0.smithi090.stdout:3/318: dwrite d2/d1f/f35 [0,4194304] 0 2024-03-11T19:31:36.594 INFO:tasks.workunit.client.0.smithi090.stdout:8/379: dread d0/d3/d3e/f4e [0,4194304] 0 2024-03-11T19:31:37.005 INFO:tasks.workunit.client.0.smithi090.stdout:5/346: truncate d1/d17/f30 1456768 0 2024-03-11T19:31:37.387 INFO:tasks.workunit.client.0.smithi090.stdout:0/361: dread d2/d11/d15/d29/f5b [0,4194304] 0 2024-03-11T19:31:37.482 INFO:tasks.workunit.client.0.smithi090.stdout:4/377: truncate d4/d18/f37 560896 0 2024-03-11T19:31:37.703 INFO:tasks.workunit.client.0.smithi090.stdout:9/340: truncate d1/f9 994374 0 2024-03-11T19:31:37.885 INFO:tasks.workunit.client.0.smithi090.stdout:6/382: unlink d4/d14/d44/c73 0 2024-03-11T19:31:38.560 INFO:tasks.workunit.client.0.smithi090.stdout:0/362: creat d2/d11/d3a/f70 x:0 0 0 2024-03-11T19:31:38.794 INFO:tasks.workunit.client.0.smithi090.stdout:4/378: creat d4/d18/d6f/f87 x:0 0 0 2024-03-11T19:31:39.092 INFO:tasks.workunit.client.0.smithi090.stdout:9/341: read d1/dd/f61 [3744211,69237] 0 2024-03-11T19:31:39.127 INFO:tasks.workunit.client.0.smithi090.stdout:6/383: unlink d4/d13/l30 0 2024-03-11T19:31:39.234 INFO:tasks.workunit.client.0.smithi090.stdout:6/384: dread d4/d14/f78 [0,4194304] 0 2024-03-11T19:31:39.387 INFO:tasks.workunit.client.0.smithi090.stdout:8/380: symlink d0/d43/l70 0 2024-03-11T19:31:39.444 INFO:tasks.workunit.client.0.smithi090.stdout:7/350: link d3/dc/d17/d34/c43 d3/d49/c73 0 2024-03-11T19:31:39.857 INFO:tasks.workunit.client.0.smithi090.stdout:5/347: getdents d1/d17/d39/d51/d73 0 2024-03-11T19:31:39.974 INFO:tasks.workunit.client.0.smithi090.stdout:0/363: truncate d2/d11/f41 959768 0 2024-03-11T19:31:40.132 INFO:tasks.workunit.client.0.smithi090.stdout:2/342: write d0/f4 [8402275,54523] 0 2024-03-11T19:31:40.696 INFO:tasks.workunit.client.0.smithi090.stdout:6/385: mknod d4/d13/d22/d42/d48/d4e/d61/d79/c7a 0 2024-03-11T19:31:40.740 INFO:tasks.workunit.client.0.smithi090.stdout:6/386: readlink d4/d13/d22/d34/d37/l57 0 2024-03-11T19:31:40.788 INFO:tasks.workunit.client.0.smithi090.stdout:5/348: fsync d1/d26/f54 0 2024-03-11T19:31:41.071 INFO:tasks.workunit.client.0.smithi090.stdout:4/379: rename d4/d27/d28/d38/c42 to d4/d18/c88 0 2024-03-11T19:31:41.148 INFO:tasks.workunit.client.0.smithi090.stdout:7/351: dread d3/dc/d17/d32/d71/d5d/f53 [0,4194304] 0 2024-03-11T19:31:41.733 INFO:tasks.workunit.client.0.smithi090.stdout:1/322: creat d5/da/f68 x:0 0 0 2024-03-11T19:31:42.629 INFO:tasks.workunit.client.0.smithi090.stdout:3/319: write d2/db/d15/f17 [7451694,112401] 0 2024-03-11T19:31:43.034 INFO:tasks.workunit.client.0.smithi090.stdout:0/364: creat d2/d11/d15/d29/f71 x:0 0 0 2024-03-11T19:31:43.421 INFO:tasks.workunit.client.0.smithi090.stdout:1/323: dread d5/fc [4194304,4194304] 0 2024-03-11T19:31:43.667 INFO:tasks.workunit.client.0.smithi090.stdout:3/320: mknod d2/d45/c6b 0 2024-03-11T19:31:43.751 INFO:tasks.workunit.client.0.smithi090.stdout:1/324: dwrite d5/da/f68 [0,4194304] 0 2024-03-11T19:31:43.752 INFO:tasks.workunit.client.0.smithi090.stdout:0/365: dread - d2/d11/d15/d29/f6a zero size 2024-03-11T19:31:44.289 INFO:tasks.workunit.client.0.smithi090.stdout:2/343: dwrite d0/de/f17 [0,4194304] 0 2024-03-11T19:31:44.679 INFO:tasks.workunit.client.0.smithi090.stdout:8/381: rename d0/f2a to d0/d53/f71 0 2024-03-11T19:31:44.682 INFO:tasks.workunit.client.0.smithi090.stdout:6/387: write d4/d13/d22/d3a/f54 [3140326,48663] 0 2024-03-11T19:31:45.147 INFO:tasks.workunit.client.0.smithi090.stdout:7/352: write d3/dc/d17/d32/f4a [142554,6873] 0 2024-03-11T19:31:45.474 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:31:45.478 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:31:45.541 INFO:tasks.workunit.client.0.smithi090.stdout:5/349: mkdir d1/d75 0 2024-03-11T19:31:45.689 INFO:tasks.workunit.client.0.smithi090.stdout:9/342: mknod d1/c6b 0 2024-03-11T19:31:45.838 INFO:tasks.workunit.client.0.smithi090.stdout:4/380: rename d4/d27/d28/d38/d79 to d4/d51/d52/d89 0 2024-03-11T19:31:46.127 INFO:tasks.workunit.client.0.smithi090.stdout:8/382: creat d0/d53/d59/f72 x:0 0 0 2024-03-11T19:31:46.426 INFO:tasks.workunit.client.0.smithi090.stdout:1/325: creat d5/d65/f69 x:0 0 0 2024-03-11T19:31:46.769 INFO:tasks.workunit.client.0.smithi090.stdout:7/353: truncate d3/dc/d17/d32/d45/f5b 452290 0 2024-03-11T19:31:47.163 INFO:tasks.workunit.client.0.smithi090.stdout:7/354: sync 2024-03-11T19:31:47.255 INFO:tasks.workunit.client.0.smithi090.stdout:9/343: mknod d1/dd/d18/d3a/c6c 0 2024-03-11T19:31:47.283 INFO:tasks.workunit.client.0.smithi090.stdout:9/344: chown d1/dd/df/c27 4613995 1 2024-03-11T19:31:47.367 INFO:tasks.workunit.client.0.smithi090.stdout:4/381: fdatasync d4/d18/f6e 0 2024-03-11T19:31:47.478 INFO:tasks.workunit.client.0.smithi090.stdout:6/388: creat d4/d13/d22/d42/d48/d4e/d61/d79/d74/f7b x:0 0 0 2024-03-11T19:31:47.555 INFO:tasks.workunit.client.0.smithi090.stdout:6/389: rename d4/d13/d22/d42/d48/d4e/d61 to d4/d13/d22/d42/d48/d4e/d61/d7c 22 2024-03-11T19:31:47.577 INFO:tasks.workunit.client.0.smithi090.stdout:8/383: dread - d0/d3/d4c/f69 zero size 2024-03-11T19:31:47.722 INFO:tasks.workunit.client.0.smithi090.stdout:1/326: fsync d5/da/d1c/f30 0 2024-03-11T19:31:47.843 INFO:tasks.workunit.client.0.smithi090.stdout:3/321: link d2/d1f/f3f d2/d45/d55/f6c 0 2024-03-11T19:31:48.266 INFO:tasks.workunit.client.0.smithi090.stdout:2/344: dwrite d0/d21/d30/f31 [0,4194304] 0 2024-03-11T19:31:48.393 INFO:tasks.workunit.client.0.smithi090.stdout:8/384: truncate d0/d43/f64 1130180 0 2024-03-11T19:31:48.435 INFO:tasks.workunit.client.0.smithi090.stdout:2/345: chown d0/d21/c43 0 1 2024-03-11T19:31:48.578 INFO:tasks.workunit.client.0.smithi090.stdout:1/327: truncate d5/f4f 3935882 0 2024-03-11T19:31:48.659 INFO:tasks.workunit.client.0.smithi090.stdout:3/322: unlink d2/de/l10 0 2024-03-11T19:31:48.679 INFO:tasks.workunit.client.0.smithi090.stdout:3/323: chown d2/d4f/d59 1 1 2024-03-11T19:31:48.792 INFO:tasks.workunit.client.0.smithi090.stdout:3/324: sync 2024-03-11T19:31:48.850 INFO:tasks.workunit.client.0.smithi090.stdout:9/345: fdatasync d1/f3 0 2024-03-11T19:31:49.191 INFO:tasks.workunit.client.0.smithi090.stdout:4/382: fsync d4/d18/f37 0 2024-03-11T19:31:49.351 INFO:tasks.workunit.client.0.smithi090.stdout:0/366: mknod d2/d11/c72 0 2024-03-11T19:31:49.560 INFO:tasks.workunit.client.0.smithi090.stdout:5/350: write d1/d17/d39/d51/d53/d64/f6d [169883,123842] 0 2024-03-11T19:31:50.232 INFO:tasks.workunit.client.0.smithi090.stdout:3/325: dread d2/db/d15/f1e [0,4194304] 0 2024-03-11T19:31:50.482 INFO:tasks.workunit.client.0.smithi090.stdout:4/383: fdatasync d4/d27/d44/f4e 0 2024-03-11T19:31:50.835 INFO:tasks.workunit.client.0.smithi090.stdout:6/390: dwrite d4/d13/d22/d3a/f65 [0,4194304] 0 2024-03-11T19:31:51.535 INFO:tasks.workunit.client.0.smithi090.stdout:2/346: write d0/f7 [6495956,59363] 0 2024-03-11T19:31:51.606 INFO:tasks.workunit.client.0.smithi090.stdout:9/346: write d1/dd/df/f4d [940182,117952] 0 2024-03-11T19:31:51.612 INFO:tasks.workunit.client.0.smithi090.stdout:1/328: write d5/d31/d4c/f59 [411018,9334] 0 2024-03-11T19:31:51.737 INFO:tasks.workunit.client.0.smithi090.stdout:9/347: chown d1/dd/d18/d3a/d48 7180574 1 2024-03-11T19:31:51.800 INFO:tasks.workunit.client.0.smithi090.stdout:7/355: link d3/dc/d17/d32/d71/d5d/l2f d3/dc/d17/d32/d71/l74 0 2024-03-11T19:31:52.510 INFO:tasks.workunit.client.0.smithi090.stdout:0/367: creat d2/d11/d15/d29/d2e/d6d/f73 x:0 0 0 2024-03-11T19:31:52.552 INFO:tasks.workunit.client.0.smithi090.stdout:0/368: chown d2/d11/d15/d29/d43/f4d 3782 1 2024-03-11T19:31:52.568 INFO:tasks.workunit.client.0.smithi090.stdout:0/369: chown d2/cc 55235035 1 2024-03-11T19:31:52.592 INFO:tasks.workunit.client.0.smithi090.stdout:0/370: dread - d2/d11/f24 zero size 2024-03-11T19:31:52.795 INFO:tasks.workunit.client.0.smithi090.stdout:6/391: mkdir d4/d13/d22/d34/d40/d7d 0 2024-03-11T19:31:52.965 INFO:tasks.workunit.client.0.smithi090.stdout:8/385: link d0/d43/dc/l4f d0/d3/d4c/l73 0 2024-03-11T19:31:53.563 INFO:tasks.workunit.client.0.smithi090.stdout:7/356: dread - d3/dc/d29/f6b zero size 2024-03-11T19:31:53.608 INFO:tasks.workunit.client.0.smithi090.stdout:1/329: dread d5/da/f68 [0,4194304] 0 2024-03-11T19:31:53.688 INFO:tasks.workunit.client.0.smithi090.stdout:1/330: chown d5/d31/c46 3081330 1 2024-03-11T19:31:53.739 INFO:tasks.workunit.client.0.smithi090.stdout:3/326: mknod d2/d1d/d47/d6a/c6d 0 2024-03-11T19:31:54.428 INFO:tasks.workunit.client.0.smithi090.stdout:8/386: symlink d0/d53/l74 0 2024-03-11T19:31:55.138 INFO:tasks.workunit.client.0.smithi090.stdout:3/327: symlink d2/l6e 0 2024-03-11T19:31:55.608 INFO:tasks.workunit.client.0.smithi090.stdout:5/351: getdents d1/d26 0 2024-03-11T19:31:55.647 INFO:tasks.workunit.client.0.smithi090.stdout:2/347: dwrite d0/de/d12/d46/f5b [0,4194304] 0 2024-03-11T19:31:55.786 INFO:tasks.workunit.client.0.smithi090.stdout:8/387: creat d0/d43/dc/f75 x:0 0 0 2024-03-11T19:31:55.820 INFO:tasks.workunit.client.0.smithi090.stdout:2/348: readlink d0/l9 0 2024-03-11T19:31:56.182 INFO:tasks.workunit.client.0.smithi090.stdout:7/357: write d3/dc/d17/d32/d71/d5d/f21 [1314427,124718] 0 2024-03-11T19:31:56.248 INFO:tasks.workunit.client.0.smithi090.stdout:4/384: link d4/c22 d4/d27/d28/d38/c8a 0 2024-03-11T19:31:56.720 INFO:tasks.workunit.client.0.smithi090.stdout:6/392: creat d4/d13/f7e x:0 0 0 2024-03-11T19:31:56.853 INFO:tasks.workunit.client.0.smithi090.stdout:9/348: getdents d1/d49 0 2024-03-11T19:31:57.083 INFO:tasks.workunit.client.0.smithi090.stdout:5/352: mkdir d1/d26/d19/d27/d3b/d76 0 2024-03-11T19:31:57.106 INFO:tasks.workunit.client.0.smithi090.stdout:2/349: creat d0/de/d12/f5e x:0 0 0 2024-03-11T19:31:57.148 INFO:tasks.workunit.client.0.smithi090.stdout:5/353: readlink d1/d26/l47 0 2024-03-11T19:31:57.424 INFO:tasks.workunit.client.0.smithi090.stdout:4/385: fsync d4/f12 0 2024-03-11T19:31:57.566 INFO:tasks.workunit.client.0.smithi090.stdout:0/371: link d2/d11/d15/d36/f46 d2/d11/d15/f74 0 2024-03-11T19:31:58.180 INFO:tasks.workunit.client.0.smithi090.stdout:8/388: mknod d0/c76 0 2024-03-11T19:31:58.484 INFO:tasks.workunit.client.0.smithi090.stdout:9/349: mkdir d1/dd/df/d1c/d6d 0 2024-03-11T19:31:58.544 INFO:tasks.workunit.client.0.smithi090.stdout:2/350: creat d0/f5f x:0 0 0 2024-03-11T19:31:59.414 INFO:tasks.workunit.client.0.smithi090.stdout:8/389: rmdir d0/d43/dc 39 2024-03-11T19:31:59.430 INFO:tasks.workunit.client.0.smithi090.stdout:6/393: mkdir d4/d13/d22/d34/d37/d7f 0 2024-03-11T19:31:59.659 INFO:tasks.workunit.client.0.smithi090.stdout:9/350: dread d1/dd/df/f4d [0,4194304] 0 2024-03-11T19:31:59.706 INFO:tasks.workunit.client.0.smithi090.stdout:5/354: creat d1/d26/d19/d27/d3b/d76/f77 x:0 0 0 2024-03-11T19:32:00.096 INFO:tasks.workunit.client.0.smithi090.stdout:1/331: link d5/d31/l5c d5/da/d23/l6a 0 2024-03-11T19:32:00.411 INFO:tasks.workunit.client.0.smithi090.stdout:4/386: write d4/d27/d44/f4e [1004835,70725] 0 2024-03-11T19:32:00.648 INFO:tasks.workunit.client.0.smithi090.stdout:0/372: dwrite d2/f8 [0,4194304] 0 2024-03-11T19:32:00.650 INFO:tasks.workunit.client.0.smithi090.stdout:6/394: mkdir d4/d24/d80 0 2024-03-11T19:32:01.091 INFO:tasks.workunit.client.0.smithi090.stdout:5/355: symlink d1/d26/l78 0 2024-03-11T19:32:01.290 INFO:tasks.workunit.client.0.smithi090.stdout:7/358: getdents d3/dc/d29 0 2024-03-11T19:32:01.500 INFO:tasks.workunit.client.0.smithi090.stdout:1/332: dread d5/da/d23/f3a [0,4194304] 0 2024-03-11T19:32:01.876 INFO:tasks.workunit.client.0.smithi090.stdout:9/351: dwrite d1/dd/df/f4d [0,4194304] 0 2024-03-11T19:32:01.906 INFO:tasks.workunit.client.0.smithi090.stdout:6/395: truncate d4/f46 2069042 0 2024-03-11T19:32:01.907 INFO:tasks.workunit.client.0.smithi090.stdout:4/387: creat d4/d27/d28/f8b x:0 0 0 2024-03-11T19:32:01.923 INFO:tasks.workunit.client.0.smithi090.stdout:0/373: truncate d2/d11/f47 607922 0 2024-03-11T19:32:02.162 INFO:tasks.workunit.client.0.smithi090.stdout:6/396: chown d4/d13/d22/d34/f36 6 1 2024-03-11T19:32:02.384 INFO:tasks.workunit.client.0.smithi090.stdout:1/333: dread d5/da/f4a [0,4194304] 0 2024-03-11T19:32:02.767 INFO:tasks.workunit.client.0.smithi090.stdout:3/328: getdents d2/d1d/d47 0 2024-03-11T19:32:02.972 INFO:tasks.workunit.client.0.smithi090.stdout:8/390: rename d0/c76 to d0/d43/c77 0 2024-03-11T19:32:03.002 INFO:tasks.workunit.client.0.smithi090.stdout:5/356: write d1/f1e [3780013,112416] 0 2024-03-11T19:32:03.111 INFO:tasks.workunit.client.0.smithi090.stdout:5/357: chown d1/d26/d19/d27/f5b 40636 1 2024-03-11T19:32:03.127 INFO:tasks.workunit.client.0.smithi090.stdout:7/359: symlink d3/dc/d17/d32/d67/l75 0 2024-03-11T19:32:03.290 INFO:tasks.workunit.client.0.smithi090.stdout:5/358: dread d1/d26/f20 [4194304,4194304] 0 2024-03-11T19:32:03.527 INFO:tasks.workunit.client.0.smithi090.stdout:2/351: symlink d0/d21/d30/l60 0 2024-03-11T19:32:03.648 INFO:tasks.workunit.client.0.smithi090.stdout:3/329: read d2/d1d/fc [2277699,58349] 0 2024-03-11T19:32:03.884 INFO:tasks.workunit.client.0.smithi090.stdout:2/352: sync 2024-03-11T19:32:03.889 INFO:tasks.workunit.client.0.smithi090.stdout:0/374: rename d2/d11/d15/d29/f6a to d2/d11/d3a/f75 0 2024-03-11T19:32:04.185 INFO:tasks.workunit.client.0.smithi090.stdout:1/334: fsync d5/f4f 0 2024-03-11T19:32:04.312 INFO:tasks.workunit.client.0.smithi090.stdout:9/352: write d1/dd/f3f [1698390,128285] 0 2024-03-11T19:32:04.514 INFO:tasks.workunit.client.0.smithi090.stdout:4/388: write d4/d27/d28/f68 [1279616,47711] 0 2024-03-11T19:32:04.554 INFO:tasks.workunit.client.0.smithi090.stdout:6/397: write d4/fb [3769514,88875] 0 2024-03-11T19:32:04.914 INFO:tasks.workunit.client.0.smithi090.stdout:1/335: mkdir d5/d65/d6b 0 2024-03-11T19:32:05.474 INFO:tasks.workunit.client.0.smithi090.stdout:2/353: dread d0/de/d12/d46/d4c/f53 [0,4194304] 0 2024-03-11T19:32:05.504 INFO:tasks.workunit.client.0.smithi090.stdout:2/354: readlink d0/de/d12/l19 0 2024-03-11T19:32:05.619 INFO:tasks.workunit.client.0.smithi090.stdout:4/389: rename d4/d51/d52 to d4/d51/d8c 0 2024-03-11T19:32:05.885 INFO:tasks.workunit.client.0.smithi090.stdout:6/398: symlink d4/d13/d22/d34/d40/l81 0 2024-03-11T19:32:05.966 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:57:25.723 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:32:06.015 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:57:25.711 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:32:06.020 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:57:25.709 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:32:06.027 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:57:25.724 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:32:06.028 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:57:25.716 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:32:06.049 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:57:25.807 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:32:06.199 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:57:25.895 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:32:06.208 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:57:25.897 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:32:06.220 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:57:25.978 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:32:06.301 INFO:tasks.workunit.client.0.smithi090.stdout:3/330: dwrite d2/db/f1c [0,4194304] 0 2024-03-11T19:32:06.523 INFO:tasks.workunit.client.0.smithi090.stdout:8/391: link d0/d53/d59/l62 d0/l78 0 2024-03-11T19:32:06.539 INFO:tasks.workunit.client.0.smithi090.stdout:7/360: rename d3/c1c to d3/dc/c76 0 2024-03-11T19:32:06.624 INFO:tasks.workunit.client.0.smithi090.stdout:2/355: chown d0/de/d12/d46/f48 11625523 1 2024-03-11T19:32:06.639 INFO:tasks.workunit.client.0.smithi090.stdout:4/390: truncate d4/d18/f25 2283512 0 2024-03-11T19:32:06.910 INFO:tasks.workunit.client.0.smithi090.stdout:6/399: mknod d4/d14/d44/c82 0 2024-03-11T19:32:07.309 INFO:tasks.workunit.client.0.smithi090.stdout:1/336: write d5/da/dd/f1a [12801051,30742] 0 2024-03-11T19:32:07.488 INFO:tasks.workunit.client.0.smithi090.stdout:0/375: rename d2/f8 to d2/d11/d3f/f76 0 2024-03-11T19:32:07.495 INFO:tasks.workunit.client.0.smithi090.stdout:8/392: rmdir d0/d3/d3e 39 2024-03-11T19:32:07.890 INFO:tasks.workunit.client.0.smithi090.stdout:5/359: link d1/f4f d1/d17/f79 0 2024-03-11T19:32:08.073 INFO:tasks.workunit.client.0.smithi090.stdout:3/331: creat d2/db/d15/d69/f6f x:0 0 0 2024-03-11T19:32:08.346 INFO:tasks.workunit.client.0.smithi090.stdout:0/376: mknod d2/d11/d3f/c77 0 2024-03-11T19:32:08.962 INFO:tasks.workunit.client.0.smithi090.stdout:5/360: mknod d1/d26/c7a 0 2024-03-11T19:32:09.127 INFO:tasks.workunit.client.0.smithi090.stdout:4/391: write d4/d27/d28/d69/f85 [171876,68374] 0 2024-03-11T19:32:09.287 INFO:tasks.workunit.client.0.smithi090.stdout:9/353: rename d1/dd/d18/d25/l3b to d1/dd/df/d1c/d21/l6e 0 2024-03-11T19:32:09.369 INFO:tasks.workunit.client.0.smithi090.stdout:7/361: dwrite d3/dc/d17/d32/d71/d5d/f25 [0,4194304] 0 2024-03-11T19:32:09.673 INFO:tasks.workunit.client.0.smithi090.stdout:3/332: truncate d2/de/f58 63749 0 2024-03-11T19:32:09.701 INFO:tasks.workunit.client.0.smithi090.stdout:6/400: truncate d4/fb 2325031 0 2024-03-11T19:32:09.802 INFO:tasks.workunit.client.0.smithi090.stdout:1/337: dwrite d5/f37 [0,4194304] 0 2024-03-11T19:32:09.815 INFO:tasks.workunit.client.0.smithi090.stdout:4/392: mkdir d4/d51/d8c/d6a/d8d 0 2024-03-11T19:32:09.969 INFO:tasks.workunit.client.0.smithi090.stdout:6/401: chown d4/d14/f6a 169585 1 2024-03-11T19:32:10.052 INFO:tasks.workunit.client.0.smithi090.stdout:3/333: rmdir d2/d1d 39 2024-03-11T19:32:10.056 INFO:tasks.workunit.client.0.smithi090.stdout:1/338: chown d5/fc 2974 1 2024-03-11T19:32:10.406 INFO:tasks.workunit.client.0.smithi090.stdout:4/393: dwrite d4/d27/d28/d69/f85 [0,4194304] 0 2024-03-11T19:32:10.561 INFO:tasks.workunit.client.0.smithi090.stdout:4/394: chown d4/d27/d28/d69 10 1 2024-03-11T19:32:11.040 INFO:tasks.workunit.client.0.smithi090.stdout:5/361: dwrite d1/d26/f3c [0,4194304] 0 2024-03-11T19:32:11.275 INFO:tasks.workunit.client.0.smithi090.stdout:6/402: mkdir d4/d13/d22/d34/d83 0 2024-03-11T19:32:11.302 INFO:tasks.workunit.client.0.smithi090.stdout:1/339: mknod d5/d3c/d45/c6c 0 2024-03-11T19:32:11.389 INFO:tasks.workunit.client.0.smithi090.stdout:6/403: read d4/d14/f78 [722126,72750] 0 2024-03-11T19:32:11.428 INFO:tasks.workunit.client.0.smithi090.stdout:0/377: write d2/d11/d3a/f75 [194471,79104] 0 2024-03-11T19:32:11.841 INFO:tasks.workunit.client.0.smithi090.stdout:7/362: dwrite d3/dc/d1e/d2a/f66 [0,4194304] 0 2024-03-11T19:32:11.955 INFO:tasks.workunit.client.0.smithi090.stdout:2/356: rename d0/f1b to d0/de/d12/d46/d4c/f61 0 2024-03-11T19:32:12.438 INFO:tasks.workunit.client.0.smithi090.stdout:6/404: symlink d4/d13/d22/d34/d40/l84 0 2024-03-11T19:32:12.483 INFO:tasks.workunit.client.0.smithi090.stdout:3/334: creat d2/d45/f70 x:0 0 0 2024-03-11T19:32:13.048 INFO:tasks.workunit.client.0.smithi090.stdout:9/354: dwrite d1/dd/f13 [0,4194304] 0 2024-03-11T19:32:13.309 INFO:tasks.workunit.client.0.smithi090.stdout:4/395: getdents d4/d27/d28/d38 0 2024-03-11T19:32:13.993 INFO:tasks.workunit.client.0.smithi090.stdout:9/355: fsync d1/f58 0 2024-03-11T19:32:14.296 INFO:tasks.workunit.client.0.smithi090.stdout:1/340: write d5/d31/d4c/f59 [624064,20109] 0 2024-03-11T19:32:14.343 INFO:tasks.workunit.client.0.smithi090.stdout:1/341: stat d5 0 2024-03-11T19:32:14.533 INFO:tasks.workunit.client.0.smithi090.stdout:4/396: creat d4/d51/d8c/d6a/d8d/f8e x:0 0 0 2024-03-11T19:32:14.885 INFO:tasks.workunit.client.0.smithi090.stdout:8/393: rename d0/f31 to d0/d43/f79 0 2024-03-11T19:32:15.077 INFO:tasks.workunit.client.0.smithi090.stdout:3/335: mknod d2/db/d15/d1a/d3c/c71 0 2024-03-11T19:32:15.193 INFO:tasks.workunit.client.0.smithi090.stdout:1/342: unlink d5/da/d23/f3a 0 2024-03-11T19:32:15.217 INFO:tasks.workunit.client.0.smithi090.stdout:1/343: chown d5/da/f47 3087 1 2024-03-11T19:32:15.537 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:32:15.540 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:32:15.556 INFO:tasks.workunit.client.0.smithi090.stdout:7/363: dwrite d3/dc/d17/d32/d71/d5d/f4c [0,4194304] 0 2024-03-11T19:32:15.694 INFO:tasks.workunit.client.0.smithi090.stdout:5/362: rename d1/d26/d19/d27 to d1/d17/d39/d51/d53/d7b 0 2024-03-11T19:32:15.750 INFO:tasks.workunit.client.0.smithi090.stdout:6/405: rmdir d4 39 2024-03-11T19:32:15.915 INFO:tasks.workunit.client.0.smithi090.stdout:0/378: dwrite d2/d11/d15/d36/f57 [0,4194304] 0 2024-03-11T19:32:16.039 INFO:tasks.workunit.client.0.smithi090.stdout:0/379: chown d2/d11/l13 12127 1 2024-03-11T19:32:16.098 INFO:tasks.workunit.client.0.smithi090.stdout:0/380: chown d2/d11/d15/d29/d43 47115092 1 2024-03-11T19:32:16.320 INFO:tasks.workunit.client.0.smithi090.stdout:1/344: mkdir d5/da/d1c/d6d 0 2024-03-11T19:32:16.790 INFO:tasks.workunit.client.0.smithi090.stdout:9/356: write d1/dd/f30 [391958,121544] 0 2024-03-11T19:32:17.306 INFO:tasks.workunit.client.0.smithi090.stdout:2/357: creat d0/de/d12/d46/d5d/f62 x:0 0 0 2024-03-11T19:32:17.380 INFO:tasks.workunit.client.0.smithi090.stdout:6/406: dread d4/f11 [0,4194304] 0 2024-03-11T19:32:17.789 INFO:tasks.workunit.client.0.smithi090.stdout:0/381: mkdir d2/d11/d15/d29/d6b/d78 0 2024-03-11T19:32:18.024 INFO:tasks.workunit.client.0.smithi090.stdout:3/336: symlink d2/d45/d55/l72 0 2024-03-11T19:32:18.031 INFO:tasks.workunit.client.0.smithi090.stdout:8/394: link d0/d43/f2e d0/d53/f7a 0 2024-03-11T19:32:18.539 INFO:tasks.workunit.client.0.smithi090.stdout:2/358: mkdir d0/de/d12/d46/d5d/d63 0 2024-03-11T19:32:18.706 INFO:tasks.workunit.client.0.smithi090.stdout:6/407: mknod d4/d14/c85 0 2024-03-11T19:32:18.931 INFO:tasks.workunit.client.0.smithi090.stdout:0/382: read d2/d11/d15/d29/f68 [5042,108305] 0 2024-03-11T19:32:19.325 INFO:tasks.workunit.client.0.smithi090.stdout:8/395: chown d0/d43/dc/fd 129 1 2024-03-11T19:32:19.355 INFO:tasks.workunit.client.0.smithi090.stdout:8/396: chown d0/d43/dc/l16 40205790 1 2024-03-11T19:32:19.642 INFO:tasks.workunit.client.0.smithi090.stdout:6/408: unlink d4/d14/c5c 0 2024-03-11T19:32:19.797 INFO:tasks.workunit.client.0.smithi090.stdout:8/397: sync 2024-03-11T19:32:19.931 INFO:tasks.workunit.client.0.smithi090.stdout:3/337: creat d2/d1d/d47/d6a/f73 x:0 0 0 2024-03-11T19:32:20.043 INFO:tasks.workunit.client.0.smithi090.stdout:3/338: sync 2024-03-11T19:32:20.109 INFO:tasks.workunit.client.0.smithi090.stdout:6/409: dread - d4/d13/d22/d34/f6d zero size 2024-03-11T19:32:20.189 INFO:tasks.workunit.client.0.smithi090.stdout:7/364: getdents d3/dc/d17/d34 0 2024-03-11T19:32:20.269 INFO:tasks.workunit.client.0.smithi090.stdout:8/398: mknod d0/d43/dc/c7b 0 2024-03-11T19:32:20.560 INFO:tasks.workunit.client.0.smithi090.stdout:9/357: dwrite d1/d1d/f66 [0,4194304] 0 2024-03-11T19:32:21.125 INFO:tasks.workunit.client.0.smithi090.stdout:3/339: dread d2/db/f62 [0,4194304] 0 2024-03-11T19:32:21.145 INFO:tasks.workunit.client.0.smithi090.stdout:8/399: read d0/f17 [1031097,45432] 0 2024-03-11T19:32:21.161 INFO:tasks.workunit.client.0.smithi090.stdout:4/397: rename d4/l2f to d4/d51/d8c/d6a/d8d/l8f 0 2024-03-11T19:32:21.259 INFO:tasks.workunit.client.0.smithi090.stdout:0/383: creat d2/d11/f79 x:0 0 0 2024-03-11T19:32:21.375 INFO:tasks.workunit.client.0.smithi090.stdout:5/363: dwrite d1/d26/f71 [0,4194304] 0 2024-03-11T19:32:22.113 INFO:tasks.workunit.client.0.smithi090.stdout:2/359: write d0/fc [2057575,55194] 0 2024-03-11T19:32:22.855 INFO:tasks.workunit.client.0.smithi090.stdout:7/365: creat d3/dc/d17/d32/d71/f77 x:0 0 0 2024-03-11T19:32:22.981 INFO:tasks.workunit.client.0.smithi090.stdout:3/340: truncate d2/f12 5066682 0 2024-03-11T19:32:23.101 INFO:tasks.workunit.client.0.smithi090.stdout:8/400: symlink d0/d3/l7c 0 2024-03-11T19:32:23.482 INFO:tasks.workunit.client.0.smithi090.stdout:9/358: write d1/f17 [3035901,81286] 0 2024-03-11T19:32:23.555 INFO:tasks.workunit.client.0.smithi090.stdout:9/359: stat d1/dd/d18/d3a 0 2024-03-11T19:32:23.573 INFO:tasks.workunit.client.0.smithi090.stdout:3/341: read d2/f1b [2416605,76948] 0 2024-03-11T19:32:24.324 INFO:tasks.workunit.client.0.smithi090.stdout:3/342: mkdir d2/db/d74 0 2024-03-11T19:32:24.587 INFO:tasks.workunit.client.0.smithi090.stdout:1/345: rename d5/fc to d5/d4e/d54/f6e 0 2024-03-11T19:32:24.631 INFO:tasks.workunit.client.0.smithi090.stdout:0/384: write d2/d11/d3a/f70 [925630,37981] 0 2024-03-11T19:32:24.771 INFO:tasks.workunit.client.0.smithi090.stdout:7/366: creat d3/dc/d17/d32/d71/f78 x:0 0 0 2024-03-11T19:32:24.951 INFO:tasks.workunit.client.0.smithi090.stdout:0/385: dread d2/d11/d15/d29/d2e/f52 [0,4194304] 0 2024-03-11T19:32:24.957 INFO:tasks.workunit.client.0.smithi090.stdout:3/343: mkdir d2/db/d15/d1a/d75 0 2024-03-11T19:32:25.106 INFO:tasks.workunit.client.0.smithi090.stdout:6/410: rename d4/l3d to d4/d14/d25/l86 0 2024-03-11T19:32:25.234 INFO:tasks.workunit.client.0.smithi090.stdout:1/346: fdatasync d5/da/dd/f1a 0 2024-03-11T19:32:25.394 INFO:tasks.workunit.client.0.smithi090.stdout:4/398: dwrite d4/d27/d44/d74/f84 [0,4194304] 0 2024-03-11T19:32:25.595 INFO:tasks.workunit.client.0.smithi090.stdout:5/364: dwrite d1/d26/f54 [0,4194304] 0 2024-03-11T19:32:25.768 INFO:tasks.workunit.client.0.smithi090.stdout:2/360: write d0/d21/d30/d25/f5c [687914,75174] 0 2024-03-11T19:32:25.781 INFO:tasks.workunit.client.0.smithi090.stdout:9/360: write d1/d1d/f2b [183853,59878] 0 2024-03-11T19:32:25.791 INFO:tasks.workunit.client.0.smithi090.stdout:8/401: write d0/d43/dc/f50 [1728704,106844] 0 2024-03-11T19:32:25.872 INFO:tasks.workunit.client.0.smithi090.stdout:4/399: dread - d4/d18/f3c zero size 2024-03-11T19:32:25.903 INFO:tasks.workunit.client.0.smithi090.stdout:5/365: stat d1/d26/l78 0 2024-03-11T19:32:25.907 INFO:tasks.workunit.client.0.smithi090.stdout:8/402: stat d0/d3/c5c 0 2024-03-11T19:32:26.531 INFO:tasks.workunit.client.0.smithi090.stdout:0/386: creat d2/d11/d3f/f7a x:0 0 0 2024-03-11T19:32:26.630 INFO:tasks.workunit.client.0.smithi090.stdout:0/387: chown d2/d11/f18 88 1 2024-03-11T19:32:26.750 INFO:tasks.workunit.client.0.smithi090.stdout:7/367: dread d3/dc/d17/d32/d71/d5d/f35 [0,4194304] 0 2024-03-11T19:32:27.023 INFO:tasks.workunit.client.0.smithi090.stdout:4/400: fdatasync d4/d18/f2a 0 2024-03-11T19:32:27.089 INFO:tasks.workunit.client.0.smithi090.stdout:6/411: dread d4/d13/d22/d42/d48/d4e/d61/d79/f63 [0,4194304] 0 2024-03-11T19:32:27.140 INFO:tasks.workunit.client.0.smithi090.stdout:6/412: stat d4/d13/d22/d3a/l4b 0 2024-03-11T19:32:27.192 INFO:tasks.workunit.client.0.smithi090.stdout:2/361: mkdir d0/de/d12/d46/d4c/d64 0 2024-03-11T19:32:27.255 INFO:tasks.workunit.client.0.smithi090.stdout:9/361: truncate d1/dd/d18/d32/f34 4886167 0 2024-03-11T19:32:27.287 INFO:tasks.workunit.client.0.smithi090.stdout:9/362: chown d1/d47/l53 12598785 1 2024-03-11T19:32:27.772 INFO:tasks.workunit.client.0.smithi090.stdout:0/388: mkdir d2/d11/d15/d29/d3d/d7b 0 2024-03-11T19:32:27.911 INFO:tasks.workunit.client.0.smithi090.stdout:1/347: write d5/da/d1c/d3b/d40/f50 [1016191,65521] 0 2024-03-11T19:32:28.153 INFO:tasks.workunit.client.0.smithi090.stdout:1/348: sync 2024-03-11T19:32:28.293 INFO:tasks.workunit.client.0.smithi090.stdout:8/403: write d0/d43/dc/f12 [125035,87202] 0 2024-03-11T19:32:28.395 INFO:tasks.workunit.client.0.smithi090.stdout:8/404: sync 2024-03-11T19:32:28.679 INFO:tasks.workunit.client.0.smithi090.stdout:3/344: write d2/d1f/f3b [269908,128645] 0 2024-03-11T19:32:28.686 INFO:tasks.workunit.client.0.smithi090.stdout:2/362: truncate d0/de/f16 729589 0 2024-03-11T19:32:28.724 INFO:tasks.workunit.client.0.smithi090.stdout:0/389: creat d2/d11/d15/d36/f7c x:0 0 0 2024-03-11T19:32:28.748 INFO:tasks.workunit.client.0.smithi090.stdout:7/368: mknod d3/dc/c79 0 2024-03-11T19:32:28.764 INFO:tasks.workunit.client.0.smithi090.stdout:8/405: dread d0/d43/dc/f50 [4194304,4194304] 0 2024-03-11T19:32:28.790 INFO:tasks.workunit.client.0.smithi090.stdout:5/366: creat d1/f7c x:0 0 0 2024-03-11T19:32:28.795 INFO:tasks.workunit.client.0.smithi090.stdout:2/363: sync 2024-03-11T19:32:28.834 INFO:tasks.workunit.client.0.smithi090.stdout:4/401: write d4/d18/d6f/f87 [649798,17232] 0 2024-03-11T19:32:28.921 INFO:tasks.workunit.client.0.smithi090.stdout:6/413: rename d4/d13/f1b to d4/d13/d58/f87 0 2024-03-11T19:32:28.986 INFO:tasks.workunit.client.0.smithi090.stdout:9/363: fsync d1/d1d/f67 0 2024-03-11T19:32:29.125 INFO:tasks.workunit.client.0.smithi090.stdout:5/367: chown d1/d17/d39/d51/c6e 400705743 1 2024-03-11T19:32:29.212 INFO:tasks.workunit.client.0.smithi090.stdout:5/368: chown d1/d26/d19/l56 3299392 1 2024-03-11T19:32:29.475 INFO:tasks.workunit.client.0.smithi090.stdout:4/402: dread d4/d27/f4d [0,4194304] 0 2024-03-11T19:32:29.727 INFO:tasks.workunit.client.0.smithi090.stdout:3/345: creat d2/db/d15/d1a/d3c/f76 x:0 0 0 2024-03-11T19:32:29.966 INFO:tasks.workunit.client.0.smithi090.stdout:6/414: mknod d4/d13/d22/d3a/d67/c88 0 2024-03-11T19:32:30.175 INFO:tasks.workunit.client.0.smithi090.stdout:7/369: truncate d3/dc/d17/d32/d71/d5d/f25 2194110 0 2024-03-11T19:32:30.244 INFO:tasks.workunit.client.0.smithi090.stdout:5/369: dread - d1/d17/d39/d51/d53/f67 zero size 2024-03-11T19:32:30.283 INFO:tasks.workunit.client.0.smithi090.stdout:0/390: dread d2/d11/d15/d29/f4a [0,4194304] 0 2024-03-11T19:32:30.340 INFO:tasks.workunit.client.0.smithi090.stdout:8/406: unlink d0/f19 0 2024-03-11T19:32:30.372 INFO:tasks.workunit.client.0.smithi090.stdout:2/364: dread d0/d21/f2a [0,4194304] 0 2024-03-11T19:32:30.611 INFO:tasks.workunit.client.0.smithi090.stdout:1/349: dwrite d5/d65/f69 [0,4194304] 0 2024-03-11T19:32:30.863 INFO:tasks.workunit.client.0.smithi090.stdout:2/365: dread d0/de/d12/d46/d4c/f53 [0,4194304] 0 2024-03-11T19:32:30.943 INFO:tasks.workunit.client.0.smithi090.stdout:4/403: fsync d4/f2e 0 2024-03-11T19:32:30.950 INFO:tasks.workunit.client.0.smithi090.stdout:9/364: mknod d1/dd/df/d1c/d21/d5c/c6f 0 2024-03-11T19:32:31.091 INFO:tasks.workunit.client.0.smithi090.stdout:7/370: creat d3/dc/d17/d32/f7a x:0 0 0 2024-03-11T19:32:31.522 INFO:tasks.workunit.client.0.smithi090.stdout:8/407: dread d0/f2d [0,4194304] 0 2024-03-11T19:32:31.645 INFO:tasks.workunit.client.0.smithi090.stdout:2/366: creat d0/de/d12/d46/d4c/f65 x:0 0 0 2024-03-11T19:32:32.040 INFO:tasks.workunit.client.0.smithi090.stdout:3/346: fsync d2/d4f/f5d 0 2024-03-11T19:32:32.298 INFO:tasks.workunit.client.0.smithi090.stdout:7/371: truncate d3/d55/f6a 726237 0 2024-03-11T19:32:32.409 INFO:tasks.workunit.client.0.smithi090.stdout:1/350: symlink d5/d65/d6b/l6f 0 2024-03-11T19:32:32.564 INFO:tasks.workunit.client.0.smithi090.stdout:6/415: dwrite d4/d14/f6a [0,4194304] 0 2024-03-11T19:32:33.156 INFO:tasks.workunit.client.0.smithi090.stdout:5/370: dwrite d1/f2c [0,4194304] 0 2024-03-11T19:32:33.312 INFO:tasks.workunit.client.0.smithi090.stdout:4/404: fsync d4/d18/f25 0 2024-03-11T19:32:34.086 INFO:tasks.workunit.client.0.smithi090.stdout:7/372: mkdir d3/dc/d17/d32/d7b 0 2024-03-11T19:32:34.185 INFO:tasks.workunit.client.0.smithi090.stdout:1/351: read d5/da/f44 [882,15470] 0 2024-03-11T19:32:34.441 INFO:tasks.workunit.client.0.smithi090.stdout:6/416: symlink d4/d13/d22/d42/d48/d4e/d61/d79/d74/l89 0 2024-03-11T19:32:34.974 INFO:tasks.workunit.client.0.smithi090.stdout:4/405: chown d4/c17 14 1 2024-03-11T19:32:35.064 INFO:tasks.workunit.client.0.smithi090.stdout:0/391: write d2/d11/d15/d29/d2e/f32 [2489550,124315] 0 2024-03-11T19:32:35.515 INFO:tasks.workunit.client.0.smithi090.stdout:8/408: write d0/d43/dc/f50 [152785,28856] 0 2024-03-11T19:32:35.554 INFO:tasks.workunit.client.0.smithi090.stdout:1/352: dread - d5/d4e/f5e zero size 2024-03-11T19:32:35.781 INFO:tasks.workunit.client.0.smithi090.stdout:2/367: dwrite d0/de/d1f/f55 [0,4194304] 0 2024-03-11T19:32:35.919 INFO:tasks.workunit.client.0.smithi090.stdout:9/365: write d1/dd/d18/d32/f34 [2408693,60948] 0 2024-03-11T19:32:36.282 INFO:tasks.workunit.client.0.smithi090.stdout:5/371: truncate d1/d17/f79 1694351 0 2024-03-11T19:32:36.299 INFO:tasks.workunit.client.0.smithi090.stdout:5/372: readlink d1/d17/l74 0 2024-03-11T19:32:36.434 INFO:tasks.workunit.client.0.smithi090.stdout:4/406: rmdir d4/d27/d28/d7a 39 2024-03-11T19:32:36.449 INFO:tasks.workunit.client.0.smithi090.stdout:0/392: truncate d2/f27 2504717 0 2024-03-11T19:32:36.730 INFO:tasks.workunit.client.0.smithi090.stdout:6/417: rename d4/d13/d22/d42/d48/d4e/d61/d79 to d4/d14/d8a 0 2024-03-11T19:32:37.286 INFO:tasks.workunit.client.0.smithi090.stdout:8/409: symlink d0/d3/d4c/l7d 0 2024-03-11T19:32:37.661 INFO:tasks.workunit.client.0.smithi090.stdout:9/366: mkdir d1/dd/d18/d3a/d70 0 2024-03-11T19:32:37.696 INFO:tasks.workunit.client.0.smithi090.stdout:7/373: write d3/dc/d1e/f26 [2478606,75658] 0 2024-03-11T19:32:37.843 INFO:tasks.workunit.client.0.smithi090.stdout:5/373: dread d1/d26/f3c [0,4194304] 0 2024-03-11T19:32:37.923 INFO:tasks.workunit.client.0.smithi090.stdout:3/347: link d2/f1b d2/d1d/f77 0 2024-03-11T19:32:37.963 INFO:tasks.workunit.client.0.smithi090.stdout:8/410: creat d0/d43/dc/f7e x:0 0 0 2024-03-11T19:32:38.041 INFO:tasks.workunit.client.0.smithi090.stdout:4/407: dread d4/d27/d44/f4e [0,4194304] 0 2024-03-11T19:32:38.668 INFO:tasks.workunit.client.0.smithi090.stdout:0/393: mknod d2/d11/d3a/d64/c7d 0 2024-03-11T19:32:39.448 INFO:tasks.workunit.client.0.smithi090.stdout:1/353: write d5/da/d1c/f30 [2749023,77232] 0 2024-03-11T19:32:39.738 INFO:tasks.workunit.client.0.smithi090.stdout:4/408: fsync d4/d27/f3a 0 2024-03-11T19:32:39.869 INFO:tasks.workunit.client.0.smithi090.stdout:7/374: mkdir d3/dc/d1e/d7c 0 2024-03-11T19:32:39.985 INFO:tasks.workunit.client.0.smithi090.stdout:5/374: mknod d1/d75/c7d 0 2024-03-11T19:32:40.153 INFO:tasks.workunit.client.0.smithi090.stdout:9/367: dwrite d1/dd/df/d1c/d21/d2a/f3d [0,4194304] 0 2024-03-11T19:32:40.298 INFO:tasks.workunit.client.0.smithi090.stdout:6/418: dwrite d4/d14/f78 [0,4194304] 0 2024-03-11T19:32:40.448 INFO:tasks.workunit.client.0.smithi090.stdout:9/368: sync 2024-03-11T19:32:40.486 INFO:tasks.workunit.client.0.smithi090.stdout:1/354: creat d5/d31/d36/f70 x:0 0 0 2024-03-11T19:32:40.525 INFO:tasks.workunit.client.0.smithi090.stdout:6/419: chown d4/d24/l71 241006904 1 2024-03-11T19:32:40.577 INFO:tasks.workunit.client.0.smithi090.stdout:2/368: rename d0/f4 to d0/f66 0 2024-03-11T19:32:41.122 INFO:tasks.workunit.client.0.smithi090.stdout:0/394: write d2/d11/f1a [1451252,119733] 0 2024-03-11T19:32:41.324 INFO:tasks.workunit.client.0.smithi090.stdout:3/348: write d2/d1d/f25 [2754719,72444] 0 2024-03-11T19:32:41.520 INFO:tasks.workunit.client.0.smithi090.stdout:4/409: dread d4/f24 [0,4194304] 0 2024-03-11T19:32:41.625 INFO:tasks.workunit.client.0.smithi090.stdout:8/411: link d0/d43/l5b d0/d43/dc/l7f 0 2024-03-11T19:32:42.735 INFO:tasks.workunit.client.0.smithi090.stdout:5/375: symlink d1/l7e 0 2024-03-11T19:32:43.221 INFO:tasks.workunit.client.0.smithi090.stdout:1/355: write d5/da/dd/f14 [3384460,94468] 0 2024-03-11T19:32:43.294 INFO:tasks.workunit.client.0.smithi090.stdout:9/369: dwrite d1/f17 [0,4194304] 0 2024-03-11T19:32:43.604 INFO:tasks.workunit.client.0.smithi090.stdout:6/420: fsync d4/d13/f3b 0 2024-03-11T19:32:43.649 INFO:tasks.workunit.client.0.smithi090.stdout:6/421: chown d4/d13/d22/d3a/l4b 7 1 2024-03-11T19:32:44.022 INFO:tasks.workunit.client.0.smithi090.stdout:7/375: rename d3/dc/d17/d32/f51 to d3/d55/f7d 0 2024-03-11T19:32:44.077 INFO:tasks.workunit.client.0.smithi090.stdout:4/410: getdents d4/d27/d28/d38/d75 0 2024-03-11T19:32:44.181 INFO:tasks.workunit.client.0.smithi090.stdout:7/376: sync 2024-03-11T19:32:44.288 INFO:tasks.workunit.client.0.smithi090.stdout:3/349: write d2/db/d15/d1a/f3a [398038,88956] 0 2024-03-11T19:32:44.323 INFO:tasks.workunit.client.0.smithi090.stdout:0/395: dwrite d2/f1c [0,4194304] 0 2024-03-11T19:32:44.330 INFO:tasks.workunit.client.0.smithi090.stdout:7/377: read d3/f4 [1964479,110667] 0 2024-03-11T19:32:44.474 INFO:tasks.workunit.client.0.smithi090.stdout:6/422: unlink d4/d14/d44/f53 0 2024-03-11T19:32:44.868 INFO:tasks.workunit.client.0.smithi090.stdout:8/412: creat d0/d3/f80 x:0 0 0 2024-03-11T19:32:45.066 INFO:tasks.workunit.client.0.smithi090.stdout:4/411: creat d4/d18/f90 x:0 0 0 2024-03-11T19:32:45.277 INFO:tasks.workunit.client.0.smithi090.stdout:8/413: dread d0/d43/dc/f3b [0,4194304] 0 2024-03-11T19:32:45.305 INFO:tasks.workunit.client.0.smithi090.stdout:8/414: chown d0/d43/l35 3 1 2024-03-11T19:32:45.335 INFO:tasks.workunit.client.0.smithi090.stdout:3/350: creat d2/d1f/f78 x:0 0 0 2024-03-11T19:32:45.423 INFO:tasks.workunit.client.0.smithi090.stdout:2/369: write d0/f3 [600585,110733] 0 2024-03-11T19:32:45.572 INFO:tasks.workunit.client.0.smithi090.stdout:7/378: symlink d3/dc/d29/l7e 0 2024-03-11T19:32:45.590 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:32:45.592 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:32:45.679 INFO:tasks.workunit.client.0.smithi090.stdout:8/415: dread d0/d43/dc/fd [0,4194304] 0 2024-03-11T19:32:45.831 INFO:tasks.workunit.client.0.smithi090.stdout:0/396: mkdir d2/d11/d15/d29/d3d/d7e 0 2024-03-11T19:32:45.924 INFO:tasks.workunit.client.0.smithi090.stdout:5/376: link d1/d26/d4d/l69 d1/d17/d39/d51/d73/l7f 0 2024-03-11T19:32:45.978 INFO:tasks.workunit.client.0.smithi090.stdout:5/377: stat d1/d17/d39/d51/d53/d7b/d3b/d76/f77 0 2024-03-11T19:32:46.293 INFO:tasks.workunit.client.0.smithi090.stdout:1/356: write d5/da/d23/f2a [207784,77835] 0 2024-03-11T19:32:46.294 INFO:tasks.workunit.client.0.smithi090.stdout:9/370: creat d1/dd/d18/f71 x:0 0 0 2024-03-11T19:32:46.613 INFO:tasks.workunit.client.0.smithi090.stdout:1/357: dwrite d5/da/d23/f2a [0,4194304] 0 2024-03-11T19:32:47.019 INFO:tasks.workunit.client.0.smithi090.stdout:6/423: fsync d4/ff 0 2024-03-11T19:32:47.102 INFO:tasks.workunit.client.0.smithi090.stdout:6/424: dread - d4/d13/d22/d34/f6d zero size 2024-03-11T19:32:47.403 INFO:tasks.workunit.client.0.smithi090.stdout:4/412: write d4/d18/f2a [3969306,8791] 0 2024-03-11T19:32:47.504 INFO:tasks.workunit.client.0.smithi090.stdout:7/379: dread d3/f7 [0,4194304] 0 2024-03-11T19:32:47.758 INFO:tasks.workunit.client.0.smithi090.stdout:5/378: rmdir d1/d17/d39/d51/d73 39 2024-03-11T19:32:48.009 INFO:tasks.workunit.client.0.smithi090.stdout:1/358: rmdir d5/d65 39 2024-03-11T19:32:48.311 INFO:tasks.workunit.client.0.smithi090.stdout:3/351: getdents d2/db/d74 0 2024-03-11T19:32:48.313 INFO:tasks.workunit.client.0.smithi090.stdout:2/370: truncate d0/d21/f3d 4159164 0 2024-03-11T19:32:48.650 INFO:tasks.workunit.client.0.smithi090.stdout:8/416: dwrite d0/d43/f64 [0,4194304] 0 2024-03-11T19:32:48.738 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:58:08.496 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:32:48.744 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:58:08.503 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:32:48.748 INFO:tasks.workunit.client.0.smithi090.stdout:4/413: creat d4/d27/d28/d38/f91 x:0 0 0 2024-03-11T19:32:48.792 INFO:tasks.workunit.client.0.smithi090.stdout:7/380: mkdir d3/dc/d17/d7f 0 2024-03-11T19:32:48.794 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:58:08.552 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:32:48.806 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:58:08.503 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:32:48.812 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:58:08.509 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:32:48.843 INFO:tasks.workunit.client.0.smithi090.stdout:0/397: symlink d2/d11/d15/l7f 0 2024-03-11T19:32:48.844 INFO:tasks.workunit.client.0.smithi090.stdout:4/414: chown d4/c2d 193 1 2024-03-11T19:32:48.859 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:58:08.555 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:32:48.867 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:58:08.556 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:32:48.874 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:58:08.563 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:32:48.887 INFO:tasks.workunit.client.0.smithi090.stdout:1/359: mknod d5/da/d1c/d3b/c71 0 2024-03-11T19:32:48.918 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:58:08.606 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:32:48.932 INFO:tasks.workunit.client.0.smithi090.stdout:4/415: read - d4/d18/f3c zero size 2024-03-11T19:32:48.935 INFO:tasks.workunit.client.0.smithi090.stdout:9/371: dwrite d1/d1d/f2b [0,4194304] 0 2024-03-11T19:32:49.032 INFO:tasks.workunit.client.0.smithi090.stdout:3/352: fsync d2/d1f/f3b 0 2024-03-11T19:32:49.052 INFO:tasks.workunit.client.0.smithi090.stdout:0/398: chown d2/d11/c72 1 1 2024-03-11T19:32:49.058 INFO:tasks.workunit.client.0.smithi090.stdout:2/371: creat d0/d21/d30/d25/f67 x:0 0 0 2024-03-11T19:32:49.114 INFO:tasks.workunit.client.0.smithi090.stdout:5/379: dread d1/d26/f71 [0,4194304] 0 2024-03-11T19:32:49.120 INFO:tasks.workunit.client.0.smithi090.stdout:4/416: chown d4/d27/d44/f7e 1 1 2024-03-11T19:32:49.180 INFO:tasks.workunit.client.0.smithi090.stdout:8/417: stat d0/d53/f6e 0 2024-03-11T19:32:49.344 INFO:tasks.workunit.client.0.smithi090.stdout:0/399: sync 2024-03-11T19:32:49.353 INFO:tasks.workunit.client.0.smithi090.stdout:6/425: link d4/d13/d22/d3a/c4f d4/d13/d22/d34/c8b 0 2024-03-11T19:32:49.409 INFO:tasks.workunit.client.0.smithi090.stdout:6/426: chown d4/d13/d22/d3a/c51 71488531 1 2024-03-11T19:32:49.636 INFO:tasks.workunit.client.0.smithi090.stdout:9/372: read d1/dd/df/f10 [1876133,76225] 0 2024-03-11T19:32:49.751 INFO:tasks.workunit.client.0.smithi090.stdout:3/353: mknod d2/db/d15/d1a/d3c/c79 0 2024-03-11T19:32:50.034 INFO:tasks.workunit.client.0.smithi090.stdout:4/417: creat d4/d51/f92 x:0 0 0 2024-03-11T19:32:50.388 INFO:tasks.workunit.client.0.smithi090.stdout:1/360: dread d5/d65/f69 [0,4194304] 0 2024-03-11T19:32:50.430 INFO:tasks.workunit.client.0.smithi090.stdout:0/400: creat d2/d11/d3a/f80 x:0 0 0 2024-03-11T19:32:50.452 INFO:tasks.workunit.client.0.smithi090.stdout:0/401: chown d2/f14 1371327069 1 2024-03-11T19:32:50.494 INFO:tasks.workunit.client.0.smithi090.stdout:6/427: unlink d4/d13/f7e 0 2024-03-11T19:32:50.507 INFO:tasks.workunit.client.0.smithi090.stdout:1/361: sync 2024-03-11T19:32:50.592 INFO:tasks.workunit.client.0.smithi090.stdout:2/372: fdatasync d0/fc 0 2024-03-11T19:32:50.641 INFO:tasks.workunit.client.0.smithi090.stdout:9/373: creat d1/d1d/f72 x:0 0 0 2024-03-11T19:32:51.032 INFO:tasks.workunit.client.0.smithi090.stdout:7/381: dwrite d3/f59 [0,4194304] 0 2024-03-11T19:32:51.059 INFO:tasks.workunit.client.0.smithi090.stdout:4/418: creat d4/d27/d28/d38/d64/f93 x:0 0 0 2024-03-11T19:32:51.301 INFO:tasks.workunit.client.0.smithi090.stdout:7/382: dread d3/dc/f33 [0,4194304] 0 2024-03-11T19:32:51.349 INFO:tasks.workunit.client.0.smithi090.stdout:7/383: chown d3/dc/d1e/ff 13820 1 2024-03-11T19:32:51.638 INFO:tasks.workunit.client.0.smithi090.stdout:6/428: unlink d4/d13/d22/d34/d40/l81 0 2024-03-11T19:32:51.733 INFO:tasks.workunit.client.0.smithi090.stdout:5/380: dwrite d1/d26/f54 [4194304,4194304] 0 2024-03-11T19:32:52.842 INFO:tasks.workunit.client.0.smithi090.stdout:0/402: dwrite d2/d11/d15/d29/d2e/d6d/f73 [0,4194304] 0 2024-03-11T19:32:53.247 INFO:tasks.workunit.client.0.smithi090.stdout:1/362: dwrite d5/d4e/f5e [0,4194304] 0 2024-03-11T19:32:53.375 INFO:tasks.workunit.client.0.smithi090.stdout:4/419: truncate d4/d51/d8c/d6a/f7f 538157 0 2024-03-11T19:32:53.658 INFO:tasks.workunit.client.0.smithi090.stdout:3/354: dwrite d2/f9 [0,4194304] 0 2024-03-11T19:32:53.845 INFO:tasks.workunit.client.0.smithi090.stdout:9/374: write d1/dd/df/d1c/d21/d2a/f50 [369139,55257] 0 2024-03-11T19:32:53.845 INFO:tasks.workunit.client.0.smithi090.stdout:6/429: creat d4/d13/d22/d34/d40/f8c x:0 0 0 2024-03-11T19:32:54.109 INFO:tasks.workunit.client.0.smithi090.stdout:5/381: dread d1/f4f [0,4194304] 0 2024-03-11T19:32:54.701 INFO:tasks.workunit.client.0.smithi090.stdout:0/403: symlink d2/d11/d15/d29/d6b/l81 0 2024-03-11T19:32:54.890 INFO:tasks.workunit.client.0.smithi090.stdout:1/363: rename d5/da/dd/f1a to d5/da/dd/f72 0 2024-03-11T19:32:54.899 INFO:tasks.workunit.client.0.smithi090.stdout:0/404: sync 2024-03-11T19:32:54.990 INFO:tasks.workunit.client.0.smithi090.stdout:7/384: dwrite d3/dc/d17/d32/f7a [0,4194304] 0 2024-03-11T19:32:55.150 INFO:tasks.workunit.client.0.smithi090.stdout:4/420: creat d4/d27/d44/f94 x:0 0 0 2024-03-11T19:32:55.175 INFO:tasks.workunit.client.0.smithi090.stdout:3/355: creat d2/d45/f7a x:0 0 0 2024-03-11T19:32:55.276 INFO:tasks.workunit.client.0.smithi090.stdout:9/375: fdatasync d1/f3 0 2024-03-11T19:32:55.489 INFO:tasks.workunit.client.0.smithi090.stdout:2/373: write d0/f4e [210184,101734] 0 2024-03-11T19:32:55.799 INFO:tasks.workunit.client.0.smithi090.stdout:8/418: getdents d0/d3/d3e 0 2024-03-11T19:32:55.825 INFO:tasks.workunit.client.0.smithi090.stdout:8/419: readlink d0/d43/l33 0 2024-03-11T19:32:56.665 INFO:tasks.workunit.client.0.smithi090.stdout:1/364: creat d5/d31/d4c/f73 x:0 0 0 2024-03-11T19:32:56.954 INFO:tasks.workunit.client.0.smithi090.stdout:3/356: truncate d2/db/d15/f48 380811 0 2024-03-11T19:32:57.042 INFO:tasks.workunit.client.0.smithi090.stdout:4/421: dread d4/f4a [0,4194304] 0 2024-03-11T19:32:57.072 INFO:tasks.workunit.client.0.smithi090.stdout:4/422: readlink d4/d18/d6f/l73 0 2024-03-11T19:32:57.206 INFO:tasks.workunit.client.0.smithi090.stdout:7/385: truncate d3/dc/d17/d32/d71/d5d/f18 4626990 0 2024-03-11T19:32:57.781 INFO:tasks.workunit.client.0.smithi090.stdout:6/430: mkdir d4/d13/d22/d34/d83/d8d 0 2024-03-11T19:32:57.787 INFO:tasks.workunit.client.0.smithi090.stdout:9/376: creat d1/dd/d18/d3a/d48/f73 x:0 0 0 2024-03-11T19:32:57.812 INFO:tasks.workunit.client.0.smithi090.stdout:0/405: write d2/d11/d3a/f75 [989272,19967] 0 2024-03-11T19:32:57.921 INFO:tasks.workunit.client.0.smithi090.stdout:5/382: mkdir d1/d17/d39/d51/d53/d7b/d3b/d63/d80 0 2024-03-11T19:32:57.942 INFO:tasks.workunit.client.0.smithi090.stdout:9/377: sync 2024-03-11T19:32:58.033 INFO:tasks.workunit.client.0.smithi090.stdout:2/374: dread d0/fd [4194304,4194304] 0 2024-03-11T19:32:58.065 INFO:tasks.workunit.client.0.smithi090.stdout:0/406: rename d2/d11/d15/d29 to d2/d11/d15/d29/d6b/d82 22 2024-03-11T19:32:58.118 INFO:tasks.workunit.client.0.smithi090.stdout:9/378: read d1/dd/df/d1c/d21/d2a/f29 [1264046,54014] 0 2024-03-11T19:32:59.838 INFO:tasks.workunit.client.0.smithi090.stdout:9/379: chown d1/dd/d18/d32/l56 1484 1 2024-03-11T19:33:00.120 INFO:tasks.workunit.client.0.smithi090.stdout:3/357: symlink d2/d2f/l7b 0 2024-03-11T19:33:00.592 INFO:tasks.workunit.client.0.smithi090.stdout:1/365: dwrite d5/da/d23/f2a [0,4194304] 0 2024-03-11T19:33:00.710 INFO:tasks.workunit.client.0.smithi090.stdout:7/386: mkdir d3/dc/d17/d32/d71/d80 0 2024-03-11T19:33:00.821 INFO:tasks.workunit.client.0.smithi090.stdout:8/420: creat d0/d3/f81 x:0 0 0 2024-03-11T19:33:00.917 INFO:tasks.workunit.client.0.smithi090.stdout:5/383: mknod d1/c81 0 2024-03-11T19:33:01.172 INFO:tasks.workunit.client.0.smithi090.stdout:6/431: write d4/d14/d25/f41 [608584,52139] 0 2024-03-11T19:33:01.278 INFO:tasks.workunit.client.0.smithi090.stdout:0/407: write d2/d11/d15/f28 [316300,38361] 0 2024-03-11T19:33:01.325 INFO:tasks.workunit.client.0.smithi090.stdout:0/408: readlink d2/d11/d15/d29/d2e/l30 0 2024-03-11T19:33:01.383 INFO:tasks.workunit.client.0.smithi090.stdout:0/409: write d2/d11/d3a/f75 [1640459,76112] 0 2024-03-11T19:33:01.625 INFO:tasks.workunit.client.0.smithi090.stdout:1/366: mknod d5/d65/c74 0 2024-03-11T19:33:01.654 INFO:tasks.workunit.client.0.smithi090.stdout:8/421: mkdir d0/d53/d59/d82 0 2024-03-11T19:33:01.765 INFO:tasks.workunit.client.0.smithi090.stdout:5/384: creat d1/d17/d39/f82 x:0 0 0 2024-03-11T19:33:02.236 INFO:tasks.workunit.client.0.smithi090.stdout:4/423: link d4/d27/d28/d7a/c80 d4/d27/d44/d66/c95 0 2024-03-11T19:33:02.464 INFO:tasks.workunit.client.0.smithi090.stdout:2/375: creat d0/de/f68 x:0 0 0 2024-03-11T19:33:02.716 INFO:tasks.workunit.client.0.smithi090.stdout:7/387: fsync d3/d55/f6a 0 2024-03-11T19:33:02.719 INFO:tasks.workunit.client.0.smithi090.stdout:1/367: mknod d5/d3c/c75 0 2024-03-11T19:33:02.814 INFO:tasks.workunit.client.0.smithi090.stdout:8/422: read - d0/d3/f80 zero size 2024-03-11T19:33:03.026 INFO:tasks.workunit.client.0.smithi090.stdout:3/358: rename d2/d1d/l7 to d2/d1d/l7c 0 2024-03-11T19:33:03.220 INFO:tasks.workunit.client.0.smithi090.stdout:6/432: truncate d4/d24/f5f 854842 0 2024-03-11T19:33:03.967 INFO:tasks.workunit.client.0.smithi090.stdout:1/368: rename d5/d3c/d45 to d5/d31/d76 0 2024-03-11T19:33:04.022 INFO:tasks.workunit.client.0.smithi090.stdout:8/423: symlink d0/d60/l83 0 2024-03-11T19:33:04.068 INFO:tasks.workunit.client.0.smithi090.stdout:7/388: dread d3/dc/f3a [0,4194304] 0 2024-03-11T19:33:04.143 INFO:tasks.workunit.client.0.smithi090.stdout:8/424: chown d0/d3/d4c/l63 88660 1 2024-03-11T19:33:04.192 INFO:tasks.workunit.client.0.smithi090.stdout:6/433: dread - d4/d14/d8a/d74/f7b zero size 2024-03-11T19:33:04.389 INFO:tasks.workunit.client.0.smithi090.stdout:2/376: symlink d0/d21/d30/l69 0 2024-03-11T19:33:04.403 INFO:tasks.workunit.client.0.smithi090.stdout:0/410: write d2/d11/d3a/f70 [572624,43911] 0 2024-03-11T19:33:04.893 INFO:tasks.workunit.client.0.smithi090.stdout:3/359: unlink d2/f46 0 2024-03-11T19:33:04.904 INFO:tasks.workunit.client.0.smithi090.stdout:7/389: unlink d3/dc/d17/d32/d71/l74 0 2024-03-11T19:33:05.117 INFO:tasks.workunit.client.0.smithi090.stdout:8/425: read d0/d43/dc/f12 [790388,20521] 0 2024-03-11T19:33:05.149 INFO:tasks.workunit.client.0.smithi090.stdout:6/434: truncate d4/f11 1610435 0 2024-03-11T19:33:05.462 INFO:tasks.workunit.client.0.smithi090.stdout:0/411: dread d2/d11/d3f/f40 [0,4194304] 0 2024-03-11T19:33:05.596 INFO:tasks.workunit.client.0.smithi090.stdout:5/385: getdents d1/d26 0 2024-03-11T19:33:05.896 INFO:tasks.workunit.client.0.smithi090.stdout:4/424: dwrite d4/d27/f3b [0,4194304] 0 2024-03-11T19:33:06.020 INFO:tasks.workunit.client.0.smithi090.stdout:8/426: fsync d0/d3/d3e/f4e 0 2024-03-11T19:33:06.054 INFO:tasks.workunit.client.0.smithi090.stdout:6/435: fsync d4/d13/d22/d42/f72 0 2024-03-11T19:33:06.073 INFO:tasks.workunit.client.0.smithi090.stdout:4/425: readlink d4/d27/d28/l32 0 2024-03-11T19:33:06.557 INFO:tasks.workunit.client.0.smithi090.stdout:5/386: read d1/f1e [2377294,98237] 0 2024-03-11T19:33:06.601 INFO:tasks.workunit.client.0.smithi090.stdout:7/390: getdents d3/dc/d17/d32/d71/d80 0 2024-03-11T19:33:06.632 INFO:tasks.workunit.client.0.smithi090.stdout:7/391: readlink d3/dc/d29/l7e 0 2024-03-11T19:33:06.832 INFO:tasks.workunit.client.0.smithi090.stdout:1/369: write d5/f15 [2250453,102799] 0 2024-03-11T19:33:06.910 INFO:tasks.workunit.client.0.smithi090.stdout:9/380: mknod d1/c74 0 2024-03-11T19:33:06.976 INFO:tasks.workunit.client.0.smithi090.stdout:8/427: dread d0/f58 [0,4194304] 0 2024-03-11T19:33:07.476 INFO:tasks.workunit.client.0.smithi090.stdout:6/436: dread d4/fb [0,4194304] 0 2024-03-11T19:33:07.489 INFO:tasks.workunit.client.0.smithi090.stdout:4/426: rmdir d4/d18/d6f 39 2024-03-11T19:33:08.450 INFO:tasks.workunit.client.0.smithi090.stdout:9/381: unlink d1/dd/d18/f71 0 2024-03-11T19:33:08.676 INFO:tasks.workunit.client.0.smithi090.stdout:0/412: dwrite d2/d11/f1a [0,4194304] 0 2024-03-11T19:33:08.821 INFO:tasks.workunit.client.0.smithi090.stdout:2/377: getdents d0/de/d12/d46/d4c 0 2024-03-11T19:33:09.100 INFO:tasks.workunit.client.0.smithi090.stdout:3/360: link d2/l2a d2/d4f/l7d 0 2024-03-11T19:33:09.561 INFO:tasks.workunit.client.0.smithi090.stdout:7/392: rename d3/dc/c76 to d3/dc/d17/d32/c81 0 2024-03-11T19:33:09.782 INFO:tasks.workunit.client.0.smithi090.stdout:2/378: readlink d0/l8 0 2024-03-11T19:33:09.843 INFO:tasks.workunit.client.0.smithi090.stdout:0/413: rmdir d2/d11/d15/d29/d6b 39 2024-03-11T19:33:09.857 INFO:tasks.workunit.client.0.smithi090.stdout:0/414: chown f0 71399737 1 2024-03-11T19:33:09.894 INFO:tasks.workunit.client.0.smithi090.stdout:0/415: chown d2/d11/d15/d29/d43/c4e 1 1 2024-03-11T19:33:10.011 INFO:tasks.workunit.client.0.smithi090.stdout:4/427: truncate d4/d27/d44/f65 347034 0 2024-03-11T19:33:10.488 INFO:tasks.workunit.client.0.smithi090.stdout:8/428: creat d0/d43/f84 x:0 0 0 2024-03-11T19:33:10.953 INFO:tasks.workunit.client.0.smithi090.stdout:5/387: getdents d1/d75 0 2024-03-11T19:33:11.058 INFO:tasks.workunit.client.0.smithi090.stdout:4/428: read d4/d18/f41 [1836817,9469] 0 2024-03-11T19:33:11.171 INFO:tasks.workunit.client.0.smithi090.stdout:8/429: truncate d0/d43/f61 390415 0 2024-03-11T19:33:11.272 INFO:tasks.workunit.client.0.smithi090.stdout:1/370: write d5/da/d23/f34 [4490537,69508] 0 2024-03-11T19:33:11.535 INFO:tasks.workunit.client.0.smithi090.stdout:2/379: mknod d0/de/d12/d46/d5d/d63/c6a 0 2024-03-11T19:33:11.554 INFO:tasks.workunit.client.0.smithi090.stdout:9/382: write d1/d1d/f67 [1055793,16712] 0 2024-03-11T19:33:11.688 INFO:tasks.workunit.client.0.smithi090.stdout:5/388: dread - d1/d17/d39/d51/d53/d7b/d3b/d76/f77 zero size 2024-03-11T19:33:12.015 INFO:tasks.workunit.client.0.smithi090.stdout:8/430: mkdir d0/d3/d3e/d85 0 2024-03-11T19:33:12.031 INFO:tasks.workunit.client.0.smithi090.stdout:3/361: write d2/d1d/fc [641042,117148] 0 2024-03-11T19:33:12.043 INFO:tasks.workunit.client.0.smithi090.stdout:4/429: dread - d4/d51/f72 zero size 2024-03-11T19:33:12.314 INFO:tasks.workunit.client.0.smithi090.stdout:1/371: mkdir d5/d31/d76/d77 0 2024-03-11T19:33:12.344 INFO:tasks.workunit.client.0.smithi090.stdout:3/362: sync 2024-03-11T19:33:12.701 INFO:tasks.workunit.client.0.smithi090.stdout:1/372: dread d5/da/f68 [0,4194304] 0 2024-03-11T19:33:12.706 INFO:tasks.workunit.client.0.smithi090.stdout:5/389: truncate d1/f50 258966 0 2024-03-11T19:33:12.706 INFO:tasks.workunit.client.0.smithi090.stdout:9/383: creat d1/dd/df/d1c/d21/d2a/f75 x:0 0 0 2024-03-11T19:33:12.857 INFO:tasks.workunit.client.0.smithi090.stdout:8/431: truncate d0/d43/dc/f12 3883199 0 2024-03-11T19:33:13.082 INFO:tasks.workunit.client.0.smithi090.stdout:0/416: dwrite d2/d11/f18 [0,4194304] 0 2024-03-11T19:33:13.194 INFO:tasks.workunit.client.0.smithi090.stdout:7/393: link d3/d49/c73 d3/d55/c82 0 2024-03-11T19:33:13.530 INFO:tasks.workunit.client.0.smithi090.stdout:0/417: dread d2/f1b [0,4194304] 0 2024-03-11T19:33:13.848 INFO:tasks.workunit.client.0.smithi090.stdout:8/432: truncate d0/f65 707798 0 2024-03-11T19:33:14.101 INFO:tasks.workunit.client.0.smithi090.stdout:2/380: write d0/d21/d30/f22 [4381014,125551] 0 2024-03-11T19:33:14.189 INFO:tasks.workunit.client.0.smithi090.stdout:2/381: read d0/de/d1f/f1c [5458928,87767] 0 2024-03-11T19:33:14.367 INFO:tasks.workunit.client.0.smithi090.stdout:0/418: truncate d2/d11/d15/d29/d43/d58/f65 1006200 0 2024-03-11T19:33:14.381 INFO:tasks.workunit.client.0.smithi090.stdout:1/373: unlink d5/da/dd/f72 0 2024-03-11T19:33:14.608 INFO:tasks.workunit.client.0.smithi090.stdout:7/394: mknod d3/dc/d17/d32/d71/d80/c83 0 2024-03-11T19:33:15.043 INFO:tasks.workunit.client.0.smithi090.stdout:6/437: rename d4/fb to d4/d13/d22/d34/d37/f8e 0 2024-03-11T19:33:15.093 INFO:tasks.workunit.client.0.smithi090.stdout:1/374: creat d5/d31/d76/f78 x:0 0 0 2024-03-11T19:33:15.212 INFO:tasks.workunit.client.0.smithi090.stdout:3/363: write d2/d1f/f78 [681984,608] 0 2024-03-11T19:33:15.417 INFO:tasks.workunit.client.0.smithi090.stdout:7/395: mkdir d3/dc/d17/d32/d45/d84 0 2024-03-11T19:33:15.451 INFO:tasks.workunit.client.0.smithi090.stdout:7/396: chown d3/dc/d1e/f26 972783 1 2024-03-11T19:33:15.643 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:33:15.646 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:33:15.987 INFO:tasks.workunit.client.0.smithi090.stdout:9/384: rename d1/dd/d18/f31 to d1/d49/f76 0 2024-03-11T19:33:16.147 INFO:tasks.workunit.client.0.smithi090.stdout:6/438: mknod d4/d13/d22/d34/d83/c8f 0 2024-03-11T19:33:16.729 INFO:tasks.workunit.client.0.smithi090.stdout:2/382: creat d0/de/d12/d46/f6b x:0 0 0 2024-03-11T19:33:16.730 INFO:tasks.workunit.client.0.smithi090.stdout:3/364: dread - d2/db/d15/d1a/d3c/f52 zero size 2024-03-11T19:33:16.730 INFO:tasks.workunit.client.0.smithi090.stdout:8/433: truncate d0/f65 765449 0 2024-03-11T19:33:16.756 INFO:tasks.workunit.client.0.smithi090.stdout:1/375: fsync d5/d31/d76/f78 0 2024-03-11T19:33:16.774 INFO:tasks.workunit.client.0.smithi090.stdout:5/390: write d1/d17/f30 [240182,130919] 0 2024-03-11T19:33:16.813 INFO:tasks.workunit.client.0.smithi090.stdout:9/385: unlink d1/d1d/f66 0 2024-03-11T19:33:16.966 INFO:tasks.workunit.client.0.smithi090.stdout:0/419: dwrite d2/d11/d15/d29/d2e/f52 [0,4194304] 0 2024-03-11T19:33:17.367 INFO:tasks.workunit.client.0.smithi090.stdout:4/430: symlink d4/d27/l96 0 2024-03-11T19:33:17.589 INFO:tasks.workunit.client.0.smithi090.stdout:3/365: fdatasync d2/db/f1c 0 2024-03-11T19:33:17.699 INFO:tasks.workunit.client.0.smithi090.stdout:1/376: mkdir d5/d4e/d79 0 2024-03-11T19:33:17.758 INFO:tasks.workunit.client.0.smithi090.stdout:1/377: dread d5/da/d1c/f28 [0,4194304] 0 2024-03-11T19:33:17.817 INFO:tasks.workunit.client.0.smithi090.stdout:5/391: creat d1/d26/f83 x:0 0 0 2024-03-11T19:33:17.849 INFO:tasks.workunit.client.0.smithi090.stdout:5/392: read - d1/d26/f83 zero size 2024-03-11T19:33:18.014 INFO:tasks.workunit.client.0.smithi090.stdout:2/383: creat d0/de/d12/d46/d49/f6c x:0 0 0 2024-03-11T19:33:18.263 INFO:tasks.workunit.client.0.smithi090.stdout:9/386: unlink d1/dd/d18/f52 0 2024-03-11T19:33:18.288 INFO:tasks.workunit.client.0.smithi090.stdout:6/439: mkdir d4/d13/d22/d34/d37/d7f/d90 0 2024-03-11T19:33:18.293 INFO:tasks.workunit.client.0.smithi090.stdout:7/397: write d3/dc/d17/d32/f7a [1099466,31649] 0 2024-03-11T19:33:18.343 INFO:tasks.workunit.client.0.smithi090.stdout:6/440: stat d4/c16 0 2024-03-11T19:33:18.365 INFO:tasks.workunit.client.0.smithi090.stdout:9/387: stat d1/dd/d18/d25/d4b 0 2024-03-11T19:33:18.448 INFO:tasks.workunit.client.0.smithi090.stdout:0/420: creat d2/d11/d3f/f83 x:0 0 0 2024-03-11T19:33:18.642 INFO:tasks.workunit.client.0.smithi090.stdout:4/431: dread d4/d27/d28/f30 [0,4194304] 0 2024-03-11T19:33:18.973 INFO:tasks.workunit.client.0.smithi090.stdout:3/366: creat d2/db/f7e x:0 0 0 2024-03-11T19:33:19.463 INFO:tasks.workunit.client.0.smithi090.stdout:8/434: mkdir d0/d3/d86 0 2024-03-11T19:33:19.468 INFO:tasks.workunit.client.0.smithi090.stdout:2/384: rmdir d0/d21/d30/d25 39 2024-03-11T19:33:19.496 INFO:tasks.workunit.client.0.smithi090.stdout:6/441: symlink d4/d13/d22/d3a/l91 0 2024-03-11T19:33:19.515 INFO:tasks.workunit.client.0.smithi090.stdout:9/388: mkdir d1/dd/df/d1c/d77 0 2024-03-11T19:33:19.598 INFO:tasks.workunit.client.0.smithi090.stdout:8/435: stat d0/d43/c4b 0 2024-03-11T19:33:19.673 INFO:tasks.workunit.client.0.smithi090.stdout:8/436: chown d0/d43/dc/l41 9945681 1 2024-03-11T19:33:19.904 INFO:tasks.workunit.client.0.smithi090.stdout:1/378: symlink d5/da/d23/l7a 0 2024-03-11T19:33:20.090 INFO:tasks.workunit.client.0.smithi090.stdout:3/367: truncate d2/d1f/f35 1360669 0 2024-03-11T19:33:20.327 INFO:tasks.workunit.client.0.smithi090.stdout:9/389: dread d1/dd/f30 [0,4194304] 0 2024-03-11T19:33:20.487 INFO:tasks.workunit.client.0.smithi090.stdout:2/385: creat d0/d21/d30/d29/f6d x:0 0 0 2024-03-11T19:33:20.792 INFO:tasks.workunit.client.0.smithi090.stdout:8/437: unlink d0/d3/f6c 0 2024-03-11T19:33:21.018 INFO:tasks.workunit.client.0.smithi090.stdout:1/379: mknod d5/d65/d6b/c7b 0 2024-03-11T19:33:21.042 INFO:tasks.workunit.client.0.smithi090.stdout:1/380: chown d5/d31/d4c 1610 1 2024-03-11T19:33:21.359 INFO:tasks.workunit.client.0.smithi090.stdout:3/368: truncate d2/db/d15/f17 135153 0 2024-03-11T19:33:21.367 INFO:tasks.workunit.client.0.smithi090.stdout:9/390: truncate d1/dd/df/d1c/d21/d2a/f29 1492817 0 2024-03-11T19:33:21.723 INFO:tasks.workunit.client.0.smithi090.stdout:8/438: fsync d0/d3/d4c/f69 0 2024-03-11T19:33:21.957 INFO:tasks.workunit.client.0.smithi090.stdout:1/381: dread d5/f60 [0,4194304] 0 2024-03-11T19:33:22.052 INFO:tasks.workunit.client.0.smithi090.stdout:0/421: symlink d2/l84 0 2024-03-11T19:33:22.245 INFO:tasks.workunit.client.0.smithi090.stdout:3/369: rmdir d2/db/d15/d69 39 2024-03-11T19:33:22.558 INFO:tasks.workunit.client.0.smithi090.stdout:7/398: write d3/dc/d17/d32/d71/f78 [388221,82793] 0 2024-03-11T19:33:22.636 INFO:tasks.workunit.client.0.smithi090.stdout:5/393: dwrite d1/d26/f43 [0,4194304] 0 2024-03-11T19:33:22.828 INFO:tasks.workunit.client.0.smithi090.stdout:6/442: getdents d4/d13/d22/d34/d40 0 2024-03-11T19:33:22.918 INFO:tasks.workunit.client.0.smithi090.stdout:5/394: dread d1/d17/f79 [0,4194304] 0 2024-03-11T19:33:22.943 INFO:tasks.workunit.client.0.smithi090.stdout:8/439: rename d0/d3/d3e/c5f to d0/d43/dc/c87 0 2024-03-11T19:33:23.493 INFO:tasks.workunit.client.0.smithi090.stdout:4/432: dwrite d4/d27/d28/f8b [0,4194304] 0 2024-03-11T19:33:23.614 INFO:tasks.workunit.client.0.smithi090.stdout:7/399: creat d3/d55/f85 x:0 0 0 2024-03-11T19:33:24.105 INFO:tasks.workunit.client.0.smithi090.stdout:5/395: creat d1/d17/d39/d51/d53/d64/f84 x:0 0 0 2024-03-11T19:33:24.193 INFO:tasks.workunit.client.0.smithi090.stdout:1/382: mknod d5/da/d23/c7c 0 2024-03-11T19:33:24.532 INFO:tasks.workunit.client.0.smithi090.stdout:0/422: rename d2/d11/d15/f60 to d2/d11/d15/d29/d6b/d78/f85 0 2024-03-11T19:33:24.872 INFO:tasks.workunit.client.0.smithi090.stdout:4/433: truncate d4/d27/d44/f4e 3310014 0 2024-03-11T19:33:25.145 INFO:tasks.workunit.client.0.smithi090.stdout:2/386: dwrite d0/de/d12/d46/d4c/f65 [0,4194304] 0 2024-03-11T19:33:25.362 INFO:tasks.workunit.client.0.smithi090.stdout:6/443: truncate d4/d13/d22/d34/d37/f8e 1946771 0 2024-03-11T19:33:25.368 INFO:tasks.workunit.client.0.smithi090.stdout:9/391: write d1/dd/f30 [1155796,128621] 0 2024-03-11T19:33:25.424 INFO:tasks.workunit.client.0.smithi090.stdout:9/392: chown d1/f3 1987027282 1 2024-03-11T19:33:25.645 INFO:tasks.workunit.client.0.smithi090.stdout:5/396: creat d1/d17/d39/d51/d53/d64/f85 x:0 0 0 2024-03-11T19:33:25.835 INFO:tasks.workunit.client.0.smithi090.stdout:1/383: symlink d5/d3c/l7d 0 2024-03-11T19:33:26.123 INFO:tasks.workunit.client.0.smithi090.stdout:3/370: creat d2/db/d15/d49/f7f x:0 0 0 2024-03-11T19:33:26.163 INFO:tasks.workunit.client.0.smithi090.stdout:3/371: read d2/d2f/f30 [206647,6611] 0 2024-03-11T19:33:26.190 INFO:tasks.workunit.client.0.smithi090.stdout:3/372: chown d2/d1d/f27 7412130 1 2024-03-11T19:33:26.244 INFO:tasks.workunit.client.0.smithi090.stdout:3/373: write d2/db/d15/d49/f7f [193858,43669] 0 2024-03-11T19:33:26.678 INFO:tasks.workunit.client.0.smithi090.stdout:4/434: mknod d4/d27/d28/d38/c97 0 2024-03-11T19:33:26.891 INFO:tasks.workunit.client.0.smithi090.stdout:6/444: fsync d4/d14/d8a/f63 0 2024-03-11T19:33:26.933 INFO:tasks.workunit.client.0.smithi090.stdout:9/393: fdatasync d1/dd/d18/d3a/d48/f73 0 2024-03-11T19:33:27.602 INFO:tasks.workunit.client.0.smithi090.stdout:3/374: read - d2/d45/f7a zero size 2024-03-11T19:33:28.181 INFO:tasks.workunit.client.0.smithi090.stdout:9/394: fdatasync d1/d1d/f2b 0 2024-03-11T19:33:28.525 INFO:tasks.workunit.client.0.smithi090.stdout:8/440: link d0/f65 d0/d3/d4c/f88 0 2024-03-11T19:33:28.658 INFO:tasks.workunit.client.0.smithi090.stdout:5/397: symlink d1/d17/d39/d51/d53/d7b/d3b/d63/d80/l86 0 2024-03-11T19:33:28.800 INFO:tasks.workunit.client.0.smithi090.stdout:1/384: rename d5/da/d23/l6a to d5/d31/d4c/l7e 0 2024-03-11T19:33:28.814 INFO:tasks.workunit.client.0.smithi090.stdout:1/385: chown d5/d4e/d54 115148418 1 2024-03-11T19:33:28.865 INFO:tasks.workunit.client.0.smithi090.stdout:7/400: getdents d3/dc/d17/d32/d45 0 2024-03-11T19:33:29.897 INFO:tasks.workunit.client.0.smithi090.stdout:0/423: rmdir d2/d11/d15/d29/d2e 39 2024-03-11T19:33:30.066 INFO:tasks.workunit.client.0.smithi090.stdout:8/441: symlink d0/d3/d3e/l89 0 2024-03-11T19:33:30.517 INFO:tasks.workunit.client.0.smithi090.stdout:1/386: dread d5/f1b [0,4194304] 0 2024-03-11T19:33:31.367 INFO:tasks.workunit.client.0.smithi090.stdout:0/424: rmdir d2/d11/d15/d29/d2e/d6d 39 2024-03-11T19:33:31.663 INFO:tasks.workunit.client.0.smithi090.stdout:2/387: dwrite d0/de/d1f/f55 [4194304,4194304] 0 2024-03-11T19:33:31.778 INFO:tasks.workunit.client.0.smithi090.stdout:4/435: write d4/d18/f90 [911794,91533] 0 2024-03-11T19:33:31.821 INFO:tasks.workunit.client.0.smithi090.stdout:2/388: stat d0/de/d12/d46/d4c/c4d 0 2024-03-11T19:33:32.114 INFO:tasks.workunit.client.0.smithi090.stdout:5/398: getdents d1/d17 0 2024-03-11T19:33:32.354 INFO:tasks.workunit.client.0.smithi090.stdout:4/436: unlink d4/d18/c5e 0 2024-03-11T19:33:32.482 INFO:tasks.workunit.client.0.smithi090.stdout:2/389: rmdir d0/de 39 2024-03-11T19:33:32.792 INFO:tasks.workunit.client.0.smithi090.stdout:5/399: truncate d1/d26/f3e 682410 0 2024-03-11T19:33:33.458 INFO:tasks.workunit.client.0.smithi090.stdout:2/390: rmdir d0/de/d12/d46 39 2024-03-11T19:33:33.498 INFO:tasks.workunit.client.0.smithi090.stdout:3/375: dwrite d2/db/d15/d1a/d3c/f52 [0,4194304] 0 2024-03-11T19:33:33.969 INFO:tasks.workunit.client.0.smithi090.stdout:7/401: write d3/dc/d17/d32/d45/f5b [981136,42482] 0 2024-03-11T19:33:33.999 INFO:tasks.workunit.client.0.smithi090.stdout:1/387: write d5/f37 [1528304,59336] 0 2024-03-11T19:33:34.061 INFO:tasks.workunit.client.0.smithi090.stdout:8/442: write d0/d43/dc/f75 [111819,6541] 0 2024-03-11T19:33:34.130 INFO:tasks.workunit.client.0.smithi090.stdout:6/445: dwrite d4/f46 [0,4194304] 0 2024-03-11T19:33:34.398 INFO:tasks.workunit.client.0.smithi090.stdout:0/425: write d2/d11/d3f/f83 [222819,48582] 0 2024-03-11T19:33:34.668 INFO:tasks.workunit.client.0.smithi090.stdout:9/395: truncate d1/dd/df/d1c/d21/d2a/f29 138119 0 2024-03-11T19:33:34.691 INFO:tasks.workunit.client.0.smithi090.stdout:9/396: chown d1/dd/f69 80146 1 2024-03-11T19:33:34.776 INFO:tasks.workunit.client.0.smithi090.stdout:1/388: readlink d5/d31/d4c/l7e 0 2024-03-11T19:33:34.959 INFO:tasks.workunit.client.0.smithi090.stdout:8/443: symlink d0/d43/dc/l8a 0 2024-03-11T19:33:35.230 INFO:tasks.workunit.client.0.smithi090.stdout:2/391: rename d0/fb to d0/d21/f6e 0 2024-03-11T19:33:35.342 INFO:tasks.workunit.client.0.smithi090.stdout:5/400: write d1/d17/d39/d51/d53/d7b/f5b [401417,23211] 0 2024-03-11T19:33:35.477 INFO:tasks.workunit.client.0.smithi090.stdout:4/437: dwrite d4/d27/d28/d38/d64/f93 [0,4194304] 0 2024-03-11T19:33:35.489 INFO:tasks.workunit.client.0.smithi090.stdout:5/401: stat d1/l12 0 2024-03-11T19:33:35.908 INFO:tasks.workunit.client.0.smithi090.stdout:2/392: creat d0/d21/f6f x:0 0 0 2024-03-11T19:33:36.227 INFO:tasks.workunit.client.0.smithi090.stdout:3/376: rmdir d2/db 39 2024-03-11T19:33:36.459 INFO:tasks.workunit.client.0.smithi090.stdout:4/438: creat d4/d51/d8c/d89/f98 x:0 0 0 2024-03-11T19:33:36.618 INFO:tasks.workunit.client.0.smithi090.stdout:5/402: dread d1/f32 [0,4194304] 0 2024-03-11T19:33:36.620 INFO:tasks.workunit.client.0.smithi090.stdout:8/444: symlink d0/d3/d86/l8b 0 2024-03-11T19:33:36.651 INFO:tasks.workunit.client.0.smithi090.stdout:7/402: link d3/dc/d1e/f4f d3/dc/d17/d32/d7b/f86 0 2024-03-11T19:33:36.802 INFO:tasks.workunit.client.0.smithi090.stdout:2/393: truncate d0/d21/f42 992679 0 2024-03-11T19:33:36.827 INFO:tasks.workunit.client.0.smithi090.stdout:2/394: readlink d0/de/d1f/l2d 0 2024-03-11T19:33:36.848 INFO:tasks.workunit.client.0.smithi090.stdout:2/395: readlink d0/de/d12/l19 0 2024-03-11T19:33:37.149 INFO:tasks.workunit.client.0.smithi090.stdout:4/439: dread - d4/d51/d8c/d6a/d8d/f8e zero size 2024-03-11T19:33:37.392 INFO:tasks.workunit.client.0.smithi090.stdout:4/440: dread d4/d27/d28/d38/d64/f93 [0,4194304] 0 2024-03-11T19:33:37.474 INFO:tasks.workunit.client.0.smithi090.stdout:8/445: rename d0/d3/d3e/c52 to d0/d53/d59/d82/c8c 0 2024-03-11T19:33:37.606 INFO:tasks.workunit.client.0.smithi090.stdout:6/446: dwrite d4/d14/d8a/f63 [0,4194304] 0 2024-03-11T19:33:37.763 INFO:tasks.workunit.client.0.smithi090.stdout:6/447: readlink d4/d13/d22/d42/l55 0 2024-03-11T19:33:37.784 INFO:tasks.workunit.client.0.smithi090.stdout:6/448: stat d4/d13/d22/d3a/d6f 0 2024-03-11T19:33:37.930 INFO:tasks.workunit.client.0.smithi090.stdout:0/426: write d2/d11/d15/d29/f68 [511987,104685] 0 2024-03-11T19:33:38.261 INFO:tasks.workunit.client.0.smithi090.stdout:9/397: dwrite d1/dd/df/d1c/d21/d2a/f50 [0,4194304] 0 2024-03-11T19:33:38.465 INFO:tasks.workunit.client.0.smithi090.stdout:5/403: mknod d1/d17/d39/d51/c87 0 2024-03-11T19:33:38.699 INFO:tasks.workunit.client.0.smithi090.stdout:4/441: creat d4/d27/d28/d69/f99 x:0 0 0 2024-03-11T19:33:38.829 INFO:tasks.workunit.client.0.smithi090.stdout:8/446: fsync d0/d43/dc/f50 0 2024-03-11T19:33:38.953 INFO:tasks.workunit.client.0.smithi090.stdout:3/377: creat d2/db/d15/d1a/d75/f80 x:0 0 0 2024-03-11T19:33:38.970 INFO:tasks.workunit.client.0.smithi090.stdout:5/404: dread d1/d26/f54 [0,4194304] 0 2024-03-11T19:33:39.043 INFO:tasks.workunit.client.0.smithi090.stdout:6/449: mkdir d4/d14/d92 0 2024-03-11T19:33:39.243 INFO:tasks.workunit.client.0.smithi090.stdout:1/389: dwrite d5/d4e/d54/f6e [0,4194304] 0 2024-03-11T19:33:39.680 INFO:tasks.workunit.client.0.smithi090.stdout:4/442: symlink d4/d27/d28/d69/l9a 0 2024-03-11T19:33:39.726 INFO:tasks.workunit.client.0.smithi090.stdout:8/447: mkdir d0/d3/d4c/d8d 0 2024-03-11T19:33:40.330 INFO:tasks.workunit.client.0.smithi090.stdout:2/396: write d0/f4e [1061298,8684] 0 2024-03-11T19:33:40.395 INFO:tasks.workunit.client.0.smithi090.stdout:3/378: dread - d2/db/d15/d1a/d3c/f76 zero size 2024-03-11T19:33:40.565 INFO:tasks.workunit.client.0.smithi090.stdout:9/398: fsync d1/d49/f76 0 2024-03-11T19:33:40.665 INFO:tasks.workunit.client.0.smithi090.stdout:6/450: mknod d4/d13/d22/d42/d48/d4e/c93 0 2024-03-11T19:33:40.782 INFO:tasks.workunit.client.0.smithi090.stdout:1/390: mkdir d5/d65/d6b/d7f 0 2024-03-11T19:33:41.083 INFO:tasks.workunit.client.0.smithi090.stdout:7/403: getdents d3/dc/d17 0 2024-03-11T19:33:41.415 INFO:tasks.workunit.client.0.smithi090.stdout:8/448: creat d0/d43/dc/f8e x:0 0 0 2024-03-11T19:33:41.478 INFO:tasks.workunit.client.0.smithi090.stdout:4/443: dread d4/d27/d44/d74/f84 [0,4194304] 0 2024-03-11T19:33:41.831 INFO:tasks.workunit.client.0.smithi090.stdout:4/444: sync 2024-03-11T19:33:41.859 INFO:tasks.workunit.client.0.smithi090.stdout:4/445: stat d4/d27/d28/d69/l71 0 2024-03-11T19:33:42.417 INFO:tasks.workunit.client.0.smithi090.stdout:0/427: link d2/d11/d15/d29/d2e/l30 d2/l86 0 2024-03-11T19:33:42.456 INFO:tasks.workunit.client.0.smithi090.stdout:8/449: mknod d0/d53/d59/c8f 0 2024-03-11T19:33:42.601 INFO:tasks.workunit.client.0.smithi090.stdout:4/446: mkdir d4/d27/d28/d7a/d9b 0 2024-03-11T19:33:42.632 INFO:tasks.workunit.client.0.smithi090.stdout:4/447: chown d4/d27/d28/c3e 59419042 1 2024-03-11T19:33:43.091 INFO:tasks.workunit.client.0.smithi090.stdout:6/451: fdatasync d4/d13/d22/d34/d37/f8e 0 2024-03-11T19:33:43.416 INFO:tasks.workunit.client.0.smithi090.stdout:1/391: mkdir d5/da/d1c/d6d/d80 0 2024-03-11T19:33:43.676 INFO:tasks.workunit.client.0.smithi090.stdout:0/428: read d2/d11/f41 [824007,68423] 0 2024-03-11T19:33:43.702 INFO:tasks.workunit.client.0.smithi090.stdout:5/405: link d1/d26/d19/l23 d1/l88 0 2024-03-11T19:33:43.856 INFO:tasks.workunit.client.0.smithi090.stdout:2/397: mknod d0/de/d12/d46/c70 0 2024-03-11T19:33:43.917 INFO:tasks.workunit.client.0.smithi090.stdout:8/450: fdatasync d0/d43/dc/f75 0 2024-03-11T19:33:43.948 INFO:tasks.workunit.client.0.smithi090.stdout:8/451: readlink d0/d3/d3e/l89 0 2024-03-11T19:33:44.107 INFO:tasks.workunit.client.0.smithi090.stdout:4/448: unlink d4/d27/d28/f8b 0 2024-03-11T19:33:44.364 INFO:tasks.workunit.client.0.smithi090.stdout:6/452: symlink d4/d13/d22/d42/l94 0 2024-03-11T19:33:44.457 INFO:tasks.workunit.client.0.smithi090.stdout:6/453: dread d4/d13/d22/d34/d37/f8e [0,4194304] 0 2024-03-11T19:33:44.688 INFO:tasks.workunit.client.0.smithi090.stdout:1/392: truncate d5/f37 767709 0 2024-03-11T19:33:44.826 INFO:tasks.workunit.client.0.smithi090.stdout:1/393: dread d5/f1b [4194304,4194304] 0 2024-03-11T19:33:45.048 INFO:tasks.workunit.client.0.smithi090.stdout:5/406: dread - d1/f49 zero size 2024-03-11T19:33:45.271 INFO:tasks.workunit.client.0.smithi090.stdout:2/398: mkdir d0/de/d12/d46/d4c/d71 0 2024-03-11T19:33:45.294 INFO:tasks.workunit.client.0.smithi090.stdout:8/452: truncate d0/d3/f80 511904 0 2024-03-11T19:33:45.405 INFO:tasks.workunit.client.0.smithi090.stdout:2/399: sync 2024-03-11T19:33:45.465 INFO:tasks.workunit.client.0.smithi090.stdout:3/379: write d2/d1d/f27 [4615544,10342] 0 2024-03-11T19:33:45.734 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:33:45.737 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:33:45.868 INFO:tasks.workunit.client.0.smithi090.stdout:1/394: symlink d5/d65/l81 0 2024-03-11T19:33:45.950 INFO:tasks.workunit.client.0.smithi090.stdout:0/429: fdatasync d2/f5 0 2024-03-11T19:33:46.340 INFO:tasks.workunit.client.0.smithi090.stdout:8/453: mkdir d0/d43/dc/d90 0 2024-03-11T19:33:46.755 INFO:tasks.workunit.client.0.smithi090.stdout:2/400: creat d0/de/d12/d46/d5d/d63/f72 x:0 0 0 2024-03-11T19:33:46.779 INFO:tasks.workunit.client.0.smithi090.stdout:3/380: rename d2/d45/c51 to d2/d2f/c81 0 2024-03-11T19:33:46.785 INFO:tasks.workunit.client.0.smithi090.stdout:9/399: write d1/dd/d18/d3a/d57/f5b [121849,7846] 0 2024-03-11T19:33:46.866 INFO:tasks.workunit.client.0.smithi090.stdout:9/400: chown d1/dd/f3f 2 1 2024-03-11T19:33:46.870 INFO:tasks.workunit.client.0.smithi090.stdout:3/381: chown d2/db/d15/f37 585320 1 2024-03-11T19:33:47.433 INFO:tasks.workunit.client.0.smithi090.stdout:3/382: symlink d2/db/d15/d49/l82 0 2024-03-11T19:33:47.441 INFO:tasks.workunit.client.0.smithi090.stdout:2/401: dread - d0/de/f68 zero size 2024-03-11T19:33:47.460 INFO:tasks.workunit.client.0.smithi090.stdout:0/430: mkdir d2/d11/d15/d29/d87 0 2024-03-11T19:33:47.700 INFO:tasks.workunit.client.0.smithi090.stdout:3/383: sync 2024-03-11T19:33:47.891 INFO:tasks.workunit.client.0.smithi090.stdout:5/407: link d1/c46 d1/d17/d39/d51/d53/d7b/d3b/d63/c89 0 2024-03-11T19:33:48.000 INFO:tasks.workunit.client.0.smithi090.stdout:0/431: creat d2/d11/d15/d29/d43/d58/f88 x:0 0 0 2024-03-11T19:33:48.517 INFO:tasks.workunit.client.0.smithi090.stdout:2/402: rename d0/d21/c59 to d0/de/d12/d46/d5d/d63/c73 0 2024-03-11T19:33:48.520 INFO:tasks.workunit.client.0.smithi090.stdout:7/404: mkdir d3/dc/d87 0 2024-03-11T19:33:48.632 INFO:tasks.workunit.client.0.smithi090.stdout:3/384: truncate d2/d1d/d47/d6a/f73 428396 0 2024-03-11T19:33:48.758 INFO:tasks.workunit.client.0.smithi090.stdout:8/454: link d0/d43/dc/l4f d0/d53/d59/l91 0 2024-03-11T19:33:48.980 INFO:tasks.workunit.client.0.smithi090.stdout:5/408: chown d1/d26/d4d/l61 286 1 2024-03-11T19:33:49.133 INFO:tasks.workunit.client.0.smithi090.stdout:0/432: mknod d2/d11/d3a/d64/c89 0 2024-03-11T19:33:49.375 INFO:tasks.workunit.client.0.smithi090.stdout:7/405: rename d3/dc/d17/f1b to d3/dc/d17/d32/d45/f88 0 2024-03-11T19:33:49.844 INFO:tasks.workunit.client.0.smithi090.stdout:8/455: creat d0/d3/d3e/f92 x:0 0 0 2024-03-11T19:33:49.857 INFO:tasks.workunit.client.0.smithi090.stdout:9/401: getdents d1/dd/df/d1c 0 2024-03-11T19:33:50.218 INFO:tasks.workunit.client.0.smithi090.stdout:6/454: dwrite d4/ff [0,4194304] 0 2024-03-11T19:33:50.344 INFO:tasks.workunit.client.0.smithi090.stdout:7/406: symlink d3/dc/d29/l89 0 2024-03-11T19:33:50.428 INFO:tasks.workunit.client.0.smithi090.stdout:3/385: mknod d2/c83 0 2024-03-11T19:33:50.774 INFO:tasks.workunit.client.0.smithi090.stdout:4/449: dwrite d4/d51/d8c/d6a/f7f [0,4194304] 0 2024-03-11T19:33:50.884 INFO:tasks.workunit.client.0.smithi090.stdout:9/402: mknod d1/dd/d18/d32/c78 0 2024-03-11T19:33:50.931 INFO:tasks.workunit.client.0.smithi090.stdout:9/403: readlink d1/dd/df/d1c/l55 0 2024-03-11T19:33:51.061 INFO:tasks.workunit.client.0.smithi090.stdout:1/395: write d5/da/f44 [955895,96305] 0 2024-03-11T19:33:51.194 INFO:tasks.workunit.client.0.smithi090.stdout:3/386: creat d2/db/f84 x:0 0 0 2024-03-11T19:33:52.544 INFO:tasks.workunit.client.0.smithi090.stdout:6/455: rmdir d4/d13/d22/d42/d75/d77 0 2024-03-11T19:33:52.806 INFO:tasks.workunit.client.0.smithi090.stdout:1/396: creat d5/d4e/d79/f82 x:0 0 0 2024-03-11T19:33:53.395 INFO:tasks.workunit.client.0.smithi090.stdout:9/404: getdents d1 0 2024-03-11T19:33:53.542 INFO:tasks.workunit.client.0.smithi090.stdout:1/397: unlink d5/d65/l81 0 2024-03-11T19:33:54.127 INFO:tasks.workunit.client.0.smithi090.stdout:5/409: unlink d1/d17/f2a 0 2024-03-11T19:33:54.140 INFO:tasks.workunit.client.0.smithi090.stdout:0/433: write d2/f14 [1095232,49114] 0 2024-03-11T19:33:54.146 INFO:tasks.workunit.client.0.smithi090.stdout:1/398: rename d5/da/d1c/c1d to d5/d4e/c83 0 2024-03-11T19:33:54.811 INFO:tasks.workunit.client.0.smithi090.stdout:7/407: write d3/dc/d1e/f57 [867729,38281] 0 2024-03-11T19:33:54.819 INFO:tasks.workunit.client.0.smithi090.stdout:8/456: dwrite d0/d3/d3e/f66 [0,4194304] 0 2024-03-11T19:33:54.832 INFO:tasks.workunit.client.0.smithi090.stdout:6/456: link d4/d13/d22/d42/l55 d4/d13/d22/d34/d40/d7d/l95 0 2024-03-11T19:33:54.988 INFO:tasks.workunit.client.0.smithi090.stdout:2/403: dwrite d0/f3e [0,4194304] 0 2024-03-11T19:33:55.111 INFO:tasks.workunit.client.0.smithi090.stdout:7/408: sync 2024-03-11T19:33:55.181 INFO:tasks.workunit.client.0.smithi090.stdout:2/404: sync 2024-03-11T19:33:55.308 INFO:tasks.workunit.client.0.smithi090.stdout:6/457: chown d4/d14/f78 31597611 1 2024-03-11T19:33:55.356 INFO:tasks.workunit.client.0.smithi090.stdout:3/387: write d2/f67 [372649,11413] 0 2024-03-11T19:33:55.521 INFO:tasks.workunit.client.0.smithi090.stdout:8/457: write d0/d3/d3e/f92 [292106,27791] 0 2024-03-11T19:33:55.602 INFO:tasks.workunit.client.0.smithi090.stdout:4/450: dwrite d4/d18/f3c [0,4194304] 0 2024-03-11T19:33:55.812 INFO:tasks.workunit.client.0.smithi090.stdout:1/399: dread d5/da/d23/f2a [0,4194304] 0 2024-03-11T19:33:55.839 INFO:tasks.workunit.client.0.smithi090.stdout:8/458: chown d0/d3/d4c/c6d 7525 1 2024-03-11T19:33:56.189 INFO:tasks.workunit.client.0.smithi090.stdout:9/405: write d1/dd/d18/f65 [1042210,93443] 0 2024-03-11T19:33:56.570 INFO:tasks.workunit.client.0.smithi090.stdout:4/451: creat d4/d51/d8c/f9c x:0 0 0 2024-03-11T19:33:56.730 INFO:tasks.workunit.client.0.smithi090.stdout:7/409: symlink d3/dc/d1e/l8a 0 2024-03-11T19:33:56.747 INFO:tasks.workunit.client.0.smithi090.stdout:8/459: unlink d0/d43/dc/l16 0 2024-03-11T19:33:56.876 INFO:tasks.workunit.client.0.smithi090.stdout:5/410: dwrite d1/d26/f3c [4194304,4194304] 0 2024-03-11T19:33:56.999 INFO:tasks.workunit.client.0.smithi090.stdout:8/460: read d0/d53/f71 [1282697,92377] 0 2024-03-11T19:33:57.026 INFO:tasks.workunit.client.0.smithi090.stdout:1/400: truncate d5/d31/d4c/f73 188729 0 2024-03-11T19:33:57.055 INFO:tasks.workunit.client.0.smithi090.stdout:0/434: write d2/f27 [237645,107597] 0 2024-03-11T19:33:57.105 INFO:tasks.workunit.client.0.smithi090.stdout:1/401: readlink d5/da/d1c/l2f 0 2024-03-11T19:33:57.784 INFO:tasks.workunit.client.0.smithi090.stdout:8/461: mkdir d0/d3/d86/d93 0 2024-03-11T19:33:58.058 INFO:tasks.workunit.client.0.smithi090.stdout:0/435: rmdir d2/d11/d3a 39 2024-03-11T19:33:58.084 INFO:tasks.workunit.client.0.smithi090.stdout:6/458: dwrite d4/d13/d22/d3a/f54 [4194304,4194304] 0 2024-03-11T19:33:58.108 INFO:tasks.workunit.client.0.smithi090.stdout:3/388: dwrite d2/db/d15/d49/f7f [0,4194304] 0 2024-03-11T19:33:58.529 INFO:tasks.workunit.client.0.smithi090.stdout:9/406: dwrite d1/dd/d18/d32/f45 [0,4194304] 0 2024-03-11T19:33:58.642 INFO:tasks.workunit.client.0.smithi090.stdout:2/405: rename d0/l2c to d0/d21/d30/l74 0 2024-03-11T19:33:58.839 INFO:tasks.workunit.client.0.smithi090.stdout:7/410: write d3/dc/d1e/f63 [11675,69482] 0 2024-03-11T19:33:58.954 INFO:tasks.workunit.client.0.smithi090.stdout:4/452: dwrite d4/d27/d44/f5c [0,4194304] 0 2024-03-11T19:33:59.090 INFO:tasks.workunit.client.0.smithi090.stdout:1/402: fdatasync d5/da/d1c/f39 0 2024-03-11T19:33:59.127 INFO:tasks.workunit.client.0.smithi090.stdout:1/403: readlink d5/l9 0 2024-03-11T19:34:00.419 INFO:tasks.workunit.client.0.smithi090.stdout:1/404: creat d5/d31/d76/f84 x:0 0 0 2024-03-11T19:34:00.475 INFO:tasks.workunit.client.0.smithi090.stdout:7/411: dread d3/dc/d1e/d2a/f66 [0,4194304] 0 2024-03-11T19:34:00.505 INFO:tasks.workunit.client.0.smithi090.stdout:5/411: link d1/c10 d1/c8a 0 2024-03-11T19:34:00.583 INFO:tasks.workunit.client.0.smithi090.stdout:0/436: fdatasync d2/d11/d3a/f80 0 2024-03-11T19:34:00.733 INFO:tasks.workunit.client.0.smithi090.stdout:7/412: sync 2024-03-11T19:34:01.150 INFO:tasks.workunit.client.0.smithi090.stdout:2/406: mkdir d0/de/d12/d46/d75 0 2024-03-11T19:34:01.249 INFO:tasks.workunit.client.0.smithi090.stdout:4/453: mknod d4/d27/d28/d38/d75/c9d 0 2024-03-11T19:34:01.406 INFO:tasks.workunit.client.0.smithi090.stdout:8/462: dwrite d0/d43/dc/f50 [0,4194304] 0 2024-03-11T19:34:01.585 INFO:tasks.workunit.client.0.smithi090.stdout:6/459: write d4/d14/d8a/d74/f7b [702615,499] 0 2024-03-11T19:34:01.592 INFO:tasks.workunit.client.0.smithi090.stdout:5/412: mknod d1/d26/c8b 0 2024-03-11T19:34:01.767 INFO:tasks.workunit.client.0.smithi090.stdout:0/437: mknod d2/d11/d15/d29/d43/d58/c8a 0 2024-03-11T19:34:02.137 INFO:tasks.workunit.client.0.smithi090.stdout:9/407: write d1/dd/f3c [1038441,20427] 0 2024-03-11T19:34:02.234 INFO:tasks.workunit.client.0.smithi090.stdout:9/408: sync 2024-03-11T19:34:02.303 INFO:tasks.workunit.client.0.smithi090.stdout:3/389: link d2/db/d15/d1a/d3c/c79 d2/d2f/c85 0 2024-03-11T19:34:02.401 INFO:tasks.workunit.client.0.smithi090.stdout:2/407: read d0/d21/f3d [61288,72328] 0 2024-03-11T19:34:02.535 INFO:tasks.workunit.client.0.smithi090.stdout:4/454: fdatasync d4/d18/f90 0 2024-03-11T19:34:02.809 INFO:tasks.workunit.client.0.smithi090.stdout:6/460: creat d4/d14/d8a/f96 x:0 0 0 2024-03-11T19:34:03.364 INFO:tasks.workunit.client.0.smithi090.stdout:1/405: write d5/d65/f69 [1069109,41302] 0 2024-03-11T19:34:03.412 INFO:tasks.workunit.client.0.smithi090.stdout:1/406: readlink d5/da/d1c/l2f 0 2024-03-11T19:34:03.698 INFO:tasks.workunit.client.0.smithi090.stdout:3/390: mknod d2/db/c86 0 2024-03-11T19:34:03.932 INFO:tasks.workunit.client.0.smithi090.stdout:2/408: creat d0/de/d1f/f76 x:0 0 0 2024-03-11T19:34:03.968 INFO:tasks.workunit.client.0.smithi090.stdout:2/409: stat d0/d21/d30/d29/f6d 0 2024-03-11T19:34:04.317 INFO:tasks.workunit.client.0.smithi090.stdout:4/455: mknod d4/d27/d28/d38/d75/c9e 0 2024-03-11T19:34:04.365 INFO:tasks.workunit.client.0.smithi090.stdout:4/456: rename d4/d27 to d4/d27/d28/d7a/d9f 22 2024-03-11T19:34:04.972 INFO:tasks.workunit.client.0.smithi090.stdout:7/413: link d3/dc/d17/d32/d71/d5d/l14 d3/dc/d17/d32/d71/d5d/d48/l8b 0 2024-03-11T19:34:05.135 INFO:tasks.workunit.client.0.smithi090.stdout:3/391: stat d2/f1b 0 2024-03-11T19:34:05.209 INFO:tasks.workunit.client.0.smithi090.stdout:9/409: write d1/dd/f3f [547342,26135] 0 2024-03-11T19:34:05.976 INFO:tasks.workunit.client.0.smithi090.stdout:5/413: dread d1/d17/d39/d51/d53/d7b/f37 [0,4194304] 0 2024-03-11T19:34:06.275 INFO:tasks.workunit.client.0.smithi090.stdout:1/407: symlink d5/d65/d6b/d7f/l85 0 2024-03-11T19:34:06.440 INFO:tasks.workunit.client.0.smithi090.stdout:7/414: truncate d3/dc/d17/d32/d71/d5d/f4c 1979680 0 2024-03-11T19:34:06.461 INFO:tasks.workunit.client.0.smithi090.stdout:7/415: readlink d3/dc/d29/l89 0 2024-03-11T19:34:06.617 INFO:tasks.workunit.client.0.smithi090.stdout:9/410: unlink d1/dd/df/c3e 0 2024-03-11T19:34:06.632 INFO:tasks.workunit.client.0.smithi090.stdout:8/463: write d0/d43/dc/f12 [2676058,128415] 0 2024-03-11T19:34:06.733 INFO:tasks.workunit.client.0.smithi090.stdout:2/410: mknod d0/d21/d30/d25/c77 0 2024-03-11T19:34:06.831 INFO:tasks.workunit.client.0.smithi090.stdout:2/411: dread d0/de/d12/f24 [0,4194304] 0 2024-03-11T19:34:06.960 INFO:tasks.workunit.client.0.smithi090.stdout:6/461: link d4/c6 d4/d14/d44/c97 0 2024-03-11T19:34:07.038 INFO:tasks.workunit.client.0.smithi090.stdout:0/438: write d2/d11/d15/f74 [215805,58064] 0 2024-03-11T19:34:07.109 INFO:tasks.workunit.client.0.smithi090.stdout:5/414: creat d1/d75/f8c x:0 0 0 2024-03-11T19:34:07.126 INFO:tasks.workunit.client.0.smithi090.stdout:6/462: sync 2024-03-11T19:34:07.357 INFO:tasks.workunit.client.0.smithi090.stdout:3/392: mkdir d2/db/d74/d87 0 2024-03-11T19:34:07.579 INFO:tasks.workunit.client.0.smithi090.stdout:9/411: truncate d1/dd/d18/f65 1591654 0 2024-03-11T19:34:07.767 INFO:tasks.workunit.client.0.smithi090.stdout:8/464: truncate d0/d3/d3e/f66 4040600 0 2024-03-11T19:34:08.250 INFO:tasks.workunit.client.0.smithi090.stdout:2/412: fsync d0/de/d1f/f55 0 2024-03-11T19:34:08.340 INFO:tasks.workunit.client.0.smithi090.stdout:0/439: mknod d2/d11/d3a/c8b 0 2024-03-11T19:34:08.948 INFO:tasks.workunit.client.0.smithi090.stdout:3/393: dread d2/d45/d55/f6c [0,4194304] 0 2024-03-11T19:34:09.410 INFO:tasks.workunit.client.0.smithi090.stdout:4/457: write d4/d18/d6f/f87 [598342,109736] 0 2024-03-11T19:34:09.646 INFO:tasks.workunit.client.0.smithi090.stdout:5/415: mknod d1/d17/c8d 0 2024-03-11T19:34:09.885 INFO:tasks.workunit.client.0.smithi090.stdout:6/463: rename d4/d14/d44/c97 to d4/d13/d22/d42/d48/d4e/c98 0 2024-03-11T19:34:09.967 INFO:tasks.workunit.client.0.smithi090.stdout:1/408: write d5/da/dd/f2c [441446,104479] 0 2024-03-11T19:34:10.290 INFO:tasks.workunit.client.0.smithi090.stdout:4/458: rmdir d4/d51 39 2024-03-11T19:34:10.292 INFO:tasks.workunit.client.0.smithi090.stdout:0/440: getdents d2/d11/d15/d29/d3d/d7e 0 2024-03-11T19:34:10.330 INFO:tasks.workunit.client.0.smithi090.stdout:4/459: chown d4/f46 50525959 1 2024-03-11T19:34:10.359 INFO:tasks.workunit.client.0.smithi090.stdout:0/441: stat d2/d11/d15/d29/d43/d58/c8a 0 2024-03-11T19:34:10.697 INFO:tasks.workunit.client.0.smithi090.stdout:5/416: fsync d1/d26/f83 0 2024-03-11T19:34:10.774 INFO:tasks.workunit.client.0.smithi090.stdout:7/416: dwrite d3/dc/d17/d32/f4a [0,4194304] 0 2024-03-11T19:34:10.807 INFO:tasks.workunit.client.0.smithi090.stdout:6/464: truncate d4/d14/d25/f41 739215 0 2024-03-11T19:34:10.975 INFO:tasks.workunit.client.0.smithi090.stdout:8/465: fsync d0/d3/d3e/f66 0 2024-03-11T19:34:11.011 INFO:tasks.workunit.client.0.smithi090.stdout:1/409: truncate d5/da/dd/f14 3557313 0 2024-03-11T19:34:11.046 INFO:tasks.workunit.client.0.smithi090.stdout:3/394: unlink d2/db/l60 0 2024-03-11T19:34:11.185 INFO:tasks.workunit.client.0.smithi090.stdout:0/442: dwrite d2/d11/d15/f74 [0,4194304] 0 2024-03-11T19:34:11.329 INFO:tasks.workunit.client.0.smithi090.stdout:4/460: symlink d4/d27/d28/d69/la0 0 2024-03-11T19:34:12.113 INFO:tasks.workunit.client.0.smithi090.stdout:5/417: unlink d1/f7c 0 2024-03-11T19:34:12.221 INFO:tasks.workunit.client.0.smithi090.stdout:8/466: dread d0/d43/f79 [0,4194304] 0 2024-03-11T19:34:12.248 INFO:tasks.workunit.client.0.smithi090.stdout:3/395: readlink d2/d4f/l7d 0 2024-03-11T19:34:12.262 INFO:tasks.workunit.client.0.smithi090.stdout:5/418: truncate d1/d75/f8c 669865 0 2024-03-11T19:34:12.319 INFO:tasks.workunit.client.0.smithi090.stdout:6/465: rmdir d4/d13/d22/d3a/d67 39 2024-03-11T19:34:12.559 INFO:tasks.workunit.client.0.smithi090.stdout:7/417: chown d3/dc/d17/d32/d71/d5d/f18 43107399 1 2024-03-11T19:34:12.727 INFO:tasks.workunit.client.0.smithi090.stdout:0/443: mkdir d2/d11/d15/d29/d3d/d8c 0 2024-03-11T19:34:12.854 INFO:tasks.workunit.client.0.smithi090.stdout:4/461: mkdir d4/d27/d44/d66/da1 0 2024-03-11T19:34:12.914 INFO:tasks.workunit.client.0.smithi090.stdout:1/410: dread d5/da/d1c/f30 [0,4194304] 0 2024-03-11T19:34:13.136 INFO:tasks.workunit.client.0.smithi090.stdout:2/413: rename d0/d21/d30/f39 to d0/de/d12/f78 0 2024-03-11T19:34:13.181 INFO:tasks.workunit.client.0.smithi090.stdout:2/414: readlink d0/de/d1f/l2d 0 2024-03-11T19:34:13.217 INFO:tasks.workunit.client.0.smithi090.stdout:2/415: chown d0/d21/d30/d25/f5c 21657 1 2024-03-11T19:34:13.415 INFO:tasks.workunit.client.0.smithi090.stdout:6/466: chown d4/d13/d22/d42/d48/d4e/l5b 1 1 2024-03-11T19:34:13.440 INFO:tasks.workunit.client.0.smithi090.stdout:6/467: chown d4/d13/d22/d3a/d6f 36687703 1 2024-03-11T19:34:13.722 INFO:tasks.workunit.client.0.smithi090.stdout:7/418: fdatasync d3/dc/d1e/f12 0 2024-03-11T19:34:13.833 INFO:tasks.workunit.client.0.smithi090.stdout:0/444: read d2/f3 [3348092,34343] 0 2024-03-11T19:34:14.222 INFO:tasks.workunit.client.0.smithi090.stdout:8/467: rename d0/d3/d3e/f66 to d0/d3/d4c/d8d/f94 0 2024-03-11T19:34:14.595 INFO:tasks.workunit.client.0.smithi090.stdout:2/416: mknod d0/d21/d30/d25/c79 0 2024-03-11T19:34:15.297 INFO:tasks.workunit.client.0.smithi090.stdout:4/462: fdatasync d4/d51/f72 0 2024-03-11T19:34:15.535 INFO:tasks.workunit.client.0.smithi090.stdout:4/463: sync 2024-03-11T19:34:15.654 INFO:tasks.workunit.client.0.smithi090.stdout:9/412: creat d1/f79 x:0 0 0 2024-03-11T19:34:15.661 INFO:tasks.workunit.client.0.smithi090.stdout:8/468: dread - d0/d3/f68 zero size 2024-03-11T19:34:15.667 INFO:tasks.workunit.client.0.smithi090.stdout:4/464: sync 2024-03-11T19:34:15.776 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:34:15.779 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:34:15.872 INFO:tasks.workunit.client.0.smithi090.stdout:5/419: link d1/f60 d1/d17/d39/d51/d53/d7b/d6f/f8e 0 2024-03-11T19:34:15.872 INFO:tasks.workunit.client.0.smithi090.stdout:2/417: rename d0/de/d12/d46/d4c/f65 to d0/d21/d30/d29/f7a 0 2024-03-11T19:34:16.044 INFO:tasks.workunit.client.0.smithi090.stdout:8/469: dread d0/d43/dc/fd [0,4194304] 0 2024-03-11T19:34:16.048 INFO:tasks.workunit.client.0.smithi090.stdout:2/418: sync 2024-03-11T19:34:16.088 INFO:tasks.workunit.client.0.smithi090.stdout:7/419: getdents d3/dc/d17/d32/d45/d84 0 2024-03-11T19:34:16.666 INFO:tasks.workunit.client.0.smithi090.stdout:4/465: symlink d4/d51/d8c/d6a/la2 0 2024-03-11T19:34:16.719 INFO:tasks.workunit.client.0.smithi090.stdout:5/420: unlink d1/d26/d19/c5f 0 2024-03-11T19:34:17.014 INFO:tasks.workunit.client.0.smithi090.stdout:2/419: symlink d0/d21/d30/d25/l7b 0 2024-03-11T19:34:17.115 INFO:tasks.workunit.client.0.smithi090.stdout:3/396: write d2/db/d15/d1a/d3c/f52 [1916421,66374] 0 2024-03-11T19:34:17.366 INFO:tasks.workunit.client.0.smithi090.stdout:8/470: mknod d0/d3/d4c/d8d/c95 0 2024-03-11T19:34:17.910 INFO:tasks.workunit.client.0.smithi090.stdout:1/411: dwrite d5/da/d23/f2a [0,4194304] 0 2024-03-11T19:34:18.174 INFO:tasks.workunit.client.0.smithi090.stdout:8/471: mkdir d0/d53/d59/d96 0 2024-03-11T19:34:18.194 INFO:tasks.workunit.client.0.smithi090.stdout:6/468: write d4/d24/f3f [4028356,22131] 0 2024-03-11T19:34:18.284 INFO:tasks.workunit.client.0.smithi090.stdout:6/469: fdatasync d4/d13/d22/d3a/f56 0 2024-03-11T19:34:18.487 INFO:tasks.workunit.client.0.smithi090.stdout:1/412: mkdir d5/d31/d86 0 2024-03-11T19:34:18.824 INFO:tasks.workunit.client.0.smithi090.stdout:0/445: write d2/d11/d15/d29/d2e/d6d/f73 [1681029,23450] 0 2024-03-11T19:34:19.754 INFO:tasks.workunit.client.0.smithi090.stdout:1/413: symlink d5/da/d1c/d3b/l87 0 2024-03-11T19:34:20.037 INFO:tasks.workunit.client.0.smithi090.stdout:7/420: getdents d3/dc/d17/d32/d7b 0 2024-03-11T19:34:20.122 INFO:tasks.workunit.client.0.smithi090.stdout:6/470: dread d4/d13/d22/d34/f36 [0,4194304] 0 2024-03-11T19:34:20.233 INFO:tasks.workunit.client.0.smithi090.stdout:2/420: write d0/d21/d30/d25/f5c [721955,40055] 0 2024-03-11T19:34:20.267 INFO:tasks.workunit.client.0.smithi090.stdout:4/466: write d4/d18/f25 [1880454,21594] 0 2024-03-11T19:34:20.289 INFO:tasks.workunit.client.0.smithi090.stdout:5/421: write d1/d26/fc [5233058,102117] 0 2024-03-11T19:34:20.524 INFO:tasks.workunit.client.0.smithi090.stdout:8/472: mknod d0/d43/dc/d90/c97 0 2024-03-11T19:34:20.533 INFO:tasks.workunit.client.0.smithi090.stdout:4/467: stat d4/d27/d28/d38/f4b 0 2024-03-11T19:34:20.645 INFO:tasks.workunit.client.0.smithi090.stdout:1/414: creat d5/d31/d4c/f88 x:0 0 0 2024-03-11T19:34:20.893 INFO:tasks.workunit.client.0.smithi090.stdout:3/397: dwrite d2/d1f/f3f [0,4194304] 0 2024-03-11T19:34:21.088 INFO:tasks.workunit.client.0.smithi090.stdout:5/422: dread d1/f1e [0,4194304] 0 2024-03-11T19:34:21.198 INFO:tasks.workunit.client.0.smithi090.stdout:6/471: creat d4/d14/d92/f99 x:0 0 0 2024-03-11T19:34:21.224 INFO:tasks.workunit.client.0.smithi090.stdout:4/468: dread - d4/d27/d28/d69/f99 zero size 2024-03-11T19:34:21.531 INFO:tasks.workunit.client.0.smithi090.stdout:8/473: mkdir d0/d3/d98 0 2024-03-11T19:34:21.672 INFO:tasks.workunit.client.0.smithi090.stdout:5/423: creat d1/d26/f8f x:0 0 0 2024-03-11T19:34:21.786 INFO:tasks.workunit.client.0.smithi090.stdout:0/446: dwrite d2/d11/d15/d29/f4a [0,4194304] 0 2024-03-11T19:34:21.881 INFO:tasks.workunit.client.0.smithi090.stdout:1/415: truncate d5/d31/d36/f70 529253 0 2024-03-11T19:34:21.972 INFO:tasks.workunit.client.0.smithi090.stdout:6/472: creat d4/d14/d44/f9a x:0 0 0 2024-03-11T19:34:22.096 INFO:tasks.workunit.client.0.smithi090.stdout:1/416: dread d5/da/d23/f2a [0,4194304] 0 2024-03-11T19:34:22.097 INFO:tasks.workunit.client.0.smithi090.stdout:6/473: dread - d4/d13/d22/d34/f6d zero size 2024-03-11T19:34:22.267 INFO:tasks.workunit.client.0.smithi090.stdout:6/474: dread d4/d13/d22/d34/d37/f8e [0,4194304] 0 2024-03-11T19:34:22.290 INFO:tasks.workunit.client.0.smithi090.stdout:2/421: link d0/d21/f2a d0/de/d12/d46/d75/f7c 0 2024-03-11T19:34:22.464 INFO:tasks.workunit.client.0.smithi090.stdout:8/474: mknod d0/d3/d4c/d8d/c99 0 2024-03-11T19:34:22.487 INFO:tasks.workunit.client.0.smithi090.stdout:7/421: rename d3/dc/l1a to d3/dc/d17/d32/d7b/l8c 0 2024-03-11T19:34:22.629 INFO:tasks.workunit.client.0.smithi090.stdout:5/424: mknod d1/d17/d39/d51/d53/d7b/d6f/c90 0 2024-03-11T19:34:22.695 INFO:tasks.workunit.client.0.smithi090.stdout:9/413: dwrite d1/dd/f30 [0,4194304] 0 2024-03-11T19:34:23.067 INFO:tasks.workunit.client.0.smithi090.stdout:1/417: unlink d5/f60 0 2024-03-11T19:34:23.104 INFO:tasks.workunit.client.0.smithi090.stdout:1/418: dread - d5/d31/d4c/f88 zero size 2024-03-11T19:34:23.793 INFO:tasks.workunit.client.0.smithi090.stdout:7/422: chown d3/dc/d1e/l2e 1310152891 1 2024-03-11T19:34:24.042 INFO:tasks.workunit.client.0.smithi090.stdout:3/398: dwrite d2/f2c [0,4194304] 0 2024-03-11T19:34:24.178 INFO:tasks.workunit.client.0.smithi090.stdout:9/414: creat d1/dd/d18/d32/f7a x:0 0 0 2024-03-11T19:34:24.258 INFO:tasks.workunit.client.0.smithi090.stdout:3/399: chown d2/d1f/f3b 26668 1 2024-03-11T19:34:24.309 INFO:tasks.workunit.client.0.smithi090.stdout:9/415: write d1/dd/d18/d32/f7a [30255,94875] 0 2024-03-11T19:34:24.336 INFO:tasks.workunit.client.0.smithi090.stdout:3/400: chown d2/d2f/f30 985 1 2024-03-11T19:34:24.357 INFO:tasks.workunit.client.0.smithi090.stdout:3/401: chown d2/d1d 68305 1 2024-03-11T19:34:24.463 INFO:tasks.workunit.client.0.smithi090.stdout:4/469: link d4/d27/d28/d38/f91 d4/d27/d28/d38/d64/fa3 0 2024-03-11T19:34:24.483 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:59:44.241 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:34:24.495 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:59:44.253 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:34:24.567 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:00:59:44.325 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:34:24.571 INFO:tasks.workunit.client.0.smithi090.stdout:9/416: dread d1/dd/df/f10 [0,4194304] 0 2024-03-11T19:34:24.589 INFO:tasks.workunit.client.0.smithi090.stdout:4/470: chown d4/d18/f3c 24937 1 2024-03-11T19:34:24.609 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:59:44.306 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:34:24.615 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:59:44.304 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:34:24.616 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:59:44.313 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:34:24.617 INFO:tasks.workunit.client.0.smithi090.stdout:4/471: chown d4/d51/d8c/d89/f98 13 1 2024-03-11T19:34:24.624 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:59:44.314 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:34:24.678 INFO:tasks.workunit.client.0.smithi090.stdout:1/419: mkdir d5/da/d1c/d3b/d40/d89 0 2024-03-11T19:34:24.692 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:00:59:44.390 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:34:24.708 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:00:59:44.398 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:34:24.781 INFO:tasks.workunit.client.0.smithi090.stdout:6/475: mkdir d4/d9b 0 2024-03-11T19:34:24.946 INFO:tasks.workunit.client.0.smithi090.stdout:0/447: dwrite d2/f5 [0,4194304] 0 2024-03-11T19:34:25.084 INFO:tasks.workunit.client.0.smithi090.stdout:2/422: mknod d0/de/d12/c7d 0 2024-03-11T19:34:25.307 INFO:tasks.workunit.client.0.smithi090.stdout:2/423: dread d0/de/d12/f1d [0,4194304] 0 2024-03-11T19:34:25.449 INFO:tasks.workunit.client.0.smithi090.stdout:5/425: mknod d1/d17/d39/d51/c91 0 2024-03-11T19:34:25.570 INFO:tasks.workunit.client.0.smithi090.stdout:5/426: stat d1/d17/d39/c59 0 2024-03-11T19:34:25.693 INFO:tasks.workunit.client.0.smithi090.stdout:8/475: dwrite d0/d3/f37 [0,4194304] 0 2024-03-11T19:34:26.297 INFO:tasks.workunit.client.0.smithi090.stdout:6/476: mkdir d4/d13/d22/d34/d83/d9c 0 2024-03-11T19:34:26.810 INFO:tasks.workunit.client.0.smithi090.stdout:2/424: dread d0/d21/d30/f22 [0,4194304] 0 2024-03-11T19:34:26.888 INFO:tasks.workunit.client.0.smithi090.stdout:3/402: unlink d2/d1d/l7c 0 2024-03-11T19:34:26.906 INFO:tasks.workunit.client.0.smithi090.stdout:7/423: write d3/f3b [3553840,26133] 0 2024-03-11T19:34:26.943 INFO:tasks.workunit.client.0.smithi090.stdout:4/472: getdents d4/d27/d28/d7a/d9b 0 2024-03-11T19:34:27.005 INFO:tasks.workunit.client.0.smithi090.stdout:3/403: chown d2/db/f24 2 1 2024-03-11T19:34:27.033 INFO:tasks.workunit.client.0.smithi090.stdout:4/473: read d4/d18/f90 [825826,38285] 0 2024-03-11T19:34:27.092 INFO:tasks.workunit.client.0.smithi090.stdout:8/476: symlink d0/d53/l9a 0 2024-03-11T19:34:27.093 INFO:tasks.workunit.client.0.smithi090.stdout:9/417: rmdir d1/d1d 39 2024-03-11T19:34:27.292 INFO:tasks.workunit.client.0.smithi090.stdout:1/420: symlink d5/da/d1c/d6d/d80/l8a 0 2024-03-11T19:34:27.882 INFO:tasks.workunit.client.0.smithi090.stdout:6/477: fdatasync d4/d13/d22/d34/f6d 0 2024-03-11T19:34:28.169 INFO:tasks.workunit.client.0.smithi090.stdout:2/425: chown d0/d21/f6e 190863 1 2024-03-11T19:34:28.262 INFO:tasks.workunit.client.0.smithi090.stdout:0/448: write d2/d11/d3a/f80 [190099,99689] 0 2024-03-11T19:34:28.399 INFO:tasks.workunit.client.0.smithi090.stdout:0/449: dread d2/d11/d3a/f80 [0,4194304] 0 2024-03-11T19:34:28.499 INFO:tasks.workunit.client.0.smithi090.stdout:4/474: rmdir d4/d27/d28/d38 39 2024-03-11T19:34:28.564 INFO:tasks.workunit.client.0.smithi090.stdout:5/427: write d1/d17/d39/d51/d53/d7b/f44 [227267,107314] 0 2024-03-11T19:34:29.560 INFO:tasks.workunit.client.0.smithi090.stdout:7/424: mkdir d3/dc/d17/d32/d45/d84/d8d 0 2024-03-11T19:34:30.286 INFO:tasks.workunit.client.0.smithi090.stdout:4/475: creat d4/d27/d44/fa4 x:0 0 0 2024-03-11T19:34:30.425 INFO:tasks.workunit.client.0.smithi090.stdout:8/477: write d0/d3/d3e/f4e [147846,42997] 0 2024-03-11T19:34:30.598 INFO:tasks.workunit.client.0.smithi090.stdout:1/421: write d5/f4f [610129,103402] 0 2024-03-11T19:34:30.661 INFO:tasks.workunit.client.0.smithi090.stdout:1/422: chown d5 27 1 2024-03-11T19:34:30.784 INFO:tasks.workunit.client.0.smithi090.stdout:6/478: dwrite d4/d13/d22/d34/f6d [0,4194304] 0 2024-03-11T19:34:31.020 INFO:tasks.workunit.client.0.smithi090.stdout:5/428: dread d1/d26/fc [0,4194304] 0 2024-03-11T19:34:31.376 INFO:tasks.workunit.client.0.smithi090.stdout:3/404: rmdir d2/db/d74/d87 0 2024-03-11T19:34:31.417 INFO:tasks.workunit.client.0.smithi090.stdout:9/418: creat d1/dd/d18/d3a/f7b x:0 0 0 2024-03-11T19:34:31.800 INFO:tasks.workunit.client.0.smithi090.stdout:8/478: rmdir d0/d3/d3e 39 2024-03-11T19:34:31.934 INFO:tasks.workunit.client.0.smithi090.stdout:4/476: dread d4/d27/d28/d69/f85 [0,4194304] 0 2024-03-11T19:34:32.026 INFO:tasks.workunit.client.0.smithi090.stdout:0/450: write d2/d11/d15/d29/d2e/f52 [4725268,1060] 0 2024-03-11T19:34:32.094 INFO:tasks.workunit.client.0.smithi090.stdout:2/426: write d0/f66 [2135924,115463] 0 2024-03-11T19:34:32.405 INFO:tasks.workunit.client.0.smithi090.stdout:7/425: dwrite d3/d55/f85 [0,4194304] 0 2024-03-11T19:34:32.406 INFO:tasks.workunit.client.0.smithi090.stdout:1/423: rename d5/l48 to d5/da/dd/l8b 0 2024-03-11T19:34:32.922 INFO:tasks.workunit.client.0.smithi090.stdout:5/429: dread d1/d26/f71 [0,4194304] 0 2024-03-11T19:34:32.952 INFO:tasks.workunit.client.0.smithi090.stdout:5/430: chown d1/d26/f43 15 1 2024-03-11T19:34:33.389 INFO:tasks.workunit.client.0.smithi090.stdout:2/427: mknod d0/d21/d30/d29/c7e 0 2024-03-11T19:34:33.414 INFO:tasks.workunit.client.0.smithi090.stdout:9/419: write d1/dd/d18/d32/f34 [1345057,95444] 0 2024-03-11T19:34:33.427 INFO:tasks.workunit.client.0.smithi090.stdout:7/426: mkdir d3/d49/d8e 0 2024-03-11T19:34:33.438 INFO:tasks.workunit.client.0.smithi090.stdout:1/424: mkdir d5/da/d1c/d3b/d8c 0 2024-03-11T19:34:33.492 INFO:tasks.workunit.client.0.smithi090.stdout:6/479: creat d4/d13/d22/d3a/d67/f9d x:0 0 0 2024-03-11T19:34:33.675 INFO:tasks.workunit.client.0.smithi090.stdout:9/420: dread d1/dd/df/f10 [0,4194304] 0 2024-03-11T19:34:33.938 INFO:tasks.workunit.client.0.smithi090.stdout:5/431: mkdir d1/d17/d39/d51/d53/d7b/d3b/d76/d92 0 2024-03-11T19:34:33.964 INFO:tasks.workunit.client.0.smithi090.stdout:8/479: write d0/d53/d59/f72 [693053,20239] 0 2024-03-11T19:34:33.968 INFO:tasks.workunit.client.0.smithi090.stdout:3/405: rename d2/d2f/c81 to d2/db/d15/d1a/d3c/c88 0 2024-03-11T19:34:34.026 INFO:tasks.workunit.client.0.smithi090.stdout:1/425: fsync d5/d31/d4c/f88 0 2024-03-11T19:34:34.063 INFO:tasks.workunit.client.0.smithi090.stdout:7/427: dread - d3/d55/f64 zero size 2024-03-11T19:34:34.136 INFO:tasks.workunit.client.0.smithi090.stdout:4/477: link d4/f8 d4/d27/fa5 0 2024-03-11T19:34:34.357 INFO:tasks.workunit.client.0.smithi090.stdout:7/428: chown d3/dc/d17/d32/d71/d5d/f42 14 1 2024-03-11T19:34:34.564 INFO:tasks.workunit.client.0.smithi090.stdout:9/421: dread d1/f3 [4194304,4194304] 0 2024-03-11T19:34:34.673 INFO:tasks.workunit.client.0.smithi090.stdout:7/429: chown d3/dc/d17/d34/c3d 147289508 1 2024-03-11T19:34:34.829 INFO:tasks.workunit.client.0.smithi090.stdout:3/406: dread d2/d1d/fc [0,4194304] 0 2024-03-11T19:34:34.850 INFO:tasks.workunit.client.0.smithi090.stdout:3/407: chown d2/d1d/f77 14 1 2024-03-11T19:34:35.131 INFO:tasks.workunit.client.0.smithi090.stdout:5/432: creat d1/d17/d39/d51/d53/d64/f93 x:0 0 0 2024-03-11T19:34:35.635 INFO:tasks.workunit.client.0.smithi090.stdout:6/480: dwrite d4/d13/d22/d3a/f65 [0,4194304] 0 2024-03-11T19:34:35.877 INFO:tasks.workunit.client.0.smithi090.stdout:5/433: fdatasync d1/d17/d39/d51/d53/d7b/d3b/d76/f77 0 2024-03-11T19:34:36.123 INFO:tasks.workunit.client.0.smithi090.stdout:1/426: dwrite d5/f15 [0,4194304] 0 2024-03-11T19:34:36.340 INFO:tasks.workunit.client.0.smithi090.stdout:5/434: dread - d1/d17/d39/d51/d53/d64/f93 zero size 2024-03-11T19:34:36.365 INFO:tasks.workunit.client.0.smithi090.stdout:4/478: dwrite d4/d27/d28/d38/d64/f93 [0,4194304] 0 2024-03-11T19:34:36.503 INFO:tasks.workunit.client.0.smithi090.stdout:0/451: rename d2/d11/d15/d29/d2e/l30 to d2/d11/l8d 0 2024-03-11T19:34:36.508 INFO:tasks.workunit.client.0.smithi090.stdout:8/480: write d0/d3/d4c/d8d/f94 [3944650,10056] 0 2024-03-11T19:34:36.597 INFO:tasks.workunit.client.0.smithi090.stdout:0/452: chown d2/d11/d15/d36/f46 215827134 1 2024-03-11T19:34:36.606 INFO:tasks.workunit.client.0.smithi090.stdout:0/453: readlink d2/d11/l13 0 2024-03-11T19:34:36.859 INFO:tasks.workunit.client.0.smithi090.stdout:4/479: sync 2024-03-11T19:34:36.883 INFO:tasks.workunit.client.0.smithi090.stdout:6/481: symlink d4/d13/d22/l9e 0 2024-03-11T19:34:37.074 INFO:tasks.workunit.client.0.smithi090.stdout:5/435: creat d1/d17/d39/d51/d73/f94 x:0 0 0 2024-03-11T19:34:37.227 INFO:tasks.workunit.client.0.smithi090.stdout:5/436: chown d1/d26/f3f 36705951 1 2024-03-11T19:34:37.246 INFO:tasks.workunit.client.0.smithi090.stdout:7/430: dwrite d3/dc/d1e/f26 [0,4194304] 0 2024-03-11T19:34:37.539 INFO:tasks.workunit.client.0.smithi090.stdout:4/480: creat d4/d27/d28/d7a/fa6 x:0 0 0 2024-03-11T19:34:37.921 INFO:tasks.workunit.client.0.smithi090.stdout:2/428: rename d0/de/d1f/l3c to d0/l7f 0 2024-03-11T19:34:37.986 INFO:tasks.workunit.client.0.smithi090.stdout:8/481: dread d0/d43/dc/f50 [4194304,4194304] 0 2024-03-11T19:34:38.040 INFO:tasks.workunit.client.0.smithi090.stdout:0/454: dread d2/d11/f2b [0,4194304] 0 2024-03-11T19:34:38.123 INFO:tasks.workunit.client.0.smithi090.stdout:9/422: dwrite d1/dd/d18/d25/f60 [0,4194304] 0 2024-03-11T19:34:38.195 INFO:tasks.workunit.client.0.smithi090.stdout:7/431: stat d3/dc/d17/d32/d7b/l8c 0 2024-03-11T19:34:38.299 INFO:tasks.workunit.client.0.smithi090.stdout:5/437: fdatasync d1/d17/d39/d51/d53/d64/f85 0 2024-03-11T19:34:38.321 INFO:tasks.workunit.client.0.smithi090.stdout:6/482: write d4/d24/f26 [1437326,95795] 0 2024-03-11T19:34:38.327 INFO:tasks.workunit.client.0.smithi090.stdout:9/423: stat d1/dd/d18/d3a/c6c 0 2024-03-11T19:34:38.443 INFO:tasks.workunit.client.0.smithi090.stdout:2/429: dread d0/de/f52 [0,4194304] 0 2024-03-11T19:34:38.545 INFO:tasks.workunit.client.0.smithi090.stdout:3/408: rename d2/d4f/f5d to d2/d45/f89 0 2024-03-11T19:34:38.706 INFO:tasks.workunit.client.0.smithi090.stdout:4/481: truncate d4/d27/d44/f7e 845196 0 2024-03-11T19:34:38.721 INFO:tasks.workunit.client.0.smithi090.stdout:8/482: truncate d0/f5a 1464422 0 2024-03-11T19:34:38.996 INFO:tasks.workunit.client.0.smithi090.stdout:7/432: dread d3/dc/d1e/d2a/f5e [0,4194304] 0 2024-03-11T19:34:38.997 INFO:tasks.workunit.client.0.smithi090.stdout:0/455: creat d2/d11/d15/d29/d6b/d78/f8e x:0 0 0 2024-03-11T19:34:39.023 INFO:tasks.workunit.client.0.smithi090.stdout:2/430: fdatasync d0/de/f17 0 2024-03-11T19:34:39.107 INFO:tasks.workunit.client.0.smithi090.stdout:0/456: readlink d2/d11/l13 0 2024-03-11T19:34:39.143 INFO:tasks.workunit.client.0.smithi090.stdout:7/433: dread - d3/dc/d17/f56 zero size 2024-03-11T19:34:39.519 INFO:tasks.workunit.client.0.smithi090.stdout:1/427: rename d5/d31/f66 to d5/da/d1c/d6d/f8d 0 2024-03-11T19:34:39.559 INFO:tasks.workunit.client.0.smithi090.stdout:4/482: truncate d4/d18/f90 304638 0 2024-03-11T19:34:39.571 INFO:tasks.workunit.client.0.smithi090.stdout:9/424: mkdir d1/d47/d7c 0 2024-03-11T19:34:39.622 INFO:tasks.workunit.client.0.smithi090.stdout:2/431: rmdir d0/de/d12/d46 39 2024-03-11T19:34:39.748 INFO:tasks.workunit.client.0.smithi090.stdout:9/425: chown d1/dd/df/f10 189231559 1 2024-03-11T19:34:40.020 INFO:tasks.workunit.client.0.smithi090.stdout:8/483: creat d0/d3/d86/d93/f9b x:0 0 0 2024-03-11T19:34:40.025 INFO:tasks.workunit.client.0.smithi090.stdout:5/438: rename d1/d17/d39/d51/f5e to d1/d17/d39/d51/d73/f95 0 2024-03-11T19:34:40.080 INFO:tasks.workunit.client.0.smithi090.stdout:8/484: stat d0/d43/c4b 0 2024-03-11T19:34:40.151 INFO:tasks.workunit.client.0.smithi090.stdout:4/483: creat d4/d27/d44/fa7 x:0 0 0 2024-03-11T19:34:40.168 INFO:tasks.workunit.client.0.smithi090.stdout:7/434: getdents d3/dc/d17/d7f 0 2024-03-11T19:34:40.241 INFO:tasks.workunit.client.0.smithi090.stdout:3/409: rename d2/d45/d55/f6c to d2/db/f8a 0 2024-03-11T19:34:40.356 INFO:tasks.workunit.client.0.smithi090.stdout:8/485: creat d0/d3/d86/f9c x:0 0 0 2024-03-11T19:34:40.384 INFO:tasks.workunit.client.0.smithi090.stdout:2/432: mknod d0/d21/d30/c80 0 2024-03-11T19:34:40.558 INFO:tasks.workunit.client.0.smithi090.stdout:4/484: dread - d4/d27/d44/f94 zero size 2024-03-11T19:34:41.623 INFO:tasks.workunit.client.0.smithi090.stdout:3/410: unlink d2/d1d/l57 0 2024-03-11T19:34:41.697 INFO:tasks.workunit.client.0.smithi090.stdout:8/486: dread d0/d3/d3e/f92 [0,4194304] 0 2024-03-11T19:34:41.949 INFO:tasks.workunit.client.0.smithi090.stdout:7/435: creat d3/dc/d17/d32/d7b/f8f x:0 0 0 2024-03-11T19:34:42.069 INFO:tasks.workunit.client.0.smithi090.stdout:6/483: truncate d4/d13/d22/d3a/f65 2909032 0 2024-03-11T19:34:42.179 INFO:tasks.workunit.client.0.smithi090.stdout:0/457: rename d2/d11/f2b to d2/d11/d15/d29/f8f 0 2024-03-11T19:34:42.273 INFO:tasks.workunit.client.0.smithi090.stdout:6/484: sync 2024-03-11T19:34:42.466 INFO:tasks.workunit.client.0.smithi090.stdout:1/428: write d5/da/d23/f2a [2098386,2829] 0 2024-03-11T19:34:42.515 INFO:tasks.workunit.client.0.smithi090.stdout:1/429: chown d5/da/d1c/l2f 70292 1 2024-03-11T19:34:42.824 INFO:tasks.workunit.client.0.smithi090.stdout:1/430: dread d5/da/f68 [0,4194304] 0 2024-03-11T19:34:43.186 INFO:tasks.workunit.client.0.smithi090.stdout:0/458: readlink d2/l86 0 2024-03-11T19:34:43.257 INFO:tasks.workunit.client.0.smithi090.stdout:6/485: truncate d4/d14/f21 5304880 0 2024-03-11T19:34:43.578 INFO:tasks.workunit.client.0.smithi090.stdout:5/439: write d1/d17/f66 [885501,45889] 0 2024-03-11T19:34:43.722 INFO:tasks.workunit.client.0.smithi090.stdout:7/436: creat d3/dc/d87/f90 x:0 0 0 2024-03-11T19:34:43.940 INFO:tasks.workunit.client.0.smithi090.stdout:0/459: mknod d2/d11/d3f/c90 0 2024-03-11T19:34:44.384 INFO:tasks.workunit.client.0.smithi090.stdout:4/485: dwrite d4/d18/f41 [4194304,4194304] 0 2024-03-11T19:34:44.771 INFO:tasks.workunit.client.0.smithi090.stdout:2/433: write d0/de/d12/d46/d49/f6c [273038,72941] 0 2024-03-11T19:34:44.785 INFO:tasks.workunit.client.0.smithi090.stdout:0/460: creat d2/d11/d15/d29/d43/d58/f91 x:0 0 0 2024-03-11T19:34:44.882 INFO:tasks.workunit.client.0.smithi090.stdout:0/461: chown d2/d11/d15/f19 117184290 1 2024-03-11T19:34:45.185 INFO:tasks.workunit.client.0.smithi090.stdout:4/486: truncate d4/d18/f2a 4906956 0 2024-03-11T19:34:45.423 INFO:tasks.workunit.client.0.smithi090.stdout:1/431: getdents d5 0 2024-03-11T19:34:45.467 INFO:tasks.workunit.client.0.smithi090.stdout:3/411: dwrite d2/d1d/f25 [4194304,4194304] 0 2024-03-11T19:34:45.610 INFO:tasks.workunit.client.0.smithi090.stdout:1/432: chown d5/d31/d4c/c57 556 1 2024-03-11T19:34:45.904 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:34:45.907 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:34:45.953 INFO:tasks.workunit.client.0.smithi090.stdout:2/434: mkdir d0/de/d12/d81 0 2024-03-11T19:34:46.145 INFO:tasks.workunit.client.0.smithi090.stdout:0/462: mknod d2/d11/c92 0 2024-03-11T19:34:46.145 INFO:tasks.workunit.client.0.smithi090.stdout:1/433: mkdir d5/d4e/d54/d8e 0 2024-03-11T19:34:46.215 INFO:tasks.workunit.client.0.smithi090.stdout:0/463: chown d2/d11/d15/d29/d2e/c31 1330649445 1 2024-03-11T19:34:46.311 INFO:tasks.workunit.client.0.smithi090.stdout:3/412: symlink d2/l8b 0 2024-03-11T19:34:46.693 INFO:tasks.workunit.client.0.smithi090.stdout:6/486: write d4/d13/d22/d34/d40/f8c [23208,60407] 0 2024-03-11T19:34:47.075 INFO:tasks.workunit.client.0.smithi090.stdout:4/487: creat d4/d18/fa8 x:0 0 0 2024-03-11T19:34:47.093 INFO:tasks.workunit.client.0.smithi090.stdout:7/437: write d3/f8 [3574196,120938] 0 2024-03-11T19:34:47.286 INFO:tasks.workunit.client.0.smithi090.stdout:3/413: dread d2/d1f/f78 [0,4194304] 0 2024-03-11T19:34:47.313 INFO:tasks.workunit.client.0.smithi090.stdout:6/487: fsync d4/d24/f26 0 2024-03-11T19:34:47.344 INFO:tasks.workunit.client.0.smithi090.stdout:6/488: readlink d4/d24/l71 0 2024-03-11T19:34:47.381 INFO:tasks.workunit.client.0.smithi090.stdout:6/489: write d4/d24/f26 [290670,126448] 0 2024-03-11T19:34:47.470 INFO:tasks.workunit.client.0.smithi090.stdout:2/435: fsync d0/de/f16 0 2024-03-11T19:34:47.787 INFO:tasks.workunit.client.0.smithi090.stdout:0/464: mknod d2/d11/d15/d29/d3d/d7e/c93 0 2024-03-11T19:34:48.167 INFO:tasks.workunit.client.0.smithi090.stdout:3/414: readlink d2/l6e 0 2024-03-11T19:34:48.593 INFO:tasks.workunit.client.0.smithi090.stdout:7/438: mknod d3/dc/c91 0 2024-03-11T19:34:48.615 INFO:tasks.workunit.client.0.smithi090.stdout:7/439: stat d3/dc/d1e/ff 0 2024-03-11T19:34:49.053 INFO:tasks.workunit.client.0.smithi090.stdout:6/490: symlink d4/d13/d64/l9f 0 2024-03-11T19:34:49.281 INFO:tasks.workunit.client.0.smithi090.stdout:3/415: mkdir d2/db/d15/d1a/d75/d8c 0 2024-03-11T19:34:49.351 INFO:tasks.workunit.client.0.smithi090.stdout:6/491: sync 2024-03-11T19:34:49.512 INFO:tasks.workunit.client.0.smithi090.stdout:0/465: mkdir d2/d94 0 2024-03-11T19:34:49.975 INFO:tasks.workunit.client.0.smithi090.stdout:3/416: unlink d2/f1b 0 2024-03-11T19:34:50.002 INFO:tasks.workunit.client.0.smithi090.stdout:6/492: fsync d4/ff 0 2024-03-11T19:34:50.293 INFO:tasks.workunit.client.0.smithi090.stdout:4/488: link d4/d27/d44/f4e d4/d27/d28/d38/d75/fa9 0 2024-03-11T19:34:50.380 INFO:tasks.workunit.client.0.smithi090.stdout:1/434: dwrite d5/da/dd/f2c [4194304,4194304] 0 2024-03-11T19:34:50.500 INFO:tasks.workunit.client.0.smithi090.stdout:9/426: rename d1/dd/d18/d25/l44 to d1/l7d 0 2024-03-11T19:34:50.502 INFO:tasks.workunit.client.0.smithi090.stdout:4/489: chown d4/f46 21 1 2024-03-11T19:34:50.531 INFO:tasks.workunit.client.0.smithi090.stdout:1/435: sync 2024-03-11T19:34:50.846 INFO:tasks.workunit.client.0.smithi090.stdout:3/417: dread d2/db/d15/f37 [0,4194304] 0 2024-03-11T19:34:50.870 INFO:tasks.workunit.client.0.smithi090.stdout:3/418: chown d2/f67 196934020 1 2024-03-11T19:34:51.132 INFO:tasks.workunit.client.0.smithi090.stdout:6/493: symlink d4/d13/d22/d42/d48/d4e/la0 0 2024-03-11T19:34:51.394 INFO:tasks.workunit.client.0.smithi090.stdout:4/490: fdatasync d4/d51/d8c/d6a/d8d/f8e 0 2024-03-11T19:34:51.443 INFO:tasks.workunit.client.0.smithi090.stdout:9/427: read d1/dd/df/d1c/d21/f54 [6300761,78137] 0 2024-03-11T19:34:52.053 INFO:tasks.workunit.client.0.smithi090.stdout:2/436: dwrite d0/fd [0,4194304] 0 2024-03-11T19:34:52.230 INFO:tasks.workunit.client.0.smithi090.stdout:8/487: rename d0/d43/dc/l54 to d0/d43/l9d 0 2024-03-11T19:34:52.326 INFO:tasks.workunit.client.0.smithi090.stdout:4/491: dread d4/d18/f6e [0,4194304] 0 2024-03-11T19:34:52.443 INFO:tasks.workunit.client.0.smithi090.stdout:3/419: dread d2/d1d/f27 [0,4194304] 0 2024-03-11T19:34:52.541 INFO:tasks.workunit.client.0.smithi090.stdout:0/466: write d2/d11/d15/f19 [1705280,43291] 0 2024-03-11T19:34:52.576 INFO:tasks.workunit.client.0.smithi090.stdout:0/467: chown d2/d94 130384649 1 2024-03-11T19:34:53.001 INFO:tasks.workunit.client.0.smithi090.stdout:1/436: link d5/d65/d6b/l6f d5/da/d1c/d3b/d40/d89/l8f 0 2024-03-11T19:34:53.072 INFO:tasks.workunit.client.0.smithi090.stdout:1/437: chown d5/d31/d76/d77 1756403 1 2024-03-11T19:34:53.143 INFO:tasks.workunit.client.0.smithi090.stdout:9/428: symlink d1/d47/d7c/l7e 0 2024-03-11T19:34:53.290 INFO:tasks.workunit.client.0.smithi090.stdout:5/440: rename d1/d26/l3a to d1/d17/d39/d51/d53/d7b/d6f/l96 0 2024-03-11T19:34:53.417 INFO:tasks.workunit.client.0.smithi090.stdout:8/488: creat d0/d43/dc/d90/f9e x:0 0 0 2024-03-11T19:34:53.463 INFO:tasks.workunit.client.0.smithi090.stdout:2/437: truncate d0/de/f52 573727 0 2024-03-11T19:34:53.657 INFO:tasks.workunit.client.0.smithi090.stdout:4/492: fdatasync d4/d51/d8c/d6a/f7f 0 2024-03-11T19:34:54.269 INFO:tasks.workunit.client.0.smithi090.stdout:1/438: unlink d5/da/f44 0 2024-03-11T19:34:54.694 INFO:tasks.workunit.client.0.smithi090.stdout:6/494: write d4/d13/d22/d42/f72 [3655660,54191] 0 2024-03-11T19:34:55.036 INFO:tasks.workunit.client.0.smithi090.stdout:0/468: truncate d2/d11/d15/d29/d43/d58/f65 223822 0 2024-03-11T19:34:55.155 INFO:tasks.workunit.client.0.smithi090.stdout:1/439: fdatasync d5/da/d23/f2a 0 2024-03-11T19:34:55.261 INFO:tasks.workunit.client.0.smithi090.stdout:7/440: rename d3/dc/l62 to d3/d55/l92 0 2024-03-11T19:34:55.851 INFO:tasks.workunit.client.0.smithi090.stdout:6/495: creat d4/d13/d22/d34/d40/fa1 x:0 0 0 2024-03-11T19:34:56.295 INFO:tasks.workunit.client.0.smithi090.stdout:1/440: symlink d5/da/d1c/d6d/l90 0 2024-03-11T19:34:56.626 INFO:tasks.workunit.client.0.smithi090.stdout:9/429: symlink d1/d1d/d6a/l7f 0 2024-03-11T19:34:56.707 INFO:tasks.workunit.client.0.smithi090.stdout:9/430: dread d1/f3 [4194304,4194304] 0 2024-03-11T19:34:56.789 INFO:tasks.workunit.client.0.smithi090.stdout:7/441: symlink d3/dc/d17/d32/d45/d84/l93 0 2024-03-11T19:34:57.131 INFO:tasks.workunit.client.0.smithi090.stdout:0/469: mknod d2/d11/d15/d29/d2e/c95 0 2024-03-11T19:34:57.399 INFO:tasks.workunit.client.0.smithi090.stdout:3/420: write d2/db/f1c [2608192,48674] 0 2024-03-11T19:34:57.544 INFO:tasks.workunit.client.0.smithi090.stdout:9/431: rename d1/dd/f30 to d1/dd/f80 0 2024-03-11T19:34:57.800 INFO:tasks.workunit.client.0.smithi090.stdout:5/441: getdents d1/d17/d39/d51/d53/d64 0 2024-03-11T19:34:58.275 INFO:tasks.workunit.client.0.smithi090.stdout:4/493: write d4/d27/d28/d69/f99 [920756,16378] 0 2024-03-11T19:34:58.892 INFO:tasks.workunit.client.0.smithi090.stdout:2/438: write d0/de/d12/d46/d75/f7c [4551858,76097] 0 2024-03-11T19:34:59.178 INFO:tasks.workunit.client.0.smithi090.stdout:4/494: symlink d4/d18/laa 0 2024-03-11T19:34:59.387 INFO:tasks.workunit.client.0.smithi090.stdout:3/421: getdents d2/db/d15/d1a/d75 0 2024-03-11T19:34:59.549 INFO:tasks.workunit.client.0.smithi090.stdout:4/495: rmdir d4/d27/d28/d38/d75 39 2024-03-11T19:34:59.571 INFO:tasks.workunit.client.0.smithi090.stdout:4/496: chown d4/l49 1849071775 1 2024-03-11T19:35:00.256 INFO:tasks.workunit.client.0.smithi090.stdout:3/422: creat d2/db/d15/d1a/d75/d8c/f8d x:0 0 0 2024-03-11T19:35:00.278 INFO:tasks.workunit.client.0.smithi090.stdout:1/441: write d5/d4e/d54/f6e [566404,35092] 0 2024-03-11T19:35:00.721 INFO:tasks.workunit.client.0.smithi090.stdout:7/442: dwrite d3/dc/d1e/f46 [0,4194304] 0 2024-03-11T19:35:00.880 INFO:tasks.workunit.client.0.smithi090.stdout:1/442: fsync d5/d65/f69 0 2024-03-11T19:35:01.030 INFO:tasks.workunit.client.0.smithi090.stdout:4/497: link d4/f2e d4/d18/fab 0 2024-03-11T19:35:01.348 INFO:tasks.workunit.client.0.smithi090.stdout:1/443: dread d5/f4f [0,4194304] 0 2024-03-11T19:35:01.418 INFO:tasks.workunit.client.0.smithi090.stdout:0/470: dwrite d2/d11/d15/f74 [0,4194304] 0 2024-03-11T19:35:01.553 INFO:tasks.workunit.client.0.smithi090.stdout:5/442: write d1/d17/d39/d51/d53/d7b/f44 [384455,79257] 0 2024-03-11T19:35:01.939 INFO:tasks.workunit.client.0.smithi090.stdout:9/432: write d1/dd/d18/f65 [1419654,44375] 0 2024-03-11T19:35:01.950 INFO:tasks.workunit.client.0.smithi090.stdout:2/439: write d0/d21/d30/f22 [5485373,4333] 0 2024-03-11T19:35:01.999 INFO:tasks.workunit.client.0.smithi090.stdout:8/489: rename d0/l78 to d0/d60/l9f 0 2024-03-11T19:35:02.090 INFO:tasks.workunit.client.0.smithi090.stdout:8/490: dread d0/d43/f79 [0,4194304] 0 2024-03-11T19:35:02.296 INFO:tasks.workunit.client.0.smithi090.stdout:5/443: read d1/f50 [82656,90457] 0 2024-03-11T19:35:02.405 INFO:tasks.workunit.client.0.smithi090.stdout:1/444: truncate d5/d31/d76/f78 797566 0 2024-03-11T19:35:02.730 INFO:tasks.workunit.client.0.smithi090.stdout:7/443: write d3/dc/d17/d32/d71/d5d/f5a [5170973,31710] 0 2024-03-11T19:35:02.840 INFO:tasks.workunit.client.0.smithi090.stdout:8/491: dread d0/d3/d4c/f88 [0,4194304] 0 2024-03-11T19:35:02.878 INFO:tasks.workunit.client.0.smithi090.stdout:1/445: read d5/d31/d36/f70 [201052,76146] 0 2024-03-11T19:35:02.890 INFO:tasks.workunit.client.0.smithi090.stdout:5/444: dread d1/d75/f8c [0,4194304] 0 2024-03-11T19:35:02.916 INFO:tasks.workunit.client.0.smithi090.stdout:2/440: creat d0/de/d12/d46/d4c/d64/f82 x:0 0 0 2024-03-11T19:35:02.948 INFO:tasks.workunit.client.0.smithi090.stdout:7/444: write d3/dc/d1e/f46 [3689246,30144] 0 2024-03-11T19:35:03.149 INFO:tasks.workunit.client.0.smithi090.stdout:5/445: chown d1/d17/d39/d51/d53/d64/c70 321 1 2024-03-11T19:35:03.489 INFO:tasks.workunit.client.0.smithi090.stdout:8/492: symlink d0/d53/d59/d82/la0 0 2024-03-11T19:35:03.521 INFO:tasks.workunit.client.0.smithi090.stdout:0/471: write d2/d11/d3f/f40 [1430716,115861] 0 2024-03-11T19:35:03.601 INFO:tasks.workunit.client.0.smithi090.stdout:1/446: dwrite d5/d4e/d54/f6e [8388608,4194304] 0 2024-03-11T19:35:03.884 INFO:tasks.workunit.client.0.smithi090.stdout:0/472: readlink d2/d11/l4f 0 2024-03-11T19:35:03.889 INFO:tasks.workunit.client.0.smithi090.stdout:3/423: dwrite d2/d1d/f25 [4194304,4194304] 0 2024-03-11T19:35:03.912 INFO:tasks.workunit.client.0.smithi090.stdout:2/441: read d0/de/d12/d46/d49/f6c [150681,65659] 0 2024-03-11T19:35:04.040 INFO:tasks.workunit.client.0.smithi090.stdout:7/445: write d3/dc/d1e/f57 [1015757,79557] 0 2024-03-11T19:35:04.101 INFO:tasks.workunit.client.0.smithi090.stdout:6/496: rename d4/d13/d58/f87 to d4/d13/fa2 0 2024-03-11T19:35:04.328 INFO:tasks.workunit.client.0.smithi090.stdout:1/447: creat d5/d31/d4c/f91 x:0 0 0 2024-03-11T19:35:04.455 INFO:tasks.workunit.client.0.smithi090.stdout:3/424: write d2/d1d/f25 [1472538,90988] 0 2024-03-11T19:35:04.492 INFO:tasks.workunit.client.0.smithi090.stdout:5/446: dread d1/d26/f6 [0,4194304] 0 2024-03-11T19:35:04.602 INFO:tasks.workunit.client.0.smithi090.stdout:4/498: rename d4/d27/f3b to d4/d51/d8c/fac 0 2024-03-11T19:35:04.737 INFO:tasks.workunit.client.0.smithi090.stdout:2/442: truncate d0/de/d1f/f1c 6633541 0 2024-03-11T19:35:04.771 INFO:tasks.workunit.client.0.smithi090.stdout:8/493: dwrite d0/f17 [0,4194304] 0 2024-03-11T19:35:04.802 INFO:tasks.workunit.client.0.smithi090.stdout:1/448: sync 2024-03-11T19:35:04.909 INFO:tasks.workunit.client.0.smithi090.stdout:9/433: rename d1/dd/d18/d3a/f7b to d1/dd/df/d1c/f81 0 2024-03-11T19:35:04.921 INFO:tasks.workunit.client.0.smithi090.stdout:4/499: sync 2024-03-11T19:35:04.981 INFO:tasks.workunit.client.0.smithi090.stdout:1/449: stat d5/da/l2e 0 2024-03-11T19:35:05.205 INFO:tasks.workunit.client.0.smithi090.stdout:3/425: mknod d2/d1d/c8e 0 2024-03-11T19:35:05.225 INFO:tasks.workunit.client.0.smithi090.stdout:6/497: truncate d4/d13/d22/d34/d40/fa1 220011 0 2024-03-11T19:35:05.282 INFO:tasks.workunit.client.0.smithi090.stdout:3/426: chown d2/db/d74 0 1 2024-03-11T19:35:05.446 INFO:tasks.workunit.client.0.smithi090.stdout:5/447: dread d1/d17/d39/d51/d53/d7b/f5b [0,4194304] 0 2024-03-11T19:35:05.511 INFO:tasks.workunit.client.0.smithi090.stdout:8/494: mkdir d0/d43/dc/da1 0 2024-03-11T19:35:05.566 INFO:tasks.workunit.client.0.smithi090.stdout:0/473: getdents d2/d11/d15/d36 0 2024-03-11T19:35:05.591 INFO:tasks.workunit.client.0.smithi090.stdout:9/434: creat d1/dd/d18/d25/f82 x:0 0 0 2024-03-11T19:35:05.618 INFO:tasks.workunit.client.0.smithi090.stdout:2/443: truncate d0/de/f17 3136277 0 2024-03-11T19:35:05.804 INFO:tasks.workunit.client.0.smithi090.stdout:2/444: dread d0/de/d12/f1d [0,4194304] 0 2024-03-11T19:35:05.833 INFO:tasks.workunit.client.0.smithi090.stdout:6/498: unlink d4/d13/d22/d3a/d67/l6e 0 2024-03-11T19:35:06.444 INFO:tasks.workunit.client.0.smithi090.stdout:3/427: mknod d2/d2f/c8f 0 2024-03-11T19:35:06.470 INFO:tasks.workunit.client.0.smithi090.stdout:4/500: unlink d4/d27/d44/f4e 0 2024-03-11T19:35:06.542 INFO:tasks.workunit.client.0.smithi090.stdout:9/435: creat d1/d47/d7c/f83 x:0 0 0 2024-03-11T19:35:06.551 INFO:tasks.workunit.client.0.smithi090.stdout:7/446: link d3/dc/d17/d34/c6e d3/dc/d17/d34/d4d/c94 0 2024-03-11T19:35:06.595 INFO:tasks.workunit.client.0.smithi090.stdout:1/450: creat d5/da/f92 x:0 0 0 2024-03-11T19:35:06.714 INFO:tasks.workunit.client.0.smithi090.stdout:7/447: dread d3/d55/f6a [0,4194304] 0 2024-03-11T19:35:07.155 INFO:tasks.workunit.client.0.smithi090.stdout:2/445: mkdir d0/d21/d30/d83 0 2024-03-11T19:35:07.379 INFO:tasks.workunit.client.0.smithi090.stdout:9/436: unlink d1/d1d/l2f 0 2024-03-11T19:35:07.814 INFO:tasks.workunit.client.0.smithi090.stdout:4/501: truncate d4/fa 5587745 0 2024-03-11T19:35:07.985 INFO:tasks.workunit.client.0.smithi090.stdout:6/499: link d4/d13/l4a d4/d14/d92/la3 0 2024-03-11T19:35:08.079 INFO:tasks.workunit.client.0.smithi090.stdout:2/446: truncate d0/de/d12/f5e 566371 0 2024-03-11T19:35:08.444 INFO:tasks.workunit.client.0.smithi090.stdout:8/495: write d0/d3/d3e/f92 [889716,37897] 0 2024-03-11T19:35:08.467 INFO:tasks.workunit.client.0.smithi090.stdout:5/448: dwrite d1/d26/d4d/f55 [0,4194304] 0 2024-03-11T19:35:08.959 INFO:tasks.workunit.client.0.smithi090.stdout:6/500: truncate d4/f62 2647534 0 2024-03-11T19:35:09.042 INFO:tasks.workunit.client.0.smithi090.stdout:9/437: mknod d1/d49/c84 0 2024-03-11T19:35:10.216 INFO:tasks.workunit.client.0.smithi090.stdout:0/474: rename d2/d11/f79 to d2/d11/d15/d29/d3d/d7b/f96 0 2024-03-11T19:35:10.260 INFO:tasks.workunit.client.0.smithi090.stdout:1/451: creat d5/da/f93 x:0 0 0 2024-03-11T19:35:10.301 INFO:tasks.workunit.client.0.smithi090.stdout:8/496: truncate d0/d3/f37 1272009 0 2024-03-11T19:35:10.430 INFO:tasks.workunit.client.0.smithi090.stdout:4/502: creat d4/d27/d28/d7a/d9b/fad x:0 0 0 2024-03-11T19:35:10.708 INFO:tasks.workunit.client.0.smithi090.stdout:3/428: dwrite d2/d1f/f35 [0,4194304] 0 2024-03-11T19:35:11.601 INFO:tasks.workunit.client.0.smithi090.stdout:8/497: read - d0/d43/dc/f7e zero size 2024-03-11T19:35:11.832 INFO:tasks.workunit.client.0.smithi090.stdout:2/447: dwrite d0/de/d12/f1a [0,4194304] 0 2024-03-11T19:35:12.224 INFO:tasks.workunit.client.0.smithi090.stdout:7/448: rename d3/d55/f7d to d3/dc/d17/d32/d71/d80/f95 0 2024-03-11T19:35:12.232 INFO:tasks.workunit.client.0.smithi090.stdout:2/448: dwrite d0/de/d12/f1a [4194304,4194304] 0 2024-03-11T19:35:12.671 INFO:tasks.workunit.client.0.smithi090.stdout:9/438: creat d1/dd/df/f85 x:0 0 0 2024-03-11T19:35:12.870 INFO:tasks.workunit.client.0.smithi090.stdout:6/501: rename d4/d14/d44 to d4/d13/d22/d34/d40/da4 0 2024-03-11T19:35:13.044 INFO:tasks.workunit.client.0.smithi090.stdout:2/449: mknod d0/de/d1f/c84 0 2024-03-11T19:35:13.140 INFO:tasks.workunit.client.0.smithi090.stdout:8/498: mkdir d0/d3/d98/da2 0 2024-03-11T19:35:13.933 INFO:tasks.workunit.client.0.smithi090.stdout:3/429: truncate d2/d1d/f25 1286538 0 2024-03-11T19:35:14.025 INFO:tasks.workunit.client.0.smithi090.stdout:8/499: creat d0/d43/dc/fa3 x:0 0 0 2024-03-11T19:35:14.625 INFO:tasks.workunit.client.0.smithi090.stdout:6/502: creat d4/d13/d22/d34/d83/d9c/fa5 x:0 0 0 2024-03-11T19:35:14.654 INFO:tasks.workunit.client.0.smithi090.stdout:1/452: write d5/f1b [8262193,66406] 0 2024-03-11T19:35:14.736 INFO:tasks.workunit.client.0.smithi090.stdout:3/430: read d2/db/f24 [3744556,89812] 0 2024-03-11T19:35:15.118 INFO:tasks.workunit.client.0.smithi090.stdout:2/450: creat d0/d21/d30/f85 x:0 0 0 2024-03-11T19:35:15.436 INFO:tasks.workunit.client.0.smithi090.stdout:1/453: mkdir d5/d4e/d54/d94 0 2024-03-11T19:35:15.451 INFO:tasks.workunit.client.0.smithi090.stdout:3/431: mknod d2/db/d15/d49/c90 0 2024-03-11T19:35:15.692 INFO:tasks.workunit.client.0.smithi090.stdout:0/475: rename d2/d11/d15/d29/d6b/d78/f85 to d2/f97 0 2024-03-11T19:35:15.770 INFO:tasks.workunit.client.0.smithi090.stdout:6/503: truncate d4/d24/f5f 1854121 0 2024-03-11T19:35:15.949 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:35:15.952 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:35:15.974 INFO:tasks.workunit.client.0.smithi090.stdout:1/454: dread d5/da/d23/f2d [0,4194304] 0 2024-03-11T19:35:16.179 INFO:tasks.workunit.client.0.smithi090.stdout:3/432: dread d2/db/d15/d49/f7f [0,4194304] 0 2024-03-11T19:35:16.198 INFO:tasks.workunit.client.0.smithi090.stdout:4/503: rename d4/d51/d8c/l83 to d4/d51/d8c/d89/lae 0 2024-03-11T19:35:17.144 INFO:tasks.workunit.client.0.smithi090.stdout:6/504: creat d4/d13/d22/d34/d40/d7d/fa6 x:0 0 0 2024-03-11T19:35:17.354 INFO:tasks.workunit.client.0.smithi090.stdout:6/505: read - d4/d13/d22/d34/d83/d9c/fa5 zero size 2024-03-11T19:35:17.411 INFO:tasks.workunit.client.0.smithi090.stdout:9/439: dwrite d1/d1d/f67 [0,4194304] 0 2024-03-11T19:35:17.553 INFO:tasks.workunit.client.0.smithi090.stdout:1/455: truncate d5/d31/d4c/f73 323471 0 2024-03-11T19:35:17.570 INFO:tasks.workunit.client.0.smithi090.stdout:3/433: creat d2/de/f91 x:0 0 0 2024-03-11T19:35:17.824 INFO:tasks.workunit.client.0.smithi090.stdout:6/506: rmdir d4/d14/d8a/d74 39 2024-03-11T19:35:17.928 INFO:tasks.workunit.client.0.smithi090.stdout:0/476: getdents d2/d11/d15/d29/d43/d58 0 2024-03-11T19:35:17.996 INFO:tasks.workunit.client.0.smithi090.stdout:9/440: creat d1/dd/d18/d3a/f86 x:0 0 0 2024-03-11T19:35:18.103 INFO:tasks.workunit.client.0.smithi090.stdout:1/456: creat d5/d31/d36/f95 x:0 0 0 2024-03-11T19:35:18.155 INFO:tasks.workunit.client.0.smithi090.stdout:1/457: write d5/d31/d36/f95 [432050,20790] 0 2024-03-11T19:35:18.197 INFO:tasks.workunit.client.0.smithi090.stdout:1/458: chown d5/da/d23 31 1 2024-03-11T19:35:18.239 INFO:tasks.workunit.client.0.smithi090.stdout:8/500: write d0/d53/f6e [4201560,5718] 0 2024-03-11T19:35:18.323 INFO:tasks.workunit.client.0.smithi090.stdout:6/507: truncate d4/d13/d22/d34/f6d 1599308 0 2024-03-11T19:35:18.484 INFO:tasks.workunit.client.0.smithi090.stdout:5/449: rename d1/f50 to d1/f97 0 2024-03-11T19:35:18.852 INFO:tasks.workunit.client.0.smithi090.stdout:9/441: truncate d1/f9 2001512 0 2024-03-11T19:35:18.980 INFO:tasks.workunit.client.0.smithi090.stdout:5/450: sync 2024-03-11T19:35:19.024 INFO:tasks.workunit.client.0.smithi090.stdout:6/508: dread d4/d13/d22/d3a/f54 [4194304,4194304] 0 2024-03-11T19:35:19.033 INFO:tasks.workunit.client.0.smithi090.stdout:8/501: creat d0/d3/d98/da2/fa4 x:0 0 0 2024-03-11T19:35:19.734 INFO:tasks.workunit.client.0.smithi090.stdout:5/451: creat d1/d17/d39/d51/d53/f98 x:0 0 0 2024-03-11T19:35:19.735 INFO:tasks.workunit.client.0.smithi090.stdout:8/502: truncate d0/d43/dc/fd 147524 0 2024-03-11T19:35:19.852 INFO:tasks.workunit.client.0.smithi090.stdout:5/452: sync 2024-03-11T19:35:20.144 INFO:tasks.workunit.client.0.smithi090.stdout:8/503: creat d0/d3/d3e/fa5 x:0 0 0 2024-03-11T19:35:20.402 INFO:tasks.workunit.client.0.smithi090.stdout:6/509: getdents d4/d14/d25 0 2024-03-11T19:35:21.108 INFO:tasks.workunit.client.0.smithi090.stdout:7/449: rename d3/dc/d17 to d3/dc/d1e/d7c/d96 0 2024-03-11T19:35:21.185 INFO:tasks.workunit.client.0.smithi090.stdout:2/451: rename d0 to d0/d21/d86 22 2024-03-11T19:35:21.287 INFO:tasks.workunit.client.0.smithi090.stdout:3/434: dwrite d2/db/d15/d1a/d75/d8c/f8d [0,4194304] 0 2024-03-11T19:35:21.301 INFO:tasks.workunit.client.0.smithi090.stdout:0/477: dwrite d2/f1c [4194304,4194304] 0 2024-03-11T19:35:21.618 INFO:tasks.workunit.client.0.smithi090.stdout:9/442: write d1/f79 [209636,37888] 0 2024-03-11T19:35:21.653 INFO:tasks.workunit.client.0.smithi090.stdout:9/443: chown d1/dd/df/d1c/d21/d2a 486 1 2024-03-11T19:35:21.764 INFO:tasks.workunit.client.0.smithi090.stdout:4/504: rename d4/d18/laa to d4/d27/d28/d38/d64/laf 0 2024-03-11T19:35:21.933 INFO:tasks.workunit.client.0.smithi090.stdout:2/452: readlink d0/de/l51 0 2024-03-11T19:35:22.183 INFO:tasks.workunit.client.0.smithi090.stdout:5/453: dwrite d1/d17/d39/d51/d53/d64/f85 [0,4194304] 0 2024-03-11T19:35:22.376 INFO:tasks.workunit.client.0.smithi090.stdout:7/450: unlink d3/dc/d1e/d7c/d96/d32/d67/l75 0 2024-03-11T19:35:22.405 INFO:tasks.workunit.client.0.smithi090.stdout:1/459: rename d5/da/d1c/f30 to d5/d4e/d54/d94/f96 0 2024-03-11T19:35:22.442 INFO:tasks.workunit.client.0.smithi090.stdout:6/510: write d4/d14/d92/f99 [652188,92052] 0 2024-03-11T19:35:22.778 INFO:tasks.workunit.client.0.smithi090.stdout:5/454: mkdir d1/d26/d4d/d99 0 2024-03-11T19:35:23.034 INFO:tasks.workunit.client.0.smithi090.stdout:9/444: mknod d1/dd/d18/d3a/d70/c87 0 2024-03-11T19:35:23.099 INFO:tasks.workunit.client.0.smithi090.stdout:5/455: dread d1/d26/d19/f21 [0,4194304] 0 2024-03-11T19:35:23.275 INFO:tasks.workunit.client.0.smithi090.stdout:5/456: dread d1/d17/d39/d51/d53/d64/f85 [0,4194304] 0 2024-03-11T19:35:23.453 INFO:tasks.workunit.client.0.smithi090.stdout:5/457: sync 2024-03-11T19:35:23.616 INFO:tasks.workunit.client.0.smithi090.stdout:1/460: fsync d5/f15 0 2024-03-11T19:35:24.011 INFO:tasks.workunit.client.0.smithi090.stdout:8/504: rename d0/d3/c5c to d0/d53/d59/ca6 0 2024-03-11T19:35:24.059 INFO:tasks.workunit.client.0.smithi090.stdout:7/451: dread d3/dc/d1e/d7c/d96/d32/d71/d5d/f25 [0,4194304] 0 2024-03-11T19:35:24.189 INFO:tasks.workunit.client.0.smithi090.stdout:3/435: write d2/db/f24 [5068786,26711] 0 2024-03-11T19:35:24.258 INFO:tasks.workunit.client.0.smithi090.stdout:0/478: write d2/d11/d3f/f76 [4046141,74452] 0 2024-03-11T19:35:24.726 INFO:tasks.workunit.client.0.smithi090.stdout:2/453: dwrite d0/d21/f6f [0,4194304] 0 2024-03-11T19:35:25.610 INFO:tasks.workunit.client.0.smithi090.stdout:7/452: mknod d3/dc/d1e/d7c/d96/d32/d71/d5d/c97 0 2024-03-11T19:35:25.777 INFO:tasks.workunit.client.0.smithi090.stdout:0/479: creat d2/d11/d15/d29/d43/f98 x:0 0 0 2024-03-11T19:35:26.132 INFO:tasks.workunit.client.0.smithi090.stdout:6/511: dwrite d4/d13/d22/d3a/d67/f9d [0,4194304] 0 2024-03-11T19:35:26.149 INFO:tasks.workunit.client.0.smithi090.stdout:4/505: getdents d4/d27/d44/d66 0 2024-03-11T19:35:26.308 INFO:tasks.workunit.client.0.smithi090.stdout:6/512: readlink d4/d14/d25/l86 0 2024-03-11T19:35:26.474 INFO:tasks.workunit.client.0.smithi090.stdout:1/461: mkdir d5/da/d97 0 2024-03-11T19:35:26.495 INFO:tasks.workunit.client.0.smithi090.stdout:1/462: chown d5/f1b 5 1 2024-03-11T19:35:26.520 INFO:tasks.workunit.client.0.smithi090.stdout:9/445: write d1/dd/df/f4d [221393,49678] 0 2024-03-11T19:35:26.945 INFO:tasks.workunit.client.0.smithi090.stdout:3/436: fdatasync d2/d45/f89 0 2024-03-11T19:35:27.136 INFO:tasks.workunit.client.0.smithi090.stdout:7/453: fdatasync d3/dc/d29/f4e 0 2024-03-11T19:35:27.668 INFO:tasks.workunit.client.0.smithi090.stdout:8/505: dwrite d0/f47 [0,4194304] 0 2024-03-11T19:35:27.780 INFO:tasks.workunit.client.0.smithi090.stdout:2/454: creat d0/de/d12/d81/f87 x:0 0 0 2024-03-11T19:35:27.849 INFO:tasks.workunit.client.0.smithi090.stdout:6/513: fsync d4/d13/d22/d34/d37/f8e 0 2024-03-11T19:35:28.024 INFO:tasks.workunit.client.0.smithi090.stdout:1/463: chown d5/d31/c3e 1728 1 2024-03-11T19:35:28.227 INFO:tasks.workunit.client.0.smithi090.stdout:5/458: rename d1/d17/d39/d51/d53/d7b/d3b/d63 to d1/d17/d39/d51/d9a 0 2024-03-11T19:35:28.255 INFO:tasks.workunit.client.0.smithi090.stdout:5/459: chown d1/d26/d19/f21 32 1 2024-03-11T19:35:28.292 INFO:tasks.workunit.client.0.smithi090.stdout:5/460: chown d1/c81 3 1 2024-03-11T19:35:29.075 INFO:tasks.workunit.client.0.smithi090.stdout:3/437: read d2/db/d15/f48 [140859,98381] 0 2024-03-11T19:35:30.044 INFO:tasks.workunit.client.0.smithi090.stdout:6/514: dread d4/d24/f3f [0,4194304] 0 2024-03-11T19:35:30.135 INFO:tasks.workunit.client.0.smithi090.stdout:4/506: dwrite d4/d27/d44/d74/f84 [0,4194304] 0 2024-03-11T19:35:30.276 INFO:tasks.workunit.client.0.smithi090.stdout:3/438: unlink d2/db/f7e 0 2024-03-11T19:35:30.319 INFO:tasks.workunit.client.0.smithi090.stdout:4/507: chown d4/d27/d44/f94 431159386 1 2024-03-11T19:35:30.815 INFO:tasks.workunit.client.0.smithi090.stdout:8/506: truncate d0/d3/f37 1761011 0 2024-03-11T19:35:31.300 INFO:tasks.workunit.client.0.smithi090.stdout:1/464: unlink d5/da/d23/l33 0 2024-03-11T19:35:31.309 INFO:tasks.workunit.client.0.smithi090.stdout:1/465: chown d5 1105652 1 2024-03-11T19:35:31.523 INFO:tasks.workunit.client.0.smithi090.stdout:6/515: mknod d4/d13/d22/d42/d48/d4e/ca7 0 2024-03-11T19:35:31.567 INFO:tasks.workunit.client.0.smithi090.stdout:6/516: readlink d4/d13/d22/d34/d40/l84 0 2024-03-11T19:35:31.690 INFO:tasks.workunit.client.0.smithi090.stdout:3/439: dread d2/db/f24 [4194304,4194304] 0 2024-03-11T19:35:31.804 INFO:tasks.workunit.client.0.smithi090.stdout:0/480: getdents d2/d11/d15/d29/d6b/d78 0 2024-03-11T19:35:32.221 INFO:tasks.workunit.client.0.smithi090.stdout:7/454: symlink d3/l98 0 2024-03-11T19:35:32.246 INFO:tasks.workunit.client.0.smithi090.stdout:1/466: creat d5/d4e/d79/f98 x:0 0 0 2024-03-11T19:35:32.347 INFO:tasks.workunit.client.0.smithi090.stdout:6/517: readlink d4/d13/d22/d42/l55 0 2024-03-11T19:35:32.347 INFO:tasks.workunit.client.0.smithi090.stdout:9/446: dwrite d1/dd/f80 [0,4194304] 0 2024-03-11T19:35:32.475 INFO:tasks.workunit.client.0.smithi090.stdout:3/440: fsync d2/f67 0 2024-03-11T19:35:32.551 INFO:tasks.workunit.client.0.smithi090.stdout:0/481: mknod d2/d11/d15/d29/d43/d58/c99 0 2024-03-11T19:35:32.572 INFO:tasks.workunit.client.0.smithi090.stdout:1/467: readlink d5/d4e/l61 0 2024-03-11T19:35:32.760 INFO:tasks.workunit.client.0.smithi090.stdout:6/518: chown d4/d14/f6a 13211 1 2024-03-11T19:35:32.885 INFO:tasks.workunit.client.0.smithi090.stdout:5/461: dwrite d1/d17/d39/d51/d53/d7b/d3b/f5a [0,4194304] 0 2024-03-11T19:35:32.909 INFO:tasks.workunit.client.0.smithi090.stdout:4/508: rename d4/d27/f35 to d4/d18/fb0 0 2024-03-11T19:35:33.094 INFO:tasks.workunit.client.0.smithi090.stdout:6/519: chown d4/d13/d22/d3a/l4b 261925 1 2024-03-11T19:35:33.124 INFO:tasks.workunit.client.0.smithi090.stdout:6/520: chown d4/d13 161192 1 2024-03-11T19:35:33.167 INFO:tasks.workunit.client.0.smithi090.stdout:6/521: chown d4/d13/d22/d34/d40/da4/f9a 7 1 2024-03-11T19:35:33.515 INFO:tasks.workunit.client.0.smithi090.stdout:2/455: dwrite d0/de/d12/f5e [0,4194304] 0 2024-03-11T19:35:33.894 INFO:tasks.workunit.client.0.smithi090.stdout:5/462: mknod d1/d75/c9b 0 2024-03-11T19:35:33.932 INFO:tasks.workunit.client.0.smithi090.stdout:0/482: dwrite d2/d11/d15/d29/d43/f98 [0,4194304] 0 2024-03-11T19:35:34.142 INFO:tasks.workunit.client.0.smithi090.stdout:0/483: chown d2/d11/f24 5837486 1 2024-03-11T19:35:34.395 INFO:tasks.workunit.client.0.smithi090.stdout:7/455: mkdir d3/dc/d1e/d7c/d96/d32/d7b/d99 0 2024-03-11T19:35:34.501 INFO:tasks.workunit.client.0.smithi090.stdout:3/441: rename d2/db/d15/d69/f6f to d2/db/f92 0 2024-03-11T19:35:34.555 INFO:tasks.workunit.client.0.smithi090.stdout:3/442: stat d2/db/d15/d1a/f33 0 2024-03-11T19:35:34.802 INFO:tasks.workunit.client.0.smithi090.stdout:8/507: dwrite d0/d3/f68 [0,4194304] 0 2024-03-11T19:35:34.878 INFO:tasks.workunit.client.0.smithi090.stdout:3/443: sync 2024-03-11T19:35:35.723 INFO:tasks.workunit.client.0.smithi090.stdout:7/456: fsync d3/dc/d1e/d7c/d96/d32/d71/d5d/f40 0 2024-03-11T19:35:36.394 INFO:tasks.workunit.client.0.smithi090.stdout:7/457: truncate d3/dc/d1e/d7c/d96/d32/f3f 4321442 0 2024-03-11T19:35:36.404 INFO:tasks.workunit.client.0.smithi090.stdout:1/468: write d5/da/d23/f2a [4971118,100090] 0 2024-03-11T19:35:36.558 INFO:tasks.workunit.client.0.smithi090.stdout:3/444: fdatasync d2/ff 0 2024-03-11T19:35:36.815 INFO:tasks.workunit.client.0.smithi090.stdout:4/509: getdents d4/d27/d28 0 2024-03-11T19:35:37.035 INFO:tasks.workunit.client.0.smithi090.stdout:9/447: write d1/dd/d18/f38 [795062,119333] 0 2024-03-11T19:35:37.637 INFO:tasks.workunit.client.0.smithi090.stdout:6/522: dwrite d4/d13/d22/d3a/f54 [8388608,4194304] 0 2024-03-11T19:35:37.640 INFO:tasks.workunit.client.0.smithi090.stdout:0/484: rename d2/d11/f47 to d2/d11/d15/d29/f9a 0 2024-03-11T19:35:38.241 INFO:tasks.workunit.client.0.smithi090.stdout:2/456: dwrite d0/de/d1f/f40 [0,4194304] 0 2024-03-11T19:35:38.397 INFO:tasks.workunit.client.0.smithi090.stdout:5/463: unlink d1/f60 0 2024-03-11T19:35:38.649 INFO:tasks.workunit.client.0.smithi090.stdout:8/508: rename d0/d3/d4c/c6d to d0/d43/dc/d90/ca7 0 2024-03-11T19:35:38.734 INFO:tasks.workunit.client.0.smithi090.stdout:6/523: creat d4/d24/fa8 x:0 0 0 2024-03-11T19:35:39.471 INFO:tasks.workunit.client.0.smithi090.stdout:4/510: fdatasync d4/d27/d28/d38/d75/fa9 0 2024-03-11T19:35:39.717 INFO:tasks.workunit.client.0.smithi090.stdout:5/464: rmdir d1/d26/d4d 39 2024-03-11T19:35:39.837 INFO:tasks.workunit.client.0.smithi090.stdout:9/448: rmdir d1/dd/df/d1c/d6d 0 2024-03-11T19:35:39.975 INFO:tasks.workunit.client.0.smithi090.stdout:1/469: dwrite d5/d31/d36/f95 [0,4194304] 0 2024-03-11T19:35:40.097 INFO:tasks.workunit.client.0.smithi090.stdout:6/524: rename d4/d14/l19 to d4/d24/d80/la9 0 2024-03-11T19:35:40.507 INFO:tasks.workunit.client.0.smithi090.stdout:3/445: dwrite d2/d45/f70 [0,4194304] 0 2024-03-11T19:35:40.766 INFO:tasks.workunit.client.0.smithi090.stdout:4/511: dread - d4/d27/d28/d7a/d9b/fad zero size 2024-03-11T19:35:40.924 INFO:tasks.workunit.client.0.smithi090.stdout:7/458: dwrite d3/dc/d1e/d7c/d96/d32/d71/d5d/f4c [0,4194304] 0 2024-03-11T19:35:41.106 INFO:tasks.workunit.client.0.smithi090.stdout:7/459: dread - d3/dc/d29/f4e zero size 2024-03-11T19:35:41.384 INFO:tasks.workunit.client.0.smithi090.stdout:7/460: rename d3 to d3/dc/d1e/d7c/d96/d7f/d9a 22 2024-03-11T19:35:41.400 INFO:tasks.workunit.client.0.smithi090.stdout:9/449: dread d1/dd/d18/d32/f34 [0,4194304] 0 2024-03-11T19:35:41.516 INFO:tasks.workunit.client.0.smithi090.stdout:7/461: chown d3/dc/d1e/d7c/d96/d32/d71/d5d/f25 38861886 1 2024-03-11T19:35:41.622 INFO:tasks.workunit.client.0.smithi090.stdout:0/485: link d2/f1b d2/d11/d6e/f9b 0 2024-03-11T19:35:42.439 INFO:tasks.workunit.client.0.smithi090.stdout:8/509: write d0/d53/f6e [370699,103230] 0 2024-03-11T19:35:42.804 INFO:tasks.workunit.client.0.smithi090.stdout:9/450: fsync d1/dd/df/d1c/d21/d2a/f50 0 2024-03-11T19:35:43.034 INFO:tasks.workunit.client.0.smithi090.stdout:7/462: rename d3/dc/d1e/f12 to d3/d49/d8e/f9b 0 2024-03-11T19:35:43.239 INFO:tasks.workunit.client.0.smithi090.stdout:0/486: symlink d2/d11/d3a/l9c 0 2024-03-11T19:35:43.879 INFO:tasks.workunit.client.0.smithi090.stdout:2/457: dwrite d0/f7 [0,4194304] 0 2024-03-11T19:35:43.994 INFO:tasks.workunit.client.0.smithi090.stdout:9/451: mkdir d1/dd/df/d1c/d88 0 2024-03-11T19:35:44.041 INFO:tasks.workunit.client.0.smithi090.stdout:7/463: fsync d3/dc/d1e/d7c/d96/d32/f7a 0 2024-03-11T19:35:44.464 INFO:tasks.workunit.client.0.smithi090.stdout:0/487: rename d2/d11/d3a/f53 to d2/d11/d15/d29/d3d/d7e/f9d 0 2024-03-11T19:35:44.494 INFO:tasks.workunit.client.0.smithi090.stdout:6/525: write d4/d13/f1f [338864,91890] 0 2024-03-11T19:35:44.524 INFO:tasks.workunit.client.0.smithi090.stdout:0/488: stat d2/d11/l13 0 2024-03-11T19:35:44.599 INFO:tasks.workunit.client.0.smithi090.stdout:5/465: getdents d1/d75 0 2024-03-11T19:35:44.640 INFO:tasks.workunit.client.0.smithi090.stdout:5/466: stat d1/d26/c7a 0 2024-03-11T19:35:44.763 INFO:tasks.workunit.client.0.smithi090.stdout:3/446: unlink d2/c83 0 2024-03-11T19:35:45.098 INFO:tasks.workunit.client.0.smithi090.stdout:2/458: dread d0/de/d12/d46/f5b [0,4194304] 0 2024-03-11T19:35:45.291 INFO:tasks.workunit.client.0.smithi090.stdout:9/452: dread d1/dd/d18/d3a/d57/f5b [0,4194304] 0 2024-03-11T19:35:45.519 INFO:tasks.workunit.client.0.smithi090.stdout:0/489: fdatasync d2/d11/d3f/f76 0 2024-03-11T19:35:45.606 INFO:tasks.workunit.client.0.smithi090.stdout:2/459: dread d0/de/d12/d46/d4c/f53 [0,4194304] 0 2024-03-11T19:35:45.688 INFO:tasks.workunit.client.0.smithi090.stdout:6/526: rename d4/d14/d8a/c7a to d4/d13/d22/d3a/d67/caa 0 2024-03-11T19:35:45.973 INFO:tasks.workunit.client.0.smithi090.stdout:3/447: symlink d2/db/d15/d1a/d3c/l93 0 2024-03-11T19:35:45.994 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:35:45.997 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:35:46.001 INFO:tasks.workunit.client.0.smithi090.stdout:3/448: chown d2/d1f/f78 14 1 2024-03-11T19:35:46.161 INFO:tasks.workunit.client.0.smithi090.stdout:7/464: mkdir d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/d9c 0 2024-03-11T19:35:46.440 INFO:tasks.workunit.client.0.smithi090.stdout:9/453: symlink d1/dd/d18/d25/d4b/l89 0 2024-03-11T19:35:46.557 INFO:tasks.workunit.client.0.smithi090.stdout:6/527: read - d4/d14/d8a/f96 zero size 2024-03-11T19:35:46.759 INFO:tasks.workunit.client.0.smithi090.stdout:4/512: dwrite d4/ff [0,4194304] 0 2024-03-11T19:35:46.871 INFO:tasks.workunit.client.0.smithi090.stdout:2/460: rename d0/de/d12/d46/d5d/d63/f72 to d0/de/d12/d46/d5d/d63/f88 0 2024-03-11T19:35:46.915 INFO:tasks.workunit.client.0.smithi090.stdout:8/510: write d0/d43/dc/d90/f9e [713900,76755] 0 2024-03-11T19:35:47.003 INFO:tasks.workunit.client.0.smithi090.stdout:7/465: creat d3/dc/d1e/d7c/d96/d32/d45/f9d x:0 0 0 2024-03-11T19:35:47.085 INFO:tasks.workunit.client.0.smithi090.stdout:3/449: read d2/f29 [4864596,43480] 0 2024-03-11T19:35:47.219 INFO:tasks.workunit.client.0.smithi090.stdout:2/461: chown d0/d21/d30/d25/f5c 1037198034 1 2024-03-11T19:35:47.225 INFO:tasks.workunit.client.0.smithi090.stdout:3/450: chown d2/db/d74 338800827 1 2024-03-11T19:35:47.234 INFO:tasks.workunit.client.0.smithi090.stdout:1/470: dwrite d5/d31/d76/f78 [0,4194304] 0 2024-03-11T19:35:47.247 INFO:tasks.workunit.client.0.smithi090.stdout:8/511: write d0/d43/dc/d90/f9e [1084295,105849] 0 2024-03-11T19:35:47.473 INFO:tasks.workunit.client.0.smithi090.stdout:5/467: unlink d1/d17/d39/d51/d53/d7b/d6f/c90 0 2024-03-11T19:35:47.554 INFO:tasks.workunit.client.0.smithi090.stdout:8/512: chown d0/d43/l1e 3 1 2024-03-11T19:35:47.698 INFO:tasks.workunit.client.0.smithi090.stdout:7/466: creat d3/dc/d1e/d7c/d96/f9e x:0 0 0 2024-03-11T19:35:47.906 INFO:tasks.workunit.client.0.smithi090.stdout:3/451: creat d2/d45/d55/f94 x:0 0 0 2024-03-11T19:35:47.926 INFO:tasks.workunit.client.0.smithi090.stdout:5/468: sync 2024-03-11T19:35:48.243 INFO:tasks.workunit.client.0.smithi090.stdout:7/467: fdatasync d3/dc/d1e/ff 0 2024-03-11T19:35:48.596 INFO:tasks.workunit.client.0.smithi090.stdout:6/528: rename d4/d14/d8a/c5d to d4/cab 0 2024-03-11T19:35:48.607 INFO:tasks.workunit.client.0.smithi090.stdout:2/462: symlink d0/d21/d30/d83/l89 0 2024-03-11T19:35:48.691 INFO:tasks.workunit.client.0.smithi090.stdout:1/471: symlink d5/da/d1c/d3b/d40/d89/l99 0 2024-03-11T19:35:48.872 INFO:tasks.workunit.client.0.smithi090.stdout:5/469: truncate d1/d17/d39/d51/d53/d7b/f5b 681865 0 2024-03-11T19:35:49.150 INFO:tasks.workunit.client.0.smithi090.stdout:0/490: truncate d2/d11/d15/d29/d3d/d7e/f9d 942523 0 2024-03-11T19:35:49.181 INFO:tasks.workunit.client.0.smithi090.stdout:0/491: readlink d2/d11/d15/d36/l61 0 2024-03-11T19:35:49.320 INFO:tasks.workunit.client.0.smithi090.stdout:7/468: truncate d3/dc/f3a 4433638 0 2024-03-11T19:35:49.398 INFO:tasks.workunit.client.0.smithi090.stdout:7/469: truncate d3/dc/d1e/d7c/d96/f9e 1002919 0 2024-03-11T19:35:49.455 INFO:tasks.workunit.client.0.smithi090.stdout:7/470: chown d3/d49/f65 11 1 2024-03-11T19:35:49.640 INFO:tasks.workunit.client.0.smithi090.stdout:4/513: rename d4/d51/c61 to d4/d51/d8c/d6a/cb1 0 2024-03-11T19:35:49.657 INFO:tasks.workunit.client.0.smithi090.stdout:3/452: creat d2/d1d/d47/d6a/f95 x:0 0 0 2024-03-11T19:35:50.059 INFO:tasks.workunit.client.0.smithi090.stdout:2/463: truncate d0/de/d12/d46/d49/f6c 21245 0 2024-03-11T19:35:50.084 INFO:tasks.workunit.client.0.smithi090.stdout:5/470: dread - d1/d17/f41 zero size 2024-03-11T19:35:50.182 INFO:tasks.workunit.client.0.smithi090.stdout:2/464: sync 2024-03-11T19:35:50.229 INFO:tasks.workunit.client.0.smithi090.stdout:8/513: creat d0/d3/fa8 x:0 0 0 2024-03-11T19:35:50.417 INFO:tasks.workunit.client.0.smithi090.stdout:0/492: creat d2/d11/d15/d29/d43/d58/f9e x:0 0 0 2024-03-11T19:35:50.672 INFO:tasks.workunit.client.0.smithi090.stdout:6/529: rename d4/d13/d22/d34/d40/f8c to d4/d13/d22/d42/fac 0 2024-03-11T19:35:51.010 INFO:tasks.workunit.client.0.smithi090.stdout:9/454: dwrite d1/d1d/f67 [4194304,4194304] 0 2024-03-11T19:35:51.158 INFO:tasks.workunit.client.0.smithi090.stdout:5/471: symlink d1/d17/d39/d51/d73/l9c 0 2024-03-11T19:35:51.584 INFO:tasks.workunit.client.0.smithi090.stdout:0/493: rmdir d2/d11/d6e 39 2024-03-11T19:35:51.604 INFO:tasks.workunit.client.0.smithi090.stdout:7/471: symlink d3/dc/l9f 0 2024-03-11T19:35:51.647 INFO:tasks.workunit.client.0.smithi090.stdout:6/530: rmdir d4/d14/d8a 39 2024-03-11T19:35:51.671 INFO:tasks.workunit.client.0.smithi090.stdout:7/472: dread - d3/dc/d29/f6b zero size 2024-03-11T19:35:51.746 INFO:tasks.workunit.client.0.smithi090.stdout:7/473: chown d3/dc/d29/f4e 267578084 1 2024-03-11T19:35:51.800 INFO:tasks.workunit.client.0.smithi090.stdout:7/474: chown d3/dc/d1e/d7c/d96/d32/d71/d5d/c44 4128916 1 2024-03-11T19:35:52.044 INFO:tasks.workunit.client.0.smithi090.stdout:2/465: creat d0/de/d12/d46/d4c/d71/f8a x:0 0 0 2024-03-11T19:35:52.892 INFO:tasks.workunit.client.0.smithi090.stdout:9/455: fdatasync d1/f4f 0 2024-03-11T19:35:53.125 INFO:tasks.workunit.client.0.smithi090.stdout:8/514: link d0/d43/l1e d0/d3/d98/da2/la9 0 2024-03-11T19:35:53.307 INFO:tasks.workunit.client.0.smithi090.stdout:6/531: creat d4/d13/d22/d34/d37/d7f/d90/fad x:0 0 0 2024-03-11T19:35:53.525 INFO:tasks.workunit.client.0.smithi090.stdout:2/466: dread d0/d21/f2a [0,4194304] 0 2024-03-11T19:35:53.993 INFO:tasks.workunit.client.0.smithi090.stdout:1/472: dwrite d5/d4e/f5e [0,4194304] 0 2024-03-11T19:35:54.403 INFO:tasks.workunit.client.0.smithi090.stdout:9/456: creat d1/d49/f8a x:0 0 0 2024-03-11T19:35:54.431 INFO:tasks.workunit.client.0.smithi090.stdout:8/515: creat d0/d53/faa x:0 0 0 2024-03-11T19:35:54.492 INFO:tasks.workunit.client.0.smithi090.stdout:9/457: chown d1/dd/df/d1c/d21/d5c/c6f 14 1 2024-03-11T19:35:54.675 INFO:tasks.workunit.client.0.smithi090.stdout:6/532: rmdir d4/d13/d22/d42 39 2024-03-11T19:35:54.851 INFO:tasks.workunit.client.0.smithi090.stdout:2/467: mknod d0/d21/d30/d29/c8b 0 2024-03-11T19:35:55.156 INFO:tasks.workunit.client.0.smithi090.stdout:8/516: fsync d0/d43/f23 0 2024-03-11T19:35:55.288 INFO:tasks.workunit.client.0.smithi090.stdout:3/453: dwrite d2/f2c [0,4194304] 0 2024-03-11T19:35:55.594 INFO:tasks.workunit.client.0.smithi090.stdout:2/468: rmdir d0/de/d12/d46/d49 39 2024-03-11T19:35:55.884 INFO:tasks.workunit.client.0.smithi090.stdout:4/514: dwrite d4/d51/d8c/f9c [0,4194304] 0 2024-03-11T19:35:56.169 INFO:tasks.workunit.client.0.smithi090.stdout:0/494: write d2/d11/d3f/f83 [822264,126560] 0 2024-03-11T19:35:56.230 INFO:tasks.workunit.client.0.smithi090.stdout:8/517: dread d0/d43/dc/f28 [0,4194304] 0 2024-03-11T19:35:56.388 INFO:tasks.workunit.client.0.smithi090.stdout:0/495: readlink d2/d11/d15/d29/l2a 0 2024-03-11T19:35:56.441 INFO:tasks.workunit.client.0.smithi090.stdout:5/472: dwrite d1/f4c [0,4194304] 0 2024-03-11T19:35:56.620 INFO:tasks.workunit.client.0.smithi090.stdout:7/475: write d3/f3b [1810604,63693] 0 2024-03-11T19:35:56.625 INFO:tasks.workunit.client.0.smithi090.stdout:6/533: truncate d4/d14/d8a/f63 5236998 0 2024-03-11T19:35:56.825 INFO:tasks.workunit.client.0.smithi090.stdout:1/473: write d5/d31/d4c/f59 [1194453,1016] 0 2024-03-11T19:35:57.042 INFO:tasks.workunit.client.0.smithi090.stdout:3/454: dread d2/d45/f70 [0,4194304] 0 2024-03-11T19:35:57.087 INFO:tasks.workunit.client.0.smithi090.stdout:4/515: unlink d4/d18/f3c 0 2024-03-11T19:35:57.311 INFO:tasks.workunit.client.0.smithi090.stdout:8/518: dread - d0/d43/dc/f8e zero size 2024-03-11T19:35:57.425 INFO:tasks.workunit.client.0.smithi090.stdout:2/469: dread d0/d21/d30/d29/f2e [0,4194304] 0 2024-03-11T19:35:57.468 INFO:tasks.workunit.client.0.smithi090.stdout:9/458: dwrite d1/dd/f13 [0,4194304] 0 2024-03-11T19:35:57.634 INFO:tasks.workunit.client.0.smithi090.stdout:9/459: chown d1/f4f 11 1 2024-03-11T19:35:57.691 INFO:tasks.workunit.client.0.smithi090.stdout:2/470: sync 2024-03-11T19:35:57.786 INFO:tasks.workunit.client.0.smithi090.stdout:6/534: readlink d4/d13/d22/d42/d48/d4e/la0 0 2024-03-11T19:35:58.290 INFO:tasks.workunit.client.0.smithi090.stdout:3/455: dread d2/d1d/d47/d6a/f73 [0,4194304] 0 2024-03-11T19:35:58.405 INFO:tasks.workunit.client.0.smithi090.stdout:4/516: mknod d4/d27/d44/d66/cb2 0 2024-03-11T19:35:58.821 INFO:tasks.workunit.client.0.smithi090.stdout:9/460: fsync d1/dd/f3c 0 2024-03-11T19:35:59.037 INFO:tasks.workunit.client.0.smithi090.stdout:7/476: mkdir d3/dc/d1e/d7c/d96/d32/da0 0 2024-03-11T19:35:59.104 INFO:tasks.workunit.client.0.smithi090.stdout:6/535: creat d4/d13/d22/d42/d48/d4e/fae x:0 0 0 2024-03-11T19:35:59.301 INFO:tasks.workunit.client.0.smithi090.stdout:3/456: truncate d2/d2f/f30 1107101 0 2024-03-11T19:35:59.380 INFO:tasks.workunit.client.0.smithi090.stdout:4/517: symlink d4/d27/d44/d66/lb3 0 2024-03-11T19:35:59.408 INFO:tasks.workunit.client.0.smithi090.stdout:4/518: readlink d4/d27/d28/d69/l71 0 2024-03-11T19:35:59.678 INFO:tasks.workunit.client.0.smithi090.stdout:1/474: rename d5/da/d1c/f28 to d5/f9a 0 2024-03-11T19:36:00.419 INFO:tasks.workunit.client.0.smithi090.stdout:3/457: rmdir d2/db/d15/d1a/d75/d8c 39 2024-03-11T19:36:00.858 INFO:tasks.workunit.client.0.smithi090.stdout:0/496: dwrite d2/d11/d3a/f69 [0,4194304] 0 2024-03-11T19:36:00.956 INFO:tasks.workunit.client.0.smithi090.stdout:9/461: rename d1/f3 to d1/dd/df/d1c/d21/d5c/f8b 0 2024-03-11T19:36:01.016 INFO:tasks.workunit.client.0.smithi090.stdout:0/497: chown d2/d11/d15/d29/d3d/d7e 18167543 1 2024-03-11T19:36:01.271 INFO:tasks.workunit.client.0.smithi090.stdout:6/536: mknod d4/d14/d8a/caf 0 2024-03-11T19:36:01.314 INFO:tasks.workunit.client.0.smithi090.stdout:5/473: getdents d1/d17/d39/d51/d53/d7b 0 2024-03-11T19:36:01.540 INFO:tasks.workunit.client.0.smithi090.stdout:8/519: link d0/d53/l9a d0/d3/lab 0 2024-03-11T19:36:01.739 INFO:tasks.workunit.client.0.smithi090.stdout:9/462: creat d1/d1d/d6a/f8c x:0 0 0 2024-03-11T19:36:01.916 INFO:tasks.workunit.client.0.smithi090.stdout:0/498: rename d2/d11/d3f/f7a to d2/d11/d15/d29/d43/d58/f9f 0 2024-03-11T19:36:01.929 INFO:tasks.workunit.client.0.smithi090.stdout:2/471: write d0/f66 [4492471,30932] 0 2024-03-11T19:36:01.982 INFO:tasks.workunit.client.0.smithi090.stdout:0/499: chown d2/ca 15292582 1 2024-03-11T19:36:02.230 INFO:tasks.workunit.client.0.smithi090.stdout:6/537: creat d4/d13/d22/d34/d83/d9c/fb0 x:0 0 0 2024-03-11T19:36:02.368 INFO:tasks.workunit.client.0.smithi090.stdout:8/520: creat d0/d43/dc/d90/fac x:0 0 0 2024-03-11T19:36:03.083 INFO:tasks.workunit.client.0.smithi090.stdout:9/463: truncate d1/dd/d18/f65 1440124 0 2024-03-11T19:36:03.200 INFO:tasks.workunit.client.0.smithi090.stdout:2/472: chown d0/l7f 0 1 2024-03-11T19:36:03.350 INFO:tasks.workunit.client.0.smithi090.stdout:0/500: fdatasync d2/d11/d15/d29/d43/d58/f65 0 2024-03-11T19:36:03.448 INFO:tasks.workunit.client.0.smithi090.stdout:7/477: dwrite d3/dc/d1e/d7c/d96/d32/d71/d5d/f53 [0,4194304] 0 2024-03-11T19:36:03.649 INFO:tasks.workunit.client.0.smithi090.stdout:6/538: rmdir d4/d13/d22/d34/d37/d7f/d90 39 2024-03-11T19:36:03.783 INFO:tasks.workunit.client.0.smithi090.stdout:4/519: dwrite d4/d27/d28/d38/d64/f93 [0,4194304] 0 2024-03-11T19:36:03.948 INFO:tasks.workunit.client.0.smithi090.stdout:6/539: read d4/d13/d22/d34/f36 [3449203,119379] 0 2024-03-11T19:36:04.517 INFO:tasks.workunit.client.0.smithi090.stdout:2/473: truncate d0/d21/f6e 4509300 0 2024-03-11T19:36:04.585 INFO:tasks.workunit.client.0.smithi090.stdout:0/501: rmdir d2/d11/d15/d29/d2e 39 2024-03-11T19:36:04.610 INFO:tasks.workunit.client.0.smithi090.stdout:7/478: readlink d3/d55/l92 0 2024-03-11T19:36:04.759 INFO:tasks.workunit.client.0.smithi090.stdout:3/458: write d2/db/f92 [522038,66925] 0 2024-03-11T19:36:04.826 INFO:tasks.workunit.client.0.smithi090.stdout:0/502: sync 2024-03-11T19:36:04.840 INFO:tasks.workunit.client.0.smithi090.stdout:5/474: write d1/f49 [539898,80321] 0 2024-03-11T19:36:04.890 INFO:tasks.workunit.client.0.smithi090.stdout:0/503: readlink d2/l3e 0 2024-03-11T19:36:04.935 INFO:tasks.workunit.client.0.smithi090.stdout:3/459: chown d2/db/d15/d1a/d3c/c71 184017537 1 2024-03-11T19:36:04.975 INFO:tasks.workunit.client.0.smithi090.stdout:5/475: sync 2024-03-11T19:36:05.027 INFO:tasks.workunit.client.0.smithi090.stdout:6/540: fdatasync d4/d14/f6a 0 2024-03-11T19:36:05.273 INFO:tasks.workunit.client.0.smithi090.stdout:2/474: mkdir d0/d21/d30/d25/d8c 0 2024-03-11T19:36:05.506 INFO:tasks.workunit.client.0.smithi090.stdout:4/520: mknod d4/d18/cb4 0 2024-03-11T19:36:05.734 INFO:tasks.workunit.client.0.smithi090.stdout:1/475: rename d5/da/l63 to d5/d4e/d54/d8e/l9b 0 2024-03-11T19:36:05.842 INFO:tasks.workunit.client.0.smithi090.stdout:3/460: mknod d2/d1d/d47/c96 0 2024-03-11T19:36:05.872 INFO:tasks.workunit.client.0.smithi090.stdout:3/461: dread - d2/d45/f7a zero size 2024-03-11T19:36:06.090 INFO:tasks.workunit.client.0.smithi090.stdout:6/541: stat d4/d13/d22/d3a/f65 0 2024-03-11T19:36:06.456 INFO:tasks.workunit.client.0.smithi090.stdout:7/479: creat d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/d9c/fa1 x:0 0 0 2024-03-11T19:36:06.604 INFO:tasks.workunit.client.0.smithi090.stdout:2/475: mkdir d0/de/d12/d46/d5d/d8d 0 2024-03-11T19:36:06.699 INFO:tasks.workunit.client.0.smithi090.stdout:9/464: dwrite d1/dd/df/d1c/d21/d2a/f50 [0,4194304] 0 2024-03-11T19:36:06.861 INFO:tasks.workunit.client.0.smithi090.stdout:8/521: dwrite d0/d53/d59/f72 [0,4194304] 0 2024-03-11T19:36:07.010 INFO:tasks.workunit.client.0.smithi090.stdout:2/476: chown d0/de/d12/d46/d5d/f62 6 1 2024-03-11T19:36:07.077 INFO:tasks.workunit.client.0.smithi090.stdout:3/462: truncate d2/d1f/f78 1305691 0 2024-03-11T19:36:07.103 INFO:tasks.workunit.client.0.smithi090.stdout:3/463: chown d2/d1d/c8e 7 1 2024-03-11T19:36:07.137 INFO:tasks.workunit.client.0.smithi090.stdout:0/504: write d2/d11/f1a [2879347,63575] 0 2024-03-11T19:36:07.955 INFO:tasks.workunit.client.0.smithi090.stdout:8/522: unlink d0/d3/d3e/f92 0 2024-03-11T19:36:08.101 INFO:tasks.workunit.client.0.smithi090.stdout:9/465: creat d1/dd/d18/d25/d4b/f8d x:0 0 0 2024-03-11T19:36:08.279 INFO:tasks.workunit.client.0.smithi090.stdout:2/477: truncate d0/d21/d30/d25/f67 642532 0 2024-03-11T19:36:09.213 INFO:tasks.workunit.client.0.smithi090.stdout:6/542: write d4/fe [6705761,17257] 0 2024-03-11T19:36:09.420 INFO:tasks.workunit.client.0.smithi090.stdout:9/466: creat d1/dd/d18/d3a/f8e x:0 0 0 2024-03-11T19:36:09.468 INFO:tasks.workunit.client.0.smithi090.stdout:9/467: dread - d1/dd/d18/d3a/d48/f73 zero size 2024-03-11T19:36:09.496 INFO:tasks.workunit.client.0.smithi090.stdout:9/468: chown d1/dd/d18/d25 16006308 1 2024-03-11T19:36:09.722 INFO:tasks.workunit.client.0.smithi090.stdout:2/478: mkdir d0/de/d12/d46/d5d/d63/d8e 0 2024-03-11T19:36:10.040 INFO:tasks.workunit.client.0.smithi090.stdout:5/476: rename d1/d26/d4d/d99 to d1/d17/d39/d51/d9d 0 2024-03-11T19:36:10.065 INFO:tasks.workunit.client.0.smithi090.stdout:4/521: write d4/d27/d28/d38/d64/fa3 [622487,66400] 0 2024-03-11T19:36:10.183 INFO:tasks.workunit.client.0.smithi090.stdout:4/522: chown d4/d27/d28/d38/d64/f93 0 1 2024-03-11T19:36:10.484 INFO:tasks.workunit.client.0.smithi090.stdout:1/476: write d5/f1b [2216759,48831] 0 2024-03-11T19:36:11.123 INFO:tasks.workunit.client.0.smithi090.stdout:0/505: write d2/d11/d15/f28 [395458,89700] 0 2024-03-11T19:36:11.154 INFO:tasks.workunit.client.0.smithi090.stdout:9/469: dread d1/dd/f3f [0,4194304] 0 2024-03-11T19:36:11.259 INFO:tasks.workunit.client.0.smithi090.stdout:3/464: dwrite d2/de/f91 [0,4194304] 0 2024-03-11T19:36:11.419 INFO:tasks.workunit.client.0.smithi090.stdout:4/523: unlink d4/d18/c81 0 2024-03-11T19:36:11.481 INFO:tasks.workunit.client.0.smithi090.stdout:7/480: getdents d3/dc/d1e/d7c 0 2024-03-11T19:36:11.484 INFO:tasks.workunit.client.0.smithi090.stdout:4/524: chown d4/d27/d28/d7a/fa6 24498 1 2024-03-11T19:36:12.756 INFO:tasks.workunit.client.0.smithi090.stdout:4/525: dread d4/d27/d28/d69/f99 [0,4194304] 0 2024-03-11T19:36:12.868 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:01:32.625 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:36:12.897 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:01:32.655 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:36:12.916 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:01:32.605 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:36:12.920 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:01:32.618 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:36:12.932 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:01:32.622 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:36:12.935 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:01:32.632 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:36:13.031 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:01:32.789 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:36:13.060 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:01:32.757 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:36:13.060 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:01:32.750 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:36:13.218 INFO:tasks.workunit.client.0.smithi090.stdout:8/523: rename d0/d43/f23 to d0/d3/fad 0 2024-03-11T19:36:13.605 INFO:tasks.workunit.client.0.smithi090.stdout:6/543: dwrite d4/d13/d22/d3a/f65 [0,4194304] 0 2024-03-11T19:36:13.720 INFO:tasks.workunit.client.0.smithi090.stdout:5/477: link d1/d17/d39/d51/c87 d1/d17/d39/d51/d9d/c9e 0 2024-03-11T19:36:13.728 INFO:tasks.workunit.client.0.smithi090.stdout:3/465: link d2/de/f91 d2/d1d/d47/d6a/f97 0 2024-03-11T19:36:13.934 INFO:tasks.workunit.client.0.smithi090.stdout:4/526: rmdir d4/d18/d6f 39 2024-03-11T19:36:14.277 INFO:tasks.workunit.client.0.smithi090.stdout:6/544: fsync d4/d14/d25/f41 0 2024-03-11T19:36:14.368 INFO:tasks.workunit.client.0.smithi090.stdout:2/479: write d0/de/f17 [963339,61889] 0 2024-03-11T19:36:14.566 INFO:tasks.workunit.client.0.smithi090.stdout:5/478: dread - d1/d17/d39/d51/d53/f98 zero size 2024-03-11T19:36:14.595 INFO:tasks.workunit.client.0.smithi090.stdout:1/477: dwrite d5/da/d1c/f39 [0,4194304] 0 2024-03-11T19:36:14.722 INFO:tasks.workunit.client.0.smithi090.stdout:4/527: mkdir d4/d27/d44/db5 0 2024-03-11T19:36:15.010 INFO:tasks.workunit.client.0.smithi090.stdout:8/524: symlink d0/d43/lae 0 2024-03-11T19:36:15.061 INFO:tasks.workunit.client.0.smithi090.stdout:8/525: readlink d0/d43/dc/l41 0 2024-03-11T19:36:15.183 INFO:tasks.workunit.client.0.smithi090.stdout:3/466: unlink d2/d1d/l22 0 2024-03-11T19:36:15.194 INFO:tasks.workunit.client.0.smithi090.stdout:9/470: write d1/d47/d7c/f83 [93840,74249] 0 2024-03-11T19:36:15.247 INFO:tasks.workunit.client.0.smithi090.stdout:9/471: chown d1/f79 44811600 1 2024-03-11T19:36:15.250 INFO:tasks.workunit.client.0.smithi090.stdout:3/467: chown d2/db/c86 100 1 2024-03-11T19:36:15.512 INFO:tasks.workunit.client.0.smithi090.stdout:7/481: dwrite d3/dc/d1e/ff [4194304,4194304] 0 2024-03-11T19:36:15.615 INFO:tasks.workunit.client.0.smithi090.stdout:6/545: symlink d4/d24/d80/lb1 0 2024-03-11T19:36:15.680 INFO:tasks.workunit.client.0.smithi090.stdout:1/478: mkdir d5/d4e/d54/d8e/d9c 0 2024-03-11T19:36:15.845 INFO:tasks.workunit.client.0.smithi090.stdout:5/479: dread - d1/d17/d39/d51/d73/f94 zero size 2024-03-11T19:36:16.048 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:36:16.051 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:36:16.138 INFO:tasks.workunit.client.0.smithi090.stdout:5/480: dread d1/f38 [0,4194304] 0 2024-03-11T19:36:16.436 INFO:tasks.workunit.client.0.smithi090.stdout:9/472: truncate d1/dd/d18/d3a/d57/f5b 426452 0 2024-03-11T19:36:16.565 INFO:tasks.workunit.client.0.smithi090.stdout:7/482: symlink d3/dc/d29/la2 0 2024-03-11T19:36:17.314 INFO:tasks.workunit.client.0.smithi090.stdout:3/468: truncate d2/d45/f63 6706785 0 2024-03-11T19:36:17.333 INFO:tasks.workunit.client.0.smithi090.stdout:0/506: rename d2/f5 to d2/d11/d15/d29/d2e/fa0 0 2024-03-11T19:36:17.519 INFO:tasks.workunit.client.0.smithi090.stdout:6/546: mknod d4/cb2 0 2024-03-11T19:36:17.624 INFO:tasks.workunit.client.0.smithi090.stdout:7/483: rmdir d3/d49/d8e 39 2024-03-11T19:36:17.828 INFO:tasks.workunit.client.0.smithi090.stdout:1/479: mknod d5/d65/d6b/d7f/c9d 0 2024-03-11T19:36:18.439 INFO:tasks.workunit.client.0.smithi090.stdout:9/473: mknod d1/dd/df/d1c/d77/c8f 0 2024-03-11T19:36:18.693 INFO:tasks.workunit.client.0.smithi090.stdout:4/528: rmdir d4 39 2024-03-11T19:36:18.697 INFO:tasks.workunit.client.0.smithi090.stdout:3/469: stat d2/db/d15/l39 0 2024-03-11T19:36:18.797 INFO:tasks.workunit.client.0.smithi090.stdout:2/480: link d0/de/d12/c36 d0/de/d12/c8f 0 2024-03-11T19:36:18.883 INFO:tasks.workunit.client.0.smithi090.stdout:6/547: mknod d4/d24/cb3 0 2024-03-11T19:36:19.290 INFO:tasks.workunit.client.0.smithi090.stdout:1/480: chown d5/d31/d76/f78 2 1 2024-03-11T19:36:19.375 INFO:tasks.workunit.client.0.smithi090.stdout:8/526: dwrite d0/d43/f84 [0,4194304] 0 2024-03-11T19:36:20.475 INFO:tasks.workunit.client.0.smithi090.stdout:1/481: mknod d5/da/dd/c9e 0 2024-03-11T19:36:20.541 INFO:tasks.workunit.client.0.smithi090.stdout:0/507: link d2/d11/l59 d2/d11/d3a/la1 0 2024-03-11T19:36:20.913 INFO:tasks.workunit.client.0.smithi090.stdout:2/481: mknod d0/de/c90 0 2024-03-11T19:36:21.026 INFO:tasks.workunit.client.0.smithi090.stdout:8/527: dread d0/d3/d4c/f5e [0,4194304] 0 2024-03-11T19:36:21.066 INFO:tasks.workunit.client.0.smithi090.stdout:2/482: chown d0/d21/d30/d25 360799980 1 2024-03-11T19:36:21.104 INFO:tasks.workunit.client.0.smithi090.stdout:4/529: dread d4/d27/d28/d38/f91 [0,4194304] 0 2024-03-11T19:36:21.331 INFO:tasks.workunit.client.0.smithi090.stdout:8/528: dread d0/d53/f71 [0,4194304] 0 2024-03-11T19:36:21.378 INFO:tasks.workunit.client.0.smithi090.stdout:9/474: link d1/f79 d1/d47/f90 0 2024-03-11T19:36:21.500 INFO:tasks.workunit.client.0.smithi090.stdout:9/475: readlink d1/d1d/l41 0 2024-03-11T19:36:21.607 INFO:tasks.workunit.client.0.smithi090.stdout:0/508: creat d2/d11/d15/d29/d6b/fa2 x:0 0 0 2024-03-11T19:36:21.668 INFO:tasks.workunit.client.0.smithi090.stdout:5/481: dwrite d1/d17/f79 [0,4194304] 0 2024-03-11T19:36:22.085 INFO:tasks.workunit.client.0.smithi090.stdout:8/529: creat d0/d53/faf x:0 0 0 2024-03-11T19:36:22.233 INFO:tasks.workunit.client.0.smithi090.stdout:4/530: rename d4/f24 to d4/d27/d28/d38/d64/fb6 0 2024-03-11T19:36:22.640 INFO:tasks.workunit.client.0.smithi090.stdout:0/509: read d2/f14 [272411,28736] 0 2024-03-11T19:36:22.655 INFO:tasks.workunit.client.0.smithi090.stdout:7/484: link d3/dc/f3a d3/dc/d1e/fa3 0 2024-03-11T19:36:22.876 INFO:tasks.workunit.client.0.smithi090.stdout:6/548: write d4/d13/d22/d34/f36 [1988401,9321] 0 2024-03-11T19:36:23.537 INFO:tasks.workunit.client.0.smithi090.stdout:4/531: fsync d4/d27/d28/d7a/fa6 0 2024-03-11T19:36:23.817 INFO:tasks.workunit.client.0.smithi090.stdout:7/485: creat d3/d55/fa4 x:0 0 0 2024-03-11T19:36:23.929 INFO:tasks.workunit.client.0.smithi090.stdout:0/510: mknod d2/d11/d15/d29/d3d/d7e/ca3 0 2024-03-11T19:36:24.102 INFO:tasks.workunit.client.0.smithi090.stdout:4/532: sync 2024-03-11T19:36:24.193 INFO:tasks.workunit.client.0.smithi090.stdout:6/549: mknod d4/d14/d92/cb4 0 2024-03-11T19:36:24.332 INFO:tasks.workunit.client.0.smithi090.stdout:8/530: mkdir d0/d3/db0 0 2024-03-11T19:36:24.435 INFO:tasks.workunit.client.0.smithi090.stdout:3/470: chown d2/d1d/l23 63774 1 2024-03-11T19:36:24.565 INFO:tasks.workunit.client.0.smithi090.stdout:7/486: creat d3/dc/d1e/d2a/fa5 x:0 0 0 2024-03-11T19:36:24.946 INFO:tasks.workunit.client.0.smithi090.stdout:6/550: truncate d4/d13/d22/d34/d37/f8e 1937102 0 2024-03-11T19:36:25.420 INFO:tasks.workunit.client.0.smithi090.stdout:7/487: truncate d3/dc/d29/f4e 49532 0 2024-03-11T19:36:25.632 INFO:tasks.workunit.client.0.smithi090.stdout:1/482: dwrite d5/f9a [0,4194304] 0 2024-03-11T19:36:25.752 INFO:tasks.workunit.client.0.smithi090.stdout:4/533: mkdir d4/d18/d6f/db7 0 2024-03-11T19:36:25.928 INFO:tasks.workunit.client.0.smithi090.stdout:3/471: unlink d2/d1d/f25 0 2024-03-11T19:36:26.112 INFO:tasks.workunit.client.0.smithi090.stdout:5/482: write d1/f49 [285385,69215] 0 2024-03-11T19:36:26.144 INFO:tasks.workunit.client.0.smithi090.stdout:9/476: dwrite d1/d1d/d6a/f8c [0,4194304] 0 2024-03-11T19:36:26.769 INFO:tasks.workunit.client.0.smithi090.stdout:2/483: rename d0/l9 to d0/l91 0 2024-03-11T19:36:26.935 INFO:tasks.workunit.client.0.smithi090.stdout:7/488: chown d3/dc/d29/f4e 250 1 2024-03-11T19:36:27.066 INFO:tasks.workunit.client.0.smithi090.stdout:9/477: symlink d1/d49/l91 0 2024-03-11T19:36:27.337 INFO:tasks.workunit.client.0.smithi090.stdout:2/484: unlink d0/d21/d30/d25/f5c 0 2024-03-11T19:36:27.715 INFO:tasks.workunit.client.0.smithi090.stdout:4/534: rename d4/d27/d28/c43 to d4/d27/d28/d38/d75/cb8 0 2024-03-11T19:36:27.769 INFO:tasks.workunit.client.0.smithi090.stdout:4/535: chown d4/d51/c62 141 1 2024-03-11T19:36:28.382 INFO:tasks.workunit.client.0.smithi090.stdout:8/531: write d0/d3/f81 [1047970,31178] 0 2024-03-11T19:36:28.436 INFO:tasks.workunit.client.0.smithi090.stdout:0/511: dwrite d2/d11/d15/d29/d43/d58/f88 [0,4194304] 0 2024-03-11T19:36:28.526 INFO:tasks.workunit.client.0.smithi090.stdout:2/485: fsync d0/de/d12/d46/d4c/d71/f8a 0 2024-03-11T19:36:28.570 INFO:tasks.workunit.client.0.smithi090.stdout:2/486: chown d0/l8 188847944 1 2024-03-11T19:36:28.587 INFO:tasks.workunit.client.0.smithi090.stdout:0/512: chown d2/d11/d15/d29/d43/d58/l67 60436773 1 2024-03-11T19:36:28.948 INFO:tasks.workunit.client.0.smithi090.stdout:1/483: rename d5/d3c/l7d to d5/da/d1c/d6d/d80/l9f 0 2024-03-11T19:36:29.709 INFO:tasks.workunit.client.0.smithi090.stdout:8/532: dread d0/f5a [0,4194304] 0 2024-03-11T19:36:30.075 INFO:tasks.workunit.client.0.smithi090.stdout:5/483: rename d1/d26/f3e to d1/d26/f9f 0 2024-03-11T19:36:30.146 INFO:tasks.workunit.client.0.smithi090.stdout:0/513: fdatasync d2/d11/d3a/f69 0 2024-03-11T19:36:30.835 INFO:tasks.workunit.client.0.smithi090.stdout:3/472: dwrite d2/d2f/f30 [0,4194304] 0 2024-03-11T19:36:31.353 INFO:tasks.workunit.client.0.smithi090.stdout:7/489: mkdir d3/dc/da6 0 2024-03-11T19:36:31.605 INFO:tasks.workunit.client.0.smithi090.stdout:0/514: dread d2/d11/d15/d29/d2e/f32 [0,4194304] 0 2024-03-11T19:36:31.872 INFO:tasks.workunit.client.0.smithi090.stdout:6/551: link d4/d13/d22/d34/d37/f8e d4/d13/d22/d3a/d6f/fb5 0 2024-03-11T19:36:31.877 INFO:tasks.workunit.client.0.smithi090.stdout:4/536: write d4/ff [3663272,121570] 0 2024-03-11T19:36:31.980 INFO:tasks.workunit.client.0.smithi090.stdout:9/478: dwrite d1/dd/df/d1c/d21/d2a/f29 [0,4194304] 0 2024-03-11T19:36:32.761 INFO:tasks.workunit.client.0.smithi090.stdout:2/487: dwrite d0/de/d12/f1a [0,4194304] 0 2024-03-11T19:36:32.871 INFO:tasks.workunit.client.0.smithi090.stdout:0/515: rename d2/d11/d15/d29/d6b to d2/d11/d15/d29/d2e/d6d/da4 0 2024-03-11T19:36:32.966 INFO:tasks.workunit.client.0.smithi090.stdout:4/537: mknod d4/d51/d8c/d6a/d8d/cb9 0 2024-03-11T19:36:33.061 INFO:tasks.workunit.client.0.smithi090.stdout:3/473: truncate d2/de/f91 2695323 0 2024-03-11T19:36:33.088 INFO:tasks.workunit.client.0.smithi090.stdout:9/479: dread d1/dd/f13 [0,4194304] 0 2024-03-11T19:36:33.274 INFO:tasks.workunit.client.0.smithi090.stdout:1/484: dwrite d5/da/f4a [0,4194304] 0 2024-03-11T19:36:33.941 INFO:tasks.workunit.client.0.smithi090.stdout:9/480: truncate d1/f4f 340376 0 2024-03-11T19:36:34.035 INFO:tasks.workunit.client.0.smithi090.stdout:1/485: chown d5/d31/d76/l58 11457489 1 2024-03-11T19:36:34.261 INFO:tasks.workunit.client.0.smithi090.stdout:3/474: truncate d2/d45/f89 376015 0 2024-03-11T19:36:34.553 INFO:tasks.workunit.client.0.smithi090.stdout:9/481: creat d1/dd/d18/d3a/d48/f92 x:0 0 0 2024-03-11T19:36:34.579 INFO:tasks.workunit.client.0.smithi090.stdout:1/486: creat d5/da/d1c/d6d/d80/fa0 x:0 0 0 2024-03-11T19:36:34.656 INFO:tasks.workunit.client.0.smithi090.stdout:4/538: truncate d4/d27/d28/d69/f99 699765 0 2024-03-11T19:36:34.777 INFO:tasks.workunit.client.0.smithi090.stdout:9/482: chown d1/dd/df/c27 179948952 1 2024-03-11T19:36:34.889 INFO:tasks.workunit.client.0.smithi090.stdout:8/533: dwrite d0/d43/f2e [0,4194304] 0 2024-03-11T19:36:34.992 INFO:tasks.workunit.client.0.smithi090.stdout:9/483: chown d1/dd/c15 4937 1 2024-03-11T19:36:35.259 INFO:tasks.workunit.client.0.smithi090.stdout:5/484: unlink d1/f97 0 2024-03-11T19:36:35.266 INFO:tasks.workunit.client.0.smithi090.stdout:2/488: write d0/de/d12/f24 [1424389,70356] 0 2024-03-11T19:36:35.270 INFO:tasks.workunit.client.0.smithi090.stdout:7/490: write d3/dc/d1e/d7c/d96/d32/d71/d5d/f53 [3084249,65758] 0 2024-03-11T19:36:35.283 INFO:tasks.workunit.client.0.smithi090.stdout:6/552: write d4/d14/d25/f41 [1044771,18194] 0 2024-03-11T19:36:35.568 INFO:tasks.workunit.client.0.smithi090.stdout:4/539: unlink d4/d51/c53 0 2024-03-11T19:36:35.936 INFO:tasks.workunit.client.0.smithi090.stdout:9/484: rename d1/d49/l91 to d1/dd/df/d1c/d88/l93 0 2024-03-11T19:36:36.013 INFO:tasks.workunit.client.0.smithi090.stdout:2/489: chown d0/f18 4458 1 2024-03-11T19:36:36.271 INFO:tasks.workunit.client.0.smithi090.stdout:6/553: dread d4/d14/d92/f99 [0,4194304] 0 2024-03-11T19:36:36.360 INFO:tasks.workunit.client.0.smithi090.stdout:2/490: sync 2024-03-11T19:36:36.992 INFO:tasks.workunit.client.0.smithi090.stdout:9/485: mknod d1/dd/df/d1c/d21/c94 0 2024-03-11T19:36:37.075 INFO:tasks.workunit.client.0.smithi090.stdout:5/485: truncate d1/d26/f71 3495039 0 2024-03-11T19:36:37.113 INFO:tasks.workunit.client.0.smithi090.stdout:5/486: stat d1/d17/d39/d51/d53/d7b/d3b 0 2024-03-11T19:36:37.391 INFO:tasks.workunit.client.0.smithi090.stdout:6/554: truncate d4/d13/d22/d34/d40/da4/f9a 1024508 0 2024-03-11T19:36:37.433 INFO:tasks.workunit.client.0.smithi090.stdout:3/475: write d2/f12 [1244478,88578] 0 2024-03-11T19:36:37.692 INFO:tasks.workunit.client.0.smithi090.stdout:8/534: link d0/d43/l5b d0/d3/d3e/d85/lb1 0 2024-03-11T19:36:37.955 INFO:tasks.workunit.client.0.smithi090.stdout:9/486: fsync d1/d1d/f72 0 2024-03-11T19:36:38.201 INFO:tasks.workunit.client.0.smithi090.stdout:5/487: mkdir d1/d17/d39/d51/d53/d7b/da0 0 2024-03-11T19:36:38.247 INFO:tasks.workunit.client.0.smithi090.stdout:7/491: getdents d3/dc/d1e/d7c/d96/d34 0 2024-03-11T19:36:38.265 INFO:tasks.workunit.client.0.smithi090.stdout:6/555: read d4/d13/d22/d3a/d67/f9d [2239795,37199] 0 2024-03-11T19:36:38.330 INFO:tasks.workunit.client.0.smithi090.stdout:5/488: read d1/d26/f3f [241264,104174] 0 2024-03-11T19:36:38.354 INFO:tasks.workunit.client.0.smithi090.stdout:2/491: rmdir d0/de/d12/d46/d49 39 2024-03-11T19:36:38.680 INFO:tasks.workunit.client.0.smithi090.stdout:1/487: dwrite d5/d31/d76/f4d [0,4194304] 0 2024-03-11T19:36:39.015 INFO:tasks.workunit.client.0.smithi090.stdout:4/540: rename d4/d51/d8c/d6a/cb1 to d4/d27/d28/d38/d64/cba 0 2024-03-11T19:36:39.179 INFO:tasks.workunit.client.0.smithi090.stdout:9/487: dread d1/dd/df/d1c/d21/d2a/f29 [0,4194304] 0 2024-03-11T19:36:39.386 INFO:tasks.workunit.client.0.smithi090.stdout:0/516: creat d2/d11/d15/d29/fa5 x:0 0 0 2024-03-11T19:36:39.406 INFO:tasks.workunit.client.0.smithi090.stdout:7/492: fdatasync d3/d55/f6a 0 2024-03-11T19:36:39.458 INFO:tasks.workunit.client.0.smithi090.stdout:7/493: readlink d3/dc/d1e/l39 0 2024-03-11T19:36:40.598 INFO:tasks.workunit.client.0.smithi090.stdout:1/488: mknod d5/da/d1c/d3b/d40/ca1 0 2024-03-11T19:36:41.190 INFO:tasks.workunit.client.0.smithi090.stdout:0/517: mkdir d2/d11/d3a/da6 0 2024-03-11T19:36:41.638 INFO:tasks.workunit.client.0.smithi090.stdout:8/535: write d0/d3/d98/da2/fa4 [971036,48871] 0 2024-03-11T19:36:42.359 INFO:tasks.workunit.client.0.smithi090.stdout:4/541: symlink d4/d18/d6f/db7/lbb 0 2024-03-11T19:36:42.384 INFO:tasks.workunit.client.0.smithi090.stdout:2/492: write d0/fd [2419355,1358] 0 2024-03-11T19:36:42.412 INFO:tasks.workunit.client.0.smithi090.stdout:1/489: truncate d5/f1b 7455912 0 2024-03-11T19:36:42.795 INFO:tasks.workunit.client.0.smithi090.stdout:0/518: mknod d2/d11/d15/d29/d2e/d6d/ca7 0 2024-03-11T19:36:42.856 INFO:tasks.workunit.client.0.smithi090.stdout:2/493: dread d0/d21/d30/f47 [0,4194304] 0 2024-03-11T19:36:42.922 INFO:tasks.workunit.client.0.smithi090.stdout:2/494: chown d0/de/d12/d46/d4c/c4d 5250 1 2024-03-11T19:36:43.033 INFO:tasks.workunit.client.0.smithi090.stdout:8/536: truncate d0/d3/f2c 578279 0 2024-03-11T19:36:43.217 INFO:tasks.workunit.client.0.smithi090.stdout:5/489: creat d1/d17/d39/d51/d53/d7b/d3b/d76/fa1 x:0 0 0 2024-03-11T19:36:43.388 INFO:tasks.workunit.client.0.smithi090.stdout:4/542: creat d4/d51/d8c/d6a/fbc x:0 0 0 2024-03-11T19:36:43.813 INFO:tasks.workunit.client.0.smithi090.stdout:3/476: getdents d2/db/d15 0 2024-03-11T19:36:44.313 INFO:tasks.workunit.client.0.smithi090.stdout:2/495: rmdir d0/de/d12/d46/d4c/d71 39 2024-03-11T19:36:44.340 INFO:tasks.workunit.client.0.smithi090.stdout:2/496: chown d0/d21/d30/d83 251053511 1 2024-03-11T19:36:44.679 INFO:tasks.workunit.client.0.smithi090.stdout:4/543: creat d4/d51/d8c/d89/fbd x:0 0 0 2024-03-11T19:36:44.988 INFO:tasks.workunit.client.0.smithi090.stdout:6/556: dwrite d4/d13/d22/d34/d37/d7f/d90/fad [0,4194304] 0 2024-03-11T19:36:45.139 INFO:tasks.workunit.client.0.smithi090.stdout:3/477: mknod d2/d1d/d47/d6a/c98 0 2024-03-11T19:36:45.139 INFO:tasks.workunit.client.0.smithi090.stdout:7/494: creat d3/dc/fa7 x:0 0 0 2024-03-11T19:36:45.159 INFO:tasks.workunit.client.0.smithi090.stdout:9/488: getdents d1/dd/df/d1c/d77 0 2024-03-11T19:36:45.180 INFO:tasks.workunit.client.0.smithi090.stdout:0/519: creat d2/d11/d15/d29/d87/fa8 x:0 0 0 2024-03-11T19:36:45.238 INFO:tasks.workunit.client.0.smithi090.stdout:6/557: stat d4/d13/d22/d3a/f54 0 2024-03-11T19:36:45.264 INFO:tasks.workunit.client.0.smithi090.stdout:0/520: chown d2/d11/d15/d29/l2a 3 1 2024-03-11T19:36:45.298 INFO:tasks.workunit.client.0.smithi090.stdout:6/558: read d4/d24/f3f [3637302,40796] 0 2024-03-11T19:36:45.642 INFO:tasks.workunit.client.0.smithi090.stdout:4/544: truncate d4/f46 749146 0 2024-03-11T19:36:45.813 INFO:tasks.workunit.client.0.smithi090.stdout:1/490: rmdir d5/d4e/d54/d8e/d9c 0 2024-03-11T19:36:46.184 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:36:46.188 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:36:46.250 INFO:tasks.workunit.client.0.smithi090.stdout:9/489: unlink d1/d49/f8a 0 2024-03-11T19:36:46.685 INFO:tasks.workunit.client.0.smithi090.stdout:0/521: creat d2/d11/d3f/fa9 x:0 0 0 2024-03-11T19:36:47.145 INFO:tasks.workunit.client.0.smithi090.stdout:3/478: fdatasync d2/f5a 0 2024-03-11T19:36:47.221 INFO:tasks.workunit.client.0.smithi090.stdout:4/545: dread d4/d18/fb0 [0,4194304] 0 2024-03-11T19:36:47.368 INFO:tasks.workunit.client.0.smithi090.stdout:2/497: link d0/d21/f3d d0/de/d12/d46/d5d/d63/d8e/f92 0 2024-03-11T19:36:47.457 INFO:tasks.workunit.client.0.smithi090.stdout:9/490: dread d1/dd/df/d1c/d21/d2a/f50 [0,4194304] 0 2024-03-11T19:36:47.861 INFO:tasks.workunit.client.0.smithi090.stdout:7/495: creat d3/dc/fa8 x:0 0 0 2024-03-11T19:36:48.110 INFO:tasks.workunit.client.0.smithi090.stdout:2/498: mkdir d0/de/d12/d81/d93 0 2024-03-11T19:36:48.134 INFO:tasks.workunit.client.0.smithi090.stdout:9/491: symlink d1/dd/df/d1c/d21/d2a/l95 0 2024-03-11T19:36:48.156 INFO:tasks.workunit.client.0.smithi090.stdout:5/490: write d1/d26/f52 [794635,34801] 0 2024-03-11T19:36:48.210 INFO:tasks.workunit.client.0.smithi090.stdout:8/537: dwrite d0/f65 [0,4194304] 0 2024-03-11T19:36:48.388 INFO:tasks.workunit.client.0.smithi090.stdout:1/491: rename d5/da/dd/f2c to d5/d31/d76/fa2 0 2024-03-11T19:36:49.281 INFO:tasks.workunit.client.0.smithi090.stdout:2/499: symlink d0/de/d12/d46/d4c/d64/l94 0 2024-03-11T19:36:49.662 INFO:tasks.workunit.client.0.smithi090.stdout:4/546: creat d4/d27/d28/fbe x:0 0 0 2024-03-11T19:36:49.701 INFO:tasks.workunit.client.0.smithi090.stdout:4/547: dread - d4/d27/d44/f94 zero size 2024-03-11T19:36:49.992 INFO:tasks.workunit.client.0.smithi090.stdout:6/559: link d4/d13/d22/d34/f6d d4/d13/fb6 0 2024-03-11T19:36:50.328 INFO:tasks.workunit.client.0.smithi090.stdout:3/479: rename d2/d45/f63 to d2/d1d/d47/d6a/f99 0 2024-03-11T19:36:50.784 INFO:tasks.workunit.client.0.smithi090.stdout:3/480: sync 2024-03-11T19:36:50.840 INFO:tasks.workunit.client.0.smithi090.stdout:4/548: mkdir d4/d51/dbf 0 2024-03-11T19:36:51.101 INFO:tasks.workunit.client.0.smithi090.stdout:6/560: mkdir d4/d13/d22/d34/d83/d9c/db7 0 2024-03-11T19:36:51.259 INFO:tasks.workunit.client.0.smithi090.stdout:0/522: dwrite d2/d11/d15/d29/d2e/d6d/da4/fa2 [0,4194304] 0 2024-03-11T19:36:51.867 INFO:tasks.workunit.client.0.smithi090.stdout:4/549: fsync d4/d51/f92 0 2024-03-11T19:36:52.177 INFO:tasks.workunit.client.0.smithi090.stdout:8/538: getdents d0/d3/d4c/d8d 0 2024-03-11T19:36:52.241 INFO:tasks.workunit.client.0.smithi090.stdout:7/496: write d3/d49/f65 [616124,55687] 0 2024-03-11T19:36:52.430 INFO:tasks.workunit.client.0.smithi090.stdout:6/561: chown d4/d14/d8a/f63 495283 1 2024-03-11T19:36:52.610 INFO:tasks.workunit.client.0.smithi090.stdout:1/492: getdents d5/da/d1c/d3b/d40/d89 0 2024-03-11T19:36:52.721 INFO:tasks.workunit.client.0.smithi090.stdout:0/523: mknod d2/d11/d15/d29/d3d/d7e/caa 0 2024-03-11T19:36:53.366 INFO:tasks.workunit.client.0.smithi090.stdout:8/539: chown d0/d3/f80 619 1 2024-03-11T19:36:53.389 INFO:tasks.workunit.client.0.smithi090.stdout:9/492: dwrite d1/dd/df/f4d [4194304,4194304] 0 2024-03-11T19:36:53.603 INFO:tasks.workunit.client.0.smithi090.stdout:4/550: creat d4/d27/d28/d7a/fc0 x:0 0 0 2024-03-11T19:36:53.745 INFO:tasks.workunit.client.0.smithi090.stdout:5/491: dwrite d1/d17/d39/d51/d53/f67 [0,4194304] 0 2024-03-11T19:36:53.950 INFO:tasks.workunit.client.0.smithi090.stdout:4/551: chown d4/d27/d28/d38/d64/f93 103644646 1 2024-03-11T19:36:54.059 INFO:tasks.workunit.client.0.smithi090.stdout:7/497: creat d3/dc/d1e/d7c/d96/d32/d71/fa9 x:0 0 0 2024-03-11T19:36:54.192 INFO:tasks.workunit.client.0.smithi090.stdout:9/493: fdatasync d1/dd/f3c 0 2024-03-11T19:36:54.340 INFO:tasks.workunit.client.0.smithi090.stdout:8/540: fsync d0/d43/dc/f4a 0 2024-03-11T19:36:54.660 INFO:tasks.workunit.client.0.smithi090.stdout:4/552: read - d4/d51/d8c/d6a/fbc zero size 2024-03-11T19:36:54.768 INFO:tasks.workunit.client.0.smithi090.stdout:2/500: write d0/d21/f3d [4281781,130643] 0 2024-03-11T19:36:55.088 INFO:tasks.workunit.client.0.smithi090.stdout:9/494: fdatasync d1/d47/d7c/f83 0 2024-03-11T19:36:55.542 INFO:tasks.workunit.client.0.smithi090.stdout:5/492: creat d1/d17/d39/d51/d53/d7b/d3b/d76/d92/fa2 x:0 0 0 2024-03-11T19:36:55.720 INFO:tasks.workunit.client.0.smithi090.stdout:3/481: dwrite d2/db/d15/f48 [0,4194304] 0 2024-03-11T19:36:55.845 INFO:tasks.workunit.client.0.smithi090.stdout:7/498: symlink d3/dc/d1e/d7c/d96/d32/d71/laa 0 2024-03-11T19:36:55.947 INFO:tasks.workunit.client.0.smithi090.stdout:3/482: sync 2024-03-11T19:36:56.421 INFO:tasks.workunit.client.0.smithi090.stdout:7/499: truncate d3/dc/d1e/f57 1679903 0 2024-03-11T19:36:56.480 INFO:tasks.workunit.client.0.smithi090.stdout:7/500: read d3/dc/d1e/d2a/f66 [426996,96558] 0 2024-03-11T19:36:56.853 INFO:tasks.workunit.client.0.smithi090.stdout:0/524: dwrite d2/d11/d15/f28 [0,4194304] 0 2024-03-11T19:36:57.135 INFO:tasks.workunit.client.0.smithi090.stdout:4/553: rename d4/d18/fa8 to d4/d18/fc1 0 2024-03-11T19:36:57.200 INFO:tasks.workunit.client.0.smithi090.stdout:4/554: chown d4/d27/d44/d66/c7c 35492649 1 2024-03-11T19:36:57.231 INFO:tasks.workunit.client.0.smithi090.stdout:4/555: chown d4/d51/d8c/d6a/f7f 84 1 2024-03-11T19:36:57.307 INFO:tasks.workunit.client.0.smithi090.stdout:7/501: mkdir d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab 0 2024-03-11T19:36:57.506 INFO:tasks.workunit.client.0.smithi090.stdout:8/541: getdents d0/d3 0 2024-03-11T19:36:57.641 INFO:tasks.workunit.client.0.smithi090.stdout:0/525: unlink d2/d11/d15/d29/d2e/d6d/da4/d78/f8e 0 2024-03-11T19:36:57.888 INFO:tasks.workunit.client.0.smithi090.stdout:9/495: getdents d1/dd/df 0 2024-03-11T19:36:57.908 INFO:tasks.workunit.client.0.smithi090.stdout:9/496: stat d1/d49/l5a 0 2024-03-11T19:36:58.028 INFO:tasks.workunit.client.0.smithi090.stdout:4/556: readlink d4/d51/d8c/d6a/d8d/l8f 0 2024-03-11T19:36:58.100 INFO:tasks.workunit.client.0.smithi090.stdout:6/562: rename d4/d13/d22/d34/f59 to d4/d14/d8a/fb8 0 2024-03-11T19:36:58.102 INFO:tasks.workunit.client.0.smithi090.stdout:7/502: creat d3/dc/d1e/d7c/d96/d34/d4d/fac x:0 0 0 2024-03-11T19:36:58.393 INFO:tasks.workunit.client.0.smithi090.stdout:9/497: truncate d1/dd/df/d1c/d21/d2a/f50 1347279 0 2024-03-11T19:36:58.881 INFO:tasks.workunit.client.0.smithi090.stdout:4/557: dread d4/d27/d28/d38/d64/fb6 [0,4194304] 0 2024-03-11T19:36:59.104 INFO:tasks.workunit.client.0.smithi090.stdout:2/501: dwrite d0/d21/d30/f31 [0,4194304] 0 2024-03-11T19:37:00.001 INFO:tasks.workunit.client.0.smithi090.stdout:1/493: rename d5/da/d1c/d3b/d40/ca1 to d5/da/d23/ca3 0 2024-03-11T19:37:00.013 INFO:tasks.workunit.client.0.smithi090.stdout:2/502: symlink d0/de/d12/d46/d5d/d63/l95 0 2024-03-11T19:37:00.069 INFO:tasks.workunit.client.0.smithi090.stdout:0/526: link d2/d11/d15/d29/d43/f98 d2/fab 0 2024-03-11T19:37:00.275 INFO:tasks.workunit.client.0.smithi090.stdout:5/493: write d1/d26/f71 [1618382,73881] 0 2024-03-11T19:37:00.309 INFO:tasks.workunit.client.0.smithi090.stdout:5/494: stat d1/d26/c1a 0 2024-03-11T19:37:00.713 INFO:tasks.workunit.client.0.smithi090.stdout:6/563: link d4/d24/l76 d4/d13/d22/lb9 0 2024-03-11T19:37:00.850 INFO:tasks.workunit.client.0.smithi090.stdout:8/542: rename d0/d43/l33 to d0/d3/d4c/d8d/lb2 0 2024-03-11T19:37:01.118 INFO:tasks.workunit.client.0.smithi090.stdout:1/494: creat d5/d65/d6b/fa4 x:0 0 0 2024-03-11T19:37:01.298 INFO:tasks.workunit.client.0.smithi090.stdout:8/543: dread d0/d43/dc/f50 [0,4194304] 0 2024-03-11T19:37:01.358 INFO:tasks.workunit.client.0.smithi090.stdout:0/527: read d2/d11/d3a/f42 [849007,80740] 0 2024-03-11T19:37:01.359 INFO:tasks.workunit.client.0.smithi090.stdout:3/483: creat d2/f9a x:0 0 0 2024-03-11T19:37:01.419 INFO:tasks.workunit.client.0.smithi090.stdout:2/503: mknod d0/d21/d30/d83/c96 0 2024-03-11T19:37:01.487 INFO:tasks.workunit.client.0.smithi090.stdout:8/544: read d0/d43/f79 [166892,109480] 0 2024-03-11T19:37:01.696 INFO:tasks.workunit.client.0.smithi090.stdout:9/498: write d1/d49/f76 [1987287,116059] 0 2024-03-11T19:37:01.763 INFO:tasks.workunit.client.0.smithi090.stdout:8/545: chown d0/d3/d86/d93 12408842 1 2024-03-11T19:37:01.794 INFO:tasks.workunit.client.0.smithi090.stdout:1/495: dread d5/da/f68 [0,4194304] 0 2024-03-11T19:37:02.136 INFO:tasks.workunit.client.0.smithi090.stdout:2/504: mknod d0/d21/d30/d29/c97 0 2024-03-11T19:37:02.238 INFO:tasks.workunit.client.0.smithi090.stdout:6/564: creat d4/d14/d8a/d74/fba x:0 0 0 2024-03-11T19:37:02.581 INFO:tasks.workunit.client.0.smithi090.stdout:7/503: rename d3/dc/d1e/d7c/d96/d32/c81 to d3/d49/d8e/cad 0 2024-03-11T19:37:02.758 INFO:tasks.workunit.client.0.smithi090.stdout:1/496: creat d5/d3c/fa5 x:0 0 0 2024-03-11T19:37:03.792 INFO:tasks.workunit.client.0.smithi090.stdout:8/546: unlink d0/d53/d59/l91 0 2024-03-11T19:37:04.441 INFO:tasks.workunit.client.0.smithi090.stdout:3/484: fdatasync d2/d2f/f31 0 2024-03-11T19:37:04.710 INFO:tasks.workunit.client.0.smithi090.stdout:6/565: truncate d4/d13/fa2 3234515 0 2024-03-11T19:37:04.737 INFO:tasks.workunit.client.0.smithi090.stdout:6/566: readlink d4/d13/d22/d42/l66 0 2024-03-11T19:37:04.949 INFO:tasks.workunit.client.0.smithi090.stdout:4/558: rename d4/d27/d28/d38/d75/c9e to d4/d18/d6f/db7/cc2 0 2024-03-11T19:37:05.799 INFO:tasks.workunit.client.0.smithi090.stdout:2/505: creat d0/de/d12/d46/f98 x:0 0 0 2024-03-11T19:37:06.090 INFO:tasks.workunit.client.0.smithi090.stdout:0/528: dwrite d2/f37 [0,4194304] 0 2024-03-11T19:37:06.121 INFO:tasks.workunit.client.0.smithi090.stdout:4/559: creat d4/d51/fc3 x:0 0 0 2024-03-11T19:37:06.364 INFO:tasks.workunit.client.0.smithi090.stdout:3/485: mkdir d2/d1d/d47/d6a/d9b 0 2024-03-11T19:37:06.866 INFO:tasks.workunit.client.0.smithi090.stdout:8/547: link d0/d3/c45 d0/d3/d86/cb3 0 2024-03-11T19:37:06.896 INFO:tasks.workunit.client.0.smithi090.stdout:9/499: write d1/dd/d18/d25/f60 [3026426,123017] 0 2024-03-11T19:37:06.948 INFO:tasks.workunit.client.0.smithi090.stdout:4/560: truncate d4/f60 958945 0 2024-03-11T19:37:07.528 INFO:tasks.workunit.client.0.smithi090.stdout:6/567: unlink d4/fe 0 2024-03-11T19:37:07.531 INFO:tasks.workunit.client.0.smithi090.stdout:1/497: write d5/d31/d36/f95 [872204,97254] 0 2024-03-11T19:37:07.604 INFO:tasks.workunit.client.0.smithi090.stdout:8/548: chown d0/d43/l9d 7640 1 2024-03-11T19:37:07.804 INFO:tasks.workunit.client.0.smithi090.stdout:4/561: rmdir d4/d27/d28/d69 39 2024-03-11T19:37:08.099 INFO:tasks.workunit.client.0.smithi090.stdout:3/486: fdatasync d2/f8 0 2024-03-11T19:37:08.328 INFO:tasks.workunit.client.0.smithi090.stdout:1/498: rmdir d5/d65 39 2024-03-11T19:37:08.523 INFO:tasks.workunit.client.0.smithi090.stdout:4/562: truncate d4/ff 4528750 0 2024-03-11T19:37:08.561 INFO:tasks.workunit.client.0.smithi090.stdout:4/563: chown d4/d51/d8c/d89/fbd 15537 1 2024-03-11T19:37:09.601 INFO:tasks.workunit.client.0.smithi090.stdout:4/564: fdatasync d4/d27/d28/d7a/d9b/fad 0 2024-03-11T19:37:09.722 INFO:tasks.workunit.client.0.smithi090.stdout:3/487: fdatasync d2/db/d15/f37 0 2024-03-11T19:37:10.329 INFO:tasks.workunit.client.0.smithi090.stdout:6/568: getdents d4/d13/d22/d34/d40/d7d 0 2024-03-11T19:37:10.663 INFO:tasks.workunit.client.0.smithi090.stdout:3/488: unlink d2/db/d15/f37 0 2024-03-11T19:37:10.961 INFO:tasks.workunit.client.0.smithi090.stdout:1/499: dread d5/d4e/d54/d94/f96 [0,4194304] 0 2024-03-11T19:37:10.991 INFO:tasks.workunit.client.0.smithi090.stdout:6/569: creat d4/d13/d22/d34/d40/fbb x:0 0 0 2024-03-11T19:37:11.005 INFO:tasks.workunit.client.0.smithi090.stdout:0/529: write d2/d11/d15/d29/d43/d58/f9e [222949,90107] 0 2024-03-11T19:37:11.214 INFO:tasks.workunit.client.0.smithi090.stdout:9/500: dwrite d1/dd/f13 [0,4194304] 0 2024-03-11T19:37:11.356 INFO:tasks.workunit.client.0.smithi090.stdout:1/500: sync 2024-03-11T19:37:11.356 INFO:tasks.workunit.client.0.smithi090.stdout:6/570: sync 2024-03-11T19:37:11.524 INFO:tasks.workunit.client.0.smithi090.stdout:0/530: mkdir d2/d11/d3a/d64/dac 0 2024-03-11T19:37:11.673 INFO:tasks.workunit.client.0.smithi090.stdout:3/489: dread - d2/d45/d55/f94 zero size 2024-03-11T19:37:11.834 INFO:tasks.workunit.client.0.smithi090.stdout:9/501: mkdir d1/dd/df/d1c/d88/d96 0 2024-03-11T19:37:12.221 INFO:tasks.workunit.client.0.smithi090.stdout:6/571: fsync d4/ff 0 2024-03-11T19:37:12.241 INFO:tasks.workunit.client.0.smithi090.stdout:4/565: write d4/d27/f34 [4647679,58250] 0 2024-03-11T19:37:12.390 INFO:tasks.workunit.client.0.smithi090.stdout:6/572: chown d4/d13/d22/d42/d48/d4e/c6c 83 1 2024-03-11T19:37:12.411 INFO:tasks.workunit.client.0.smithi090.stdout:6/573: chown d4/d24/cb3 92 1 2024-03-11T19:37:12.505 INFO:tasks.workunit.client.0.smithi090.stdout:0/531: creat d2/d11/d15/d29/d3d/d7b/fad x:0 0 0 2024-03-11T19:37:12.768 INFO:tasks.workunit.client.0.smithi090.stdout:9/502: dread d1/dd/d18/f38 [0,4194304] 0 2024-03-11T19:37:12.963 INFO:tasks.workunit.client.0.smithi090.stdout:1/501: creat d5/d65/fa6 x:0 0 0 2024-03-11T19:37:13.072 INFO:tasks.workunit.client.0.smithi090.stdout:4/566: mknod d4/d27/d44/d66/cc4 0 2024-03-11T19:37:13.418 INFO:tasks.workunit.client.0.smithi090.stdout:7/504: rename d3/dc/d1e/f26 to d3/dc/d29/fae 0 2024-03-11T19:37:13.428 INFO:tasks.workunit.client.0.smithi090.stdout:5/495: link d1/d17/d39/d51/d53/d7b/d6f/f8e d1/d17/d39/d51/d53/d7b/fa3 0 2024-03-11T19:37:13.506 INFO:tasks.workunit.client.0.smithi090.stdout:6/574: creat d4/d13/d22/d34/d83/fbc x:0 0 0 2024-03-11T19:37:13.549 INFO:tasks.workunit.client.0.smithi090.stdout:7/505: stat d3/dc/d1e/d7c/d96/d32/d71/d80 0 2024-03-11T19:37:14.478 INFO:tasks.workunit.client.0.smithi090.stdout:2/506: rename d0/de/l51 to d0/de/d12/d46/d49/l99 0 2024-03-11T19:37:14.642 INFO:tasks.workunit.client.0.smithi090.stdout:2/507: dread d0/de/d12/d46/d75/f7c [0,4194304] 0 2024-03-11T19:37:14.991 INFO:tasks.workunit.client.0.smithi090.stdout:3/490: unlink d2/d1f/f3f 0 2024-03-11T19:37:15.072 INFO:tasks.workunit.client.0.smithi090.stdout:9/503: truncate d1/f58 705284 0 2024-03-11T19:37:15.240 INFO:tasks.workunit.client.0.smithi090.stdout:9/504: sync 2024-03-11T19:37:15.258 INFO:tasks.workunit.client.0.smithi090.stdout:4/567: truncate d4/d27/d28/d69/f85 2995676 0 2024-03-11T19:37:15.315 INFO:tasks.workunit.client.0.smithi090.stdout:9/505: dread - d1/dd/d18/d3a/d48/f73 zero size 2024-03-11T19:37:15.506 INFO:tasks.workunit.client.0.smithi090.stdout:6/575: unlink d4/d14/f23 0 2024-03-11T19:37:16.045 INFO:tasks.workunit.client.0.smithi090.stdout:1/502: symlink d5/da/d1c/d3b/d8c/la7 0 2024-03-11T19:37:16.225 INFO:tasks.workunit.client.0.smithi090.stdout:4/568: stat d4/d27/d28/c33 0 2024-03-11T19:37:16.247 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:37:16.249 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:37:16.318 INFO:tasks.workunit.client.0.smithi090.stdout:8/549: rename d0/d43/dc/c87 to d0/d53/d59/d96/cb4 0 2024-03-11T19:37:16.423 INFO:tasks.workunit.client.0.smithi090.stdout:6/576: truncate d4/f46 4527247 0 2024-03-11T19:37:16.577 INFO:tasks.workunit.client.0.smithi090.stdout:2/508: symlink d0/de/d12/d46/d4c/d71/l9a 0 2024-03-11T19:37:16.603 INFO:tasks.workunit.client.0.smithi090.stdout:2/509: stat d0/de/d12/c27 0 2024-03-11T19:37:17.187 INFO:tasks.workunit.client.0.smithi090.stdout:8/550: creat d0/d60/fb5 x:0 0 0 2024-03-11T19:37:17.209 INFO:tasks.workunit.client.0.smithi090.stdout:8/551: chown d0/c29 60 1 2024-03-11T19:37:17.846 INFO:tasks.workunit.client.0.smithi090.stdout:1/503: fsync d5/d31/d4c/f91 0 2024-03-11T19:37:18.116 INFO:tasks.workunit.client.0.smithi090.stdout:0/532: dwrite d2/d11/d15/d29/f8f [0,4194304] 0 2024-03-11T19:37:18.235 INFO:tasks.workunit.client.0.smithi090.stdout:7/506: rename d3/dc/d29/fae to d3/dc/d1e/d7c/d96/d32/faf 0 2024-03-11T19:37:18.304 INFO:tasks.workunit.client.0.smithi090.stdout:7/507: readlink d3/dc/d1e/d7c/d96/d32/d45/l5c 0 2024-03-11T19:37:18.446 INFO:tasks.workunit.client.0.smithi090.stdout:8/552: fsync d0/d3/f68 0 2024-03-11T19:37:18.480 INFO:tasks.workunit.client.0.smithi090.stdout:8/553: stat d0/d3/d4c/d8d/c95 0 2024-03-11T19:37:18.768 INFO:tasks.workunit.client.0.smithi090.stdout:5/496: write d1/d17/d39/d51/d53/d7b/f5b [980717,86326] 0 2024-03-11T19:37:19.049 INFO:tasks.workunit.client.0.smithi090.stdout:3/491: write d2/d1f/f36 [400356,33116] 0 2024-03-11T19:37:19.198 INFO:tasks.workunit.client.0.smithi090.stdout:3/492: dread d2/d45/f70 [0,4194304] 0 2024-03-11T19:37:19.496 INFO:tasks.workunit.client.0.smithi090.stdout:0/533: rename d2/d11/d15/d29/f71 to d2/d11/d15/d29/d2e/d6d/fae 0 2024-03-11T19:37:19.689 INFO:tasks.workunit.client.0.smithi090.stdout:9/506: dwrite d1/d47/d7c/f83 [0,4194304] 0 2024-03-11T19:37:20.305 INFO:tasks.workunit.client.0.smithi090.stdout:6/577: creat d4/d14/d8a/fbd x:0 0 0 2024-03-11T19:37:20.389 INFO:tasks.workunit.client.0.smithi090.stdout:6/578: dread d4/d14/d8a/f63 [4194304,4194304] 0 2024-03-11T19:37:21.232 INFO:tasks.workunit.client.0.smithi090.stdout:7/508: mknod d3/dc/d1e/d7c/d96/d32/da0/cb0 0 2024-03-11T19:37:21.514 INFO:tasks.workunit.client.0.smithi090.stdout:0/534: write d2/d11/d15/d29/d3d/d7b/fad [286424,129199] 0 2024-03-11T19:37:21.635 INFO:tasks.workunit.client.0.smithi090.stdout:4/569: dwrite d4/f2e [0,4194304] 0 2024-03-11T19:37:21.777 INFO:tasks.workunit.client.0.smithi090.stdout:9/507: creat d1/dd/d18/d25/f97 x:0 0 0 2024-03-11T19:37:21.895 INFO:tasks.workunit.client.0.smithi090.stdout:5/497: read d1/d17/d39/d51/d53/d7b/d6f/f8e [2420592,124168] 0 2024-03-11T19:37:22.034 INFO:tasks.workunit.client.0.smithi090.stdout:6/579: truncate d4/d14/d8a/d74/fba 105598 0 2024-03-11T19:37:22.168 INFO:tasks.workunit.client.0.smithi090.stdout:2/510: link d0/fc d0/de/d12/d46/d5d/d63/d8e/f9b 0 2024-03-11T19:37:22.722 INFO:tasks.workunit.client.0.smithi090.stdout:0/535: dread d2/d11/f18 [0,4194304] 0 2024-03-11T19:37:23.075 INFO:tasks.workunit.client.0.smithi090.stdout:5/498: rmdir d1/d17/d39/d51/d73 39 2024-03-11T19:37:23.122 INFO:tasks.workunit.client.0.smithi090.stdout:6/580: creat d4/d24/fbe x:0 0 0 2024-03-11T19:37:23.190 INFO:tasks.workunit.client.0.smithi090.stdout:2/511: read - d0/de/d12/d46/d5d/f62 zero size 2024-03-11T19:37:23.287 INFO:tasks.workunit.client.0.smithi090.stdout:7/509: fsync d3/dc/d1e/d2a/f5e 0 2024-03-11T19:37:23.367 INFO:tasks.workunit.client.0.smithi090.stdout:8/554: getdents d0/d3/d4c/d8d 0 2024-03-11T19:37:23.512 INFO:tasks.workunit.client.0.smithi090.stdout:4/570: mknod d4/cc5 0 2024-03-11T19:37:24.155 INFO:tasks.workunit.client.0.smithi090.stdout:2/512: rmdir d0/de/d12/d46/d4c/d71 39 2024-03-11T19:37:24.358 INFO:tasks.workunit.client.0.smithi090.stdout:1/504: dwrite d5/f15 [0,4194304] 0 2024-03-11T19:37:24.463 INFO:tasks.workunit.client.0.smithi090.stdout:1/505: stat d5/d31/d4c/c57 0 2024-03-11T19:37:24.512 INFO:tasks.workunit.client.0.smithi090.stdout:8/555: stat d0/d3/d98/da2/la9 0 2024-03-11T19:37:24.535 INFO:tasks.workunit.client.0.smithi090.stdout:4/571: rmdir d4/d51/d8c/d89 39 2024-03-11T19:37:25.204 INFO:tasks.workunit.client.0.smithi090.stdout:5/499: truncate d1/d17/f30 1845120 0 2024-03-11T19:37:25.350 INFO:tasks.workunit.client.0.smithi090.stdout:2/513: chown d0/d21/d30/l74 1482 1 2024-03-11T19:37:26.419 INFO:tasks.workunit.client.0.smithi090.stdout:9/508: getdents d1/d1d 0 2024-03-11T19:37:26.624 INFO:tasks.workunit.client.0.smithi090.stdout:2/514: symlink d0/de/d1f/l9c 0 2024-03-11T19:37:26.708 INFO:tasks.workunit.client.0.smithi090.stdout:1/506: creat d5/da/d97/fa8 x:0 0 0 2024-03-11T19:37:27.236 INFO:tasks.workunit.client.0.smithi090.stdout:3/493: link d2/db/f8a d2/d2f/f9c 0 2024-03-11T19:37:27.312 INFO:tasks.workunit.client.0.smithi090.stdout:9/509: symlink d1/dd/d18/d25/l98 0 2024-03-11T19:37:27.657 INFO:tasks.workunit.client.0.smithi090.stdout:0/536: write d2/d11/d15/d29/d2e/d6d/da4/fa2 [2599321,12181] 0 2024-03-11T19:37:27.745 INFO:tasks.workunit.client.0.smithi090.stdout:1/507: truncate d5/d4e/d79/f82 80584 0 2024-03-11T19:37:28.034 INFO:tasks.workunit.client.0.smithi090.stdout:9/510: mknod d1/dd/df/d1c/d21/d5c/c99 0 2024-03-11T19:37:28.045 INFO:tasks.workunit.client.0.smithi090.stdout:7/510: dwrite d3/d55/f64 [0,4194304] 0 2024-03-11T19:37:28.421 INFO:tasks.workunit.client.0.smithi090.stdout:1/508: creat d5/da/dd/fa9 x:0 0 0 2024-03-11T19:37:28.465 INFO:tasks.workunit.client.0.smithi090.stdout:1/509: stat d5/d31/c46 0 2024-03-11T19:37:28.476 INFO:tasks.workunit.client.0.smithi090.stdout:1/510: stat d5/da 0 2024-03-11T19:37:28.614 INFO:tasks.workunit.client.0.smithi090.stdout:7/511: creat d3/dc/d1e/d7c/d96/fb1 x:0 0 0 2024-03-11T19:37:28.731 INFO:tasks.workunit.client.0.smithi090.stdout:8/556: write d0/d43/dc/f75 [75161,14361] 0 2024-03-11T19:37:28.798 INFO:tasks.workunit.client.0.smithi090.stdout:6/581: write d4/d14/d8a/d74/fba [431134,24334] 0 2024-03-11T19:37:29.055 INFO:tasks.workunit.client.0.smithi090.stdout:1/511: creat d5/d4e/faa x:0 0 0 2024-03-11T19:37:29.240 INFO:tasks.workunit.client.0.smithi090.stdout:8/557: mkdir d0/d60/db6 0 2024-03-11T19:37:29.358 INFO:tasks.workunit.client.0.smithi090.stdout:6/582: creat d4/d13/d22/d42/d48/d4e/fbf x:0 0 0 2024-03-11T19:37:29.408 INFO:tasks.workunit.client.0.smithi090.stdout:6/583: chown d4/d13/d22/d34/d37/d7f/d90/fad 118 1 2024-03-11T19:37:30.003 INFO:tasks.workunit.client.0.smithi090.stdout:9/511: getdents d1/dd/d18/d3a 0 2024-03-11T19:37:30.296 INFO:tasks.workunit.client.0.smithi090.stdout:7/512: rename d3/dc/d1e/d7c/d96/d32/d71/f78 to d3/dc/fb2 0 2024-03-11T19:37:30.744 INFO:tasks.workunit.client.0.smithi090.stdout:4/572: dwrite d4/f8 [4194304,4194304] 0 2024-03-11T19:37:31.011 INFO:tasks.workunit.client.0.smithi090.stdout:1/512: write d5/da/f92 [368203,38049] 0 2024-03-11T19:37:31.098 INFO:tasks.workunit.client.0.smithi090.stdout:8/558: write d0/d3/d4c/d8d/f94 [3837200,86744] 0 2024-03-11T19:37:31.121 INFO:tasks.workunit.client.0.smithi090.stdout:7/513: unlink d3/d55/f6a 0 2024-03-11T19:37:31.790 INFO:tasks.workunit.client.0.smithi090.stdout:9/512: rename d1/f79 to d1/dd/d18/d3a/d48/f9a 0 2024-03-11T19:37:31.866 INFO:tasks.workunit.client.0.smithi090.stdout:5/500: dwrite d1/d17/d39/d51/d53/d7b/f44 [0,4194304] 0 2024-03-11T19:37:32.056 INFO:tasks.workunit.client.0.smithi090.stdout:2/515: dwrite d0/f3 [0,4194304] 0 2024-03-11T19:37:32.143 INFO:tasks.workunit.client.0.smithi090.stdout:8/559: fdatasync d0/d53/f7a 0 2024-03-11T19:37:32.447 INFO:tasks.workunit.client.0.smithi090.stdout:0/537: dwrite d2/d11/d15/d29/d43/d58/f65 [0,4194304] 0 2024-03-11T19:37:32.593 INFO:tasks.workunit.client.0.smithi090.stdout:3/494: dwrite d2/db/d15/d1a/f3a [0,4194304] 0 2024-03-11T19:37:32.793 INFO:tasks.workunit.client.0.smithi090.stdout:5/501: fdatasync d1/d26/f20 0 2024-03-11T19:37:32.918 INFO:tasks.workunit.client.0.smithi090.stdout:6/584: dwrite d4/d13/d22/d42/d48/d4e/fae [0,4194304] 0 2024-03-11T19:37:32.924 INFO:tasks.workunit.client.0.smithi090.stdout:4/573: write d4/d51/d8c/d89/fbd [941352,115109] 0 2024-03-11T19:37:33.159 INFO:tasks.workunit.client.0.smithi090.stdout:8/560: creat d0/d43/dc/da1/fb7 x:0 0 0 2024-03-11T19:37:33.308 INFO:tasks.workunit.client.0.smithi090.stdout:2/516: creat d0/de/d12/d46/d49/f9d x:0 0 0 2024-03-11T19:37:33.394 INFO:tasks.workunit.client.0.smithi090.stdout:5/502: dread d1/d75/f8c [0,4194304] 0 2024-03-11T19:37:33.411 INFO:tasks.workunit.client.0.smithi090.stdout:1/513: dread d5/d4e/f5e [0,4194304] 0 2024-03-11T19:37:33.458 INFO:tasks.workunit.client.0.smithi090.stdout:5/503: chown d1/f1e 28487 1 2024-03-11T19:37:33.524 INFO:tasks.workunit.client.0.smithi090.stdout:6/585: sync 2024-03-11T19:37:33.897 INFO:tasks.workunit.client.0.smithi090.stdout:7/514: rename d3/dc/d1e/f11 to d3/dc/d1e/d7c/d96/d32/d71/d5d/fb3 0 2024-03-11T19:37:33.927 INFO:tasks.workunit.client.0.smithi090.stdout:8/561: mkdir d0/d53/db8 0 2024-03-11T19:37:33.982 INFO:tasks.workunit.client.0.smithi090.stdout:2/517: mknod d0/d21/d30/d29/c9e 0 2024-03-11T19:37:33.996 INFO:tasks.workunit.client.0.smithi090.stdout:9/513: write d1/dd/df/d1c/d21/d2a/f29 [238450,1529] 0 2024-03-11T19:37:34.300 INFO:tasks.workunit.client.0.smithi090.stdout:9/514: read - d1/d1d/f72 zero size 2024-03-11T19:37:34.442 INFO:tasks.workunit.client.0.smithi090.stdout:2/518: stat d0/de/d12/d46/d4c/d64/l94 0 2024-03-11T19:37:34.459 INFO:tasks.workunit.client.0.smithi090.stdout:6/586: write d4/d13/f3b [40457,19324] 0 2024-03-11T19:37:34.638 INFO:tasks.workunit.client.0.smithi090.stdout:0/538: dread d2/d11/d15/f19 [0,4194304] 0 2024-03-11T19:37:34.680 INFO:tasks.workunit.client.0.smithi090.stdout:4/574: unlink d4/d18/c36 0 2024-03-11T19:37:34.715 INFO:tasks.workunit.client.0.smithi090.stdout:5/504: symlink d1/d17/la4 0 2024-03-11T19:37:34.886 INFO:tasks.workunit.client.0.smithi090.stdout:3/495: write d2/db/f62 [4841978,79993] 0 2024-03-11T19:37:34.940 INFO:tasks.workunit.client.0.smithi090.stdout:1/514: dread d5/da/f47 [0,4194304] 0 2024-03-11T19:37:34.994 INFO:tasks.workunit.client.0.smithi090.stdout:9/515: write d1/dd/df/d1c/d21/d2a/f29 [4331827,81566] 0 2024-03-11T19:37:35.329 INFO:tasks.workunit.client.0.smithi090.stdout:0/539: fsync d2/d11/f41 0 2024-03-11T19:37:36.214 INFO:tasks.workunit.client.0.smithi090.stdout:5/505: creat d1/d17/d39/d51/d53/fa5 x:0 0 0 2024-03-11T19:37:36.470 INFO:tasks.workunit.client.0.smithi090.stdout:6/587: dwrite d4/d14/f78 [0,4194304] 0 2024-03-11T19:37:36.484 INFO:tasks.workunit.client.0.smithi090.stdout:7/515: dwrite d3/dc/fe [0,4194304] 0 2024-03-11T19:37:36.498 INFO:tasks.workunit.client.0.smithi090.stdout:8/562: dwrite d0/f2d [4194304,4194304] 0 2024-03-11T19:37:36.760 INFO:tasks.workunit.client.0.smithi090.stdout:1/515: creat d5/d65/d6b/d7f/fab x:0 0 0 2024-03-11T19:37:36.764 INFO:tasks.workunit.client.0.smithi090.stdout:8/563: stat d0/d53 0 2024-03-11T19:37:36.932 INFO:tasks.workunit.client.0.smithi090.stdout:0/540: mkdir d2/d11/d15/d29/d87/daf 0 2024-03-11T19:37:37.059 INFO:tasks.workunit.client.0.smithi090.stdout:2/519: mknod d0/de/d12/d46/d75/c9f 0 2024-03-11T19:37:37.292 INFO:tasks.workunit.client.0.smithi090.stdout:2/520: sync 2024-03-11T19:37:37.979 INFO:tasks.workunit.client.0.smithi090.stdout:7/516: read d3/dc/d1e/d7c/d96/d32/f4a [3079158,23299] 0 2024-03-11T19:37:38.030 INFO:tasks.workunit.client.0.smithi090.stdout:1/516: truncate d5/d31/d4c/f88 59046 0 2024-03-11T19:37:38.142 INFO:tasks.workunit.client.0.smithi090.stdout:0/541: symlink d2/d11/d3a/lb0 0 2024-03-11T19:37:38.270 INFO:tasks.workunit.client.0.smithi090.stdout:4/575: write d4/d27/f57 [3648441,83533] 0 2024-03-11T19:37:38.640 INFO:tasks.workunit.client.0.smithi090.stdout:2/521: rename d0/de/l38 to d0/d21/d30/d25/d8c/la0 0 2024-03-11T19:37:38.676 INFO:tasks.workunit.client.0.smithi090.stdout:2/522: chown d0 36885 1 2024-03-11T19:37:38.712 INFO:tasks.workunit.client.0.smithi090.stdout:5/506: mkdir d1/d17/d39/d51/d73/da6 0 2024-03-11T19:37:38.926 INFO:tasks.workunit.client.0.smithi090.stdout:9/516: dwrite d1/dd/d18/d32/f34 [0,4194304] 0 2024-03-11T19:37:39.052 INFO:tasks.workunit.client.0.smithi090.stdout:7/517: dread - d3/dc/d1e/d7c/d96/fb1 zero size 2024-03-11T19:37:40.514 INFO:tasks.workunit.client.0.smithi090.stdout:3/496: dwrite d2/db/d15/f48 [4194304,4194304] 0 2024-03-11T19:37:40.733 INFO:tasks.workunit.client.0.smithi090.stdout:3/497: chown d2/d2f/f30 3 1 2024-03-11T19:37:40.766 INFO:tasks.workunit.client.0.smithi090.stdout:8/564: dwrite d0/d53/d59/f72 [0,4194304] 0 2024-03-11T19:37:40.848 INFO:tasks.workunit.client.0.smithi090.stdout:7/518: creat d3/d55/fb4 x:0 0 0 2024-03-11T19:37:40.867 INFO:tasks.workunit.client.0.smithi090.stdout:9/517: creat d1/dd/d18/d3a/d48/f9b x:0 0 0 2024-03-11T19:37:41.071 INFO:tasks.workunit.client.0.smithi090.stdout:4/576: mknod d4/d27/d44/d66/da1/cc6 0 2024-03-11T19:37:41.090 INFO:tasks.workunit.client.0.smithi090.stdout:6/588: write d4/d13/d22/d42/f72 [402584,75206] 0 2024-03-11T19:37:41.214 INFO:tasks.workunit.client.0.smithi090.stdout:0/542: write d2/f10 [528810,15603] 0 2024-03-11T19:37:42.047 INFO:tasks.workunit.client.0.smithi090.stdout:2/523: write d0/de/d12/d46/f98 [996910,125170] 0 2024-03-11T19:37:42.375 INFO:tasks.workunit.client.0.smithi090.stdout:9/518: mkdir d1/d49/d9c 0 2024-03-11T19:37:42.525 INFO:tasks.workunit.client.0.smithi090.stdout:5/507: write d1/d17/f41 [919338,117413] 0 2024-03-11T19:37:42.601 INFO:tasks.workunit.client.0.smithi090.stdout:6/589: chown d4/d13/d22/d42/fac 1 1 2024-03-11T19:37:42.606 INFO:tasks.workunit.client.0.smithi090.stdout:8/565: dread d0/d43/f84 [0,4194304] 0 2024-03-11T19:37:42.673 INFO:tasks.workunit.client.0.smithi090.stdout:5/508: dread - d1/d17/d39/d51/d73/f94 zero size 2024-03-11T19:37:42.714 INFO:tasks.workunit.client.0.smithi090.stdout:4/577: rename d4/d27/fa5 to d4/d51/d8c/d89/fc7 0 2024-03-11T19:37:42.733 INFO:tasks.workunit.client.0.smithi090.stdout:0/543: truncate d2/d11/d15/d29/f5b 1021250 0 2024-03-11T19:37:42.882 INFO:tasks.workunit.client.0.smithi090.stdout:0/544: dread d2/d11/f41 [0,4194304] 0 2024-03-11T19:37:42.949 INFO:tasks.workunit.client.0.smithi090.stdout:1/517: truncate d5/d31/d4c/f88 395381 0 2024-03-11T19:37:43.200 INFO:tasks.workunit.client.0.smithi090.stdout:3/498: truncate d2/f2c 1101728 0 2024-03-11T19:37:43.212 INFO:tasks.workunit.client.0.smithi090.stdout:2/524: truncate d0/de/d12/d46/d4c/d64/f82 78932 0 2024-03-11T19:37:43.919 INFO:tasks.workunit.client.0.smithi090.stdout:6/590: write d4/d14/f78 [5127256,101263] 0 2024-03-11T19:37:44.271 INFO:tasks.workunit.client.0.smithi090.stdout:4/578: rename d4/d51/d8c/d6a/d8d/cb9 to d4/d51/d8c/d6a/cc8 0 2024-03-11T19:37:44.323 INFO:tasks.workunit.client.0.smithi090.stdout:4/579: stat d4/d51/d8c/d6a/d8d 0 2024-03-11T19:37:44.805 INFO:tasks.workunit.client.0.smithi090.stdout:7/519: dwrite d3/f70 [0,4194304] 0 2024-03-11T19:37:44.927 INFO:tasks.workunit.client.0.smithi090.stdout:8/566: mkdir d0/d60/db6/db9 0 2024-03-11T19:37:45.014 INFO:tasks.workunit.client.0.smithi090.stdout:9/519: truncate d1/dd/df/d1c/f81 191604 0 2024-03-11T19:37:45.133 INFO:tasks.workunit.client.0.smithi090.stdout:0/545: rename d2/d11/d15/l7f to d2/d11/d15/d29/d3d/d8c/lb1 0 2024-03-11T19:37:45.557 INFO:tasks.workunit.client.0.smithi090.stdout:4/580: mkdir d4/d51/dc9 0 2024-03-11T19:37:46.154 INFO:tasks.workunit.client.0.smithi090.stdout:7/520: fsync d3/dc/d87/f90 0 2024-03-11T19:37:46.247 INFO:tasks.workunit.client.0.smithi090.stdout:8/567: mkdir d0/d3/d86/d93/dba 0 2024-03-11T19:37:46.297 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:37:46.300 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:37:46.370 INFO:tasks.workunit.client.0.smithi090.stdout:9/520: creat d1/dd/df/d1c/d21/f9d x:0 0 0 2024-03-11T19:37:46.600 INFO:tasks.workunit.client.0.smithi090.stdout:0/546: stat d2/d11/d15/d29/d43/d58/f9f 0 2024-03-11T19:37:46.991 INFO:tasks.workunit.client.0.smithi090.stdout:1/518: symlink d5/d31/d76/d77/lac 0 2024-03-11T19:37:47.015 INFO:tasks.workunit.client.0.smithi090.stdout:2/525: write d0/d21/d30/f85 [391610,103155] 0 2024-03-11T19:37:47.518 INFO:tasks.workunit.client.0.smithi090.stdout:8/568: mknod d0/d3/d3e/cbb 0 2024-03-11T19:37:47.567 INFO:tasks.workunit.client.0.smithi090.stdout:9/521: symlink d1/dd/df/l9e 0 2024-03-11T19:37:47.792 INFO:tasks.workunit.client.0.smithi090.stdout:6/591: rename d4/d24/d80/la9 to d4/d13/d22/d3a/d6f/lc0 0 2024-03-11T19:37:47.938 INFO:tasks.workunit.client.0.smithi090.stdout:8/569: dread d0/d43/dc/f3a [0,4194304] 0 2024-03-11T19:37:48.221 INFO:tasks.workunit.client.0.smithi090.stdout:3/499: write d2/f2c [609857,129273] 0 2024-03-11T19:37:48.275 INFO:tasks.workunit.client.0.smithi090.stdout:3/500: read d2/d45/f70 [4021000,78296] 0 2024-03-11T19:37:48.928 INFO:tasks.workunit.client.0.smithi090.stdout:1/519: symlink d5/d65/d6b/lad 0 2024-03-11T19:37:48.942 INFO:tasks.workunit.client.0.smithi090.stdout:2/526: fsync d0/d21/d30/f47 0 2024-03-11T19:37:48.994 INFO:tasks.workunit.client.0.smithi090.stdout:2/527: chown d0/d21/l2f 2 1 2024-03-11T19:37:49.316 INFO:tasks.workunit.client.0.smithi090.stdout:7/521: mkdir d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/db5 0 2024-03-11T19:37:49.518 INFO:tasks.workunit.client.0.smithi090.stdout:6/592: mkdir d4/d24/dc1 0 2024-03-11T19:37:49.581 INFO:tasks.workunit.client.0.smithi090.stdout:8/570: truncate d0/d3/f2c 1319427 0 2024-03-11T19:37:49.713 INFO:tasks.workunit.client.0.smithi090.stdout:4/581: write d4/d18/fb0 [1480729,31248] 0 2024-03-11T19:37:49.903 INFO:tasks.workunit.client.0.smithi090.stdout:3/501: dread d2/f29 [0,4194304] 0 2024-03-11T19:37:49.982 INFO:tasks.workunit.client.0.smithi090.stdout:1/520: symlink d5/d31/d36/lae 0 2024-03-11T19:37:50.376 INFO:tasks.workunit.client.0.smithi090.stdout:6/593: mkdir d4/d13/d22/d34/d40/d7d/dc2 0 2024-03-11T19:37:50.414 INFO:tasks.workunit.client.0.smithi090.stdout:2/528: dread d0/f2 [4194304,4194304] 0 2024-03-11T19:37:50.736 INFO:tasks.workunit.client.0.smithi090.stdout:5/509: creat d1/fa7 x:0 0 0 2024-03-11T19:37:51.630 INFO:tasks.workunit.client.0.smithi090.stdout:1/521: mkdir d5/d31/d4c/daf 0 2024-03-11T19:37:51.662 INFO:tasks.workunit.client.0.smithi090.stdout:6/594: creat d4/d24/fc3 x:0 0 0 2024-03-11T19:37:51.713 INFO:tasks.workunit.client.0.smithi090.stdout:8/571: unlink d0/d53/d59/f6b 0 2024-03-11T19:37:51.720 INFO:tasks.workunit.client.0.smithi090.stdout:5/510: creat d1/d17/d39/d51/d9a/fa8 x:0 0 0 2024-03-11T19:37:51.804 INFO:tasks.workunit.client.0.smithi090.stdout:5/511: stat d1/d17/c8d 0 2024-03-11T19:37:52.559 INFO:tasks.workunit.client.0.smithi090.stdout:0/547: dwrite d2/d11/d3f/f62 [0,4194304] 0 2024-03-11T19:37:52.579 INFO:tasks.workunit.client.0.smithi090.stdout:9/522: rename d1/dd/d18/d25/d4b to d1/dd/df/d1c/d9f 0 2024-03-11T19:37:52.723 INFO:tasks.workunit.client.0.smithi090.stdout:1/522: creat d5/d4e/d54/d94/fb0 x:0 0 0 2024-03-11T19:37:52.760 INFO:tasks.workunit.client.0.smithi090.stdout:9/523: chown d1/d49 3120029 1 2024-03-11T19:37:52.829 INFO:tasks.workunit.client.0.smithi090.stdout:9/524: chown d1/d1d/l35 2946 1 2024-03-11T19:37:52.860 INFO:tasks.workunit.client.0.smithi090.stdout:0/548: dread d2/d11/f17 [0,4194304] 0 2024-03-11T19:37:52.983 INFO:tasks.workunit.client.0.smithi090.stdout:8/572: mknod d0/d60/db6/cbc 0 2024-03-11T19:37:52.993 INFO:tasks.workunit.client.0.smithi090.stdout:7/522: write d3/f4 [1310661,89942] 0 2024-03-11T19:37:53.178 INFO:tasks.workunit.client.0.smithi090.stdout:5/512: dread d1/d17/d39/d51/d53/d7b/f48 [0,4194304] 0 2024-03-11T19:37:53.470 INFO:tasks.workunit.client.0.smithi090.stdout:0/549: creat d2/d11/d15/d29/d3d/d7b/fb2 x:0 0 0 2024-03-11T19:37:53.697 INFO:tasks.workunit.client.0.smithi090.stdout:6/595: symlink d4/d13/d22/d34/d37/d7f/lc4 0 2024-03-11T19:37:54.049 INFO:tasks.workunit.client.0.smithi090.stdout:8/573: truncate d0/d43/dc/d90/fac 599576 0 2024-03-11T19:37:54.158 INFO:tasks.workunit.client.0.smithi090.stdout:1/523: symlink d5/d31/d86/lb1 0 2024-03-11T19:37:54.359 INFO:tasks.workunit.client.0.smithi090.stdout:4/582: write d4/d27/d28/f30 [1241698,55262] 0 2024-03-11T19:37:54.478 INFO:tasks.workunit.client.0.smithi090.stdout:6/596: creat d4/d13/d22/d42/d48/fc5 x:0 0 0 2024-03-11T19:37:54.830 INFO:tasks.workunit.client.0.smithi090.stdout:3/502: dwrite d2/db/d15/f1e [0,4194304] 0 2024-03-11T19:37:54.935 INFO:tasks.workunit.client.0.smithi090.stdout:1/524: rmdir d5/da/d1c/d3b/d40/d89 39 2024-03-11T19:37:54.962 INFO:tasks.workunit.client.0.smithi090.stdout:0/550: creat d2/d11/d15/d29/d87/daf/fb3 x:0 0 0 2024-03-11T19:37:54.998 INFO:tasks.workunit.client.0.smithi090.stdout:3/503: chown d2/d45/c6b 1983977 1 2024-03-11T19:37:55.023 INFO:tasks.workunit.client.0.smithi090.stdout:9/525: link d1/d1d/d6a/l7f d1/dd/d18/d32/la0 0 2024-03-11T19:37:55.075 INFO:tasks.workunit.client.0.smithi090.stdout:9/526: read - d1/dd/d18/d3a/f8e zero size 2024-03-11T19:37:55.281 INFO:tasks.workunit.client.0.smithi090.stdout:4/583: mknod d4/d27/d28/d7a/cca 0 2024-03-11T19:37:55.703 INFO:tasks.workunit.client.0.smithi090.stdout:0/551: read f0 [1011467,2245] 0 2024-03-11T19:37:56.139 INFO:tasks.workunit.client.0.smithi090.stdout:9/527: mknod d1/dd/d18/d3a/d70/ca1 0 2024-03-11T19:37:56.178 INFO:tasks.workunit.client.0.smithi090.stdout:1/525: read d5/da/d1c/d3b/d40/f43 [16332,30735] 0 2024-03-11T19:37:56.472 INFO:tasks.workunit.client.0.smithi090.stdout:0/552: dread d2/d11/d3f/f83 [0,4194304] 0 2024-03-11T19:37:57.094 INFO:tasks.workunit.client.0.smithi090.stdout:1/526: mkdir d5/da/d1c/d6d/d80/db2 0 2024-03-11T19:37:57.162 INFO:tasks.workunit.client.0.smithi090.stdout:0/553: dread - d2/d11/d15/d29/fa5 zero size 2024-03-11T19:37:57.336 INFO:tasks.workunit.client.0.smithi090.stdout:8/574: write d0/d3/fa8 [377452,7670] 0 2024-03-11T19:37:57.551 INFO:tasks.workunit.client.0.smithi090.stdout:5/513: dwrite d1/d17/d39/f68 [0,4194304] 0 2024-03-11T19:37:57.650 INFO:tasks.workunit.client.0.smithi090.stdout:2/529: mknod d0/ca1 0 2024-03-11T19:37:57.676 INFO:tasks.workunit.client.0.smithi090.stdout:2/530: readlink d0/l8 0 2024-03-11T19:37:57.729 INFO:tasks.workunit.client.0.smithi090.stdout:2/531: read d0/de/f16 [621589,10051] 0 2024-03-11T19:37:57.758 INFO:tasks.workunit.client.0.smithi090.stdout:1/527: creat d5/da/d1c/d3b/fb3 x:0 0 0 2024-03-11T19:37:57.966 INFO:tasks.workunit.client.0.smithi090.stdout:6/597: write d4/d13/d22/d34/d40/fbb [966802,32083] 0 2024-03-11T19:37:58.276 INFO:tasks.workunit.client.0.smithi090.stdout:5/514: dread d1/d17/d39/d51/d53/f67 [0,4194304] 0 2024-03-11T19:37:58.346 INFO:tasks.workunit.client.0.smithi090.stdout:7/523: rmdir d3/d55 39 2024-03-11T19:37:58.566 INFO:tasks.workunit.client.0.smithi090.stdout:6/598: symlink d4/d13/d22/d42/d48/d4e/lc6 0 2024-03-11T19:37:58.700 INFO:tasks.workunit.client.0.smithi090.stdout:8/575: creat d0/d3/d3e/d85/fbd x:0 0 0 2024-03-11T19:37:58.787 INFO:tasks.workunit.client.0.smithi090.stdout:1/528: truncate d5/f9a 1882552 0 2024-03-11T19:37:59.021 INFO:tasks.workunit.client.0.smithi090.stdout:7/524: symlink d3/d49/d8e/lb6 0 2024-03-11T19:37:59.252 INFO:tasks.workunit.client.0.smithi090.stdout:4/584: dwrite d4/d27/d28/d7a/fc0 [0,4194304] 0 2024-03-11T19:37:59.399 INFO:tasks.workunit.client.0.smithi090.stdout:6/599: truncate d4/d14/f78 621425 0 2024-03-11T19:37:59.418 INFO:tasks.workunit.client.0.smithi090.stdout:8/576: fdatasync d0/d43/dc/da1/fb7 0 2024-03-11T19:37:59.418 INFO:tasks.workunit.client.0.smithi090.stdout:3/504: write d2/f12 [1319045,129050] 0 2024-03-11T19:37:59.428 INFO:tasks.workunit.client.0.smithi090.stdout:9/528: write d1/dd/d18/d25/f60 [3857723,87086] 0 2024-03-11T19:37:59.525 INFO:tasks.workunit.client.0.smithi090.stdout:6/600: chown d4/f11 38 1 2024-03-11T19:37:59.867 INFO:tasks.workunit.client.0.smithi090.stdout:4/585: rename d4/d27/d28/d38/f6b to d4/d51/d8c/d6a/d8d/fcb 0 2024-03-11T19:37:59.986 INFO:tasks.workunit.client.0.smithi090.stdout:8/577: stat d0/d43/dc/l7f 0 2024-03-11T19:38:00.218 INFO:tasks.workunit.client.0.smithi090.stdout:9/529: truncate d1/dd/f69 93036 0 2024-03-11T19:38:00.675 INFO:tasks.workunit.client.0.smithi090.stdout:0/554: dwrite d2/d11/d3a/f70 [0,4194304] 0 2024-03-11T19:38:00.881 INFO:tasks.workunit.client.0.smithi090.stdout:8/578: readlink d0/d3/d4c/l73 0 2024-03-11T19:38:00.937 INFO:tasks.workunit.client.0.smithi090.stdout:3/505: symlink d2/db/d15/d1a/d75/d8c/l9d 0 2024-03-11T19:38:01.023 INFO:tasks.workunit.client.0.smithi090.stdout:2/532: write d0/de/d12/f1d [761864,78076] 0 2024-03-11T19:38:01.298 INFO:tasks.workunit.client.0.smithi090.stdout:9/530: dread d1/dd/d18/f65 [0,4194304] 0 2024-03-11T19:38:01.424 INFO:tasks.workunit.client.0.smithi090.stdout:5/515: write d1/d26/d19/f42 [230942,117356] 0 2024-03-11T19:38:01.773 INFO:tasks.workunit.client.0.smithi090.stdout:0/555: fsync d2/d11/d15/d29/d3d/d7b/fb2 0 2024-03-11T19:38:02.046 INFO:tasks.workunit.client.0.smithi090.stdout:2/533: truncate d0/de/d12/d46/d5d/d63/d8e/f92 2862762 0 2024-03-11T19:38:02.060 INFO:tasks.workunit.client.0.smithi090.stdout:9/531: mkdir d1/dd/d18/d3a/da2 0 2024-03-11T19:38:02.078 INFO:tasks.workunit.client.0.smithi090.stdout:6/601: rmdir d4/d13/d22/d42/d48/d4e/d61 0 2024-03-11T19:38:02.115 INFO:tasks.workunit.client.0.smithi090.stdout:0/556: dread d2/d11/d15/d36/f57 [0,4194304] 0 2024-03-11T19:38:02.201 INFO:tasks.workunit.client.0.smithi090.stdout:2/534: chown d0/d21/d30/d29/l5a 53854151 1 2024-03-11T19:38:02.232 INFO:tasks.workunit.client.0.smithi090.stdout:6/602: read d4/d14/d92/f99 [181916,18236] 0 2024-03-11T19:38:02.506 INFO:tasks.workunit.client.0.smithi090.stdout:7/525: dwrite d3/dc/fe [0,4194304] 0 2024-03-11T19:38:02.616 INFO:tasks.workunit.client.0.smithi090.stdout:1/529: rmdir d5/d65 39 2024-03-11T19:38:02.641 INFO:tasks.workunit.client.0.smithi090.stdout:3/506: mknod d2/db/d15/d1a/d75/c9e 0 2024-03-11T19:38:02.658 INFO:tasks.workunit.client.0.smithi090.stdout:8/579: rmdir d0 39 2024-03-11T19:38:02.728 INFO:tasks.workunit.client.0.smithi090.stdout:9/532: creat d1/dd/df/d1c/d21/d2a/fa3 x:0 0 0 2024-03-11T19:38:03.160 INFO:tasks.workunit.client.0.smithi090.stdout:5/516: rmdir d1/d17/d39/d51/d53/d7b/d3b/d76/d92 39 2024-03-11T19:38:03.486 INFO:tasks.workunit.client.0.smithi090.stdout:8/580: chown d0/d3/d4c/l7d 30 1 2024-03-11T19:38:03.722 INFO:tasks.workunit.client.0.smithi090.stdout:1/530: symlink d5/d4e/d54/d8e/lb4 0 2024-03-11T19:38:03.752 INFO:tasks.workunit.client.0.smithi090.stdout:1/531: dread - d5/d31/d4c/f91 zero size 2024-03-11T19:38:03.863 INFO:tasks.workunit.client.0.smithi090.stdout:9/533: mkdir d1/dd/df/d1c/d21/d2a/da4 0 2024-03-11T19:38:04.698 INFO:tasks.workunit.client.0.smithi090.stdout:2/535: mkdir d0/de/d12/d46/d5d/d63/d8e/da2 0 2024-03-11T19:38:04.712 INFO:tasks.workunit.client.0.smithi090.stdout:7/526: creat d3/d55/fb7 x:0 0 0 2024-03-11T19:38:04.954 INFO:tasks.workunit.client.0.smithi090.stdout:4/586: symlink d4/lcc 0 2024-03-11T19:38:05.760 INFO:tasks.workunit.client.0.smithi090.stdout:0/557: dwrite d2/d11/d15/d29/d43/d58/f88 [0,4194304] 0 2024-03-11T19:38:06.037 INFO:tasks.workunit.client.0.smithi090.stdout:2/536: truncate d0/de/f68 267425 0 2024-03-11T19:38:06.428 INFO:tasks.workunit.client.0.smithi090.stdout:1/532: mknod d5/d65/d6b/d7f/cb5 0 2024-03-11T19:38:06.439 INFO:tasks.workunit.client.0.smithi090.stdout:9/534: mknod d1/dd/d18/ca5 0 2024-03-11T19:38:06.474 INFO:tasks.workunit.client.0.smithi090.stdout:4/587: read d4/d27/f57 [2115764,89693] 0 2024-03-11T19:38:06.476 INFO:tasks.workunit.client.0.smithi090.stdout:7/527: fdatasync d3/dc/d1e/d7c/d96/d32/d45/f88 0 2024-03-11T19:38:06.497 INFO:tasks.workunit.client.0.smithi090.stdout:3/507: write d2/d2f/f30 [1675438,13505] 0 2024-03-11T19:38:06.559 INFO:tasks.workunit.client.0.smithi090.stdout:0/558: dread d2/d11/d15/d29/d43/d58/f9e [0,4194304] 0 2024-03-11T19:38:06.585 INFO:tasks.workunit.client.0.smithi090.stdout:3/508: chown d2/d1d/d47/c96 172380 1 2024-03-11T19:38:06.827 INFO:tasks.workunit.client.0.smithi090.stdout:8/581: link d0/d3/d4c/l7d d0/d53/d59/lbe 0 2024-03-11T19:38:06.870 INFO:tasks.workunit.client.0.smithi090.stdout:8/582: chown d0/d3/d4c/l63 62457 1 2024-03-11T19:38:07.206 INFO:tasks.workunit.client.0.smithi090.stdout:6/603: write d4/d14/d8a/fb8 [1361866,67209] 0 2024-03-11T19:38:07.287 INFO:tasks.workunit.client.0.smithi090.stdout:4/588: dread d4/f60 [0,4194304] 0 2024-03-11T19:38:07.433 INFO:tasks.workunit.client.0.smithi090.stdout:7/528: creat d3/dc/d1e/d7c/d96/fb8 x:0 0 0 2024-03-11T19:38:07.669 INFO:tasks.workunit.client.0.smithi090.stdout:3/509: chown d2/d4f/l7d 83809 1 2024-03-11T19:38:07.728 INFO:tasks.workunit.client.0.smithi090.stdout:0/559: fdatasync d2/d11/d15/d29/f68 0 2024-03-11T19:38:07.774 INFO:tasks.workunit.client.0.smithi090.stdout:0/560: readlink d2/d11/d15/d29/d43/l66 0 2024-03-11T19:38:07.817 INFO:tasks.workunit.client.0.smithi090.stdout:0/561: chown d2/d11/d15/d29/d43/d58/f65 940731 1 2024-03-11T19:38:08.128 INFO:tasks.workunit.client.0.smithi090.stdout:8/583: dread d0/d43/dc/d90/f9e [0,4194304] 0 2024-03-11T19:38:08.156 INFO:tasks.workunit.client.0.smithi090.stdout:8/584: chown d0/d3/d3e/f4e 2902910 1 2024-03-11T19:38:08.176 INFO:tasks.workunit.client.0.smithi090.stdout:8/585: chown d0/d43/dc/d90/c97 1334959033 1 2024-03-11T19:38:08.313 INFO:tasks.workunit.client.0.smithi090.stdout:9/535: unlink d1/dd/f3c 0 2024-03-11T19:38:09.407 INFO:tasks.workunit.client.0.smithi090.stdout:0/562: dread d2/d11/d15/f28 [0,4194304] 0 2024-03-11T19:38:09.802 INFO:tasks.workunit.client.0.smithi090.stdout:9/536: read d1/dd/df/d1c/d21/d5c/f8b [5397371,112277] 0 2024-03-11T19:38:10.069 INFO:tasks.workunit.client.0.smithi090.stdout:6/604: creat d4/d13/d22/d34/d83/d9c/db7/fc7 x:0 0 0 2024-03-11T19:38:10.217 INFO:tasks.workunit.client.0.smithi090.stdout:5/517: creat d1/d17/fa9 x:0 0 0 2024-03-11T19:38:10.545 INFO:tasks.workunit.client.0.smithi090.stdout:9/537: truncate d1/dd/f13 4349283 0 2024-03-11T19:38:10.581 INFO:tasks.workunit.client.0.smithi090.stdout:9/538: chown d1/dd/d18/d3a/l43 14 1 2024-03-11T19:38:10.678 INFO:tasks.workunit.client.0.smithi090.stdout:1/533: link d5/cb d5/d4e/d54/cb6 0 2024-03-11T19:38:10.857 INFO:tasks.workunit.client.0.smithi090.stdout:6/605: readlink d4/d13/l4a 0 2024-03-11T19:38:11.315 INFO:tasks.workunit.client.0.smithi090.stdout:9/539: symlink d1/dd/df/d1c/d88/la6 0 2024-03-11T19:38:11.451 INFO:tasks.workunit.client.0.smithi090.stdout:9/540: chown d1/dd/df/d1c/d88/d96 18903 1 2024-03-11T19:38:11.476 INFO:tasks.workunit.client.0.smithi090.stdout:2/537: dwrite d0/d21/d30/d29/f7a [0,4194304] 0 2024-03-11T19:38:11.623 INFO:tasks.workunit.client.0.smithi090.stdout:1/534: chown d5/da/d23/l64 372 1 2024-03-11T19:38:12.254 INFO:tasks.workunit.client.0.smithi090.stdout:1/535: unlink d5/da/l2e 0 2024-03-11T19:38:12.269 INFO:tasks.workunit.client.0.smithi090.stdout:9/541: dread d1/f58 [0,4194304] 0 2024-03-11T19:38:12.378 INFO:tasks.workunit.client.0.smithi090.stdout:4/589: dwrite d4/d27/d28/fbe [0,4194304] 0 2024-03-11T19:38:12.546 INFO:tasks.workunit.client.0.smithi090.stdout:5/518: creat d1/d17/d39/d51/d53/d7b/d3b/d76/faa x:0 0 0 2024-03-11T19:38:12.753 INFO:tasks.workunit.client.0.smithi090.stdout:7/529: dwrite d3/d55/f64 [0,4194304] 0 2024-03-11T19:38:12.876 INFO:tasks.workunit.client.0.smithi090.stdout:3/510: write d2/d1f/f36 [498657,87557] 0 2024-03-11T19:38:13.216 INFO:tasks.workunit.client.0.smithi090.stdout:1/536: mknod d5/da/dd/cb7 0 2024-03-11T19:38:13.469 INFO:tasks.workunit.client.0.smithi090.stdout:3/511: read d2/d1d/f77 [1072437,68500] 0 2024-03-11T19:38:13.576 INFO:tasks.workunit.client.0.smithi090.stdout:0/563: write d2/d11/d15/d36/f46 [4706853,81084] 0 2024-03-11T19:38:13.650 INFO:tasks.workunit.client.0.smithi090.stdout:8/586: dwrite d0/d53/d59/f72 [0,4194304] 0 2024-03-11T19:38:14.017 INFO:tasks.workunit.client.0.smithi090.stdout:6/606: write d4/d24/f3f [3568165,77058] 0 2024-03-11T19:38:14.072 INFO:tasks.workunit.client.0.smithi090.stdout:9/542: mknod d1/dd/d18/ca7 0 2024-03-11T19:38:14.106 INFO:tasks.workunit.client.0.smithi090.stdout:9/543: readlink d1/dd/df/d1c/d9f/l89 0 2024-03-11T19:38:14.361 INFO:tasks.workunit.client.0.smithi090.stdout:4/590: fdatasync d4/d51/d8c/d89/fc7 0 2024-03-11T19:38:14.380 INFO:tasks.workunit.client.0.smithi090.stdout:2/538: write d0/d21/d30/f47 [4941025,46208] 0 2024-03-11T19:38:14.999 INFO:tasks.workunit.client.0.smithi090.stdout:0/564: truncate d2/d11/d3a/f75 2086461 0 2024-03-11T19:38:15.641 INFO:tasks.workunit.client.0.smithi090.stdout:7/530: link d3/dc/fa8 d3/dc/d1e/d7c/d96/d32/d71/d5d/fb9 0 2024-03-11T19:38:15.713 INFO:tasks.workunit.client.0.smithi090.stdout:2/539: dread d0/fd [0,4194304] 0 2024-03-11T19:38:15.940 INFO:tasks.workunit.client.0.smithi090.stdout:0/565: chown d2/d11/d15/d29/f9a 1 1 2024-03-11T19:38:15.965 INFO:tasks.workunit.client.0.smithi090.stdout:3/512: mknod d2/db/c9f 0 2024-03-11T19:38:16.256 INFO:tasks.workunit.client.0.smithi090.stdout:5/519: link d1/d17/c1c d1/d75/cab 0 2024-03-11T19:38:16.588 INFO:tasks.workunit.client.0.smithi090.stdout:4/591: symlink d4/d18/lcd 0 2024-03-11T19:38:16.604 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:38:16.607 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:38:16.838 INFO:tasks.workunit.client.0.smithi090.stdout:2/540: chown d0/de/d12/d46/d5d/d63/f88 501043 1 2024-03-11T19:38:17.066 INFO:tasks.workunit.client.0.smithi090.stdout:8/587: rmdir d0/d53/db8 0 2024-03-11T19:38:17.085 INFO:tasks.workunit.client.0.smithi090.stdout:6/607: write d4/d14/d92/f99 [252238,81192] 0 2024-03-11T19:38:17.314 INFO:tasks.workunit.client.0.smithi090.stdout:8/588: dread - d0/d43/dc/f8e zero size 2024-03-11T19:38:17.536 INFO:tasks.workunit.client.0.smithi090.stdout:9/544: dwrite d1/dd/d18/d3a/f8e [0,4194304] 0 2024-03-11T19:38:17.680 INFO:tasks.workunit.client.0.smithi090.stdout:1/537: getdents d5/d65 0 2024-03-11T19:38:17.685 INFO:tasks.workunit.client.0.smithi090.stdout:5/520: dread - d1/d17/f6a zero size 2024-03-11T19:38:17.740 INFO:tasks.workunit.client.0.smithi090.stdout:4/592: creat d4/d27/d44/d66/fce x:0 0 0 2024-03-11T19:38:17.922 INFO:tasks.workunit.client.0.smithi090.stdout:9/545: dread d1/dd/d18/d3a/f8e [0,4194304] 0 2024-03-11T19:38:18.003 INFO:tasks.workunit.client.0.smithi090.stdout:6/608: symlink d4/d24/d80/lc8 0 2024-03-11T19:38:18.575 INFO:tasks.workunit.client.0.smithi090.stdout:8/589: creat d0/d3/fbf x:0 0 0 2024-03-11T19:38:18.720 INFO:tasks.workunit.client.0.smithi090.stdout:1/538: unlink d5/d31/d76/f78 0 2024-03-11T19:38:19.092 INFO:tasks.workunit.client.0.smithi090.stdout:8/590: sync 2024-03-11T19:38:19.107 INFO:tasks.workunit.client.0.smithi090.stdout:5/521: creat d1/d17/d39/d51/d53/fac x:0 0 0 2024-03-11T19:38:19.214 INFO:tasks.workunit.client.0.smithi090.stdout:7/531: write d3/dc/d1e/d7c/d96/d32/d71/d5d/f5a [1132832,33097] 0 2024-03-11T19:38:19.363 INFO:tasks.workunit.client.0.smithi090.stdout:9/546: rmdir d1/d47/d7c 39 2024-03-11T19:38:19.571 INFO:tasks.workunit.client.0.smithi090.stdout:0/566: rename d2/d11/d15/d29/d87 to d2/d11/d15/d29/d2e/db4 0 2024-03-11T19:38:19.621 INFO:tasks.workunit.client.0.smithi090.stdout:0/567: chown d2/d11/d3a/c8b 7896386 1 2024-03-11T19:38:20.621 INFO:tasks.workunit.client.0.smithi090.stdout:1/539: dread d5/f1b [0,4194304] 0 2024-03-11T19:38:21.402 INFO:tasks.workunit.client.0.smithi090.stdout:9/547: truncate d1/dd/d18/d32/f7a 76121 0 2024-03-11T19:38:21.848 INFO:tasks.workunit.client.0.smithi090.stdout:2/541: rename d0/de/d12/d46/d4c/f53 to d0/de/d12/d46/d4c/d64/fa3 0 2024-03-11T19:38:22.455 INFO:tasks.workunit.client.0.smithi090.stdout:3/513: dwrite d2/de/f91 [0,4194304] 0 2024-03-11T19:38:22.635 INFO:tasks.workunit.client.0.smithi090.stdout:7/532: symlink d3/dc/d1e/d7c/d96/d32/d7b/lba 0 2024-03-11T19:38:22.703 INFO:tasks.workunit.client.0.smithi090.stdout:6/609: dwrite d4/d24/fa8 [0,4194304] 0 2024-03-11T19:38:22.835 INFO:tasks.workunit.client.0.smithi090.stdout:2/542: fsync d0/f18 0 2024-03-11T19:38:22.860 INFO:tasks.workunit.client.0.smithi090.stdout:7/533: chown d3/d49/d8e 0 1 2024-03-11T19:38:23.055 INFO:tasks.workunit.client.0.smithi090.stdout:9/548: rename d1/dd/d18/d3a/f8e to d1/dd/df/d1c/d21/d2a/da4/fa8 0 2024-03-11T19:38:23.353 INFO:tasks.workunit.client.0.smithi090.stdout:5/522: write d1/d26/f3f [2527962,115091] 0 2024-03-11T19:38:23.358 INFO:tasks.workunit.client.0.smithi090.stdout:4/593: write d4/d18/f2a [2283228,10276] 0 2024-03-11T19:38:23.737 INFO:tasks.workunit.client.0.smithi090.stdout:1/540: fsync d5/da/d1c/d6d/f8d 0 2024-03-11T19:38:23.762 INFO:tasks.workunit.client.0.smithi090.stdout:2/543: creat d0/d21/d30/d29/fa4 x:0 0 0 2024-03-11T19:38:23.993 INFO:tasks.workunit.client.0.smithi090.stdout:6/610: truncate d4/d13/d22/d34/d83/d9c/db7/fc7 586567 0 2024-03-11T19:38:24.427 INFO:tasks.workunit.client.0.smithi090.stdout:8/591: rename d0/d3/d3e/cbb to d0/d53/d59/d82/cc0 0 2024-03-11T19:38:24.777 INFO:tasks.workunit.client.0.smithi090.stdout:4/594: mkdir d4/d27/d44/d66/dcf 0 2024-03-11T19:38:25.007 INFO:tasks.workunit.client.0.smithi090.stdout:5/523: dread d1/d17/d39/d51/d53/d7b/f44 [0,4194304] 0 2024-03-11T19:38:25.197 INFO:tasks.workunit.client.0.smithi090.stdout:5/524: sync 2024-03-11T19:38:25.256 INFO:tasks.workunit.client.0.smithi090.stdout:7/534: creat d3/dc/d1e/d7c/d96/d7f/fbb x:0 0 0 2024-03-11T19:38:25.475 INFO:tasks.workunit.client.0.smithi090.stdout:6/611: mknod d4/d13/d22/d42/d48/d4e/cc9 0 2024-03-11T19:38:25.491 INFO:tasks.workunit.client.0.smithi090.stdout:2/544: truncate d0/de/d12/d46/d75/f7c 4515779 0 2024-03-11T19:38:26.069 INFO:tasks.workunit.client.0.smithi090.stdout:8/592: read d0/d3/f37 [1375853,12886] 0 2024-03-11T19:38:26.102 INFO:tasks.workunit.client.0.smithi090.stdout:0/568: write d2/d11/d15/d29/d2e/d6d/fae [935209,33427] 0 2024-03-11T19:38:26.161 INFO:tasks.workunit.client.0.smithi090.stdout:0/569: dread - d2/d11/f24 zero size 2024-03-11T19:38:26.183 INFO:tasks.workunit.client.0.smithi090.stdout:0/570: chown d2/d11/d3f 1 1 2024-03-11T19:38:26.258 INFO:tasks.workunit.client.0.smithi090.stdout:1/541: rename d5/d31/c46 to d5/d4e/d79/cb8 0 2024-03-11T19:38:26.917 INFO:tasks.workunit.client.0.smithi090.stdout:2/545: symlink d0/d21/d30/d83/la5 0 2024-03-11T19:38:26.930 INFO:tasks.workunit.client.0.smithi090.stdout:8/593: fdatasync d0/d53/faf 0 2024-03-11T19:38:27.242 INFO:tasks.workunit.client.0.smithi090.stdout:1/542: creat d5/da/d1c/d6d/d80/fb9 x:0 0 0 2024-03-11T19:38:28.025 INFO:tasks.workunit.client.0.smithi090.stdout:0/571: getdents d2/d11/d15/d36 0 2024-03-11T19:38:28.820 INFO:tasks.workunit.client.0.smithi090.stdout:2/546: creat d0/de/d12/d46/d5d/d8d/fa6 x:0 0 0 2024-03-11T19:38:28.847 INFO:tasks.workunit.client.0.smithi090.stdout:6/612: rename d4/d14/d25/l86 to d4/d13/d22/lca 0 2024-03-11T19:38:29.035 INFO:tasks.workunit.client.0.smithi090.stdout:2/547: readlink d0/d21/d30/d25/l7b 0 2024-03-11T19:38:29.074 INFO:tasks.workunit.client.0.smithi090.stdout:9/549: mkdir d1/da9 0 2024-03-11T19:38:29.187 INFO:tasks.workunit.client.0.smithi090.stdout:3/514: dwrite d2/db/d15/f1e [0,4194304] 0 2024-03-11T19:38:29.305 INFO:tasks.workunit.client.0.smithi090.stdout:2/548: stat d0/d21/d30/d29/c97 0 2024-03-11T19:38:29.660 INFO:tasks.workunit.client.0.smithi090.stdout:4/595: write d4/d27/d28/f68 [2777583,51146] 0 2024-03-11T19:38:29.926 INFO:tasks.workunit.client.0.smithi090.stdout:5/525: dwrite d1/d26/d4d/f55 [0,4194304] 0 2024-03-11T19:38:29.943 INFO:tasks.workunit.client.0.smithi090.stdout:7/535: write d3/dc/d1e/d7c/d96/d32/d71/d80/f95 [836390,59738] 0 2024-03-11T19:38:30.074 INFO:tasks.workunit.client.0.smithi090.stdout:9/550: unlink d1/dd/d18/f38 0 2024-03-11T19:38:30.169 INFO:tasks.workunit.client.0.smithi090.stdout:2/549: mknod d0/de/d12/d46/d5d/d63/ca7 0 2024-03-11T19:38:30.212 INFO:tasks.workunit.client.0.smithi090.stdout:7/536: dread - d3/dc/d1e/d7c/d96/d32/d71/d5d/f40 zero size 2024-03-11T19:38:30.257 INFO:tasks.workunit.client.0.smithi090.stdout:7/537: readlink d3/dc/d1e/d7c/d96/d32/d7b/l8c 0 2024-03-11T19:38:30.305 INFO:tasks.workunit.client.0.smithi090.stdout:1/543: link d5/d31/d36/c62 d5/da/d1c/d3b/d8c/cba 0 2024-03-11T19:38:30.439 INFO:tasks.workunit.client.0.smithi090.stdout:7/538: sync 2024-03-11T19:38:30.472 INFO:tasks.workunit.client.0.smithi090.stdout:8/594: write d0/d43/dc/d90/fac [1153680,16527] 0 2024-03-11T19:38:30.796 INFO:tasks.workunit.client.0.smithi090.stdout:9/551: mknod d1/dd/d18/d3a/d57/caa 0 2024-03-11T19:38:30.801 INFO:tasks.workunit.client.0.smithi090.stdout:0/572: write d2/d11/f1a [1100242,111159] 0 2024-03-11T19:38:31.167 INFO:tasks.workunit.client.0.smithi090.stdout:4/596: rename d4/d18/d6f/db7/cc2 to d4/d51/d8c/d6a/d8d/cd0 0 2024-03-11T19:38:31.624 INFO:tasks.workunit.client.0.smithi090.stdout:6/613: link d4/d24/f3f d4/d13/d22/d34/d40/fcb 0 2024-03-11T19:38:31.690 INFO:tasks.workunit.client.0.smithi090.stdout:3/515: link d2/f29 d2/d1d/d47/d6a/fa0 0 2024-03-11T19:38:32.954 INFO:tasks.workunit.client.0.smithi090.stdout:7/539: mknod d3/d49/cbc 0 2024-03-11T19:38:33.435 INFO:tasks.workunit.client.0.smithi090.stdout:2/550: creat d0/de/d12/d46/fa8 x:0 0 0 2024-03-11T19:38:33.442 INFO:tasks.workunit.client.0.smithi090.stdout:9/552: getdents d1/da9 0 2024-03-11T19:38:33.465 INFO:tasks.workunit.client.0.smithi090.stdout:1/544: dwrite d5/d31/d36/f49 [0,4194304] 0 2024-03-11T19:38:33.851 INFO:tasks.workunit.client.0.smithi090.stdout:5/526: rename d1/d17/l24 to d1/d17/d39/d51/d53/lad 0 2024-03-11T19:38:33.863 INFO:tasks.workunit.client.0.smithi090.stdout:8/595: write d0/d53/f5d [4135840,68428] 0 2024-03-11T19:38:34.023 INFO:tasks.workunit.client.0.smithi090.stdout:5/527: sync 2024-03-11T19:38:34.109 INFO:tasks.workunit.client.0.smithi090.stdout:6/614: mknod d4/d14/ccc 0 2024-03-11T19:38:34.302 INFO:tasks.workunit.client.0.smithi090.stdout:2/551: creat d0/d21/fa9 x:0 0 0 2024-03-11T19:38:34.439 INFO:tasks.workunit.client.0.smithi090.stdout:9/553: mkdir d1/dd/df/d1c/d21/dab 0 2024-03-11T19:38:34.838 INFO:tasks.workunit.client.0.smithi090.stdout:7/540: symlink d3/dc/d1e/d7c/d96/d32/d7b/d99/lbd 0 2024-03-11T19:38:34.943 INFO:tasks.workunit.client.0.smithi090.stdout:0/573: dwrite d2/d11/d15/d29/d2e/d6d/f73 [0,4194304] 0 2024-03-11T19:38:35.100 INFO:tasks.workunit.client.0.smithi090.stdout:5/528: mknod d1/d17/d39/d51/d53/d7b/d3b/cae 0 2024-03-11T19:38:36.003 INFO:tasks.workunit.client.0.smithi090.stdout:8/596: creat d0/d3/d86/d93/fc1 x:0 0 0 2024-03-11T19:38:36.408 INFO:tasks.workunit.client.0.smithi090.stdout:3/516: getdents d2/db/d15/d1a/d75 0 2024-03-11T19:38:36.479 INFO:tasks.workunit.client.0.smithi090.stdout:4/597: dwrite d4/d27/d28/f63 [0,4194304] 0 2024-03-11T19:38:36.774 INFO:tasks.workunit.client.0.smithi090.stdout:7/541: dread d3/dc/d1e/d7c/d96/d32/d71/d5d/f24 [4194304,4194304] 0 2024-03-11T19:38:36.809 INFO:tasks.workunit.client.0.smithi090.stdout:7/542: readlink d3/l23 0 2024-03-11T19:38:36.833 INFO:tasks.workunit.client.0.smithi090.stdout:4/598: read d4/d27/d28/d38/d64/fb6 [4474986,74286] 0 2024-03-11T19:38:37.411 INFO:tasks.workunit.client.0.smithi090.stdout:1/545: rename d5/c13 to d5/d4e/d54/d94/cbb 0 2024-03-11T19:38:37.435 INFO:tasks.workunit.client.0.smithi090.stdout:5/529: symlink d1/d17/d39/d51/d53/d7b/laf 0 2024-03-11T19:38:37.572 INFO:tasks.workunit.client.0.smithi090.stdout:9/554: write d1/dd/df/d1c/d21/d2a/da4/fa8 [4728795,56201] 0 2024-03-11T19:38:37.626 INFO:tasks.workunit.client.0.smithi090.stdout:9/555: chown d1/d1d/l41 6171 1 2024-03-11T19:38:37.928 INFO:tasks.workunit.client.0.smithi090.stdout:7/543: fdatasync d3/dc/d1e/d7c/d96/d32/d71/d5d/f40 0 2024-03-11T19:38:38.359 INFO:tasks.workunit.client.0.smithi090.stdout:2/552: rename d0/f3e to d0/de/d12/d46/d5d/d63/faa 0 2024-03-11T19:38:38.529 INFO:tasks.workunit.client.0.smithi090.stdout:0/574: write d2/d11/d15/d29/d43/d58/f91 [327873,59018] 0 2024-03-11T19:38:38.984 INFO:tasks.workunit.client.0.smithi090.stdout:5/530: rename d1/d26/f3c to d1/d17/d39/d51/d73/fb0 0 2024-03-11T19:38:39.015 INFO:tasks.workunit.client.0.smithi090.stdout:6/615: dwrite d4/f62 [0,4194304] 0 2024-03-11T19:38:39.211 INFO:tasks.workunit.client.0.smithi090.stdout:1/546: dread d5/da/d23/f2d [0,4194304] 0 2024-03-11T19:38:39.304 INFO:tasks.workunit.client.0.smithi090.stdout:4/599: creat d4/d51/d8c/fd1 x:0 0 0 2024-03-11T19:38:40.032 INFO:tasks.workunit.client.0.smithi090.stdout:5/531: symlink d1/d17/d39/d51/d9d/lb1 0 2024-03-11T19:38:40.353 INFO:tasks.workunit.client.0.smithi090.stdout:3/517: unlink d2/c5 0 2024-03-11T19:38:40.388 INFO:tasks.workunit.client.0.smithi090.stdout:5/532: dread d1/d26/f6 [0,4194304] 0 2024-03-11T19:38:40.483 INFO:tasks.workunit.client.0.smithi090.stdout:0/575: mknod d2/d11/d15/d29/cb5 0 2024-03-11T19:38:40.891 INFO:tasks.workunit.client.0.smithi090.stdout:8/597: dwrite d0/d3/f81 [0,4194304] 0 2024-03-11T19:38:40.995 INFO:tasks.workunit.client.0.smithi090.stdout:6/616: mknod d4/d24/dc1/ccd 0 2024-03-11T19:38:41.122 INFO:tasks.workunit.client.0.smithi090.stdout:3/518: stat d2/db/d15/d69 0 2024-03-11T19:38:41.262 INFO:tasks.workunit.client.0.smithi090.stdout:5/533: chown d1/d26/f9f 0 1 2024-03-11T19:38:41.327 INFO:tasks.workunit.client.0.smithi090.stdout:9/556: dwrite d1/d49/f76 [0,4194304] 0 2024-03-11T19:38:41.537 INFO:tasks.workunit.client.0.smithi090.stdout:7/544: rename d3/d55/l92 to d3/dc/d1e/lbe 0 2024-03-11T19:38:41.548 INFO:tasks.workunit.client.0.smithi090.stdout:3/519: dread d2/d45/f70 [0,4194304] 0 2024-03-11T19:38:41.598 INFO:tasks.workunit.client.0.smithi090.stdout:7/545: chown d3/dc/d1e/d7c/d96/d32/d45/l69 1023843232 1 2024-03-11T19:38:41.783 INFO:tasks.workunit.client.0.smithi090.stdout:1/547: getdents d5/da/d1c/d3b/d40/d89 0 2024-03-11T19:38:41.832 INFO:tasks.workunit.client.0.smithi090.stdout:7/546: sync 2024-03-11T19:38:41.965 INFO:tasks.workunit.client.0.smithi090.stdout:8/598: creat d0/d3/d3e/fc2 x:0 0 0 2024-03-11T19:38:42.043 INFO:tasks.workunit.client.0.smithi090.stdout:2/553: dwrite d0/d21/f6f [0,4194304] 0 2024-03-11T19:38:42.131 INFO:tasks.workunit.client.0.smithi090.stdout:2/554: sync 2024-03-11T19:38:42.348 INFO:tasks.workunit.client.0.smithi090.stdout:2/555: dwrite d0/d21/f6f [0,4194304] 0 2024-03-11T19:38:42.667 INFO:tasks.workunit.client.0.smithi090.stdout:5/534: creat d1/d17/d39/d51/d53/fb2 x:0 0 0 2024-03-11T19:38:42.741 INFO:tasks.workunit.client.0.smithi090.stdout:9/557: dread d1/dd/df/f10 [0,4194304] 0 2024-03-11T19:38:42.938 INFO:tasks.workunit.client.0.smithi090.stdout:4/600: write d4/d27/d28/d7a/d9b/fad [268368,118613] 0 2024-03-11T19:38:43.157 INFO:tasks.workunit.client.0.smithi090.stdout:1/548: creat d5/d65/d6b/fbc x:0 0 0 2024-03-11T19:38:43.285 INFO:tasks.workunit.client.0.smithi090.stdout:7/547: fdatasync d3/dc/d1e/d7c/d96/fb1 0 2024-03-11T19:38:43.735 INFO:tasks.workunit.client.0.smithi090.stdout:9/558: dread d1/dd/d18/d32/f7a [0,4194304] 0 2024-03-11T19:38:43.762 INFO:tasks.workunit.client.0.smithi090.stdout:3/520: mkdir d2/d4f/d59/da1 0 2024-03-11T19:38:43.865 INFO:tasks.workunit.client.0.smithi090.stdout:9/559: sync 2024-03-11T19:38:44.097 INFO:tasks.workunit.client.0.smithi090.stdout:0/576: dwrite d2/d11/d3a/f42 [4194304,4194304] 0 2024-03-11T19:38:44.318 INFO:tasks.workunit.client.0.smithi090.stdout:7/548: mknod d3/dc/d1e/d7c/d96/d32/d45/cbf 0 2024-03-11T19:38:44.494 INFO:tasks.workunit.client.0.smithi090.stdout:0/577: dread f0 [0,4194304] 0 2024-03-11T19:38:44.729 INFO:tasks.workunit.client.0.smithi090.stdout:6/617: dwrite d4/d13/d22/d34/d37/d7f/d90/fad [0,4194304] 0 2024-03-11T19:38:44.844 INFO:tasks.workunit.client.0.smithi090.stdout:3/521: mkdir d2/db/d15/d69/da2 0 2024-03-11T19:38:45.595 INFO:tasks.workunit.client.0.smithi090.stdout:1/549: fdatasync d5/d31/d4c/f73 0 2024-03-11T19:38:45.598 INFO:tasks.workunit.client.0.smithi090.stdout:2/556: link d0/d21/d30/f31 d0/de/d12/d46/d5d/fab 0 2024-03-11T19:38:45.685 INFO:tasks.workunit.client.0.smithi090.stdout:2/557: chown d0/de/d12/d46/d5d/d63/ca7 5489 1 2024-03-11T19:38:45.711 INFO:tasks.workunit.client.0.smithi090.stdout:2/558: chown d0/d21/d30/c80 12937712 1 2024-03-11T19:38:45.772 INFO:tasks.workunit.client.0.smithi090.stdout:7/549: unlink d3/dc/d1e/d7c/d96/d32/d71/d5d/f20 0 2024-03-11T19:38:46.007 INFO:tasks.workunit.client.0.smithi090.stdout:0/578: creat d2/d11/d6e/fb6 x:0 0 0 2024-03-11T19:38:46.074 INFO:tasks.workunit.client.0.smithi090.stdout:6/618: read d4/d13/f3b [41170,75921] 0 2024-03-11T19:38:46.108 INFO:tasks.workunit.client.0.smithi090.stdout:0/579: readlink d2/d11/d15/d29/d2e/d6d/da4/l81 0 2024-03-11T19:38:46.147 INFO:tasks.workunit.client.0.smithi090.stdout:3/522: symlink d2/db/d15/d1a/d3c/la3 0 2024-03-11T19:38:46.365 INFO:tasks.workunit.client.0.smithi090.stdout:5/535: write d1/d26/f20 [3444266,113184] 0 2024-03-11T19:38:46.425 INFO:tasks.workunit.client.0.smithi090.stdout:4/601: rmdir d4/d51/dc9 0 2024-03-11T19:38:46.489 INFO:tasks.workunit.client.0.smithi090.stdout:8/599: dwrite d0/d43/dc/f12 [0,4194304] 0 2024-03-11T19:38:46.502 INFO:tasks.workunit.client.0.smithi090.stdout:9/560: creat d1/dd/d18/d3a/da2/fac x:0 0 0 2024-03-11T19:38:46.676 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:38:46.679 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:38:46.708 INFO:tasks.workunit.client.0.smithi090.stdout:1/550: mkdir d5/d31/d76/d77/dbd 0 2024-03-11T19:38:46.772 INFO:tasks.workunit.client.0.smithi090.stdout:2/559: fdatasync d0/de/d12/f1d 0 2024-03-11T19:38:47.090 INFO:tasks.workunit.client.0.smithi090.stdout:6/619: fdatasync d4/d14/d8a/f63 0 2024-03-11T19:38:47.508 INFO:tasks.workunit.client.0.smithi090.stdout:0/580: rmdir d2/d11/d15/d29/d3d/d8c 39 2024-03-11T19:38:47.562 INFO:tasks.workunit.client.0.smithi090.stdout:0/581: dread - d2/d11/d15/d36/f7c zero size 2024-03-11T19:38:47.735 INFO:tasks.workunit.client.0.smithi090.stdout:3/523: truncate d2/d1d/f77 4116063 0 2024-03-11T19:38:47.750 INFO:tasks.workunit.client.0.smithi090.stdout:4/602: fsync d4/d27/d28/d7a/fa6 0 2024-03-11T19:38:48.097 INFO:tasks.workunit.client.0.smithi090.stdout:9/561: creat d1/dd/df/d1c/fad x:0 0 0 2024-03-11T19:38:48.162 INFO:tasks.workunit.client.0.smithi090.stdout:5/536: truncate d1/d17/d39/d51/d53/d64/f6d 465817 0 2024-03-11T19:38:48.648 INFO:tasks.workunit.client.0.smithi090.stdout:2/560: dread d0/d21/d30/f47 [0,4194304] 0 2024-03-11T19:38:49.161 INFO:tasks.workunit.client.0.smithi090.stdout:0/582: dread d2/d11/d15/f74 [0,4194304] 0 2024-03-11T19:38:49.213 INFO:tasks.workunit.client.0.smithi090.stdout:0/583: read d2/d11/d3a/f42 [8011323,126899] 0 2024-03-11T19:38:49.380 INFO:tasks.workunit.client.0.smithi090.stdout:8/600: symlink d0/d60/db6/db9/lc3 0 2024-03-11T19:38:49.784 INFO:tasks.workunit.client.0.smithi090.stdout:1/551: mknod d5/da/cbe 0 2024-03-11T19:38:49.995 INFO:tasks.workunit.client.0.smithi090.stdout:5/537: rmdir d1/d75 39 2024-03-11T19:38:50.164 INFO:tasks.workunit.client.0.smithi090.stdout:2/561: rename d0/de/d12/d46/c70 to d0/de/d12/d46/d5d/d8d/cac 0 2024-03-11T19:38:50.206 INFO:tasks.workunit.client.0.smithi090.stdout:2/562: chown d0/d21/d30/d25 1 1 2024-03-11T19:38:50.457 INFO:tasks.workunit.client.0.smithi090.stdout:7/550: dwrite d3/f8 [0,4194304] 0 2024-03-11T19:38:50.885 INFO:tasks.workunit.client.0.smithi090.stdout:0/584: truncate d2/d11/d15/d29/d43/d58/f9e 576215 0 2024-03-11T19:38:50.906 INFO:tasks.workunit.client.0.smithi090.stdout:0/585: read f0 [2356335,80894] 0 2024-03-11T19:38:50.941 INFO:tasks.workunit.client.0.smithi090.stdout:8/601: truncate d0/d43/dc/f8e 838497 0 2024-03-11T19:38:51.067 INFO:tasks.workunit.client.0.smithi090.stdout:1/552: chown d5/da/c20 4229 1 2024-03-11T19:38:51.261 INFO:tasks.workunit.client.0.smithi090.stdout:9/562: truncate d1/dd/df/f85 503977 0 2024-03-11T19:38:51.471 INFO:tasks.workunit.client.0.smithi090.stdout:5/538: rename d1/d26/c8b to d1/d17/d39/d51/d73/cb3 0 2024-03-11T19:38:51.857 INFO:tasks.workunit.client.0.smithi090.stdout:6/620: rmdir d4/d13/d22/d42/d75 0 2024-03-11T19:38:52.295 INFO:tasks.workunit.client.0.smithi090.stdout:0/586: mkdir d2/d11/d3a/d64/db7 0 2024-03-11T19:38:52.531 INFO:tasks.workunit.client.0.smithi090.stdout:3/524: write d2/d1d/d47/d6a/f95 [445318,100793] 0 2024-03-11T19:38:52.675 INFO:tasks.workunit.client.0.smithi090.stdout:4/603: write d4/d27/d28/d38/d64/f93 [2542728,46856] 0 2024-03-11T19:38:54.249 INFO:tasks.workunit.client.0.smithi090.stdout:9/563: rename d1/dd/df/d1c/d21/l6e to d1/dd/df/d1c/d88/lae 0 2024-03-11T19:38:55.081 INFO:tasks.workunit.client.0.smithi090.stdout:9/564: dread - d1/dd/d18/d25/f97 zero size 2024-03-11T19:38:55.137 INFO:tasks.workunit.client.0.smithi090.stdout:2/563: write d0/de/f16 [1152958,112573] 0 2024-03-11T19:38:55.599 INFO:tasks.workunit.client.0.smithi090.stdout:7/551: dwrite d3/dc/f30 [4194304,4194304] 0 2024-03-11T19:38:55.733 INFO:tasks.workunit.client.0.smithi090.stdout:8/602: write d0/d43/f84 [3424527,11444] 0 2024-03-11T19:38:55.842 INFO:tasks.workunit.client.0.smithi090.stdout:0/587: rename d2/d11/d3f/f5c to d2/d11/d15/fb8 0 2024-03-11T19:38:56.424 INFO:tasks.workunit.client.0.smithi090.stdout:1/553: dwrite d5/d4e/d54/d94/fb0 [0,4194304] 0 2024-03-11T19:38:56.585 INFO:tasks.workunit.client.0.smithi090.stdout:6/621: write d4/d13/f1f [1821487,111811] 0 2024-03-11T19:38:57.250 INFO:tasks.workunit.client.0.smithi090.stdout:2/564: mkdir d0/dad 0 2024-03-11T19:38:57.261 INFO:tasks.workunit.client.0.smithi090.stdout:3/525: write d2/f6 [689571,3633] 0 2024-03-11T19:38:57.583 INFO:tasks.workunit.client.0.smithi090.stdout:0/588: creat d2/d11/d15/d29/fb9 x:0 0 0 2024-03-11T19:38:57.662 INFO:tasks.workunit.client.0.smithi090.stdout:3/526: sync 2024-03-11T19:38:57.960 INFO:tasks.workunit.client.0.smithi090.stdout:0/589: chown d2/d11/d6e 18542351 1 2024-03-11T19:38:58.003 INFO:tasks.workunit.client.0.smithi090.stdout:5/539: dwrite d1/d17/d39/d51/d53/d7b/f37 [0,4194304] 0 2024-03-11T19:38:58.373 INFO:tasks.workunit.client.0.smithi090.stdout:4/604: link d4/d27/d28/d69/la0 d4/ld2 0 2024-03-11T19:38:58.433 INFO:tasks.workunit.client.0.smithi090.stdout:1/554: creat d5/da/d23/fbf x:0 0 0 2024-03-11T19:38:58.443 INFO:tasks.workunit.client.0.smithi090.stdout:9/565: dwrite d1/dd/f13 [0,4194304] 0 2024-03-11T19:38:58.508 INFO:tasks.workunit.client.0.smithi090.stdout:4/605: sync 2024-03-11T19:38:58.704 INFO:tasks.workunit.client.0.smithi090.stdout:7/552: dwrite d3/dc/d1e/d7c/d96/d32/d71/fa9 [0,4194304] 0 2024-03-11T19:38:58.729 INFO:tasks.workunit.client.0.smithi090.stdout:8/603: dwrite d0/d43/dc/f28 [4194304,4194304] 0 2024-03-11T19:38:58.852 INFO:tasks.workunit.client.0.smithi090.stdout:5/540: dread - d1/d26/f83 zero size 2024-03-11T19:38:58.945 INFO:tasks.workunit.client.0.smithi090.stdout:4/606: readlink d4/d18/lcd 0 2024-03-11T19:38:59.007 INFO:tasks.workunit.client.0.smithi090.stdout:5/541: chown d1/d17/fa9 21240 1 2024-03-11T19:38:59.102 INFO:tasks.workunit.client.0.smithi090.stdout:2/565: creat d0/fae x:0 0 0 2024-03-11T19:38:59.316 INFO:tasks.workunit.client.0.smithi090.stdout:3/527: creat d2/db/d74/fa4 x:0 0 0 2024-03-11T19:38:59.829 INFO:tasks.workunit.client.0.smithi090.stdout:1/555: symlink d5/da/d1c/d6d/d80/lc0 0 2024-03-11T19:38:59.833 INFO:tasks.workunit.client.0.smithi090.stdout:6/622: write d4/d24/f5f [112537,47206] 0 2024-03-11T19:38:59.861 INFO:tasks.workunit.client.0.smithi090.stdout:8/604: fdatasync d0/d43/dc/fa3 0 2024-03-11T19:38:59.871 INFO:tasks.workunit.client.0.smithi090.stdout:4/607: rename d4/d27/d44/d74 to d4/d27/d28/d7a/d9b/dd3 0 2024-03-11T19:39:00.054 INFO:tasks.workunit.client.0.smithi090.stdout:2/566: mkdir d0/de/d12/d46/d5d/d63/d8e/daf 0 2024-03-11T19:39:00.072 INFO:tasks.workunit.client.0.smithi090.stdout:4/608: dread d4/f60 [0,4194304] 0 2024-03-11T19:39:00.089 INFO:tasks.workunit.client.0.smithi090.stdout:0/590: mkdir d2/d11/d3a/d64/dac/dba 0 2024-03-11T19:39:00.284 INFO:tasks.workunit.client.0.smithi090.stdout:1/556: dread d5/d65/f69 [0,4194304] 0 2024-03-11T19:39:00.300 INFO:tasks.workunit.client.0.smithi090.stdout:1/557: stat d5/d65/d6b/d7f/cb5 0 2024-03-11T19:39:00.566 INFO:tasks.workunit.client.0.smithi090.stdout:7/553: truncate d3/d49/d8e/f9b 3080631 0 2024-03-11T19:39:00.904 INFO:tasks.workunit.client.0.smithi090.stdout:4/609: fdatasync d4/d27/d28/d38/d75/fa9 0 2024-03-11T19:39:01.084 INFO:tasks.workunit.client.0.smithi090.stdout:3/528: truncate d2/d1d/fc 85208 0 2024-03-11T19:39:01.130 INFO:tasks.workunit.client.0.smithi090.stdout:0/591: dread d2/fab [0,4194304] 0 2024-03-11T19:39:01.314 INFO:tasks.workunit.client.0.smithi090.stdout:5/542: write d1/d17/d39/d51/d53/fb2 [770905,5137] 0 2024-03-11T19:39:01.568 INFO:tasks.workunit.client.0.smithi090.stdout:7/554: fsync d3/dc/d1e/d7c/d96/d32/d71/d80/f95 0 2024-03-11T19:39:01.597 INFO:tasks.workunit.client.0.smithi090.stdout:7/555: stat d3/l41 0 2024-03-11T19:39:01.815 INFO:tasks.workunit.client.0.smithi090.stdout:9/566: getdents d1/dd 0 2024-03-11T19:39:02.036 INFO:tasks.workunit.client.0.smithi090.stdout:4/610: fsync d4/f4a 0 2024-03-11T19:39:02.105 INFO:tasks.workunit.client.0.smithi090.stdout:8/605: write d0/d43/dc/f7e [560794,49709] 0 2024-03-11T19:39:02.218 INFO:tasks.workunit.client.0.smithi090.stdout:8/606: dread d0/d43/dc/d90/f9e [0,4194304] 0 2024-03-11T19:39:02.414 INFO:tasks.workunit.client.0.smithi090.stdout:8/607: sync 2024-03-11T19:39:02.536 INFO:tasks.workunit.client.0.smithi090.stdout:1/558: write d5/da/d97/fa8 [518181,110602] 0 2024-03-11T19:39:02.580 INFO:tasks.workunit.client.0.smithi090.stdout:5/543: dread - d1/fa7 zero size 2024-03-11T19:39:03.615 INFO:tasks.workunit.client.0.smithi090.stdout:7/556: creat d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/fc0 x:0 0 0 2024-03-11T19:39:04.574 INFO:tasks.workunit.client.0.smithi090.stdout:6/623: getdents d4/d24 0 2024-03-11T19:39:05.236 INFO:tasks.workunit.client.0.smithi090.stdout:9/567: mkdir d1/d49/d9c/daf 0 2024-03-11T19:39:05.396 INFO:tasks.workunit.client.0.smithi090.stdout:2/567: getdents d0/de/d12/d46/d4c/d64 0 2024-03-11T19:39:05.992 INFO:tasks.workunit.client.0.smithi090.stdout:0/592: creat d2/d11/d15/d29/d2e/fbb x:0 0 0 2024-03-11T19:39:06.027 INFO:tasks.workunit.client.0.smithi090.stdout:0/593: chown d2 199 1 2024-03-11T19:39:06.055 INFO:tasks.workunit.client.0.smithi090.stdout:7/557: dread d3/dc/d1e/f46 [0,4194304] 0 2024-03-11T19:39:06.767 INFO:tasks.workunit.client.0.smithi090.stdout:1/559: symlink d5/da/lc1 0 2024-03-11T19:39:06.869 INFO:tasks.workunit.client.0.smithi090.stdout:6/624: creat d4/d13/d22/d34/d83/fce x:0 0 0 2024-03-11T19:39:07.086 INFO:tasks.workunit.client.0.smithi090.stdout:9/568: dread - d1/dd/df/d1c/d21/f9d zero size 2024-03-11T19:39:07.311 INFO:tasks.workunit.client.0.smithi090.stdout:4/611: rename d4/d27/c5b to d4/d18/cd4 0 2024-03-11T19:39:07.320 INFO:tasks.workunit.client.0.smithi090.stdout:3/529: getdents d2/d2f 0 2024-03-11T19:39:07.474 INFO:tasks.workunit.client.0.smithi090.stdout:0/594: unlink d2/d11/d15/d29/d2e/d6d/da4/l6c 0 2024-03-11T19:39:07.657 INFO:tasks.workunit.client.0.smithi090.stdout:9/569: dread d1/dd/df/d1c/d21/f54 [0,4194304] 0 2024-03-11T19:39:07.714 INFO:tasks.workunit.client.0.smithi090.stdout:7/558: creat d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/fc1 x:0 0 0 2024-03-11T19:39:07.726 INFO:tasks.workunit.client.0.smithi090.stdout:8/608: write d0/d3/d98/da2/fa4 [1659761,122483] 0 2024-03-11T19:39:07.761 INFO:tasks.workunit.client.0.smithi090.stdout:0/595: fdatasync d2/d11/d15/d29/d2e/fbb 0 2024-03-11T19:39:08.264 INFO:tasks.workunit.client.0.smithi090.stdout:6/625: mknod d4/d13/d22/d34/d83/ccf 0 2024-03-11T19:39:08.274 INFO:tasks.workunit.client.0.smithi090.stdout:2/568: truncate d0/de/f68 333567 0 2024-03-11T19:39:08.309 INFO:tasks.workunit.client.0.smithi090.stdout:3/530: dread - d2/f5f zero size 2024-03-11T19:39:08.342 INFO:tasks.workunit.client.0.smithi090.stdout:2/569: chown d0/d21/l23 27818742 1 2024-03-11T19:39:08.395 INFO:tasks.workunit.client.0.smithi090.stdout:9/570: dread - d1/dd/d18/d25/f82 zero size 2024-03-11T19:39:08.445 INFO:tasks.workunit.client.0.smithi090.stdout:6/626: sync 2024-03-11T19:39:08.709 INFO:tasks.workunit.client.0.smithi090.stdout:7/559: rename d3/d49 to d3/dc/d1e/d7c/d96/d34/d4d/dc2 0 2024-03-11T19:39:09.065 INFO:tasks.workunit.client.0.smithi090.stdout:3/531: symlink d2/d4f/d59/la5 0 2024-03-11T19:39:09.281 INFO:tasks.workunit.client.0.smithi090.stdout:9/571: symlink d1/dd/df/d1c/d9f/lb0 0 2024-03-11T19:39:09.798 INFO:tasks.workunit.client.0.smithi090.stdout:2/570: symlink d0/de/d12/d46/d4c/lb0 0 2024-03-11T19:39:10.154 INFO:tasks.workunit.client.0.smithi090.stdout:3/532: mknod d2/d4f/ca6 0 2024-03-11T19:39:10.838 INFO:tasks.workunit.client.0.smithi090.stdout:7/560: mknod d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/db5/cc3 0 2024-03-11T19:39:10.972 INFO:tasks.workunit.client.0.smithi090.stdout:1/560: dwrite d5/da/f68 [0,4194304] 0 2024-03-11T19:39:10.976 INFO:tasks.workunit.client.0.smithi090.stdout:4/612: dwrite d4/d27/f57 [0,4194304] 0 2024-03-11T19:39:11.153 INFO:tasks.workunit.client.0.smithi090.stdout:8/609: dwrite d0/d43/dc/d90/fac [0,4194304] 0 2024-03-11T19:39:11.271 INFO:tasks.workunit.client.0.smithi090.stdout:5/544: creat d1/d17/fb4 x:0 0 0 2024-03-11T19:39:11.491 INFO:tasks.workunit.client.0.smithi090.stdout:9/572: creat d1/dd/df/fb1 x:0 0 0 2024-03-11T19:39:11.603 INFO:tasks.workunit.client.0.smithi090.stdout:7/561: read - d3/dc/fa7 zero size 2024-03-11T19:39:11.863 INFO:tasks.workunit.client.0.smithi090.stdout:0/596: rename d2/d11/d15/fb8 to d2/d11/d15/d29/fbc 0 2024-03-11T19:39:11.877 INFO:tasks.workunit.client.0.smithi090.stdout:6/627: rename d4/d13/d22/d34 to d4/d13/d22/d34/d37/d7f/d90/dd0 22 2024-03-11T19:39:11.951 INFO:tasks.workunit.client.0.smithi090.stdout:5/545: unlink d1/d26/c5 0 2024-03-11T19:39:12.156 INFO:tasks.workunit.client.0.smithi090.stdout:2/571: creat d0/de/d12/d46/fb1 x:0 0 0 2024-03-11T19:39:12.295 INFO:tasks.workunit.client.0.smithi090.stdout:9/573: mknod d1/dd/d18/d3a/d57/cb2 0 2024-03-11T19:39:12.509 INFO:tasks.workunit.client.0.smithi090.stdout:3/533: dwrite d2/d1f/f35 [0,4194304] 0 2024-03-11T19:39:12.657 INFO:tasks.workunit.client.0.smithi090.stdout:1/561: rename d5/d31/d86/lb1 to d5/d65/d6b/lc2 0 2024-03-11T19:39:12.661 INFO:tasks.workunit.client.0.smithi090.stdout:8/610: symlink d0/d3/d4c/d8d/lc4 0 2024-03-11T19:39:12.695 INFO:tasks.workunit.client.0.smithi090.stdout:3/534: sync 2024-03-11T19:39:12.841 INFO:tasks.workunit.client.0.smithi090.stdout:6/628: symlink d4/d13/d22/d34/d83/d9c/db7/ld1 0 2024-03-11T19:39:12.859 INFO:tasks.workunit.client.0.smithi090.stdout:2/572: symlink d0/d21/lb2 0 2024-03-11T19:39:12.984 INFO:tasks.workunit.client.0.smithi090.stdout:5/546: mkdir d1/d26/db5 0 2024-03-11T19:39:13.005 INFO:tasks.workunit.client.0.smithi090.stdout:8/611: truncate d0/d53/d59/f72 149700 0 2024-03-11T19:39:13.055 INFO:tasks.workunit.client.0.smithi090.stdout:1/562: read d5/d31/d36/f95 [799498,98485] 0 2024-03-11T19:39:13.082 INFO:tasks.workunit.client.0.smithi090.stdout:3/535: truncate d2/d1d/d47/d6a/f73 1296061 0 2024-03-11T19:39:13.126 INFO:tasks.workunit.client.0.smithi090.stdout:8/612: chown d0/d53/f71 5 1 2024-03-11T19:39:13.760 INFO:tasks.workunit.client.0.smithi090.stdout:7/562: creat d3/dc/fc4 x:0 0 0 2024-03-11T19:39:13.792 INFO:tasks.workunit.client.0.smithi090.stdout:9/574: fsync d1/dd/df/d1c/d21/f9d 0 2024-03-11T19:39:13.830 INFO:tasks.workunit.client.0.smithi090.stdout:0/597: creat d2/d11/d15/d29/fbd x:0 0 0 2024-03-11T19:39:13.865 INFO:tasks.workunit.client.0.smithi090.stdout:3/536: rmdir d2/d1f 39 2024-03-11T19:39:13.882 INFO:tasks.workunit.client.0.smithi090.stdout:5/547: creat d1/d26/d19/fb6 x:0 0 0 2024-03-11T19:39:13.897 INFO:tasks.workunit.client.0.smithi090.stdout:1/563: creat d5/d4e/d79/fc3 x:0 0 0 2024-03-11T19:39:13.902 INFO:tasks.workunit.client.0.smithi090.stdout:8/613: symlink d0/d3/d98/da2/lc5 0 2024-03-11T19:39:13.934 INFO:tasks.workunit.client.0.smithi090.stdout:2/573: read d0/d21/f42 [3258902,26938] 0 2024-03-11T19:39:14.046 INFO:tasks.workunit.client.0.smithi090.stdout:5/548: stat d1/d17/d39/c59 0 2024-03-11T19:39:14.047 INFO:tasks.workunit.client.0.smithi090.stdout:7/563: chown d3/dc/d1e/d7c/d96/d32/d71 1140 1 2024-03-11T19:39:14.105 INFO:tasks.workunit.client.0.smithi090.stdout:3/537: chown d2/db/d15/d1a/f33 14 1 2024-03-11T19:39:14.230 INFO:tasks.workunit.client.0.smithi090.stdout:5/549: chown d1/d17/d39/d51/d53/d7b/d3b/f5a 15647 1 2024-03-11T19:39:14.250 INFO:tasks.workunit.client.0.smithi090.stdout:4/613: write d4/d51/d8c/d6a/d8d/fcb [4905468,64931] 0 2024-03-11T19:39:15.022 INFO:tasks.workunit.client.0.smithi090.stdout:0/598: creat d2/d11/d15/d29/d43/fbe x:0 0 0 2024-03-11T19:39:15.279 INFO:tasks.workunit.client.0.smithi090.stdout:6/629: dwrite d4/f62 [4194304,4194304] 0 2024-03-11T19:39:15.416 INFO:tasks.workunit.client.0.smithi090.stdout:2/574: symlink d0/de/d12/d46/d5d/d63/d8e/lb3 0 2024-03-11T19:39:15.448 INFO:tasks.workunit.client.0.smithi090.stdout:7/564: creat d3/dc/d1e/d7c/d96/d32/d7b/d99/fc5 x:0 0 0 2024-03-11T19:39:15.508 INFO:tasks.workunit.client.0.smithi090.stdout:4/614: creat d4/d18/d6f/db7/fd5 x:0 0 0 2024-03-11T19:39:15.597 INFO:tasks.workunit.client.0.smithi090.stdout:4/615: chown d4/d27/c40 350 1 2024-03-11T19:39:15.605 INFO:tasks.workunit.client.0.smithi090.stdout:4/616: readlink d4/l49 0 2024-03-11T19:39:16.043 INFO:tasks.workunit.client.0.smithi090.stdout:0/599: dread d2/f37 [0,4194304] 0 2024-03-11T19:39:16.126 INFO:tasks.workunit.client.0.smithi090.stdout:3/538: symlink d2/d1d/la7 0 2024-03-11T19:39:16.170 INFO:tasks.workunit.client.0.smithi090.stdout:3/539: chown d2/db/d15/d1a/d75/d8c/f8d 187428 1 2024-03-11T19:39:16.365 INFO:tasks.workunit.client.0.smithi090.stdout:6/630: mkdir d4/d13/d22/d42/d48/dd2 0 2024-03-11T19:39:16.518 INFO:tasks.workunit.client.0.smithi090.stdout:2/575: dread d0/f3 [0,4194304] 0 2024-03-11T19:39:16.727 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:39:16.730 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:39:16.744 INFO:tasks.workunit.client.0.smithi090.stdout:7/565: rmdir d3/dc/d1e/d7c/d96/d34/d4d/dc2 39 2024-03-11T19:39:16.755 INFO:tasks.workunit.client.0.smithi090.stdout:5/550: mkdir d1/d17/d39/d51/d53/d7b/d3b/d76/db7 0 2024-03-11T19:39:17.139 INFO:tasks.workunit.client.0.smithi090.stdout:8/614: symlink d0/d3/db0/lc6 0 2024-03-11T19:39:17.512 INFO:tasks.workunit.client.0.smithi090.stdout:6/631: chown d4/c2c 380 1 2024-03-11T19:39:17.545 INFO:tasks.workunit.client.0.smithi090.stdout:6/632: chown d4/d14/d8a/d74/f7b 1610918762 1 2024-03-11T19:39:17.611 INFO:tasks.workunit.client.0.smithi090.stdout:2/576: stat d0/l91 0 2024-03-11T19:39:18.098 INFO:tasks.workunit.client.0.smithi090.stdout:1/564: dwrite d5/da/d1c/d3b/d40/f43 [0,4194304] 0 2024-03-11T19:39:18.801 INFO:tasks.workunit.client.0.smithi090.stdout:9/575: link d1/l20 d1/dd/df/d1c/d88/lb3 0 2024-03-11T19:39:19.072 INFO:tasks.workunit.client.0.smithi090.stdout:3/540: mknod d2/db/d15/ca8 0 2024-03-11T19:39:19.193 INFO:tasks.workunit.client.0.smithi090.stdout:2/577: chown d0/de/d12/c36 2465 1 2024-03-11T19:39:19.890 INFO:tasks.workunit.client.0.smithi090.stdout:7/566: mkdir d3/dc/d1e/d7c/d96/d34/d4d/dc6 0 2024-03-11T19:39:19.961 INFO:tasks.workunit.client.0.smithi090.stdout:4/617: dwrite d4/d27/d28/fbe [0,4194304] 0 2024-03-11T19:39:20.070 INFO:tasks.workunit.client.0.smithi090.stdout:9/576: symlink d1/dd/df/d1c/d77/lb4 0 2024-03-11T19:39:20.518 INFO:tasks.workunit.client.0.smithi090.stdout:0/600: dwrite d2/d11/d3f/fa9 [0,4194304] 0 2024-03-11T19:39:21.075 INFO:tasks.workunit.client.0.smithi090.stdout:3/541: truncate d2/d1d/f3d 687291 0 2024-03-11T19:39:21.264 INFO:tasks.workunit.client.0.smithi090.stdout:5/551: write d1/f4f [1076841,11609] 0 2024-03-11T19:39:21.432 INFO:tasks.workunit.client.0.smithi090.stdout:7/567: symlink d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/lc7 0 2024-03-11T19:39:21.557 INFO:tasks.workunit.client.0.smithi090.stdout:0/601: truncate d2/d11/d6e/fb6 857505 0 2024-03-11T19:39:21.582 INFO:tasks.workunit.client.0.smithi090.stdout:0/602: stat d2/d11/d15/d29/c5d 0 2024-03-11T19:39:21.740 INFO:tasks.workunit.client.0.smithi090.stdout:8/615: write d0/d43/f84 [1308220,120228] 0 2024-03-11T19:39:22.147 INFO:tasks.workunit.client.0.smithi090.stdout:3/542: symlink d2/db/d15/d1a/d75/la9 0 2024-03-11T19:39:22.247 INFO:tasks.workunit.client.0.smithi090.stdout:6/633: getdents d4/d13/d22/d34/d83 0 2024-03-11T19:39:22.571 INFO:tasks.workunit.client.0.smithi090.stdout:1/565: dwrite d5/d4e/d54/d94/f96 [4194304,4194304] 0 2024-03-11T19:39:22.818 INFO:tasks.workunit.client.0.smithi090.stdout:4/618: creat d4/fd6 x:0 0 0 2024-03-11T19:39:22.860 INFO:tasks.workunit.client.0.smithi090.stdout:9/577: creat d1/d47/fb5 x:0 0 0 2024-03-11T19:39:22.931 INFO:tasks.workunit.client.0.smithi090.stdout:6/634: mkdir d4/d13/d22/d3a/d6f/dd3 0 2024-03-11T19:39:23.172 INFO:tasks.workunit.client.0.smithi090.stdout:2/578: dwrite d0/de/d12/f1d [0,4194304] 0 2024-03-11T19:39:23.406 INFO:tasks.workunit.client.0.smithi090.stdout:0/603: stat d2/d11/d15/d29/d3d/d8c/lb1 0 2024-03-11T19:39:23.717 INFO:tasks.workunit.client.0.smithi090.stdout:0/604: sync 2024-03-11T19:39:23.773 INFO:tasks.workunit.client.0.smithi090.stdout:3/543: fdatasync d2/db/f8a 0 2024-03-11T19:39:24.089 INFO:tasks.workunit.client.0.smithi090.stdout:7/568: rename d3/dc/d1e/d7c/d96/d32/d7b/f8f to d3/fc8 0 2024-03-11T19:39:24.106 INFO:tasks.workunit.client.0.smithi090.stdout:6/635: mkdir d4/d24/d80/dd4 0 2024-03-11T19:39:24.262 INFO:tasks.workunit.client.0.smithi090.stdout:9/578: mkdir d1/dd/d18/d3a/db6 0 2024-03-11T19:39:24.465 INFO:tasks.workunit.client.0.smithi090.stdout:2/579: rmdir d0/de/d12/d46/d5d/d8d 39 2024-03-11T19:39:25.063 INFO:tasks.workunit.client.0.smithi090.stdout:3/544: creat d2/d45/faa x:0 0 0 2024-03-11T19:39:25.285 INFO:tasks.workunit.client.0.smithi090.stdout:5/552: dwrite d1/d26/f3f [0,4194304] 0 2024-03-11T19:39:25.461 INFO:tasks.workunit.client.0.smithi090.stdout:8/616: rename d0/d43/dc/da1/fb7 to d0/d60/db6/fc7 0 2024-03-11T19:39:25.683 INFO:tasks.workunit.client.0.smithi090.stdout:7/569: symlink d3/d55/lc9 0 2024-03-11T19:39:25.778 INFO:tasks.workunit.client.0.smithi090.stdout:2/580: dread - d0/f3a zero size 2024-03-11T19:39:26.459 INFO:tasks.workunit.client.0.smithi090.stdout:4/619: rename d4/d27/d28/d7a/d9b to d4/d27/d44/d66/dcf/dd7 0 2024-03-11T19:39:26.520 INFO:tasks.workunit.client.0.smithi090.stdout:8/617: symlink d0/d53/lc8 0 2024-03-11T19:39:26.550 INFO:tasks.workunit.client.0.smithi090.stdout:4/620: chown d4/d27/d28/d38/d75/fa9 199406 1 2024-03-11T19:39:26.815 INFO:tasks.workunit.client.0.smithi090.stdout:2/581: truncate d0/de/d12/d46/f5b 3162417 0 2024-03-11T19:39:26.946 INFO:tasks.workunit.client.0.smithi090.stdout:1/566: getdents d5 0 2024-03-11T19:39:27.274 INFO:tasks.workunit.client.0.smithi090.stdout:3/545: fdatasync d2/d45/f4a 0 2024-03-11T19:39:27.290 INFO:tasks.workunit.client.0.smithi090.stdout:0/605: write d2/f14 [1558198,73301] 0 2024-03-11T19:39:27.539 INFO:tasks.workunit.client.0.smithi090.stdout:4/621: read d4/d27/f57 [1943517,14847] 0 2024-03-11T19:39:27.955 INFO:tasks.workunit.client.0.smithi090.stdout:1/567: truncate d5/d4e/d79/f98 304742 0 2024-03-11T19:39:27.996 INFO:tasks.workunit.client.0.smithi090.stdout:1/568: chown d5/da/d1c/d3b/d40/f43 1 1 2024-03-11T19:39:28.582 INFO:tasks.workunit.client.0.smithi090.stdout:4/622: rmdir d4/d18/d6f 39 2024-03-11T19:39:28.617 INFO:tasks.workunit.client.0.smithi090.stdout:4/623: dread - d4/d27/d28/d7a/fa6 zero size 2024-03-11T19:39:29.196 INFO:tasks.workunit.client.0.smithi090.stdout:1/569: unlink d5/da/d23/l64 0 2024-03-11T19:39:29.282 INFO:tasks.workunit.client.0.smithi090.stdout:5/553: write d1/d17/d39/d51/d53/d7b/f44 [3273162,90726] 0 2024-03-11T19:39:29.926 INFO:tasks.workunit.client.0.smithi090.stdout:1/570: creat d5/da/d1c/d3b/d8c/fc4 x:0 0 0 2024-03-11T19:39:30.002 INFO:tasks.workunit.client.0.smithi090.stdout:4/624: dread d4/d18/f2a [0,4194304] 0 2024-03-11T19:39:30.012 INFO:tasks.workunit.client.0.smithi090.stdout:1/571: dread - d5/d65/fa6 zero size 2024-03-11T19:39:30.031 INFO:tasks.workunit.client.0.smithi090.stdout:8/618: truncate d0/d60/db6/fc7 850452 0 2024-03-11T19:39:30.160 INFO:tasks.workunit.client.0.smithi090.stdout:7/570: symlink d3/dc/d1e/d7c/d96/d32/lca 0 2024-03-11T19:39:30.379 INFO:tasks.workunit.client.0.smithi090.stdout:5/554: creat d1/d17/d39/d51/d53/d7b/d3b/d76/db7/fb8 x:0 0 0 2024-03-11T19:39:30.624 INFO:tasks.workunit.client.0.smithi090.stdout:8/619: mknod d0/d60/cc9 0 2024-03-11T19:39:30.647 INFO:tasks.workunit.client.0.smithi090.stdout:2/582: dwrite d0/f66 [4194304,4194304] 0 2024-03-11T19:39:30.974 INFO:tasks.workunit.client.0.smithi090.stdout:1/572: fdatasync d5/da/d1c/d6d/d80/fa0 0 2024-03-11T19:39:31.006 INFO:tasks.workunit.client.0.smithi090.stdout:7/571: creat d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/fcb x:0 0 0 2024-03-11T19:39:31.315 INFO:tasks.workunit.client.0.smithi090.stdout:8/620: unlink d0/d3/f37 0 2024-03-11T19:39:31.569 INFO:tasks.workunit.client.0.smithi090.stdout:6/636: rename d4/d24 to d4/d13/d22/d34/d40/dd5 0 2024-03-11T19:39:31.662 INFO:tasks.workunit.client.0.smithi090.stdout:5/555: dread d1/d17/d39/d51/d53/d7b/f37 [0,4194304] 0 2024-03-11T19:39:31.718 INFO:tasks.workunit.client.0.smithi090.stdout:3/546: dwrite d2/db/d15/d1a/f33 [0,4194304] 0 2024-03-11T19:39:32.008 INFO:tasks.workunit.client.0.smithi090.stdout:0/606: dwrite d2/d11/d15/d29/d2e/fa0 [0,4194304] 0 2024-03-11T19:39:32.118 INFO:tasks.workunit.client.0.smithi090.stdout:7/572: fsync d3/dc/d87/f90 0 2024-03-11T19:39:32.696 INFO:tasks.workunit.client.0.smithi090.stdout:4/625: dwrite d4/d27/d44/d66/fce [0,4194304] 0 2024-03-11T19:39:32.938 INFO:tasks.workunit.client.0.smithi090.stdout:5/556: dread d1/d17/d39/d51/d53/d64/f6d [0,4194304] 0 2024-03-11T19:39:33.002 INFO:tasks.workunit.client.0.smithi090.stdout:3/547: mknod d2/de/cab 0 2024-03-11T19:39:33.239 INFO:tasks.workunit.client.0.smithi090.stdout:1/573: mknod d5/da/d1c/d6d/d80/db2/cc5 0 2024-03-11T19:39:33.310 INFO:tasks.workunit.client.0.smithi090.stdout:3/548: dread d2/f8 [0,4194304] 0 2024-03-11T19:39:33.425 INFO:tasks.workunit.client.0.smithi090.stdout:7/573: truncate d3/dc/f30 8629068 0 2024-03-11T19:39:34.246 INFO:tasks.workunit.client.0.smithi090.stdout:4/626: truncate d4/d18/f37 342677 0 2024-03-11T19:39:34.713 INFO:tasks.workunit.client.0.smithi090.stdout:6/637: dwrite d4/d13/f1f [0,4194304] 0 2024-03-11T19:39:34.850 INFO:tasks.workunit.client.0.smithi090.stdout:0/607: creat d2/d11/d3a/d64/dac/dba/fbf x:0 0 0 2024-03-11T19:39:35.028 INFO:tasks.workunit.client.0.smithi090.stdout:1/574: rmdir d5/d31/d4c 39 2024-03-11T19:39:35.043 INFO:tasks.workunit.client.0.smithi090.stdout:3/549: rmdir d2/d45/d55 39 2024-03-11T19:39:35.409 INFO:tasks.workunit.client.0.smithi090.stdout:2/583: link d0/de/d1f/c84 d0/de/d12/d46/d4c/cb4 0 2024-03-11T19:39:35.688 INFO:tasks.workunit.client.0.smithi090.stdout:9/579: rename d1/dd/df/d1c/d9f/l89 to d1/dd/df/lb7 0 2024-03-11T19:39:35.982 INFO:tasks.workunit.client.0.smithi090.stdout:6/638: rmdir d4/d13/d22 39 2024-03-11T19:39:36.178 INFO:tasks.workunit.client.0.smithi090.stdout:0/608: creat d2/d11/d15/d29/d3d/d7e/fc0 x:0 0 0 2024-03-11T19:39:36.310 INFO:tasks.workunit.client.0.smithi090.stdout:6/639: dread d4/d14/f6a [0,4194304] 0 2024-03-11T19:39:36.429 INFO:tasks.workunit.client.0.smithi090.stdout:1/575: mknod d5/d65/d6b/d7f/cc6 0 2024-03-11T19:39:36.437 INFO:tasks.workunit.client.0.smithi090.stdout:5/557: write d1/fa7 [516057,70670] 0 2024-03-11T19:39:36.702 INFO:tasks.workunit.client.0.smithi090.stdout:3/550: dread d2/d1d/f77 [0,4194304] 0 2024-03-11T19:39:36.833 INFO:tasks.workunit.client.0.smithi090.stdout:2/584: truncate d0/f7 13600900 0 2024-03-11T19:39:36.906 INFO:tasks.workunit.client.0.smithi090.stdout:2/585: read d0/de/d12/f32 [109411,128823] 0 2024-03-11T19:39:37.108 INFO:tasks.workunit.client.0.smithi090.stdout:9/580: truncate d1/d1d/d6a/f8c 3864656 0 2024-03-11T19:39:37.883 INFO:tasks.workunit.client.0.smithi090.stdout:1/576: truncate d5/da/d1c/d3b/fb3 246169 0 2024-03-11T19:39:38.329 INFO:tasks.workunit.client.0.smithi090.stdout:2/586: write d0/f66 [3313735,36299] 0 2024-03-11T19:39:38.433 INFO:tasks.workunit.client.0.smithi090.stdout:7/574: dwrite d3/dc/d1e/d7c/d96/d32/d71/d5d/fb3 [0,4194304] 0 2024-03-11T19:39:38.555 INFO:tasks.workunit.client.0.smithi090.stdout:8/621: getdents d0/d3/d4c/d8d 0 2024-03-11T19:39:38.643 INFO:tasks.workunit.client.0.smithi090.stdout:3/551: creat d2/d1f/fac x:0 0 0 2024-03-11T19:39:38.673 INFO:tasks.workunit.client.0.smithi090.stdout:2/587: sync 2024-03-11T19:39:38.685 INFO:tasks.workunit.client.0.smithi090.stdout:7/575: readlink d3/l98 0 2024-03-11T19:39:38.967 INFO:tasks.workunit.client.0.smithi090.stdout:6/640: dread d4/d13/d22/d42/d48/d4e/fae [0,4194304] 0 2024-03-11T19:39:39.026 INFO:tasks.workunit.client.0.smithi090.stdout:4/627: rename d4/d27/d44 to d4/dd8 0 2024-03-11T19:39:39.122 INFO:tasks.workunit.client.0.smithi090.stdout:9/581: getdents d1/da9 0 2024-03-11T19:39:39.144 INFO:tasks.workunit.client.0.smithi090.stdout:9/582: chown d1/dd/f69 12 1 2024-03-11T19:39:39.317 INFO:tasks.workunit.client.0.smithi090.stdout:5/558: mkdir d1/d17/d39/d51/d53/d7b/da0/db9 0 2024-03-11T19:39:39.372 INFO:tasks.workunit.client.0.smithi090.stdout:0/609: write d2/d11/d15/d29/f68 [952260,102088] 0 2024-03-11T19:39:39.776 INFO:tasks.workunit.client.0.smithi090.stdout:7/576: truncate d3/dc/d1e/d7c/d96/d7f/fbb 1018144 0 2024-03-11T19:39:40.060 INFO:tasks.workunit.client.0.smithi090.stdout:9/583: symlink d1/dd/df/d1c/d9f/lb8 0 2024-03-11T19:39:40.938 INFO:tasks.workunit.client.0.smithi090.stdout:2/588: rename d0/fc to d0/de/d12/d46/d49/fb5 0 2024-03-11T19:39:41.047 INFO:tasks.workunit.client.0.smithi090.stdout:7/577: dread d3/f4b [0,4194304] 0 2024-03-11T19:39:41.094 INFO:tasks.workunit.client.0.smithi090.stdout:4/628: mknod d4/d27/d28/cd9 0 2024-03-11T19:39:41.590 INFO:tasks.workunit.client.0.smithi090.stdout:1/577: getdents d5/da/d1c 0 2024-03-11T19:39:41.876 INFO:tasks.workunit.client.0.smithi090.stdout:6/641: rename d4/d13/d22/d34/d40/dd5/fc3 to d4/d13/d22/d34/d83/fd6 0 2024-03-11T19:39:42.238 INFO:tasks.workunit.client.0.smithi090.stdout:7/578: creat d3/dc/d1e/d2a/fcc x:0 0 0 2024-03-11T19:39:42.268 INFO:tasks.workunit.client.0.smithi090.stdout:3/552: dwrite d2/f6 [0,4194304] 0 2024-03-11T19:39:42.301 INFO:tasks.workunit.client.0.smithi090.stdout:8/622: write d0/d43/dc/f3b [4714764,109303] 0 2024-03-11T19:39:42.517 INFO:tasks.workunit.client.0.smithi090.stdout:5/559: write d1/d26/f83 [934871,9611] 0 2024-03-11T19:39:42.594 INFO:tasks.workunit.client.0.smithi090.stdout:0/610: write d2/d11/d15/d29/d3d/d7b/fad [777238,81319] 0 2024-03-11T19:39:42.704 INFO:tasks.workunit.client.0.smithi090.stdout:8/623: stat d0/d43/dc/l1f 0 2024-03-11T19:39:42.945 INFO:tasks.workunit.client.0.smithi090.stdout:3/553: dread d2/d1f/f3b [0,4194304] 0 2024-03-11T19:39:43.046 INFO:tasks.workunit.client.0.smithi090.stdout:2/589: dread d0/de/d1f/f1c [0,4194304] 0 2024-03-11T19:39:43.421 INFO:tasks.workunit.client.0.smithi090.stdout:4/629: write d4/d51/d8c/d89/fc7 [6379748,83942] 0 2024-03-11T19:39:43.427 INFO:tasks.workunit.client.0.smithi090.stdout:0/611: rmdir d2/d11/d15/d29/d2e/d6d 39 2024-03-11T19:39:43.472 INFO:tasks.workunit.client.0.smithi090.stdout:8/624: creat d0/d60/db6/fca x:0 0 0 2024-03-11T19:39:43.497 INFO:tasks.workunit.client.0.smithi090.stdout:7/579: dread - d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/fcb zero size 2024-03-11T19:39:43.539 INFO:tasks.workunit.client.0.smithi090.stdout:5/560: chown d1/c10 131959667 1 2024-03-11T19:39:43.589 INFO:tasks.workunit.client.0.smithi090.stdout:9/584: getdents d1/dd/d18/d3a/d70 0 2024-03-11T19:39:43.725 INFO:tasks.workunit.client.0.smithi090.stdout:8/625: write d0/d43/dc/f3b [4211303,116392] 0 2024-03-11T19:39:43.789 INFO:tasks.workunit.client.0.smithi090.stdout:6/642: rename d4/d14/d92 to d4/d13/d22/d3a/dd7 0 2024-03-11T19:39:43.965 INFO:tasks.workunit.client.0.smithi090.stdout:7/580: truncate d3/dc/d1e/d2a/fcc 11428 0 2024-03-11T19:39:43.973 INFO:tasks.workunit.client.0.smithi090.stdout:5/561: unlink d1/c10 0 2024-03-11T19:39:44.048 INFO:tasks.workunit.client.0.smithi090.stdout:3/554: dread d2/db/f62 [0,4194304] 0 2024-03-11T19:39:44.201 INFO:tasks.workunit.client.0.smithi090.stdout:1/578: truncate d5/d31/d4c/f88 1049830 0 2024-03-11T19:39:44.546 INFO:tasks.workunit.client.0.smithi090.stdout:4/630: write d4/d18/f41 [8626668,39542] 0 2024-03-11T19:39:45.311 INFO:tasks.workunit.client.0.smithi090.stdout:6/643: creat d4/d13/d22/d34/d40/da4/fd8 x:0 0 0 2024-03-11T19:39:45.385 INFO:tasks.workunit.client.0.smithi090.stdout:7/581: dread d3/f22 [0,4194304] 0 2024-03-11T19:39:45.528 INFO:tasks.workunit.client.0.smithi090.stdout:5/562: mknod d1/d17/d39/d51/d53/d7b/d3b/d76/db7/cba 0 2024-03-11T19:39:45.565 INFO:tasks.workunit.client.0.smithi090.stdout:1/579: mknod d5/d4e/d79/cc7 0 2024-03-11T19:39:45.738 INFO:tasks.workunit.client.0.smithi090.stdout:7/582: dread d3/dc/d1e/d7c/d96/d32/f7a [0,4194304] 0 2024-03-11T19:39:45.824 INFO:tasks.workunit.client.0.smithi090.stdout:1/580: sync 2024-03-11T19:39:46.049 INFO:tasks.workunit.client.0.smithi090.stdout:8/626: symlink d0/d3/d3e/lcb 0 2024-03-11T19:39:46.782 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:39:46.785 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:39:46.790 INFO:tasks.workunit.client.0.smithi090.stdout:7/583: symlink d3/dc/d1e/d7c/d96/d34/lcd 0 2024-03-11T19:39:46.794 INFO:tasks.workunit.client.0.smithi090.stdout:1/581: creat d5/d4e/d79/fc8 x:0 0 0 2024-03-11T19:39:46.802 INFO:tasks.workunit.client.0.smithi090.stdout:2/590: rename d0/d21/d30 to d0/de/db6 0 2024-03-11T19:39:46.878 INFO:tasks.workunit.client.0.smithi090.stdout:4/631: mknod d4/d18/d6f/cda 0 2024-03-11T19:39:47.079 INFO:tasks.workunit.client.0.smithi090.stdout:0/612: rename d2/d11/d15/d29/f68 to d2/d11/d15/d29/fc1 0 2024-03-11T19:39:47.119 INFO:tasks.workunit.client.0.smithi090.stdout:4/632: sync 2024-03-11T19:39:47.186 INFO:tasks.workunit.client.0.smithi090.stdout:4/633: dread d4/dd8/d66/f76 [0,4194304] 0 2024-03-11T19:39:47.453 INFO:tasks.workunit.client.0.smithi090.stdout:1/582: dread - d5/d3c/fa5 zero size 2024-03-11T19:39:47.587 INFO:tasks.workunit.client.0.smithi090.stdout:7/584: rmdir d3/dc/d1e/d7c/d96/d32/d45/d84 39 2024-03-11T19:39:47.690 INFO:tasks.workunit.client.0.smithi090.stdout:8/627: write d0/d43/dc/f28 [2265143,57505] 0 2024-03-11T19:39:47.995 INFO:tasks.workunit.client.0.smithi090.stdout:7/585: chown d3/dc/d1e/d7c/d96/d32/d71/d5d/fb9 128477192 1 2024-03-11T19:39:48.305 INFO:tasks.workunit.client.0.smithi090.stdout:9/585: rename d1/d1d/l35 to d1/lb9 0 2024-03-11T19:39:48.321 INFO:tasks.workunit.client.0.smithi090.stdout:8/628: fdatasync d0/d43/dc/f3a 0 2024-03-11T19:39:48.672 INFO:tasks.workunit.client.0.smithi090.stdout:6/644: dwrite d4/d13/d22/d34/d40/dd5/f3f [0,4194304] 0 2024-03-11T19:39:48.848 INFO:tasks.workunit.client.0.smithi090.stdout:7/586: unlink d3/dc/d1e/d7c/d96/d34/d4d/l58 0 2024-03-11T19:39:48.883 INFO:tasks.workunit.client.0.smithi090.stdout:1/583: write d5/da/d23/f34 [2860814,3839] 0 2024-03-11T19:39:48.899 INFO:tasks.workunit.client.0.smithi090.stdout:5/563: truncate d1/d17/d39/d51/d53/d7b/d3b/d76/fa1 246272 0 2024-03-11T19:39:49.045 INFO:tasks.workunit.client.0.smithi090.stdout:3/555: rename d2/db/d74 to d2/db/d15/d69/dad 0 2024-03-11T19:39:49.196 INFO:tasks.workunit.client.0.smithi090.stdout:2/591: dwrite d0/de/f16 [0,4194304] 0 2024-03-11T19:39:49.692 INFO:tasks.workunit.client.0.smithi090.stdout:9/586: truncate d1/dd/d18/d25/f82 274043 0 2024-03-11T19:39:49.793 INFO:tasks.workunit.client.0.smithi090.stdout:0/613: link d2/d11/d3a/l9c d2/d11/lc2 0 2024-03-11T19:39:50.275 INFO:tasks.workunit.client.0.smithi090.stdout:1/584: unlink d5/da/d1c/d6d/f8d 0 2024-03-11T19:39:50.656 INFO:tasks.workunit.client.0.smithi090.stdout:4/634: rename d4/d27/d28/d38/f4b to d4/dd8/db5/fdb 0 2024-03-11T19:39:50.863 INFO:tasks.workunit.client.0.smithi090.stdout:8/629: mkdir d0/d3/d98/da2/dcc 0 2024-03-11T19:39:50.949 INFO:tasks.workunit.client.0.smithi090.stdout:2/592: truncate d0/de/d12/d46/fa8 506076 0 2024-03-11T19:39:51.794 INFO:tasks.workunit.client.0.smithi090.stdout:0/614: truncate d2/d11/d3f/f76 1023904 0 2024-03-11T19:39:51.992 INFO:tasks.workunit.client.0.smithi090.stdout:1/585: mkdir d5/da/dd/dc9 0 2024-03-11T19:39:52.014 INFO:tasks.workunit.client.0.smithi090.stdout:1/586: readlink d5/da/dd/l1e 0 2024-03-11T19:39:52.563 INFO:tasks.workunit.client.0.smithi090.stdout:3/556: unlink d2/db/d15/d1a/d75/la9 0 2024-03-11T19:39:52.650 INFO:tasks.workunit.client.0.smithi090.stdout:4/635: dread d4/f2e [0,4194304] 0 2024-03-11T19:39:52.775 INFO:tasks.workunit.client.0.smithi090.stdout:5/564: truncate d1/d17/d39/d51/d53/d7b/d6f/f72 902503 0 2024-03-11T19:39:52.917 INFO:tasks.workunit.client.0.smithi090.stdout:8/630: unlink d0/d53/f71 0 2024-03-11T19:39:53.055 INFO:tasks.workunit.client.0.smithi090.stdout:2/593: truncate d0/f66 5306706 0 2024-03-11T19:39:53.516 INFO:tasks.workunit.client.0.smithi090.stdout:0/615: rmdir d2/d11/d15/d29/d2e 39 2024-03-11T19:39:53.675 INFO:tasks.workunit.client.0.smithi090.stdout:7/587: rename d3/f27 to d3/dc/d1e/d7c/d96/d32/d71/d80/fce 0 2024-03-11T19:39:54.058 INFO:tasks.workunit.client.0.smithi090.stdout:4/636: chown d4/ld2 0 1 2024-03-11T19:39:54.111 INFO:tasks.workunit.client.0.smithi090.stdout:6/645: truncate d4/d13/d22/d34/d40/dd5/f3f 2431523 0 2024-03-11T19:39:54.185 INFO:tasks.workunit.client.0.smithi090.stdout:6/646: chown d4/d13/d22/d42/d48 3437939 1 2024-03-11T19:39:54.500 INFO:tasks.workunit.client.0.smithi090.stdout:9/587: link d1/dd/d18/d25/f97 d1/dd/df/d1c/d21/d2a/fba 0 2024-03-11T19:39:55.880 INFO:tasks.workunit.client.0.smithi090.stdout:8/631: read d0/d3/d4c/f5e [905044,68005] 0 2024-03-11T19:39:56.043 INFO:tasks.workunit.client.0.smithi090.stdout:2/594: mknod d0/dad/cb7 0 2024-03-11T19:39:56.185 INFO:tasks.workunit.client.0.smithi090.stdout:8/632: sync 2024-03-11T19:39:56.186 INFO:tasks.workunit.client.0.smithi090.stdout:1/587: dwrite d5/d65/d6b/fbc [0,4194304] 0 2024-03-11T19:39:56.508 INFO:tasks.workunit.client.0.smithi090.stdout:7/588: symlink d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/d9c/lcf 0 2024-03-11T19:39:56.634 INFO:tasks.workunit.client.0.smithi090.stdout:3/557: link d2/d45/faa d2/d1d/d47/d6a/d9b/fae 0 2024-03-11T19:39:57.158 INFO:tasks.workunit.client.0.smithi090.stdout:5/565: unlink d1/f32 0 2024-03-11T19:39:57.191 INFO:tasks.workunit.client.0.smithi090.stdout:4/637: write d4/d27/d28/d7a/fc0 [4922195,50462] 0 2024-03-11T19:39:57.306 INFO:tasks.workunit.client.0.smithi090.stdout:4/638: chown d4/d18/c29 0 1 2024-03-11T19:39:57.413 INFO:tasks.workunit.client.0.smithi090.stdout:4/639: readlink d4/d18/l1f 0 2024-03-11T19:39:57.575 INFO:tasks.workunit.client.0.smithi090.stdout:8/633: dread d0/d43/f2f [4194304,4194304] 0 2024-03-11T19:39:57.732 INFO:tasks.workunit.client.0.smithi090.stdout:1/588: rename d5/f1b to d5/d31/d36/fca 0 2024-03-11T19:39:57.735 INFO:tasks.workunit.client.0.smithi090.stdout:0/616: mknod d2/d11/d15/d29/d2e/d6d/da4/cc3 0 2024-03-11T19:39:58.058 INFO:tasks.workunit.client.0.smithi090.stdout:9/588: mkdir d1/dbb 0 2024-03-11T19:39:58.301 INFO:tasks.workunit.client.0.smithi090.stdout:2/595: getdents d0/de/d12/d46/d4c/d64 0 2024-03-11T19:39:58.460 INFO:tasks.workunit.client.0.smithi090.stdout:6/647: dwrite d4/d13/d22/d3a/dd7/f99 [0,4194304] 0 2024-03-11T19:39:58.608 INFO:tasks.workunit.client.0.smithi090.stdout:8/634: truncate d0/d43/f64 238005 0 2024-03-11T19:39:59.315 INFO:tasks.workunit.client.0.smithi090.stdout:2/596: fdatasync d0/f2 0 2024-03-11T19:39:59.367 INFO:tasks.workunit.client.0.smithi090.stdout:2/597: stat d0/de/db6/d25/l7b 0 2024-03-11T19:39:59.918 INFO:tasks.workunit.client.0.smithi090.stdout:8/635: creat d0/d3/d3e/d85/fcd x:0 0 0 2024-03-11T19:40:00.239 INFO:tasks.workunit.client.0.smithi090.stdout:7/589: dwrite d3/dc/d1e/d7c/d96/d32/d71/d5d/f24 [4194304,4194304] 0 2024-03-11T19:40:00.649 INFO:tasks.workunit.client.0.smithi090.stdout:3/558: dwrite d2/f12 [0,4194304] 0 2024-03-11T19:40:00.766 INFO:tasks.workunit.client.0.smithi090.stdout:8/636: fdatasync d0/d60/fb5 0 2024-03-11T19:40:00.840 INFO:tasks.workunit.client.0.smithi090.stdout:0/617: write d2/f10 [7517381,108132] 0 2024-03-11T19:40:00.862 INFO:tasks.workunit.client.0.smithi090.stdout:4/640: write d4/d27/d28/d38/d64/fb6 [1292405,53506] 0 2024-03-11T19:40:00.931 INFO:tasks.workunit.client.0.smithi090.stdout:5/566: write d1/d75/f8c [462675,791] 0 2024-03-11T19:40:01.043 INFO:tasks.workunit.client.0.smithi090.stdout:0/618: sync 2024-03-11T19:40:01.109 INFO:tasks.workunit.client.0.smithi090.stdout:9/589: dwrite d1/f9 [0,4194304] 0 2024-03-11T19:40:01.433 INFO:tasks.workunit.client.0.smithi090.stdout:1/589: dread d5/da/d1c/d3b/d40/f50 [0,4194304] 0 2024-03-11T19:40:01.779 INFO:tasks.workunit.client.0.smithi090.stdout:2/598: rmdir d0/de/d12/d46/d5d/d63/d8e/daf 0 2024-03-11T19:40:01.954 INFO:tasks.workunit.client.0.smithi090.stdout:6/648: getdents d4/d13/d22/d34/d40 0 2024-03-11T19:40:02.083 INFO:tasks.workunit.client.0.smithi090.stdout:4/641: truncate d4/dd8/d66/dcf/dd7/fad 1056055 0 2024-03-11T19:40:02.128 INFO:tasks.workunit.client.0.smithi090.stdout:4/642: dread d4/dd8/d66/f76 [0,4194304] 0 2024-03-11T19:40:02.156 INFO:tasks.workunit.client.0.smithi090.stdout:8/637: write d0/d43/dc/d90/fac [1337894,2348] 0 2024-03-11T19:40:02.221 INFO:tasks.workunit.client.0.smithi090.stdout:9/590: chown d1/dd/df/l5e 1 1 2024-03-11T19:40:03.058 INFO:tasks.workunit.client.0.smithi090.stdout:7/590: mkdir d3/dc/d1e/d7c/d96/d34/d4d/dc2/dd0 0 2024-03-11T19:40:03.185 INFO:tasks.workunit.client.0.smithi090.stdout:7/591: sync 2024-03-11T19:40:03.287 INFO:tasks.workunit.client.0.smithi090.stdout:3/559: dwrite d2/db/f92 [0,4194304] 0 2024-03-11T19:40:03.318 INFO:tasks.workunit.client.0.smithi090.stdout:7/592: sync 2024-03-11T19:40:03.410 INFO:tasks.workunit.client.0.smithi090.stdout:6/649: mknod d4/d14/d8a/d74/cd9 0 2024-03-11T19:40:03.695 INFO:tasks.workunit.client.0.smithi090.stdout:7/593: write d3/dc/d1e/d7c/d96/d32/d71/d5d/f24 [4153630,71292] 0 2024-03-11T19:40:03.836 INFO:tasks.workunit.client.0.smithi090.stdout:0/619: link d2/d11/d3f/c51 d2/d11/d3a/d64/dac/cc4 0 2024-03-11T19:40:04.134 INFO:tasks.workunit.client.0.smithi090.stdout:8/638: creat d0/d3/d86/fce x:0 0 0 2024-03-11T19:40:04.139 INFO:tasks.workunit.client.0.smithi090.stdout:2/599: write d0/de/db6/d29/fa4 [755457,49631] 0 2024-03-11T19:40:04.237 INFO:tasks.workunit.client.0.smithi090.stdout:5/567: dwrite d1/d17/d39/f82 [0,4194304] 0 2024-03-11T19:40:04.394 INFO:tasks.workunit.client.0.smithi090.stdout:1/590: rename d5/d31/d4c/f59 to d5/d3c/fcb 0 2024-03-11T19:40:04.533 INFO:tasks.workunit.client.0.smithi090.stdout:4/643: write d4/f4a [2211582,51504] 0 2024-03-11T19:40:04.808 INFO:tasks.workunit.client.0.smithi090.stdout:8/639: dwrite d0/d3/d3e/d85/fcd [0,4194304] 0 2024-03-11T19:40:05.155 INFO:tasks.workunit.client.0.smithi090.stdout:9/591: write d1/f4f [1380097,95280] 0 2024-03-11T19:40:05.215 INFO:tasks.workunit.client.0.smithi090.stdout:9/592: dread - d1/dd/d18/d3a/d48/f73 zero size 2024-03-11T19:40:05.299 INFO:tasks.workunit.client.0.smithi090.stdout:0/620: fsync d2/d11/d15/f19 0 2024-03-11T19:40:05.664 INFO:tasks.workunit.client.0.smithi090.stdout:6/650: write d4/d13/d22/d34/f6d [1733910,130446] 0 2024-03-11T19:40:05.901 INFO:tasks.workunit.client.0.smithi090.stdout:5/568: rmdir d1/d17 39 2024-03-11T19:40:06.037 INFO:tasks.workunit.client.0.smithi090.stdout:1/591: symlink d5/da/d1c/d6d/d80/lcc 0 2024-03-11T19:40:06.195 INFO:tasks.workunit.client.0.smithi090.stdout:1/592: sync 2024-03-11T19:40:06.392 INFO:tasks.workunit.client.0.smithi090.stdout:4/644: fsync d4/d27/d28/fbe 0 2024-03-11T19:40:07.111 INFO:tasks.workunit.client.0.smithi090.stdout:6/651: mknod d4/d13/d22/d34/d40/cda 0 2024-03-11T19:40:07.419 INFO:tasks.workunit.client.0.smithi090.stdout:2/600: dwrite d0/d21/fa9 [0,4194304] 0 2024-03-11T19:40:07.702 INFO:tasks.workunit.client.0.smithi090.stdout:0/621: dread d2/d11/d6e/f9b [0,4194304] 0 2024-03-11T19:40:07.908 INFO:tasks.workunit.client.0.smithi090.stdout:9/593: dwrite d1/dd/df/d1c/d21/f54 [4194304,4194304] 0 2024-03-11T19:40:08.256 INFO:tasks.workunit.client.0.smithi090.stdout:8/640: creat d0/d3/d86/d93/dba/fcf x:0 0 0 2024-03-11T19:40:08.273 INFO:tasks.workunit.client.0.smithi090.stdout:6/652: rmdir d4/d13/d22/d34/d40/da4 39 2024-03-11T19:40:08.285 INFO:tasks.workunit.client.0.smithi090.stdout:2/601: unlink d0/de/d12/d46/d49/f9d 0 2024-03-11T19:40:08.398 INFO:tasks.workunit.client.0.smithi090.stdout:0/622: mknod d2/d11/d3a/cc5 0 2024-03-11T19:40:08.752 INFO:tasks.workunit.client.0.smithi090.stdout:3/560: getdents d2/db/d15/d69/dad 0 2024-03-11T19:40:08.817 INFO:tasks.workunit.client.0.smithi090.stdout:7/594: getdents d3/dc/d1e/d7c/d96/d32/da0 0 2024-03-11T19:40:08.862 INFO:tasks.workunit.client.0.smithi090.stdout:7/595: read d3/dc/f33 [729167,108406] 0 2024-03-11T19:40:09.039 INFO:tasks.workunit.client.0.smithi090.stdout:4/645: write d4/d51/d8c/d89/fbd [944799,58577] 0 2024-03-11T19:40:09.176 INFO:tasks.workunit.client.0.smithi090.stdout:9/594: creat d1/dd/d18/d3a/d48/fbc x:0 0 0 2024-03-11T19:40:09.451 INFO:tasks.workunit.client.0.smithi090.stdout:6/653: dread d4/d13/d22/d34/d40/fcb [0,4194304] 0 2024-03-11T19:40:09.519 INFO:tasks.workunit.client.0.smithi090.stdout:2/602: dread - d0/de/d12/d46/fb1 zero size 2024-03-11T19:40:09.722 INFO:tasks.workunit.client.0.smithi090.stdout:1/593: link d5/da/f4a d5/da/d1c/d3b/fcd 0 2024-03-11T19:40:09.817 INFO:tasks.workunit.client.0.smithi090.stdout:3/561: creat d2/d1f/faf x:0 0 0 2024-03-11T19:40:09.994 INFO:tasks.workunit.client.0.smithi090.stdout:6/654: dwrite d4/d13/d22/d42/f72 [0,4194304] 0 2024-03-11T19:40:10.220 INFO:tasks.workunit.client.0.smithi090.stdout:0/623: dread d2/d11/d15/d29/d43/d58/f91 [0,4194304] 0 2024-03-11T19:40:10.462 INFO:tasks.workunit.client.0.smithi090.stdout:5/569: dwrite d1/d17/f6a [0,4194304] 0 2024-03-11T19:40:10.579 INFO:tasks.workunit.client.0.smithi090.stdout:5/570: chown d1/d17/d39/d51/d53/fa5 29916 1 2024-03-11T19:40:10.601 INFO:tasks.workunit.client.0.smithi090.stdout:9/595: truncate d1/dd/f1e 4182622 0 2024-03-11T19:40:10.765 INFO:tasks.workunit.client.0.smithi090.stdout:8/641: rename d0/d3/fad to d0/d53/d59/d82/fd0 0 2024-03-11T19:40:11.245 INFO:tasks.workunit.client.0.smithi090.stdout:2/603: creat d0/de/db6/d25/d8c/fb8 x:0 0 0 2024-03-11T19:40:12.135 INFO:tasks.workunit.client.0.smithi090.stdout:4/646: write d4/d18/f6e [3896686,12130] 0 2024-03-11T19:40:12.250 INFO:tasks.workunit.client.0.smithi090.stdout:3/562: truncate d2/db/d15/d1a/f33 2845031 0 2024-03-11T19:40:12.275 INFO:tasks.workunit.client.0.smithi090.stdout:0/624: read d2/d11/d15/d29/f4a [2740666,48360] 0 2024-03-11T19:40:12.462 INFO:tasks.workunit.client.0.smithi090.stdout:0/625: sync 2024-03-11T19:40:12.484 INFO:tasks.workunit.client.0.smithi090.stdout:9/596: rmdir d1/dd/df/d1c/d9f 39 2024-03-11T19:40:12.678 INFO:tasks.workunit.client.0.smithi090.stdout:5/571: dread d1/d17/d39/f82 [0,4194304] 0 2024-03-11T19:40:12.710 INFO:tasks.workunit.client.0.smithi090.stdout:1/594: fsync d5/d4e/d54/d94/fb0 0 2024-03-11T19:40:12.786 INFO:tasks.workunit.client.0.smithi090.stdout:5/572: sync 2024-03-11T19:40:12.927 INFO:tasks.workunit.client.0.smithi090.stdout:7/596: creat d3/fd1 x:0 0 0 2024-03-11T19:40:13.453 INFO:tasks.workunit.client.0.smithi090.stdout:6/655: write d4/d13/d22/d34/d83/fce [98598,68509] 0 2024-03-11T19:40:13.454 INFO:tasks.workunit.client.0.smithi090.stdout:0/626: creat d2/d11/d3a/d64/fc6 x:0 0 0 2024-03-11T19:40:13.515 INFO:tasks.workunit.client.0.smithi090.stdout:8/642: truncate d0/d53/d59/f72 536215 0 2024-03-11T19:40:13.802 INFO:tasks.workunit.client.0.smithi090.stdout:1/595: mknod d5/d31/d76/d77/cce 0 2024-03-11T19:40:14.088 INFO:tasks.workunit.client.0.smithi090.stdout:2/604: write d0/de/db6/d29/f7a [465235,16469] 0 2024-03-11T19:40:14.260 INFO:tasks.workunit.client.0.smithi090.stdout:4/647: mkdir d4/d27/d28/ddc 0 2024-03-11T19:40:14.944 INFO:tasks.workunit.client.0.smithi090.stdout:9/597: mknod d1/dd/d18/cbd 0 2024-03-11T19:40:15.080 INFO:tasks.workunit.client.0.smithi090.stdout:9/598: sync 2024-03-11T19:40:15.191 INFO:tasks.workunit.client.0.smithi090.stdout:0/627: creat d2/d11/d15/d29/d43/d58/fc7 x:0 0 0 2024-03-11T19:40:15.205 INFO:tasks.workunit.client.0.smithi090.stdout:6/656: mkdir d4/d13/d22/d34/d40/dd5/d80/ddb 0 2024-03-11T19:40:15.358 INFO:tasks.workunit.client.0.smithi090.stdout:7/597: write d3/dc/d29/f6b [1034472,5020] 0 2024-03-11T19:40:15.451 INFO:tasks.workunit.client.0.smithi090.stdout:0/628: dread d2/d11/f17 [0,4194304] 0 2024-03-11T19:40:15.587 INFO:tasks.workunit.client.0.smithi090.stdout:2/605: rename d0/de/d12/c36 to d0/de/d12/d81/d93/cb9 0 2024-03-11T19:40:15.675 INFO:tasks.workunit.client.0.smithi090.stdout:1/596: dread d5/f9a [0,4194304] 0 2024-03-11T19:40:15.694 INFO:tasks.workunit.client.0.smithi090.stdout:3/563: write d2/ff [2007927,72915] 0 2024-03-11T19:40:16.050 INFO:tasks.workunit.client.0.smithi090.stdout:4/648: read d4/d27/d28/f30 [482209,59607] 0 2024-03-11T19:40:16.837 INFO:tasks.workunit.client.0.smithi090.stdout:5/573: creat d1/fbb x:0 0 0 2024-03-11T19:40:16.911 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:40:16.914 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:40:17.095 INFO:tasks.workunit.client.0.smithi090.stdout:0/629: mkdir d2/d11/d3a/dc8 0 2024-03-11T19:40:17.106 INFO:tasks.workunit.client.0.smithi090.stdout:6/657: truncate d4/d13/d22/d3a/d67/f9d 4420957 0 2024-03-11T19:40:17.341 INFO:tasks.workunit.client.0.smithi090.stdout:3/564: mknod d2/d2f/cb0 0 2024-03-11T19:40:17.415 INFO:tasks.workunit.client.0.smithi090.stdout:8/643: write d0/d43/f61 [241964,98941] 0 2024-03-11T19:40:17.513 INFO:tasks.workunit.client.0.smithi090.stdout:7/598: truncate d3/dc/d1e/d7c/d96/d32/d71/d5d/f5a 2200512 0 2024-03-11T19:40:17.766 INFO:tasks.workunit.client.0.smithi090.stdout:9/599: getdents d1/dd/d18/d3a/d70 0 2024-03-11T19:40:17.848 INFO:tasks.workunit.client.0.smithi090.stdout:1/597: dread d5/d4e/d54/d94/f96 [0,4194304] 0 2024-03-11T19:40:18.070 INFO:tasks.workunit.client.0.smithi090.stdout:5/574: mknod d1/d17/d39/cbc 0 2024-03-11T19:40:18.370 INFO:tasks.workunit.client.0.smithi090.stdout:0/630: mkdir d2/d11/d3a/d64/dac/dba/dc9 0 2024-03-11T19:40:18.374 INFO:tasks.workunit.client.0.smithi090.stdout:6/658: fdatasync d4/d13/d22/d34/d40/dd5/f26 0 2024-03-11T19:40:18.835 INFO:tasks.workunit.client.0.smithi090.stdout:8/644: creat d0/d3/d4c/d8d/fd1 x:0 0 0 2024-03-11T19:40:18.866 INFO:tasks.workunit.client.0.smithi090.stdout:9/600: creat d1/dd/d18/d3a/fbe x:0 0 0 2024-03-11T19:40:18.882 INFO:tasks.workunit.client.0.smithi090.stdout:1/598: fdatasync d5/d31/d36/f95 0 2024-03-11T19:40:19.492 INFO:tasks.workunit.client.0.smithi090.stdout:5/575: dread d1/d26/fc [0,4194304] 0 2024-03-11T19:40:19.610 INFO:tasks.workunit.client.0.smithi090.stdout:7/599: mkdir d3/dc/d1e/d7c/d96/d34/d4d/dd2 0 2024-03-11T19:40:19.822 INFO:tasks.workunit.client.0.smithi090.stdout:3/565: truncate d2/f5a 3609025 0 2024-03-11T19:40:19.847 INFO:tasks.workunit.client.0.smithi090.stdout:4/649: dwrite d4/d27/f34 [4194304,4194304] 0 2024-03-11T19:40:20.023 INFO:tasks.workunit.client.0.smithi090.stdout:0/631: dread d2/f27 [0,4194304] 0 2024-03-11T19:40:20.109 INFO:tasks.workunit.client.0.smithi090.stdout:3/566: chown d2/d1d/d47/d6a/f95 1 1 2024-03-11T19:40:20.150 INFO:tasks.workunit.client.0.smithi090.stdout:2/606: write d0/de/d12/d46/d5d/d63/d8e/f92 [2528947,100642] 0 2024-03-11T19:40:20.151 INFO:tasks.workunit.client.0.smithi090.stdout:3/567: readlink d2/d1d/la7 0 2024-03-11T19:40:21.021 INFO:tasks.workunit.client.0.smithi090.stdout:8/645: fdatasync d0/d3/d4c/f5e 0 2024-03-11T19:40:21.178 INFO:tasks.workunit.client.0.smithi090.stdout:0/632: stat d2/f97 0 2024-03-11T19:40:21.403 INFO:tasks.workunit.client.0.smithi090.stdout:6/659: dwrite d4/d13/d22/d34/d83/d9c/fa5 [0,4194304] 0 2024-03-11T19:40:21.758 INFO:tasks.workunit.client.0.smithi090.stdout:2/607: dread - d0/f5f zero size 2024-03-11T19:40:21.903 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:05:41.662 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:40:21.916 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:05:41.674 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:40:21.935 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:05:41.634 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:40:21.945 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:05:41.644 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:40:21.950 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:05:41.641 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:40:21.970 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:05:41.661 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:40:22.111 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:05:41.869 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:40:22.138 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:05:41.836 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:40:22.171 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:05:41.861 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:40:22.205 INFO:tasks.workunit.client.0.smithi090.stdout:1/599: dwrite d5/da/dd/f1f [0,4194304] 0 2024-03-11T19:40:22.505 INFO:tasks.workunit.client.0.smithi090.stdout:5/576: mkdir d1/d17/d39/d51/d53/d7b/d3b/d76/d92/dbd 0 2024-03-11T19:40:22.631 INFO:tasks.workunit.client.0.smithi090.stdout:4/650: mknod d4/d27/d28/ddc/cdd 0 2024-03-11T19:40:22.711 INFO:tasks.workunit.client.0.smithi090.stdout:4/651: chown d4/d18/d6f/db7 47903 1 2024-03-11T19:40:22.756 INFO:tasks.workunit.client.0.smithi090.stdout:5/577: dread d1/d26/f6 [0,4194304] 0 2024-03-11T19:40:23.470 INFO:tasks.workunit.client.0.smithi090.stdout:9/601: dwrite d1/dd/df/d1c/d21/d2a/f50 [0,4194304] 0 2024-03-11T19:40:23.875 INFO:tasks.workunit.client.0.smithi090.stdout:8/646: dwrite d0/d60/db6/fca [0,4194304] 0 2024-03-11T19:40:23.988 INFO:tasks.workunit.client.0.smithi090.stdout:4/652: rmdir d4/d51/d8c/d89 39 2024-03-11T19:40:24.041 INFO:tasks.workunit.client.0.smithi090.stdout:8/647: chown d0/d43/f79 72 1 2024-03-11T19:40:24.064 INFO:tasks.workunit.client.0.smithi090.stdout:8/648: stat d0/d53/d59/d82/fd0 0 2024-03-11T19:40:24.121 INFO:tasks.workunit.client.0.smithi090.stdout:7/600: rename d3/l98 to d3/dc/d1e/d7c/d96/d34/d4d/dc2/ld3 0 2024-03-11T19:40:24.581 INFO:tasks.workunit.client.0.smithi090.stdout:3/568: creat d2/fb1 x:0 0 0 2024-03-11T19:40:24.784 INFO:tasks.workunit.client.0.smithi090.stdout:9/602: creat d1/dd/df/d1c/d21/d2a/fbf x:0 0 0 2024-03-11T19:40:24.804 INFO:tasks.workunit.client.0.smithi090.stdout:0/633: write d2/d11/d6e/f9b [3829733,13526] 0 2024-03-11T19:40:24.869 INFO:tasks.workunit.client.0.smithi090.stdout:4/653: rmdir d4/d18/d6f 39 2024-03-11T19:40:25.031 INFO:tasks.workunit.client.0.smithi090.stdout:2/608: write d0/de/db6/d29/f7a [519464,71511] 0 2024-03-11T19:40:25.033 INFO:tasks.workunit.client.0.smithi090.stdout:8/649: mknod d0/d3/d86/d93/dba/cd2 0 2024-03-11T19:40:25.114 INFO:tasks.workunit.client.0.smithi090.stdout:8/650: dread - d0/d43/dc/fa3 zero size 2024-03-11T19:40:25.471 INFO:tasks.workunit.client.0.smithi090.stdout:5/578: rename d1/d17/d39/c59 to d1/d17/d39/d51/d53/cbe 0 2024-03-11T19:40:25.516 INFO:tasks.workunit.client.0.smithi090.stdout:7/601: stat d3/dc/d29/f4e 0 2024-03-11T19:40:26.473 INFO:tasks.workunit.client.0.smithi090.stdout:4/654: unlink d4/d51/d8c/d6a/d8d/fcb 0 2024-03-11T19:40:26.498 INFO:tasks.workunit.client.0.smithi090.stdout:0/634: chown d2/d11/d15/d29/d43/d58/f9e 973997711 1 2024-03-11T19:40:27.040 INFO:tasks.workunit.client.0.smithi090.stdout:8/651: dread d0/f47 [0,4194304] 0 2024-03-11T19:40:27.113 INFO:tasks.workunit.client.0.smithi090.stdout:6/660: getdents d4/d13/d22/d34/d40 0 2024-03-11T19:40:27.351 INFO:tasks.workunit.client.0.smithi090.stdout:1/600: getdents d5/da/d1c/d6d 0 2024-03-11T19:40:27.493 INFO:tasks.workunit.client.0.smithi090.stdout:3/569: fdatasync d2/de/f58 0 2024-03-11T19:40:27.593 INFO:tasks.workunit.client.0.smithi090.stdout:9/603: fsync d1/dd/d18/d3a/d57/f5b 0 2024-03-11T19:40:28.424 INFO:tasks.workunit.client.0.smithi090.stdout:8/652: read d0/d43/dc/f28 [2279775,32705] 0 2024-03-11T19:40:28.471 INFO:tasks.workunit.client.0.smithi090.stdout:8/653: read d0/d43/dc/f50 [6142089,41387] 0 2024-03-11T19:40:28.521 INFO:tasks.workunit.client.0.smithi090.stdout:6/661: fdatasync d4/d13/d22/d3a/f65 0 2024-03-11T19:40:28.785 INFO:tasks.workunit.client.0.smithi090.stdout:3/570: chown d2/d1d/c40 104291 1 2024-03-11T19:40:28.927 INFO:tasks.workunit.client.0.smithi090.stdout:9/604: creat d1/dd/d18/d3a/da2/fc0 x:0 0 0 2024-03-11T19:40:29.433 INFO:tasks.workunit.client.0.smithi090.stdout:0/635: dwrite d2/d11/d15/d29/f8f [0,4194304] 0 2024-03-11T19:40:29.559 INFO:tasks.workunit.client.0.smithi090.stdout:7/602: rename d3/dc/d1e/d7c/d96/d32/d45/l69 to d3/dc/d1e/d7c/d96/d32/d45/ld4 0 2024-03-11T19:40:29.773 INFO:tasks.workunit.client.0.smithi090.stdout:8/654: rename d0/d43/c4b to d0/d60/cd3 0 2024-03-11T19:40:29.827 INFO:tasks.workunit.client.0.smithi090.stdout:5/579: dwrite d1/f2c [0,4194304] 0 2024-03-11T19:40:29.847 INFO:tasks.workunit.client.0.smithi090.stdout:6/662: fdatasync d4/d13/d22/d34/d40/dd5/f3f 0 2024-03-11T19:40:29.946 INFO:tasks.workunit.client.0.smithi090.stdout:5/580: stat d1/d26/f20 0 2024-03-11T19:40:30.057 INFO:tasks.workunit.client.0.smithi090.stdout:3/571: mknod d2/d1d/d47/d6a/cb2 0 2024-03-11T19:40:30.066 INFO:tasks.workunit.client.0.smithi090.stdout:1/601: write d5/da/d23/f2d [3316834,47218] 0 2024-03-11T19:40:30.279 INFO:tasks.workunit.client.0.smithi090.stdout:9/605: creat d1/dd/d18/d3a/d48/fc1 x:0 0 0 2024-03-11T19:40:30.293 INFO:tasks.workunit.client.0.smithi090.stdout:2/609: getdents d0/de/d12 0 2024-03-11T19:40:30.318 INFO:tasks.workunit.client.0.smithi090.stdout:7/603: mknod d3/dc/d1e/d7c/d96/d34/cd5 0 2024-03-11T19:40:30.500 INFO:tasks.workunit.client.0.smithi090.stdout:1/602: sync 2024-03-11T19:40:30.618 INFO:tasks.workunit.client.0.smithi090.stdout:4/655: write d4/d51/d8c/d89/f98 [9018,17648] 0 2024-03-11T19:40:31.227 INFO:tasks.workunit.client.0.smithi090.stdout:6/663: dread d4/d13/d22/d3a/dd7/f99 [0,4194304] 0 2024-03-11T19:40:31.332 INFO:tasks.workunit.client.0.smithi090.stdout:3/572: mknod d2/d4f/d59/cb3 0 2024-03-11T19:40:31.538 INFO:tasks.workunit.client.0.smithi090.stdout:2/610: creat d0/d21/fba x:0 0 0 2024-03-11T19:40:31.729 INFO:tasks.workunit.client.0.smithi090.stdout:9/606: mkdir d1/dd/df/d1c/d21/d2a/dc2 0 2024-03-11T19:40:31.984 INFO:tasks.workunit.client.0.smithi090.stdout:0/636: dwrite d2/f97 [0,4194304] 0 2024-03-11T19:40:32.114 INFO:tasks.workunit.client.0.smithi090.stdout:8/655: mknod d0/d3/db0/cd4 0 2024-03-11T19:40:32.139 INFO:tasks.workunit.client.0.smithi090.stdout:4/656: creat d4/dd8/d66/da1/fde x:0 0 0 2024-03-11T19:40:32.200 INFO:tasks.workunit.client.0.smithi090.stdout:8/656: readlink d0/d43/l35 0 2024-03-11T19:40:32.514 INFO:tasks.workunit.client.0.smithi090.stdout:4/657: dread d4/d27/d28/d38/d75/fa9 [0,4194304] 0 2024-03-11T19:40:32.712 INFO:tasks.workunit.client.0.smithi090.stdout:6/664: unlink d4/d13/d22/d3a/dd7/f99 0 2024-03-11T19:40:32.809 INFO:tasks.workunit.client.0.smithi090.stdout:5/581: write d1/d26/f20 [797354,75378] 0 2024-03-11T19:40:33.138 INFO:tasks.workunit.client.0.smithi090.stdout:2/611: dread d0/de/d12/d46/d49/fb5 [0,4194304] 0 2024-03-11T19:40:33.153 INFO:tasks.workunit.client.0.smithi090.stdout:7/604: mknod d3/dc/d1e/d7c/d96/d34/d4d/dd2/cd6 0 2024-03-11T19:40:33.161 INFO:tasks.workunit.client.0.smithi090.stdout:1/603: mkdir d5/d65/d6b/dcf 0 2024-03-11T19:40:33.383 INFO:tasks.workunit.client.0.smithi090.stdout:0/637: creat d2/d11/d3a/d64/dac/fca x:0 0 0 2024-03-11T19:40:33.467 INFO:tasks.workunit.client.0.smithi090.stdout:8/657: creat d0/d60/db6/fd5 x:0 0 0 2024-03-11T19:40:33.802 INFO:tasks.workunit.client.0.smithi090.stdout:4/658: symlink d4/d51/ldf 0 2024-03-11T19:40:34.052 INFO:tasks.workunit.client.0.smithi090.stdout:2/612: dread d0/de/db6/d25/f67 [0,4194304] 0 2024-03-11T19:40:34.379 INFO:tasks.workunit.client.0.smithi090.stdout:7/605: rename d3/f8 to d3/dc/d1e/d7c/d96/d32/d71/d5d/d48/fd7 0 2024-03-11T19:40:34.465 INFO:tasks.workunit.client.0.smithi090.stdout:9/607: getdents d1/dd/df/d1c/d21/dab 0 2024-03-11T19:40:34.496 INFO:tasks.workunit.client.0.smithi090.stdout:6/665: mkdir d4/d13/d22/d34/d40/dd5/d80/ddb/ddc 0 2024-03-11T19:40:34.695 INFO:tasks.workunit.client.0.smithi090.stdout:0/638: dread d2/d11/d6e/f9b [0,4194304] 0 2024-03-11T19:40:34.696 INFO:tasks.workunit.client.0.smithi090.stdout:6/666: chown d4/d13/d22/d34/d40/fa1 1340806315 1 2024-03-11T19:40:34.823 INFO:tasks.workunit.client.0.smithi090.stdout:5/582: creat d1/d17/d39/d51/d73/da6/fbf x:0 0 0 2024-03-11T19:40:35.021 INFO:tasks.workunit.client.0.smithi090.stdout:2/613: fsync d0/de/d12/d46/d5d/d63/f88 0 2024-03-11T19:40:35.064 INFO:tasks.workunit.client.0.smithi090.stdout:2/614: chown d0/de/d12/c45 9 1 2024-03-11T19:40:35.841 INFO:tasks.workunit.client.0.smithi090.stdout:0/639: fdatasync d2/d11/d15/d29/fbd 0 2024-03-11T19:40:35.881 INFO:tasks.workunit.client.0.smithi090.stdout:0/640: readlink d2/d11/d15/d29/d43/l66 0 2024-03-11T19:40:35.995 INFO:tasks.workunit.client.0.smithi090.stdout:6/667: creat d4/d13/d22/d3a/dd7/fdd x:0 0 0 2024-03-11T19:40:36.345 INFO:tasks.workunit.client.0.smithi090.stdout:5/583: mkdir d1/d17/d39/d51/d53/d7b/da0/dc0 0 2024-03-11T19:40:36.648 INFO:tasks.workunit.client.0.smithi090.stdout:2/615: mkdir d0/de/d12/d46/d75/dbb 0 2024-03-11T19:40:36.656 INFO:tasks.workunit.client.0.smithi090.stdout:3/573: write d2/db/d15/d1a/f33 [3244491,60418] 0 2024-03-11T19:40:36.709 INFO:tasks.workunit.client.0.smithi090.stdout:1/604: write d5/d31/d36/f49 [2149892,86759] 0 2024-03-11T19:40:36.736 INFO:tasks.workunit.client.0.smithi090.stdout:2/616: stat d0/de/l26 0 2024-03-11T19:40:37.188 INFO:tasks.workunit.client.0.smithi090.stdout:4/659: write d4/f12 [7001794,89609] 0 2024-03-11T19:40:37.757 INFO:tasks.workunit.client.0.smithi090.stdout:6/668: dread d4/d14/d8a/d74/f7b [0,4194304] 0 2024-03-11T19:40:37.790 INFO:tasks.workunit.client.0.smithi090.stdout:5/584: fsync d1/d17/f6a 0 2024-03-11T19:40:37.798 INFO:tasks.workunit.client.0.smithi090.stdout:7/606: write d3/dc/d1e/d2a/f5e [53230,121089] 0 2024-03-11T19:40:38.013 INFO:tasks.workunit.client.0.smithi090.stdout:3/574: stat d2/c13 0 2024-03-11T19:40:38.023 INFO:tasks.workunit.client.0.smithi090.stdout:2/617: rmdir d0/dad 39 2024-03-11T19:40:38.240 INFO:tasks.workunit.client.0.smithi090.stdout:8/658: getdents d0/d3/d3e 0 2024-03-11T19:40:38.270 INFO:tasks.workunit.client.0.smithi090.stdout:8/659: chown d0/d3/d86/d93 237257 1 2024-03-11T19:40:38.909 INFO:tasks.workunit.client.0.smithi090.stdout:7/607: creat d3/d55/fd8 x:0 0 0 2024-03-11T19:40:39.515 INFO:tasks.workunit.client.0.smithi090.stdout:5/585: rename d1/d17/d39/d51/d9d to d1/d17/d39/d51/d53/d7b/d3b/d76/d92/dc1 0 2024-03-11T19:40:39.830 INFO:tasks.workunit.client.0.smithi090.stdout:3/575: unlink d2/d4f/c53 0 2024-03-11T19:40:39.844 INFO:tasks.workunit.client.0.smithi090.stdout:3/576: chown d2/d2f 140245 1 2024-03-11T19:40:40.000 INFO:tasks.workunit.client.0.smithi090.stdout:2/618: creat d0/de/d12/d81/fbc x:0 0 0 2024-03-11T19:40:40.132 INFO:tasks.workunit.client.0.smithi090.stdout:8/660: unlink d0/c29 0 2024-03-11T19:40:40.264 INFO:tasks.workunit.client.0.smithi090.stdout:8/661: read d0/d3/d4c/f5e [927778,43590] 0 2024-03-11T19:40:40.391 INFO:tasks.workunit.client.0.smithi090.stdout:0/641: dwrite d2/d11/f1a [0,4194304] 0 2024-03-11T19:40:40.523 INFO:tasks.workunit.client.0.smithi090.stdout:9/608: getdents d1/d1d 0 2024-03-11T19:40:40.611 INFO:tasks.workunit.client.0.smithi090.stdout:7/608: mknod d3/dc/d1e/d7c/d96/d32/d45/d84/cd9 0 2024-03-11T19:40:40.641 INFO:tasks.workunit.client.0.smithi090.stdout:4/660: dread - d4/d18/d6f/db7/fd5 zero size 2024-03-11T19:40:40.711 INFO:tasks.workunit.client.0.smithi090.stdout:0/642: sync 2024-03-11T19:40:41.132 INFO:tasks.workunit.client.0.smithi090.stdout:1/605: write d5/da/d1c/d3b/d8c/fc4 [679644,106877] 0 2024-03-11T19:40:41.321 INFO:tasks.workunit.client.0.smithi090.stdout:3/577: symlink d2/db/d15/d69/dad/lb4 0 2024-03-11T19:40:41.346 INFO:tasks.workunit.client.0.smithi090.stdout:3/578: chown d2/d2f/l7b 35 1 2024-03-11T19:40:41.484 INFO:tasks.workunit.client.0.smithi090.stdout:2/619: fsync d0/f3a 0 2024-03-11T19:40:41.534 INFO:tasks.workunit.client.0.smithi090.stdout:2/620: dread - d0/de/d12/d46/d5d/d63/f88 zero size 2024-03-11T19:40:41.653 INFO:tasks.workunit.client.0.smithi090.stdout:8/662: creat d0/d3/d86/fd6 x:0 0 0 2024-03-11T19:40:41.895 INFO:tasks.workunit.client.0.smithi090.stdout:6/669: link d4/d13/d22/d34/d83/c8f d4/d14/d8a/d74/cde 0 2024-03-11T19:40:42.234 INFO:tasks.workunit.client.0.smithi090.stdout:7/609: creat d3/dc/d1e/d7c/d96/d32/d71/d5d/d48/fda x:0 0 0 2024-03-11T19:40:42.299 INFO:tasks.workunit.client.0.smithi090.stdout:7/610: write d3/dc/d1e/d2a/f5e [1334726,35137] 0 2024-03-11T19:40:42.898 INFO:tasks.workunit.client.0.smithi090.stdout:1/606: dread d5/d3c/fcb [0,4194304] 0 2024-03-11T19:40:42.996 INFO:tasks.workunit.client.0.smithi090.stdout:2/621: symlink d0/de/db6/d25/d8c/lbd 0 2024-03-11T19:40:43.018 INFO:tasks.workunit.client.0.smithi090.stdout:2/622: chown d0/de/d12/d46 18532033 1 2024-03-11T19:40:43.115 INFO:tasks.workunit.client.0.smithi090.stdout:8/663: creat d0/d53/d59/d96/fd7 x:0 0 0 2024-03-11T19:40:43.276 INFO:tasks.workunit.client.0.smithi090.stdout:9/609: fdatasync d1/dd/df/f85 0 2024-03-11T19:40:43.307 INFO:tasks.workunit.client.0.smithi090.stdout:9/610: chown d1/dd/f61 4 1 2024-03-11T19:40:43.950 INFO:tasks.workunit.client.0.smithi090.stdout:7/611: rmdir d3/dc/d1e/d7c 39 2024-03-11T19:40:44.148 INFO:tasks.workunit.client.0.smithi090.stdout:1/607: creat d5/da/d1c/d3b/fd0 x:0 0 0 2024-03-11T19:40:44.756 INFO:tasks.workunit.client.0.smithi090.stdout:6/670: getdents d4/d13/d22/d34/d40/dd5/d80/dd4 0 2024-03-11T19:40:44.796 INFO:tasks.workunit.client.0.smithi090.stdout:9/611: readlink d1/dd/df/d1c/d9f/lb8 0 2024-03-11T19:40:44.935 INFO:tasks.workunit.client.0.smithi090.stdout:7/612: fsync d3/dc/fa7 0 2024-03-11T19:40:45.041 INFO:tasks.workunit.client.0.smithi090.stdout:5/586: write d1/f38 [2015300,88213] 0 2024-03-11T19:40:45.129 INFO:tasks.workunit.client.0.smithi090.stdout:4/661: dwrite d4/f4a [0,4194304] 0 2024-03-11T19:40:45.286 INFO:tasks.workunit.client.0.smithi090.stdout:1/608: mkdir d5/da/d97/dd1 0 2024-03-11T19:40:45.403 INFO:tasks.workunit.client.0.smithi090.stdout:6/671: dread d4/d13/d22/d34/d40/fbb [0,4194304] 0 2024-03-11T19:40:45.670 INFO:tasks.workunit.client.0.smithi090.stdout:3/579: dwrite d2/f8 [0,4194304] 0 2024-03-11T19:40:46.357 INFO:tasks.workunit.client.0.smithi090.stdout:9/612: rename d1/dd/d18/ca5 to d1/d49/d9c/daf/cc3 0 2024-03-11T19:40:46.442 INFO:tasks.workunit.client.0.smithi090.stdout:5/587: creat d1/d17/d39/d51/d53/fc2 x:0 0 0 2024-03-11T19:40:46.866 INFO:tasks.workunit.client.0.smithi090.stdout:3/580: creat d2/d1d/d47/d6a/fb5 x:0 0 0 2024-03-11T19:40:47.039 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:40:47.042 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:40:47.240 INFO:tasks.workunit.client.0.smithi090.stdout:1/609: fdatasync d5/f4f 0 2024-03-11T19:40:47.633 INFO:tasks.workunit.client.0.smithi090.stdout:6/672: rename d4/d13/d22/d34/f36 to d4/d13/d22/d34/d40/d7d/dc2/fdf 0 2024-03-11T19:40:47.670 INFO:tasks.workunit.client.0.smithi090.stdout:9/613: creat d1/dd/df/d1c/d77/fc4 x:0 0 0 2024-03-11T19:40:47.671 INFO:tasks.workunit.client.0.smithi090.stdout:5/588: symlink d1/d17/d39/d51/d53/d64/lc3 0 2024-03-11T19:40:47.778 INFO:tasks.workunit.client.0.smithi090.stdout:8/664: getdents d0/d3/d98 0 2024-03-11T19:40:47.954 INFO:tasks.workunit.client.0.smithi090.stdout:6/673: sync 2024-03-11T19:40:48.273 INFO:tasks.workunit.client.0.smithi090.stdout:2/623: dwrite d0/de/db6/f31 [0,4194304] 0 2024-03-11T19:40:48.570 INFO:tasks.workunit.client.0.smithi090.stdout:9/614: truncate d1/f4f 514376 0 2024-03-11T19:40:48.633 INFO:tasks.workunit.client.0.smithi090.stdout:5/589: symlink d1/d17/d39/d51/d53/lc4 0 2024-03-11T19:40:48.755 INFO:tasks.workunit.client.0.smithi090.stdout:8/665: creat d0/d3/d86/d93/dba/fd8 x:0 0 0 2024-03-11T19:40:49.046 INFO:tasks.workunit.client.0.smithi090.stdout:6/674: truncate d4/d13/d22/d42/fac 264792 0 2024-03-11T19:40:49.177 INFO:tasks.workunit.client.0.smithi090.stdout:6/675: sync 2024-03-11T19:40:49.819 INFO:tasks.workunit.client.0.smithi090.stdout:0/643: link d2/d11/d3a/f75 d2/fcb 0 2024-03-11T19:40:49.834 INFO:tasks.workunit.client.0.smithi090.stdout:1/610: rename d5/d4e/d54/d8e/l9b to d5/d31/d76/d77/dbd/ld2 0 2024-03-11T19:40:49.970 INFO:tasks.workunit.client.0.smithi090.stdout:4/662: link d4/c1a d4/d18/ce0 0 2024-03-11T19:40:49.977 INFO:tasks.workunit.client.0.smithi090.stdout:2/624: mkdir d0/de/db6/d83/dbe 0 2024-03-11T19:40:49.991 INFO:tasks.workunit.client.0.smithi090.stdout:5/590: symlink d1/d17/d39/d51/d53/d7b/d3b/d76/d92/dc1/lc5 0 2024-03-11T19:40:50.017 INFO:tasks.workunit.client.0.smithi090.stdout:8/666: creat d0/d3/d86/d93/dba/fd9 x:0 0 0 2024-03-11T19:40:50.019 INFO:tasks.workunit.client.0.smithi090.stdout:9/615: dread - d1/dd/df/d1c/d9f/f8d zero size 2024-03-11T19:40:50.209 INFO:tasks.workunit.client.0.smithi090.stdout:8/667: chown d0/d53/d59/c48 13 1 2024-03-11T19:40:50.415 INFO:tasks.workunit.client.0.smithi090.stdout:1/611: rmdir d5/da/d1c/d6d/d80 39 2024-03-11T19:40:50.473 INFO:tasks.workunit.client.0.smithi090.stdout:7/613: write d3/dc/d1e/d7c/d96/d32/f7a [143342,31896] 0 2024-03-11T19:40:50.503 INFO:tasks.workunit.client.0.smithi090.stdout:3/581: link d2/db/d15/d1a/d3c/c88 d2/db/d15/d1a/cb6 0 2024-03-11T19:40:50.542 INFO:tasks.workunit.client.0.smithi090.stdout:3/582: chown d2/f9a 857 1 2024-03-11T19:40:50.738 INFO:tasks.workunit.client.0.smithi090.stdout:3/583: dread d2/d1d/f77 [0,4194304] 0 2024-03-11T19:40:51.032 INFO:tasks.workunit.client.0.smithi090.stdout:0/644: mkdir d2/d11/d15/dcc 0 2024-03-11T19:40:51.058 INFO:tasks.workunit.client.0.smithi090.stdout:1/612: truncate d5/da/f68 1543075 0 2024-03-11T19:40:51.160 INFO:tasks.workunit.client.0.smithi090.stdout:0/645: sync 2024-03-11T19:40:51.176 INFO:tasks.workunit.client.0.smithi090.stdout:7/614: creat d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/d9c/fdb x:0 0 0 2024-03-11T19:40:51.220 INFO:tasks.workunit.client.0.smithi090.stdout:7/615: chown d3/d55/f64 244 1 2024-03-11T19:40:51.950 INFO:tasks.workunit.client.0.smithi090.stdout:5/591: mkdir d1/d17/d39/d51/d53/d7b/da0/db9/dc6 0 2024-03-11T19:40:52.117 INFO:tasks.workunit.client.0.smithi090.stdout:2/625: mknod d0/de/d12/d46/cbf 0 2024-03-11T19:40:52.316 INFO:tasks.workunit.client.0.smithi090.stdout:1/613: creat d5/d65/d6b/d7f/fd3 x:0 0 0 2024-03-11T19:40:52.795 INFO:tasks.workunit.client.0.smithi090.stdout:4/663: link d4/d18/ce0 d4/d27/ce1 0 2024-03-11T19:40:53.000 INFO:tasks.workunit.client.0.smithi090.stdout:6/676: link d4/d13/d22/d3a/c4f d4/d13/d22/d34/d40/d7d/dc2/ce0 0 2024-03-11T19:40:53.505 INFO:tasks.workunit.client.0.smithi090.stdout:2/626: unlink d0/de/db6/f85 0 2024-03-11T19:40:53.726 INFO:tasks.workunit.client.0.smithi090.stdout:1/614: truncate d5/da/d1c/d3b/d8c/fc4 271615 0 2024-03-11T19:40:53.823 INFO:tasks.workunit.client.0.smithi090.stdout:9/616: dwrite d1/dd/d18/d3a/f86 [0,4194304] 0 2024-03-11T19:40:53.996 INFO:tasks.workunit.client.0.smithi090.stdout:7/616: getdents d3/dc/d1e/d7c/d96/d34/d4d/dc2/dd0 0 2024-03-11T19:40:54.259 INFO:tasks.workunit.client.0.smithi090.stdout:3/584: creat d2/d1d/fb7 x:0 0 0 2024-03-11T19:40:54.296 INFO:tasks.workunit.client.0.smithi090.stdout:2/627: mknod d0/de/d12/cc0 0 2024-03-11T19:40:54.356 INFO:tasks.workunit.client.0.smithi090.stdout:3/585: chown d2/d1d/d47/d6a/c98 491765 1 2024-03-11T19:40:54.551 INFO:tasks.workunit.client.0.smithi090.stdout:7/617: fsync d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/d9c/fdb 0 2024-03-11T19:40:54.865 INFO:tasks.workunit.client.0.smithi090.stdout:6/677: mknod d4/d13/d22/d3a/d6f/dd3/ce1 0 2024-03-11T19:40:54.988 INFO:tasks.workunit.client.0.smithi090.stdout:5/592: link d1/d26/f43 d1/d17/d39/d51/d53/d7b/d3b/fc7 0 2024-03-11T19:40:55.190 INFO:tasks.workunit.client.0.smithi090.stdout:6/678: dread d4/ff [0,4194304] 0 2024-03-11T19:40:55.523 INFO:tasks.workunit.client.0.smithi090.stdout:7/618: creat d3/dc/d1e/d7c/d96/fdc x:0 0 0 2024-03-11T19:40:55.582 INFO:tasks.workunit.client.0.smithi090.stdout:8/668: write d0/d43/dc/fd [76591,81643] 0 2024-03-11T19:40:55.625 INFO:tasks.workunit.client.0.smithi090.stdout:8/669: chown d0/d3/d3e/lcb 0 1 2024-03-11T19:40:55.674 INFO:tasks.workunit.client.0.smithi090.stdout:9/617: rename d1/dd/d18/l23 to d1/d1d/d6a/lc5 0 2024-03-11T19:40:55.853 INFO:tasks.workunit.client.0.smithi090.stdout:9/618: sync 2024-03-11T19:40:55.930 INFO:tasks.workunit.client.0.smithi090.stdout:5/593: fsync d1/d17/f41 0 2024-03-11T19:40:55.995 INFO:tasks.workunit.client.0.smithi090.stdout:6/679: unlink d4/d13/c2b 0 2024-03-11T19:40:56.025 INFO:tasks.workunit.client.0.smithi090.stdout:8/670: dread d0/d3/d4c/f5e [0,4194304] 0 2024-03-11T19:40:56.405 INFO:tasks.workunit.client.0.smithi090.stdout:0/646: dwrite d2/d11/f41 [0,4194304] 0 2024-03-11T19:40:57.062 INFO:tasks.workunit.client.0.smithi090.stdout:5/594: dread d1/f1e [0,4194304] 0 2024-03-11T19:40:57.135 INFO:tasks.workunit.client.0.smithi090.stdout:5/595: chown d1/d17/d39/d51/d53/d7b/da0/db9/dc6 8169 1 2024-03-11T19:40:57.177 INFO:tasks.workunit.client.0.smithi090.stdout:6/680: dread - d4/d13/d22/d34/d40/da4/fd8 zero size 2024-03-11T19:40:57.503 INFO:tasks.workunit.client.0.smithi090.stdout:1/615: getdents d5/da/dd 0 2024-03-11T19:40:57.535 INFO:tasks.workunit.client.0.smithi090.stdout:0/647: truncate d2/d11/d15/d29/f9a 1477865 0 2024-03-11T19:40:58.256 INFO:tasks.workunit.client.0.smithi090.stdout:1/616: symlink d5/d31/d76/d77/dbd/ld4 0 2024-03-11T19:40:58.320 INFO:tasks.workunit.client.0.smithi090.stdout:0/648: fsync d2/d11/d15/d29/d43/f4d 0 2024-03-11T19:40:58.603 INFO:tasks.workunit.client.0.smithi090.stdout:5/596: mkdir d1/d26/dc8 0 2024-03-11T19:40:58.649 INFO:tasks.workunit.client.0.smithi090.stdout:4/664: dwrite d4/dd8/f65 [0,4194304] 0 2024-03-11T19:40:59.209 INFO:tasks.workunit.client.0.smithi090.stdout:1/617: symlink d5/d31/d86/ld5 0 2024-03-11T19:40:59.546 INFO:tasks.workunit.client.0.smithi090.stdout:9/619: getdents d1/dd/d18/d3a/d48 0 2024-03-11T19:40:59.563 INFO:tasks.workunit.client.0.smithi090.stdout:3/586: write d2/db/f1c [3839092,15145] 0 2024-03-11T19:40:59.611 INFO:tasks.workunit.client.0.smithi090.stdout:9/620: chown d1/d49 69 1 2024-03-11T19:40:59.985 INFO:tasks.workunit.client.0.smithi090.stdout:0/649: mkdir d2/d11/d15/d29/d43/dcd 0 2024-03-11T19:41:00.049 INFO:tasks.workunit.client.0.smithi090.stdout:2/628: read d0/de/d12/d46/f5b [1788135,40968] 0 2024-03-11T19:41:00.055 INFO:tasks.workunit.client.0.smithi090.stdout:5/597: fsync d1/d17/fa9 0 2024-03-11T19:41:00.309 INFO:tasks.workunit.client.0.smithi090.stdout:1/618: creat d5/d31/d76/d77/dbd/fd6 x:0 0 0 2024-03-11T19:41:00.454 INFO:tasks.workunit.client.0.smithi090.stdout:8/671: write d0/d3/d98/da2/fa4 [2759691,50390] 0 2024-03-11T19:41:00.571 INFO:tasks.workunit.client.0.smithi090.stdout:9/621: rmdir d1/dd/d18/d3a/d48 39 2024-03-11T19:41:00.771 INFO:tasks.workunit.client.0.smithi090.stdout:0/650: symlink d2/d11/d3a/d64/dac/dba/lce 0 2024-03-11T19:41:00.819 INFO:tasks.workunit.client.0.smithi090.stdout:0/651: read d2/d11/d3f/f83 [92252,85242] 0 2024-03-11T19:41:01.932 INFO:tasks.workunit.client.0.smithi090.stdout:7/619: dwrite d3/f68 [0,4194304] 0 2024-03-11T19:41:02.093 INFO:tasks.workunit.client.0.smithi090.stdout:7/620: stat d3/dc/d29 0 2024-03-11T19:41:02.282 INFO:tasks.workunit.client.0.smithi090.stdout:6/681: dwrite d4/d13/d22/d34/d40/d7d/dc2/fdf [0,4194304] 0 2024-03-11T19:41:02.474 INFO:tasks.workunit.client.0.smithi090.stdout:3/587: creat d2/d45/fb8 x:0 0 0 2024-03-11T19:41:03.090 INFO:tasks.workunit.client.0.smithi090.stdout:8/672: mkdir d0/d3/d86/d93/dda 0 2024-03-11T19:41:03.113 INFO:tasks.workunit.client.0.smithi090.stdout:8/673: stat d0/d3/d3e/fc2 0 2024-03-11T19:41:03.140 INFO:tasks.workunit.client.0.smithi090.stdout:8/674: chown d0/d43/dc/l1f 23260 1 2024-03-11T19:41:03.389 INFO:tasks.workunit.client.0.smithi090.stdout:5/598: link d1/d17/d39/d51/d53/d7b/f5b d1/d26/dc8/fc9 0 2024-03-11T19:41:03.788 INFO:tasks.workunit.client.0.smithi090.stdout:0/652: rename d2/d11/l8d to d2/d11/d15/lcf 0 2024-03-11T19:41:04.009 INFO:tasks.workunit.client.0.smithi090.stdout:6/682: mknod d4/d13/d22/d42/d48/dd2/ce2 0 2024-03-11T19:41:04.020 INFO:tasks.workunit.client.0.smithi090.stdout:4/665: write d4/d51/f72 [610581,46832] 0 2024-03-11T19:41:04.146 INFO:tasks.workunit.client.0.smithi090.stdout:4/666: dread d4/d27/f4d [0,4194304] 0 2024-03-11T19:41:04.188 INFO:tasks.workunit.client.0.smithi090.stdout:5/599: truncate d1/d17/fb4 153260 0 2024-03-11T19:41:04.322 INFO:tasks.workunit.client.0.smithi090.stdout:9/622: write d1/f9 [5200728,13840] 0 2024-03-11T19:41:04.343 INFO:tasks.workunit.client.0.smithi090.stdout:1/619: write d5/d65/d6b/fbc [5092034,125629] 0 2024-03-11T19:41:04.546 INFO:tasks.workunit.client.0.smithi090.stdout:1/620: fdatasync d5/d65/d6b/fbc 0 2024-03-11T19:41:04.644 INFO:tasks.workunit.client.0.smithi090.stdout:0/653: truncate d2/d11/d3a/d64/dac/dba/fbf 416808 0 2024-03-11T19:41:04.842 INFO:tasks.workunit.client.0.smithi090.stdout:2/629: dwrite d0/de/d12/d46/d4c/f61 [0,4194304] 0 2024-03-11T19:41:05.065 INFO:tasks.workunit.client.0.smithi090.stdout:4/667: symlink d4/d27/d28/d7a/le2 0 2024-03-11T19:41:05.082 INFO:tasks.workunit.client.0.smithi090.stdout:8/675: write d0/d43/dc/fa3 [848849,109390] 0 2024-03-11T19:41:05.170 INFO:tasks.workunit.client.0.smithi090.stdout:7/621: dwrite d3/d55/fb4 [0,4194304] 0 2024-03-11T19:41:05.635 INFO:tasks.workunit.client.0.smithi090.stdout:5/600: mknod d1/d17/d39/d51/cca 0 2024-03-11T19:41:05.755 INFO:tasks.workunit.client.0.smithi090.stdout:4/668: chown d4/d51/d8c/d89/lae 27 1 2024-03-11T19:41:05.872 INFO:tasks.workunit.client.0.smithi090.stdout:1/621: dread d5/da/d1c/d3b/d8c/fc4 [0,4194304] 0 2024-03-11T19:41:05.887 INFO:tasks.workunit.client.0.smithi090.stdout:2/630: dread d0/de/d12/f32 [0,4194304] 0 2024-03-11T19:41:05.924 INFO:tasks.workunit.client.0.smithi090.stdout:3/588: rename d2/db/f8a to d2/fb9 0 2024-03-11T19:41:06.010 INFO:tasks.workunit.client.0.smithi090.stdout:3/589: readlink d2/d2f/l54 0 2024-03-11T19:41:06.093 INFO:tasks.workunit.client.0.smithi090.stdout:9/623: mkdir d1/dd/d18/d3a/d48/dc6 0 2024-03-11T19:41:06.106 INFO:tasks.workunit.client.0.smithi090.stdout:0/654: mkdir d2/d11/d3a/dc8/dd0 0 2024-03-11T19:41:06.863 INFO:tasks.workunit.client.0.smithi090.stdout:8/676: dwrite d0/d53/d59/d96/fd7 [0,4194304] 0 2024-03-11T19:41:07.014 INFO:tasks.workunit.client.0.smithi090.stdout:7/622: dwrite d3/dc/d1e/d7c/d96/fb8 [0,4194304] 0 2024-03-11T19:41:07.120 INFO:tasks.workunit.client.0.smithi090.stdout:5/601: rmdir d1/d17/d39 39 2024-03-11T19:41:07.164 INFO:tasks.workunit.client.0.smithi090.stdout:4/669: fsync d4/d51/d8c/d6a/d8d/f8e 0 2024-03-11T19:41:08.678 INFO:tasks.workunit.client.0.smithi090.stdout:1/622: rmdir d5/da/d1c/d3b/d40 39 2024-03-11T19:41:08.725 INFO:tasks.workunit.client.0.smithi090.stdout:1/623: stat d5/d31/d76/d77/dbd/ld4 0 2024-03-11T19:41:09.010 INFO:tasks.workunit.client.0.smithi090.stdout:0/655: dwrite d2/d11/d15/d29/d43/d58/f65 [0,4194304] 0 2024-03-11T19:41:09.117 INFO:tasks.workunit.client.0.smithi090.stdout:2/631: creat d0/de/db6/fc1 x:0 0 0 2024-03-11T19:41:09.223 INFO:tasks.workunit.client.0.smithi090.stdout:8/677: symlink d0/d3/d98/da2/dcc/ldb 0 2024-03-11T19:41:09.269 INFO:tasks.workunit.client.0.smithi090.stdout:8/678: chown d0/d43/dc/d90/f9e 107301 1 2024-03-11T19:41:09.363 INFO:tasks.workunit.client.0.smithi090.stdout:6/683: rename d4/d13/d22/d34/d83/c8f to d4/d13/d22/d34/d37/ce3 0 2024-03-11T19:41:09.477 INFO:tasks.workunit.client.0.smithi090.stdout:1/624: unlink d5/d4e/d79/cc7 0 2024-03-11T19:41:09.482 INFO:tasks.workunit.client.0.smithi090.stdout:7/623: write d3/f4 [243524,39938] 0 2024-03-11T19:41:09.539 INFO:tasks.workunit.client.0.smithi090.stdout:1/625: read d5/da/f47 [3079708,71972] 0 2024-03-11T19:41:09.750 INFO:tasks.workunit.client.0.smithi090.stdout:5/602: symlink d1/d17/d39/d51/d53/d7b/da0/db9/dc6/lcb 0 2024-03-11T19:41:09.759 INFO:tasks.workunit.client.0.smithi090.stdout:9/624: write d1/dd/d18/d3a/d48/f9a [833858,124072] 0 2024-03-11T19:41:09.934 INFO:tasks.workunit.client.0.smithi090.stdout:0/656: dread d2/d11/f1a [0,4194304] 0 2024-03-11T19:41:09.983 INFO:tasks.workunit.client.0.smithi090.stdout:8/679: symlink d0/d3/d98/da2/dcc/ldc 0 2024-03-11T19:41:10.124 INFO:tasks.workunit.client.0.smithi090.stdout:3/590: rename d2/db/d15/d1a/d3c/c71 to d2/d4f/d59/cba 0 2024-03-11T19:41:10.198 INFO:tasks.workunit.client.0.smithi090.stdout:6/684: mknod d4/d13/d22/d34/d83/d9c/ce4 0 2024-03-11T19:41:10.367 INFO:tasks.workunit.client.0.smithi090.stdout:1/626: fdatasync d5/d31/d76/f4d 0 2024-03-11T19:41:10.886 INFO:tasks.workunit.client.0.smithi090.stdout:9/625: rmdir d1/d49/d9c 39 2024-03-11T19:41:11.141 INFO:tasks.workunit.client.0.smithi090.stdout:4/670: getdents d4/dd8/d66/dcf/dd7 0 2024-03-11T19:41:12.275 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:06:32.034 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:41:12.289 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:06:32.047 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:41:12.342 INFO:tasks.workunit.client.0.smithi090.stdout:1/627: fdatasync d5/da/d1c/f39 0 2024-03-11T19:41:12.392 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:06:32.150 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:41:12.405 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:06:32.096 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:41:12.406 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:06:32.105 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:41:12.409 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:06:32.108 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:41:12.409 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:06:32.100 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:41:12.459 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:06:32.150 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:41:12.495 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:06:32.194 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:41:12.975 INFO:tasks.workunit.client.0.smithi090.stdout:4/671: symlink d4/dd8/le3 0 2024-03-11T19:41:13.001 INFO:tasks.workunit.client.0.smithi090.stdout:5/603: dwrite d1/d26/f3f [0,4194304] 0 2024-03-11T19:41:13.145 INFO:tasks.workunit.client.0.smithi090.stdout:6/685: write d4/d13/f1f [1556168,113316] 0 2024-03-11T19:41:13.370 INFO:tasks.workunit.client.0.smithi090.stdout:4/672: sync 2024-03-11T19:41:13.570 INFO:tasks.workunit.client.0.smithi090.stdout:2/632: dwrite d0/de/d12/d81/fbc [0,4194304] 0 2024-03-11T19:41:13.701 INFO:tasks.workunit.client.0.smithi090.stdout:0/657: dwrite d2/d11/d15/d29/fbd [0,4194304] 0 2024-03-11T19:41:13.910 INFO:tasks.workunit.client.0.smithi090.stdout:9/626: symlink d1/d47/lc7 0 2024-03-11T19:41:13.931 INFO:tasks.workunit.client.0.smithi090.stdout:3/591: dwrite d2/d1f/f35 [0,4194304] 0 2024-03-11T19:41:13.987 INFO:tasks.workunit.client.0.smithi090.stdout:8/680: creat d0/d3/fdd x:0 0 0 2024-03-11T19:41:14.262 INFO:tasks.workunit.client.0.smithi090.stdout:9/627: readlink d1/dd/df/d1c/d9f/lb0 0 2024-03-11T19:41:14.371 INFO:tasks.workunit.client.0.smithi090.stdout:7/624: getdents d3/dc/d1e/d7c/d96/d34/d4d/dd2 0 2024-03-11T19:41:14.382 INFO:tasks.workunit.client.0.smithi090.stdout:2/633: dread d0/fd [0,4194304] 0 2024-03-11T19:41:14.430 INFO:tasks.workunit.client.0.smithi090.stdout:9/628: chown d1/dd/d18/d3a/da2 41 1 2024-03-11T19:41:14.595 INFO:tasks.workunit.client.0.smithi090.stdout:4/673: creat d4/d51/d8c/d89/fe4 x:0 0 0 2024-03-11T19:41:15.245 INFO:tasks.workunit.client.0.smithi090.stdout:8/681: creat d0/d53/d59/d96/fde x:0 0 0 2024-03-11T19:41:15.272 INFO:tasks.workunit.client.0.smithi090.stdout:6/686: mknod d4/d13/d22/d34/d40/dd5/d80/dd4/ce5 0 2024-03-11T19:41:15.327 INFO:tasks.workunit.client.0.smithi090.stdout:6/687: readlink d4/d13/d22/d42/l94 0 2024-03-11T19:41:15.366 INFO:tasks.workunit.client.0.smithi090.stdout:2/634: fsync d0/f3 0 2024-03-11T19:41:15.512 INFO:tasks.workunit.client.0.smithi090.stdout:2/635: sync 2024-03-11T19:41:15.741 INFO:tasks.workunit.client.0.smithi090.stdout:0/658: symlink d2/d11/d3a/da6/ld1 0 2024-03-11T19:41:16.544 INFO:tasks.workunit.client.0.smithi090.stdout:5/604: write d1/d17/f41 [1687779,88368] 0 2024-03-11T19:41:16.579 INFO:tasks.workunit.client.0.smithi090.stdout:6/688: creat d4/d14/d25/fe6 x:0 0 0 2024-03-11T19:41:16.902 INFO:tasks.workunit.client.0.smithi090.stdout:1/628: getdents d5/da/d1c/d6d/d80/db2 0 2024-03-11T19:41:17.167 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:41:17.170 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:41:17.233 INFO:tasks.workunit.client.0.smithi090.stdout:6/689: unlink d4/d13/d22/d42/d48/d4e/la0 0 2024-03-11T19:41:17.756 INFO:tasks.workunit.client.0.smithi090.stdout:2/636: rename d0/l54 to d0/de/db6/d83/lc2 0 2024-03-11T19:41:18.073 INFO:tasks.workunit.client.0.smithi090.stdout:7/625: getdents d3/dc/d1e/d7c/d96/d32/d67 0 2024-03-11T19:41:18.222 INFO:tasks.workunit.client.0.smithi090.stdout:6/690: truncate d4/d13/d22/d3a/f54 6845208 0 2024-03-11T19:41:18.451 INFO:tasks.workunit.client.0.smithi090.stdout:4/674: dwrite d4/dd8/f5c [0,4194304] 0 2024-03-11T19:41:18.566 INFO:tasks.workunit.client.0.smithi090.stdout:5/605: creat d1/d17/d39/d51/d53/d7b/da0/dc0/fcc x:0 0 0 2024-03-11T19:41:18.567 INFO:tasks.workunit.client.0.smithi090.stdout:9/629: link d1/dd/df/d1c/d21/d2a/c1a d1/d47/d7c/cc8 0 2024-03-11T19:41:18.656 INFO:tasks.workunit.client.0.smithi090.stdout:4/675: readlink d4/d27/l96 0 2024-03-11T19:41:18.768 INFO:tasks.workunit.client.0.smithi090.stdout:7/626: rename d3/dc to d3/dc/d1e/ddd 22 2024-03-11T19:41:18.789 INFO:tasks.workunit.client.0.smithi090.stdout:6/691: creat d4/d13/d22/d34/d40/da4/fe7 x:0 0 0 2024-03-11T19:41:19.020 INFO:tasks.workunit.client.0.smithi090.stdout:0/659: write d2/d11/d15/d29/fa5 [932427,32919] 0 2024-03-11T19:41:19.034 INFO:tasks.workunit.client.0.smithi090.stdout:7/627: sync 2024-03-11T19:41:19.508 INFO:tasks.workunit.client.0.smithi090.stdout:8/682: dwrite d0/d43/dc/f50 [4194304,4194304] 0 2024-03-11T19:41:19.664 INFO:tasks.workunit.client.0.smithi090.stdout:4/676: fdatasync d4/dd8/fa4 0 2024-03-11T19:41:19.668 INFO:tasks.workunit.client.0.smithi090.stdout:3/592: dwrite d2/db/d15/d1a/d75/d8c/f8d [0,4194304] 0 2024-03-11T19:41:19.800 INFO:tasks.workunit.client.0.smithi090.stdout:4/677: readlink d4/l49 0 2024-03-11T19:41:19.817 INFO:tasks.workunit.client.0.smithi090.stdout:8/683: dread - d0/d43/dc/f4a zero size 2024-03-11T19:41:19.929 INFO:tasks.workunit.client.0.smithi090.stdout:8/684: sync 2024-03-11T19:41:20.062 INFO:tasks.workunit.client.0.smithi090.stdout:8/685: sync 2024-03-11T19:41:20.076 INFO:tasks.workunit.client.0.smithi090.stdout:8/686: readlink d0/d43/lae 0 2024-03-11T19:41:20.344 INFO:tasks.workunit.client.0.smithi090.stdout:6/692: symlink d4/d13/d22/d34/d83/d9c/db7/le8 0 2024-03-11T19:41:20.596 INFO:tasks.workunit.client.0.smithi090.stdout:0/660: symlink d2/d11/d15/d29/d2e/db4/daf/ld2 0 2024-03-11T19:41:20.603 INFO:tasks.workunit.client.0.smithi090.stdout:1/629: write d5/da/d1c/d3b/d8c/fc4 [520594,28908] 0 2024-03-11T19:41:20.666 INFO:tasks.workunit.client.0.smithi090.stdout:7/628: rename d3/dc/d1e/d7c/d96/d34 to d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/db5/dde 0 2024-03-11T19:41:20.700 INFO:tasks.workunit.client.0.smithi090.stdout:4/678: mkdir d4/dd8/d66/de5 0 2024-03-11T19:41:20.745 INFO:tasks.workunit.client.0.smithi090.stdout:2/637: write d0/de/d12/f1d [608174,128909] 0 2024-03-11T19:41:20.800 INFO:tasks.workunit.client.0.smithi090.stdout:1/630: chown d5/da/d1c/d6d 66417 1 2024-03-11T19:41:20.867 INFO:tasks.workunit.client.0.smithi090.stdout:7/629: read d3/dc/d29/f4e [37617,9025] 0 2024-03-11T19:41:20.940 INFO:tasks.workunit.client.0.smithi090.stdout:1/631: readlink d5/d31/d36/lae 0 2024-03-11T19:41:21.049 INFO:tasks.workunit.client.0.smithi090.stdout:8/687: creat d0/d3/d86/d93/dba/fdf x:0 0 0 2024-03-11T19:41:21.051 INFO:tasks.workunit.client.0.smithi090.stdout:5/606: mknod d1/d17/d39/d51/d53/d7b/d3b/d76/d92/dbd/ccd 0 2024-03-11T19:41:21.110 INFO:tasks.workunit.client.0.smithi090.stdout:7/630: readlink d3/dc/l61 0 2024-03-11T19:41:21.207 INFO:tasks.workunit.client.0.smithi090.stdout:6/693: mknod d4/d13/d22/d34/d40/dd5/d80/dd4/ce9 0 2024-03-11T19:41:21.255 INFO:tasks.workunit.client.0.smithi090.stdout:1/632: chown d5/d65/f69 7 1 2024-03-11T19:41:21.296 INFO:tasks.workunit.client.0.smithi090.stdout:9/630: creat d1/dd/df/d1c/fc9 x:0 0 0 2024-03-11T19:41:21.427 INFO:tasks.workunit.client.0.smithi090.stdout:3/593: dread d2/f9 [0,4194304] 0 2024-03-11T19:41:21.537 INFO:tasks.workunit.client.0.smithi090.stdout:6/694: chown d4/d13/d22/d34/d40/dd5/d80/lb1 15392386 1 2024-03-11T19:41:21.933 INFO:tasks.workunit.client.0.smithi090.stdout:8/688: fsync d0/d53/d59/d96/fde 0 2024-03-11T19:41:21.934 INFO:tasks.workunit.client.0.smithi090.stdout:0/661: dread d2/d11/d15/d29/d2e/d6d/f73 [0,4194304] 0 2024-03-11T19:41:21.978 INFO:tasks.workunit.client.0.smithi090.stdout:5/607: truncate d1/d26/fc 175988 0 2024-03-11T19:41:22.018 INFO:tasks.workunit.client.0.smithi090.stdout:3/594: fsync d2/d1f/f36 0 2024-03-11T19:41:22.393 INFO:tasks.workunit.client.0.smithi090.stdout:9/631: rename d1/dd/d18/d25/f60 to d1/dd/df/d1c/d21/dab/fca 0 2024-03-11T19:41:22.528 INFO:tasks.workunit.client.0.smithi090.stdout:4/679: creat d4/d27/d28/fe6 x:0 0 0 2024-03-11T19:41:22.534 INFO:tasks.workunit.client.0.smithi090.stdout:6/695: dread d4/d13/d22/d42/d48/d4e/fae [0,4194304] 0 2024-03-11T19:41:22.716 INFO:tasks.workunit.client.0.smithi090.stdout:7/631: dread d3/dc/d1e/ff [4194304,4194304] 0 2024-03-11T19:41:23.411 INFO:tasks.workunit.client.0.smithi090.stdout:3/595: readlink d2/l6e 0 2024-03-11T19:41:23.446 INFO:tasks.workunit.client.0.smithi090.stdout:2/638: link d0/de/db6/d29/c7e d0/de/db6/d25/d8c/cc3 0 2024-03-11T19:41:23.459 INFO:tasks.workunit.client.0.smithi090.stdout:1/633: write d5/d31/d76/d77/dbd/fd6 [806383,3447] 0 2024-03-11T19:41:23.536 INFO:tasks.workunit.client.0.smithi090.stdout:1/634: readlink d5/d4e/d54/d8e/lb4 0 2024-03-11T19:41:23.581 INFO:tasks.workunit.client.0.smithi090.stdout:3/596: sync 2024-03-11T19:41:23.731 INFO:tasks.workunit.client.0.smithi090.stdout:3/597: sync 2024-03-11T19:41:23.795 INFO:tasks.workunit.client.0.smithi090.stdout:5/608: chown d1/d17/f30 54 1 2024-03-11T19:41:24.126 INFO:tasks.workunit.client.0.smithi090.stdout:8/689: dread d0/d60/db6/fc7 [0,4194304] 0 2024-03-11T19:41:24.284 INFO:tasks.workunit.client.0.smithi090.stdout:4/680: rmdir d4/d18/d6f/db7 39 2024-03-11T19:41:24.291 INFO:tasks.workunit.client.0.smithi090.stdout:5/609: dread d1/d17/d39/d51/d53/d7b/f37 [0,4194304] 0 2024-03-11T19:41:24.324 INFO:tasks.workunit.client.0.smithi090.stdout:8/690: dread d0/f58 [0,4194304] 0 2024-03-11T19:41:24.502 INFO:tasks.workunit.client.0.smithi090.stdout:2/639: mknod d0/de/d12/d46/d5d/d63/cc4 0 2024-03-11T19:41:24.822 INFO:tasks.workunit.client.0.smithi090.stdout:3/598: dread d2/db/d15/d1a/f33 [0,4194304] 0 2024-03-11T19:41:25.077 INFO:tasks.workunit.client.0.smithi090.stdout:9/632: mknod d1/dbb/ccb 0 2024-03-11T19:41:25.532 INFO:tasks.workunit.client.0.smithi090.stdout:8/691: creat d0/d53/d59/d96/fe0 x:0 0 0 2024-03-11T19:41:25.609 INFO:tasks.workunit.client.0.smithi090.stdout:8/692: dread d0/d43/f79 [0,4194304] 0 2024-03-11T19:41:25.813 INFO:tasks.workunit.client.0.smithi090.stdout:2/640: mkdir d0/de/d12/d46/d75/dc5 0 2024-03-11T19:41:25.917 INFO:tasks.workunit.client.0.smithi090.stdout:8/693: dread d0/f5a [0,4194304] 0 2024-03-11T19:41:26.257 INFO:tasks.workunit.client.0.smithi090.stdout:0/662: dwrite d2/d11/d15/d29/d2e/d6d/fae [0,4194304] 0 2024-03-11T19:41:26.426 INFO:tasks.workunit.client.0.smithi090.stdout:5/610: unlink d1/d17/d39/d51/d53/d7b/d3b/d76/d92/dc1/c9e 0 2024-03-11T19:41:26.539 INFO:tasks.workunit.client.0.smithi090.stdout:1/635: rename l3 to d5/da/d1c/d3b/ld7 0 2024-03-11T19:41:26.712 INFO:tasks.workunit.client.0.smithi090.stdout:6/696: dwrite d4/f11 [0,4194304] 0 2024-03-11T19:41:26.730 INFO:tasks.workunit.client.0.smithi090.stdout:7/632: dwrite d3/d55/fb4 [0,4194304] 0 2024-03-11T19:41:27.004 INFO:tasks.workunit.client.0.smithi090.stdout:8/694: fdatasync d0/d43/f84 0 2024-03-11T19:41:27.018 INFO:tasks.workunit.client.0.smithi090.stdout:1/636: chown d5/d65/f69 342741 1 2024-03-11T19:41:27.278 INFO:tasks.workunit.client.0.smithi090.stdout:5/611: chown d1/c34 394818 1 2024-03-11T19:41:27.737 INFO:tasks.workunit.client.0.smithi090.stdout:4/681: getdents d4/dd8/d66/dcf 0 2024-03-11T19:41:28.719 INFO:tasks.workunit.client.0.smithi090.stdout:7/633: dread d3/dc/d1e/d7c/d96/d32/d45/f88 [0,4194304] 0 2024-03-11T19:41:28.828 INFO:tasks.workunit.client.0.smithi090.stdout:3/599: rename d2/c16 to d2/d1f/cbb 0 2024-03-11T19:41:28.910 INFO:tasks.workunit.client.0.smithi090.stdout:3/600: stat d2/db/d15/d1a/d3c/l43 0 2024-03-11T19:41:29.131 INFO:tasks.workunit.client.0.smithi090.stdout:6/697: creat d4/d9b/fea x:0 0 0 2024-03-11T19:41:29.479 INFO:tasks.workunit.client.0.smithi090.stdout:0/663: write d2/d11/f18 [4825486,40890] 0 2024-03-11T19:41:29.759 INFO:tasks.workunit.client.0.smithi090.stdout:9/633: rename d1/dd/df/d1c/d9f/lb8 to d1/dd/df/d1c/d21/d2a/dc2/lcc 0 2024-03-11T19:41:30.096 INFO:tasks.workunit.client.0.smithi090.stdout:2/641: getdents d0/de/db6 0 2024-03-11T19:41:30.318 INFO:tasks.workunit.client.0.smithi090.stdout:5/612: symlink d1/d17/d39/d51/d53/d7b/lce 0 2024-03-11T19:41:30.331 INFO:tasks.workunit.client.0.smithi090.stdout:5/613: stat d1/d26/f20 0 2024-03-11T19:41:30.438 INFO:tasks.workunit.client.0.smithi090.stdout:4/682: creat d4/d18/d6f/db7/fe7 x:0 0 0 2024-03-11T19:41:30.624 INFO:tasks.workunit.client.0.smithi090.stdout:6/698: unlink d4/d13/d22/d34/d40/dd5/d80/dd4/ce9 0 2024-03-11T19:41:31.426 INFO:tasks.workunit.client.0.smithi090.stdout:2/642: fsync d0/f18 0 2024-03-11T19:41:31.426 INFO:tasks.workunit.client.0.smithi090.stdout:5/614: creat d1/d17/d39/d51/d53/fcf x:0 0 0 2024-03-11T19:41:31.491 INFO:tasks.workunit.client.0.smithi090.stdout:1/637: write d5/da/d23/f2a [5957001,35907] 0 2024-03-11T19:41:31.649 INFO:tasks.workunit.client.0.smithi090.stdout:6/699: symlink d4/d13/d22/d34/d40/leb 0 2024-03-11T19:41:31.778 INFO:tasks.workunit.client.0.smithi090.stdout:8/695: dwrite d0/f47 [0,4194304] 0 2024-03-11T19:41:32.153 INFO:tasks.workunit.client.0.smithi090.stdout:4/683: getdents d4/dd8/d66/de5 0 2024-03-11T19:41:32.307 INFO:tasks.workunit.client.0.smithi090.stdout:5/615: mknod d1/d17/d39/d51/d73/cd0 0 2024-03-11T19:41:32.384 INFO:tasks.workunit.client.0.smithi090.stdout:5/616: readlink d1/d26/d4d/l69 0 2024-03-11T19:41:32.509 INFO:tasks.workunit.client.0.smithi090.stdout:1/638: dread d5/d31/d76/d77/dbd/fd6 [0,4194304] 0 2024-03-11T19:41:32.681 INFO:tasks.workunit.client.0.smithi090.stdout:6/700: unlink d4/d13/d22/d42/l66 0 2024-03-11T19:41:32.699 INFO:tasks.workunit.client.0.smithi090.stdout:6/701: stat d4/d13/l20 0 2024-03-11T19:41:33.005 INFO:tasks.workunit.client.0.smithi090.stdout:9/634: unlink d1/f4f 0 2024-03-11T19:41:33.138 INFO:tasks.workunit.client.0.smithi090.stdout:7/634: write d3/dc/d1e/d7c/d96/d32/d71/d5d/f53 [653614,123087] 0 2024-03-11T19:41:33.351 INFO:tasks.workunit.client.0.smithi090.stdout:4/684: dread d4/d27/d28/fbe [0,4194304] 0 2024-03-11T19:41:33.432 INFO:tasks.workunit.client.0.smithi090.stdout:3/601: write d2/db/f62 [4607744,106275] 0 2024-03-11T19:41:33.736 INFO:tasks.workunit.client.0.smithi090.stdout:0/664: dwrite d2/fab [4194304,4194304] 0 2024-03-11T19:41:33.839 INFO:tasks.workunit.client.0.smithi090.stdout:5/617: fsync d1/d17/d39/d51/d53/d7b/f48 0 2024-03-11T19:41:34.265 INFO:tasks.workunit.client.0.smithi090.stdout:8/696: mknod d0/d3/d86/d93/dda/ce1 0 2024-03-11T19:41:34.414 INFO:tasks.workunit.client.0.smithi090.stdout:9/635: creat d1/d1d/d6a/fcd x:0 0 0 2024-03-11T19:41:34.449 INFO:tasks.workunit.client.0.smithi090.stdout:2/643: write d0/de/db6/d29/fa4 [440055,63680] 0 2024-03-11T19:41:34.688 INFO:tasks.workunit.client.0.smithi090.stdout:3/602: mknod d2/d4f/cbc 0 2024-03-11T19:41:34.961 INFO:tasks.workunit.client.0.smithi090.stdout:4/685: dread d4/d18/f41 [4194304,4194304] 0 2024-03-11T19:41:34.978 INFO:tasks.workunit.client.0.smithi090.stdout:5/618: creat d1/d26/d4d/fd1 x:0 0 0 2024-03-11T19:41:35.025 INFO:tasks.workunit.client.0.smithi090.stdout:0/665: fdatasync d2/d11/d15/d29/d3d/d7b/fad 0 2024-03-11T19:41:35.088 INFO:tasks.workunit.client.0.smithi090.stdout:5/619: write d1/d26/d4d/fd1 [749981,7515] 0 2024-03-11T19:41:35.239 INFO:tasks.workunit.client.0.smithi090.stdout:5/620: stat d1/d17/d39/d51/d73/l9c 0 2024-03-11T19:41:35.533 INFO:tasks.workunit.client.0.smithi090.stdout:0/666: dread f0 [0,4194304] 0 2024-03-11T19:41:35.577 INFO:tasks.workunit.client.0.smithi090.stdout:0/667: chown d2/d11/d15/d36 27 1 2024-03-11T19:41:35.994 INFO:tasks.workunit.client.0.smithi090.stdout:8/697: symlink d0/d43/le2 0 2024-03-11T19:41:36.333 INFO:tasks.workunit.client.0.smithi090.stdout:8/698: sync 2024-03-11T19:41:36.450 INFO:tasks.workunit.client.0.smithi090.stdout:2/644: fdatasync d0/de/d1f/f40 0 2024-03-11T19:41:36.877 INFO:tasks.workunit.client.0.smithi090.stdout:3/603: mknod d2/d4f/d59/da1/cbd 0 2024-03-11T19:41:36.972 INFO:tasks.workunit.client.0.smithi090.stdout:0/668: chown d2/d11/d15/d29/d3d/d7b/f96 2 1 2024-03-11T19:41:37.170 INFO:tasks.workunit.client.0.smithi090.stdout:1/639: creat d5/da/d1c/d3b/d40/fd8 x:0 0 0 2024-03-11T19:41:37.271 INFO:tasks.workunit.client.0.smithi090.stdout:7/635: dwrite d3/f22 [0,4194304] 0 2024-03-11T19:41:37.520 INFO:tasks.workunit.client.0.smithi090.stdout:3/604: dread d2/d1d/f77 [0,4194304] 0 2024-03-11T19:41:37.558 INFO:tasks.workunit.client.0.smithi090.stdout:3/605: chown d2/d2f 906453738 1 2024-03-11T19:41:37.611 INFO:tasks.workunit.client.0.smithi090.stdout:7/636: chown d3/dc/d1e/d7c/d96/d32/d7b/lba 509055 1 2024-03-11T19:41:37.631 INFO:tasks.workunit.client.0.smithi090.stdout:2/645: unlink d0/d21/l2f 0 2024-03-11T19:41:38.519 INFO:tasks.workunit.client.0.smithi090.stdout:1/640: write d5/d31/d76/d77/dbd/fd6 [1707956,101165] 0 2024-03-11T19:41:38.668 INFO:tasks.workunit.client.0.smithi090.stdout:1/641: sync 2024-03-11T19:41:38.696 INFO:tasks.workunit.client.0.smithi090.stdout:1/642: readlink d5/d31/d4c/l7e 0 2024-03-11T19:41:38.808 INFO:tasks.workunit.client.0.smithi090.stdout:1/643: sync 2024-03-11T19:41:38.882 INFO:tasks.workunit.client.0.smithi090.stdout:6/702: link d4/d13/d22/d34/d40/da4/fd8 d4/d13/d22/d3a/fec 0 2024-03-11T19:41:38.889 INFO:tasks.workunit.client.0.smithi090.stdout:9/636: write d1/dd/df/d1c/d21/dab/fca [2095501,39536] 0 2024-03-11T19:41:39.411 INFO:tasks.workunit.client.0.smithi090.stdout:8/699: write d0/d53/d59/d96/fde [945700,945] 0 2024-03-11T19:41:39.419 INFO:tasks.workunit.client.0.smithi090.stdout:5/621: write d1/d17/d39/d51/d53/fb2 [1533966,7565] 0 2024-03-11T19:41:39.556 INFO:tasks.workunit.client.0.smithi090.stdout:8/700: chown d0/d53/d59/d96/fe0 35110 1 2024-03-11T19:41:39.605 INFO:tasks.workunit.client.0.smithi090.stdout:0/669: write d2/d11/d15/d29/d43/d58/fc7 [651278,114257] 0 2024-03-11T19:41:39.994 INFO:tasks.workunit.client.0.smithi090.stdout:9/637: mknod d1/dd/df/d1c/d21/d2a/cce 0 2024-03-11T19:41:40.275 INFO:tasks.workunit.client.0.smithi090.stdout:3/606: dwrite d2/f5f [0,4194304] 0 2024-03-11T19:41:40.392 INFO:tasks.workunit.client.0.smithi090.stdout:3/607: chown d2/db/d15/d1a/d75/d8c 67050770 1 2024-03-11T19:41:40.414 INFO:tasks.workunit.client.0.smithi090.stdout:3/608: chown d2/db/d15/d1a/d75 14603395 1 2024-03-11T19:41:40.830 INFO:tasks.workunit.client.0.smithi090.stdout:4/686: rename d4/fd6 to d4/d18/fe8 0 2024-03-11T19:41:40.860 INFO:tasks.workunit.client.0.smithi090.stdout:4/687: chown d4/d27/d28/f63 186 1 2024-03-11T19:41:40.890 INFO:tasks.workunit.client.0.smithi090.stdout:4/688: chown d4/d27/d28/f30 2728792 1 2024-03-11T19:41:41.141 INFO:tasks.workunit.client.0.smithi090.stdout:8/701: truncate d0/f65 2658792 0 2024-03-11T19:41:41.266 INFO:tasks.workunit.client.0.smithi090.stdout:0/670: mkdir d2/d11/d3a/d64/dac/dba/dd3 0 2024-03-11T19:41:41.345 INFO:tasks.workunit.client.0.smithi090.stdout:2/646: link d0/de/d12/d81/fbc d0/de/d12/d46/d4c/d64/fc6 0 2024-03-11T19:41:41.638 INFO:tasks.workunit.client.0.smithi090.stdout:9/638: readlink d1/dd/df/l4e 0 2024-03-11T19:41:41.746 INFO:tasks.workunit.client.0.smithi090.stdout:6/703: write d4/d13/d22/d42/d48/fc5 [348153,83432] 0 2024-03-11T19:41:41.792 INFO:tasks.workunit.client.0.smithi090.stdout:3/609: fsync d2/db/d15/f48 0 2024-03-11T19:41:42.221 INFO:tasks.workunit.client.0.smithi090.stdout:7/637: link d3/d55/f64 d3/dc/d1e/d7c/d96/d32/d71/fdf 0 2024-03-11T19:41:42.246 INFO:tasks.workunit.client.0.smithi090.stdout:7/638: stat d3/f70 0 2024-03-11T19:41:42.394 INFO:tasks.workunit.client.0.smithi090.stdout:8/702: creat d0/d3/d86/d93/fe3 x:0 0 0 2024-03-11T19:41:42.439 INFO:tasks.workunit.client.0.smithi090.stdout:0/671: fdatasync d2/f1c 0 2024-03-11T19:41:42.603 INFO:tasks.workunit.client.0.smithi090.stdout:2/647: creat d0/de/db6/d25/fc7 x:0 0 0 2024-03-11T19:41:42.695 INFO:tasks.workunit.client.0.smithi090.stdout:1/644: link d5/d31/d4c/f73 d5/d31/d86/fd9 0 2024-03-11T19:41:42.870 INFO:tasks.workunit.client.0.smithi090.stdout:1/645: sync 2024-03-11T19:41:44.034 INFO:tasks.workunit.client.0.smithi090.stdout:8/703: mknod d0/d3/d86/ce4 0 2024-03-11T19:41:44.111 INFO:tasks.workunit.client.0.smithi090.stdout:4/689: dwrite d4/d18/f41 [0,4194304] 0 2024-03-11T19:41:44.356 INFO:tasks.workunit.client.0.smithi090.stdout:0/672: unlink d2/d11/d15/d29/d2e/l63 0 2024-03-11T19:41:44.728 INFO:tasks.workunit.client.0.smithi090.stdout:9/639: dwrite d1/dd/df/d1c/d77/fc4 [0,4194304] 0 2024-03-11T19:41:45.280 INFO:tasks.workunit.client.0.smithi090.stdout:5/622: getdents d1/d17/d39/d51/d53 0 2024-03-11T19:41:45.383 INFO:tasks.workunit.client.0.smithi090.stdout:4/690: read d4/d18/f90 [3398,29963] 0 2024-03-11T19:41:45.489 INFO:tasks.workunit.client.0.smithi090.stdout:6/704: write d4/d13/d22/d34/d40/da4/fe7 [801496,99097] 0 2024-03-11T19:41:45.749 INFO:tasks.workunit.client.0.smithi090.stdout:8/704: dread d0/d3/d98/da2/fa4 [0,4194304] 0 2024-03-11T19:41:45.831 INFO:tasks.workunit.client.0.smithi090.stdout:0/673: dread d2/d11/d3f/f40 [0,4194304] 0 2024-03-11T19:41:46.029 INFO:tasks.workunit.client.0.smithi090.stdout:2/648: mkdir d0/de/d12/dc8 0 2024-03-11T19:41:46.165 INFO:tasks.workunit.client.0.smithi090.stdout:9/640: rmdir d1/dd/d18/d3a/d57 39 2024-03-11T19:41:46.331 INFO:tasks.workunit.client.0.smithi090.stdout:7/639: rename d3/dc/d1e/d7c/d96/d32/d71/d5d to d3/dc/d1e/d7c/d96/d32/d45/de0 0 2024-03-11T19:41:46.440 INFO:tasks.workunit.client.0.smithi090.stdout:5/623: fsync d1/d26/d4d/fd1 0 2024-03-11T19:41:46.746 INFO:tasks.workunit.client.0.smithi090.stdout:3/610: rmdir d2/d45 39 2024-03-11T19:41:46.775 INFO:tasks.workunit.client.0.smithi090.stdout:3/611: chown d2/d1f/faf 1687876 1 2024-03-11T19:41:47.244 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:41:47.247 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:41:47.354 INFO:tasks.workunit.client.0.smithi090.stdout:0/674: unlink d2/d11/d15/d29/d43/l66 0 2024-03-11T19:41:47.530 INFO:tasks.workunit.client.0.smithi090.stdout:1/646: creat d5/da/d1c/d3b/d40/fda x:0 0 0 2024-03-11T19:41:47.590 INFO:tasks.workunit.client.0.smithi090.stdout:2/649: stat d0/de/d12/d46/d4c 0 2024-03-11T19:41:47.811 INFO:tasks.workunit.client.0.smithi090.stdout:9/641: creat d1/dd/df/d1c/d88/fcf x:0 0 0 2024-03-11T19:41:47.976 INFO:tasks.workunit.client.0.smithi090.stdout:7/640: rename d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/db5/dde/d4d to d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/db5/de1 0 2024-03-11T19:41:48.093 INFO:tasks.workunit.client.0.smithi090.stdout:5/624: truncate d1/d75/f8c 1304377 0 2024-03-11T19:41:48.457 INFO:tasks.workunit.client.0.smithi090.stdout:4/691: creat d4/d51/dbf/fe9 x:0 0 0 2024-03-11T19:41:48.467 INFO:tasks.workunit.client.0.smithi090.stdout:3/612: stat d2/db/d15/d1a/d75/f80 0 2024-03-11T19:41:48.810 INFO:tasks.workunit.client.0.smithi090.stdout:3/613: sync 2024-03-11T19:41:48.903 INFO:tasks.workunit.client.0.smithi090.stdout:0/675: truncate d2/d11/d3f/f62 1787060 0 2024-03-11T19:41:48.927 INFO:tasks.workunit.client.0.smithi090.stdout:0/676: stat d2/cc 0 2024-03-11T19:41:49.065 INFO:tasks.workunit.client.0.smithi090.stdout:1/647: unlink d5/d31/d36/f95 0 2024-03-11T19:41:49.094 INFO:tasks.workunit.client.0.smithi090.stdout:1/648: readlink d5/da/dd/l1e 0 2024-03-11T19:41:49.692 INFO:tasks.workunit.client.0.smithi090.stdout:7/641: mknod d3/dc/d1e/d7c/d96/d32/d7b/d99/ce2 0 2024-03-11T19:41:49.812 INFO:tasks.workunit.client.0.smithi090.stdout:7/642: sync 2024-03-11T19:41:50.137 INFO:tasks.workunit.client.0.smithi090.stdout:4/692: chown d4/d51/d8c/d6a/cc8 5974 1 2024-03-11T19:41:50.571 INFO:tasks.workunit.client.0.smithi090.stdout:8/705: dwrite d0/d53/faa [0,4194304] 0 2024-03-11T19:41:50.855 INFO:tasks.workunit.client.0.smithi090.stdout:7/643: mknod d3/dc/d1e/d7c/d96/d32/da0/ce3 0 2024-03-11T19:41:50.946 INFO:tasks.workunit.client.0.smithi090.stdout:1/649: dread d5/da/d1c/d3b/fcd [0,4194304] 0 2024-03-11T19:41:52.036 INFO:tasks.workunit.client.0.smithi090.stdout:8/706: mkdir d0/d53/d59/d96/de5 0 2024-03-11T19:41:52.134 INFO:tasks.workunit.client.0.smithi090.stdout:6/705: write d4/d13/d22/d3a/d6f/fb5 [2864352,37907] 0 2024-03-11T19:41:52.296 INFO:tasks.workunit.client.0.smithi090.stdout:7/644: truncate d3/dc/d1e/d7c/d96/d32/d45/de0/f24 6789954 0 2024-03-11T19:41:52.573 INFO:tasks.workunit.client.0.smithi090.stdout:2/650: rename d0/de/db6/d83/lc2 to d0/de/d12/d46/d5d/lc9 0 2024-03-11T19:41:52.941 INFO:tasks.workunit.client.0.smithi090.stdout:8/707: creat d0/d3/d86/fe6 x:0 0 0 2024-03-11T19:41:52.971 INFO:tasks.workunit.client.0.smithi090.stdout:8/708: read d0/f58 [8485,28615] 0 2024-03-11T19:41:53.135 INFO:tasks.workunit.client.0.smithi090.stdout:7/645: mkdir d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/de4 0 2024-03-11T19:41:53.302 INFO:tasks.workunit.client.0.smithi090.stdout:6/706: rename d4/d13/d22/d34 to d4/d13/d22/d34/d37/d7f/ded 22 2024-03-11T19:41:53.785 INFO:tasks.workunit.client.0.smithi090.stdout:0/677: write d2/d11/d15/d29/d43/d58/f91 [522163,16349] 0 2024-03-11T19:41:53.842 INFO:tasks.workunit.client.0.smithi090.stdout:8/709: truncate d0/d53/f5d 2564326 0 2024-03-11T19:41:54.065 INFO:tasks.workunit.client.0.smithi090.stdout:5/625: dwrite d1/d26/f20 [0,4194304] 0 2024-03-11T19:41:54.164 INFO:tasks.workunit.client.0.smithi090.stdout:6/707: mkdir d4/d13/d22/d42/dee 0 2024-03-11T19:41:54.338 INFO:tasks.workunit.client.0.smithi090.stdout:9/642: dwrite d1/dd/df/f4d [4194304,4194304] 0 2024-03-11T19:41:54.647 INFO:tasks.workunit.client.0.smithi090.stdout:3/614: dwrite d2/db/d15/d49/f7f [0,4194304] 0 2024-03-11T19:41:54.944 INFO:tasks.workunit.client.0.smithi090.stdout:0/678: dread - d2/d11/d3a/d64/dac/fca zero size 2024-03-11T19:41:55.314 INFO:tasks.workunit.client.0.smithi090.stdout:4/693: dwrite d4/d18/d6f/db7/fe7 [0,4194304] 0 2024-03-11T19:41:55.447 INFO:tasks.workunit.client.0.smithi090.stdout:3/615: mknod d2/db/d15/d1a/d3c/cbe 0 2024-03-11T19:41:55.648 INFO:tasks.workunit.client.0.smithi090.stdout:8/710: mkdir d0/d53/d59/d96/de5/de7 0 2024-03-11T19:41:55.809 INFO:tasks.workunit.client.0.smithi090.stdout:5/626: mkdir d1/d17/d39/d51/d53/d7b/d6f/dd2 0 2024-03-11T19:41:55.863 INFO:tasks.workunit.client.0.smithi090.stdout:5/627: chown d1/d26/d4d/l61 11003788 1 2024-03-11T19:41:55.898 INFO:tasks.workunit.client.0.smithi090.stdout:2/651: rmdir d0/de/d12 39 2024-03-11T19:41:56.570 INFO:tasks.workunit.client.0.smithi090.stdout:9/643: link d1/dd/df/d1c/l55 d1/d1d/d6a/ld0 0 2024-03-11T19:41:56.704 INFO:tasks.workunit.client.0.smithi090.stdout:8/711: rmdir d0/d3/db0 39 2024-03-11T19:41:56.781 INFO:tasks.workunit.client.0.smithi090.stdout:1/650: dwrite d5/f37 [0,4194304] 0 2024-03-11T19:41:56.979 INFO:tasks.workunit.client.0.smithi090.stdout:1/651: read d5/d4e/f5e [1584639,32328] 0 2024-03-11T19:41:57.010 INFO:tasks.workunit.client.0.smithi090.stdout:2/652: creat d0/de/db6/d29/fca x:0 0 0 2024-03-11T19:41:57.564 INFO:tasks.workunit.client.0.smithi090.stdout:5/628: mknod d1/d17/d39/d51/cd3 0 2024-03-11T19:41:57.588 INFO:tasks.workunit.client.0.smithi090.stdout:8/712: symlink d0/d60/db6/db9/le8 0 2024-03-11T19:41:57.733 INFO:tasks.workunit.client.0.smithi090.stdout:1/652: creat d5/da/d1c/fdb x:0 0 0 2024-03-11T19:41:57.978 INFO:tasks.workunit.client.0.smithi090.stdout:4/694: rename d4/d51/ldf to d4/d18/d6f/lea 0 2024-03-11T19:41:58.246 INFO:tasks.workunit.client.0.smithi090.stdout:5/629: read - d1/d17/d39/d51/d73/f95 zero size 2024-03-11T19:41:58.360 INFO:tasks.workunit.client.0.smithi090.stdout:2/653: readlink d0/de/d12/l56 0 2024-03-11T19:41:58.994 INFO:tasks.workunit.client.0.smithi090.stdout:7/646: write d3/dc/d1e/d7c/d96/d32/d45/de0/f24 [2739458,52403] 0 2024-03-11T19:41:59.109 INFO:tasks.workunit.client.0.smithi090.stdout:6/708: dwrite d4/d13/d22/d3a/f56 [0,4194304] 0 2024-03-11T19:41:59.128 INFO:tasks.workunit.client.0.smithi090.stdout:1/653: symlink d5/da/ldc 0 2024-03-11T19:41:59.234 INFO:tasks.workunit.client.0.smithi090.stdout:5/630: unlink d1/d17/d39/d51/cca 0 2024-03-11T19:41:59.326 INFO:tasks.workunit.client.0.smithi090.stdout:0/679: write d2/d11/d3a/f69 [2483638,38165] 0 2024-03-11T19:42:00.106 INFO:tasks.workunit.client.0.smithi090.stdout:5/631: creat d1/d17/d39/d51/d9a/fd4 x:0 0 0 2024-03-11T19:42:00.376 INFO:tasks.workunit.client.0.smithi090.stdout:6/709: truncate d4/d14/d8a/d74/f7b 1745220 0 2024-03-11T19:42:00.432 INFO:tasks.workunit.client.0.smithi090.stdout:3/616: dwrite d2/d1f/faf [0,4194304] 0 2024-03-11T19:42:01.184 INFO:tasks.workunit.client.0.smithi090.stdout:5/632: write d1/d26/f20 [2300211,89988] 0 2024-03-11T19:42:01.365 INFO:tasks.workunit.client.0.smithi090.stdout:7/647: symlink d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/db5/de1/dc2/dd0/le5 0 2024-03-11T19:42:01.371 INFO:tasks.workunit.client.0.smithi090.stdout:9/644: write d1/dd/d18/d3a/d48/f9b [279208,126102] 0 2024-03-11T19:42:01.727 INFO:tasks.workunit.client.0.smithi090.stdout:2/654: link d0/de/f17 d0/de/d12/d46/d5d/d8d/fcb 0 2024-03-11T19:42:02.068 INFO:tasks.workunit.client.0.smithi090.stdout:0/680: creat d2/fd4 x:0 0 0 2024-03-11T19:42:02.070 INFO:tasks.workunit.client.0.smithi090.stdout:5/633: creat d1/d17/d39/d51/d9a/fd5 x:0 0 0 2024-03-11T19:42:02.144 INFO:tasks.workunit.client.0.smithi090.stdout:0/681: chown d2/d11/d15/d29/d2e/f32 1229080 1 2024-03-11T19:42:02.526 INFO:tasks.workunit.client.0.smithi090.stdout:8/713: dwrite d0/d3/d86/f9c [0,4194304] 0 2024-03-11T19:42:02.695 INFO:tasks.workunit.client.0.smithi090.stdout:2/655: creat d0/de/d12/d46/d5d/d63/fcc x:0 0 0 2024-03-11T19:42:03.707 INFO:tasks.workunit.client.0.smithi090.stdout:8/714: creat d0/d3/d3e/d85/fe9 x:0 0 0 2024-03-11T19:42:03.896 INFO:tasks.workunit.client.0.smithi090.stdout:1/654: write d5/da/d1c/d3b/d40/fd8 [509966,64035] 0 2024-03-11T19:42:04.001 INFO:tasks.workunit.client.0.smithi090.stdout:4/695: dwrite d4/d27/d28/d69/f85 [0,4194304] 0 2024-03-11T19:42:04.158 INFO:tasks.workunit.client.0.smithi090.stdout:1/655: chown d5/d31/c3e 2133169854 1 2024-03-11T19:42:04.644 INFO:tasks.workunit.client.0.smithi090.stdout:4/696: truncate d4/d51/d8c/d89/fe4 95943 0 2024-03-11T19:42:05.046 INFO:tasks.workunit.client.0.smithi090.stdout:4/697: sync 2024-03-11T19:42:05.230 INFO:tasks.workunit.client.0.smithi090.stdout:7/648: write d3/dc/d87/f90 [951496,22855] 0 2024-03-11T19:42:05.237 INFO:tasks.workunit.client.0.smithi090.stdout:3/617: write d2/f67 [392856,102807] 0 2024-03-11T19:42:05.742 INFO:tasks.workunit.client.0.smithi090.stdout:6/710: write d4/d13/d22/d34/d40/d7d/fa6 [496805,63728] 0 2024-03-11T19:42:05.795 INFO:tasks.workunit.client.0.smithi090.stdout:4/698: truncate d4/d18/f90 998919 0 2024-03-11T19:42:05.819 INFO:tasks.workunit.client.0.smithi090.stdout:5/634: write d1/d17/d39/d51/d53/fa5 [306753,115724] 0 2024-03-11T19:42:05.937 INFO:tasks.workunit.client.0.smithi090.stdout:5/635: sync 2024-03-11T19:42:06.294 INFO:tasks.workunit.client.0.smithi090.stdout:3/618: creat d2/db/d15/d69/dad/fbf x:0 0 0 2024-03-11T19:42:06.802 INFO:tasks.workunit.client.0.smithi090.stdout:2/656: dwrite d0/de/d12/d81/f87 [0,4194304] 0 2024-03-11T19:42:06.952 INFO:tasks.workunit.client.0.smithi090.stdout:0/682: dwrite d2/d11/d15/d29/d43/fbe [0,4194304] 0 2024-03-11T19:42:07.123 INFO:tasks.workunit.client.0.smithi090.stdout:9/645: dwrite d1/dd/f1e [0,4194304] 0 2024-03-11T19:42:07.487 INFO:tasks.workunit.client.0.smithi090.stdout:1/656: dwrite d5/d31/d36/fca [0,4194304] 0 2024-03-11T19:42:07.693 INFO:tasks.workunit.client.0.smithi090.stdout:8/715: dwrite d0/d60/db6/fca [0,4194304] 0 2024-03-11T19:42:07.792 INFO:tasks.workunit.client.0.smithi090.stdout:7/649: write d3/d55/f85 [109668,46853] 0 2024-03-11T19:42:07.837 INFO:tasks.workunit.client.0.smithi090.stdout:3/619: creat d2/db/d15/d69/dad/fc0 x:0 0 0 2024-03-11T19:42:07.848 INFO:tasks.workunit.client.0.smithi090.stdout:5/636: creat d1/d26/fd6 x:0 0 0 2024-03-11T19:42:08.013 INFO:tasks.workunit.client.0.smithi090.stdout:2/657: write d0/fd [4377991,14939] 0 2024-03-11T19:42:08.155 INFO:tasks.workunit.client.0.smithi090.stdout:4/699: getdents d4/d27/d28/d38 0 2024-03-11T19:42:08.161 INFO:tasks.workunit.client.0.smithi090.stdout:1/657: creat d5/da/d1c/d6d/d80/fdd x:0 0 0 2024-03-11T19:42:08.197 INFO:tasks.workunit.client.0.smithi090.stdout:3/620: mknod d2/d1d/cc1 0 2024-03-11T19:42:08.280 INFO:tasks.workunit.client.0.smithi090.stdout:9/646: truncate d1/dd/d18/d3a/fbe 445220 0 2024-03-11T19:42:08.335 INFO:tasks.workunit.client.0.smithi090.stdout:7/650: fsync d3/dc/d1e/d7c/d96/d32/f3f 0 2024-03-11T19:42:08.365 INFO:tasks.workunit.client.0.smithi090.stdout:0/683: dread d2/d11/d15/f28 [0,4194304] 0 2024-03-11T19:42:08.373 INFO:tasks.workunit.client.0.smithi090.stdout:3/621: sync 2024-03-11T19:42:09.032 INFO:tasks.workunit.client.0.smithi090.stdout:6/711: dwrite d4/d13/d22/d34/d83/d9c/fb0 [0,4194304] 0 2024-03-11T19:42:09.032 INFO:tasks.workunit.client.0.smithi090.stdout:8/716: truncate d0/d53/faa 169286 0 2024-03-11T19:42:09.032 INFO:tasks.workunit.client.0.smithi090.stdout:5/637: creat d1/d17/d39/d51/d53/d7b/da0/db9/dc6/fd7 x:0 0 0 2024-03-11T19:42:09.032 INFO:tasks.workunit.client.0.smithi090.stdout:1/658: symlink d5/d65/d6b/d7f/lde 0 2024-03-11T19:42:09.155 INFO:tasks.workunit.client.0.smithi090.stdout:1/659: sync 2024-03-11T19:42:09.240 INFO:tasks.workunit.client.0.smithi090.stdout:9/647: read - d1/dd/d18/d3a/da2/fac zero size 2024-03-11T19:42:09.508 INFO:tasks.workunit.client.0.smithi090.stdout:8/717: symlink d0/d3/d98/lea 0 2024-03-11T19:42:09.544 INFO:tasks.workunit.client.0.smithi090.stdout:8/718: chown d0/d3/d86/d93/fc1 18320966 1 2024-03-11T19:42:09.961 INFO:tasks.workunit.client.0.smithi090.stdout:4/700: rename d4/fa to d4/dd8/db5/feb 0 2024-03-11T19:42:10.165 INFO:tasks.workunit.client.0.smithi090.stdout:5/638: creat d1/d17/d39/d51/d73/da6/fd8 x:0 0 0 2024-03-11T19:42:10.399 INFO:tasks.workunit.client.0.smithi090.stdout:1/660: truncate d5/d4e/d54/d94/f96 5795781 0 2024-03-11T19:42:10.407 INFO:tasks.workunit.client.0.smithi090.stdout:0/684: unlink d2/d11/l13 0 2024-03-11T19:42:10.719 INFO:tasks.workunit.client.0.smithi090.stdout:9/648: rmdir d1/d47/d7c 39 2024-03-11T19:42:10.880 INFO:tasks.workunit.client.0.smithi090.stdout:6/712: fdatasync d4/d13/d22/d34/d40/dd5/fbe 0 2024-03-11T19:42:11.016 INFO:tasks.workunit.client.0.smithi090.stdout:6/713: sync 2024-03-11T19:42:11.157 INFO:tasks.workunit.client.0.smithi090.stdout:2/658: creat d0/de/d12/d46/d5d/fcd x:0 0 0 2024-03-11T19:42:11.241 INFO:tasks.workunit.client.0.smithi090.stdout:4/701: truncate d4/dd8/fa7 738506 0 2024-03-11T19:42:11.371 INFO:tasks.workunit.client.0.smithi090.stdout:7/651: dwrite d3/dc/d1e/ff [8388608,4194304] 0 2024-03-11T19:42:11.539 INFO:tasks.workunit.client.0.smithi090.stdout:4/702: chown d4/d51/d8c/d6a/la2 211079 1 2024-03-11T19:42:11.541 INFO:tasks.workunit.client.0.smithi090.stdout:7/652: chown d3/dc/d87 40785 1 2024-03-11T19:42:11.721 INFO:tasks.workunit.client.0.smithi090.stdout:9/649: symlink d1/dd/d18/d25/ld1 0 2024-03-11T19:42:11.974 INFO:tasks.workunit.client.0.smithi090.stdout:6/714: dread - d4/d13/d22/d34/d83/fd6 zero size 2024-03-11T19:42:12.488 INFO:tasks.workunit.client.0.smithi090.stdout:1/661: symlink d5/d31/d76/ldf 0 2024-03-11T19:42:12.587 INFO:tasks.workunit.client.0.smithi090.stdout:9/650: truncate d1/dd/df/d1c/d21/d2a/fa3 571503 0 2024-03-11T19:42:12.837 INFO:tasks.workunit.client.0.smithi090.stdout:3/622: rename d2/db/d15/d1a/d3c/l43 to d2/d1d/lc2 0 2024-03-11T19:42:12.908 INFO:tasks.workunit.client.0.smithi090.stdout:2/659: truncate d0/de/d12/d81/fbc 3173737 0 2024-03-11T19:42:12.945 INFO:tasks.workunit.client.0.smithi090.stdout:2/660: stat d0/de/d12/d46/d5d/d63/ca7 0 2024-03-11T19:42:13.118 INFO:tasks.workunit.client.0.smithi090.stdout:7/653: mkdir d3/de6 0 2024-03-11T19:42:13.795 INFO:tasks.workunit.client.0.smithi090.stdout:4/703: mknod d4/d18/cec 0 2024-03-11T19:42:13.806 INFO:tasks.workunit.client.0.smithi090.stdout:1/662: mknod d5/d65/d6b/ce0 0 2024-03-11T19:42:13.840 INFO:tasks.workunit.client.0.smithi090.stdout:8/719: rename d0/d53/d59/d82/fd0 to d0/d3/d86/d93/feb 0 2024-03-11T19:42:13.852 INFO:tasks.workunit.client.0.smithi090.stdout:3/623: mknod d2/db/d15/d1a/d75/d8c/cc3 0 2024-03-11T19:42:13.890 INFO:tasks.workunit.client.0.smithi090.stdout:4/704: stat d4/l5d 0 2024-03-11T19:42:14.213 INFO:tasks.workunit.client.0.smithi090.stdout:2/661: dread d0/de/d12/f5e [0,4194304] 0 2024-03-11T19:42:14.454 INFO:tasks.workunit.client.0.smithi090.stdout:5/639: write d1/d26/d4d/f55 [44184,36886] 0 2024-03-11T19:42:14.590 INFO:tasks.workunit.client.0.smithi090.stdout:0/685: write d2/f1c [6015092,37000] 0 2024-03-11T19:42:14.792 INFO:tasks.workunit.client.0.smithi090.stdout:7/654: symlink d3/de6/le7 0 2024-03-11T19:42:14.970 INFO:tasks.workunit.client.0.smithi090.stdout:9/651: symlink d1/d49/d9c/daf/ld2 0 2024-03-11T19:42:15.150 INFO:tasks.workunit.client.0.smithi090.stdout:3/624: mkdir d2/d2f/dc4 0 2024-03-11T19:42:15.327 INFO:tasks.workunit.client.0.smithi090.stdout:2/662: fdatasync d0/de/d12/d46/d4c/d64/fa3 0 2024-03-11T19:42:15.993 INFO:tasks.workunit.client.0.smithi090.stdout:0/686: dread - d2/d11/d15/d29/fb9 zero size 2024-03-11T19:42:16.052 INFO:tasks.workunit.client.0.smithi090.stdout:5/640: dread d1/d17/d39/f68 [0,4194304] 0 2024-03-11T19:42:16.527 INFO:tasks.workunit.client.0.smithi090.stdout:9/652: dread d1/dd/f13 [0,4194304] 0 2024-03-11T19:42:16.789 INFO:tasks.workunit.client.0.smithi090.stdout:2/663: rename d0/de/d12/d46/d5d/d63/d8e to d0/d21/dce 0 2024-03-11T19:42:16.802 INFO:tasks.workunit.client.0.smithi090.stdout:6/715: write d4/d13/d22/d3a/f56 [1412241,111841] 0 2024-03-11T19:42:16.888 INFO:tasks.workunit.client.0.smithi090.stdout:2/664: chown d0/de/db6/l74 1754 1 2024-03-11T19:42:17.324 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:42:17.327 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:42:17.384 INFO:tasks.workunit.client.0.smithi090.stdout:9/653: chown d1/dd/d18/d25/f82 11966310 1 2024-03-11T19:42:17.650 INFO:tasks.workunit.client.0.smithi090.stdout:6/716: mkdir d4/d9b/def 0 2024-03-11T19:42:18.161 INFO:tasks.workunit.client.0.smithi090.stdout:2/665: fdatasync d0/de/f68 0 2024-03-11T19:42:18.323 INFO:tasks.workunit.client.0.smithi090.stdout:5/641: rename d1/d17/d39/d51/d53/d64/c6c to d1/d17/cd9 0 2024-03-11T19:42:18.796 INFO:tasks.workunit.client.0.smithi090.stdout:5/642: symlink d1/d17/d39/d51/d53/lda 0 2024-03-11T19:42:19.658 INFO:tasks.workunit.client.0.smithi090.stdout:8/720: dwrite d0/d43/f61 [0,4194304] 0 2024-03-11T19:42:19.767 INFO:tasks.workunit.client.0.smithi090.stdout:4/705: dwrite d4/dd8/fa4 [0,4194304] 0 2024-03-11T19:42:19.938 INFO:tasks.workunit.client.0.smithi090.stdout:1/663: write d5/da/f68 [900084,109502] 0 2024-03-11T19:42:19.957 INFO:tasks.workunit.client.0.smithi090.stdout:7/655: write d3/dc/d29/f6b [395325,55399] 0 2024-03-11T19:42:20.046 INFO:tasks.workunit.client.0.smithi090.stdout:0/687: write d2/d11/d15/d29/d2e/d6d/da4/fa2 [3288734,75628] 0 2024-03-11T19:42:20.147 INFO:tasks.workunit.client.0.smithi090.stdout:5/643: read d1/d17/d39/d51/d53/d7b/d3b/d76/fa1 [79838,57761] 0 2024-03-11T19:42:20.154 INFO:tasks.workunit.client.0.smithi090.stdout:4/706: chown d4/d51/d8c/d89/lae 1954 1 2024-03-11T19:42:20.326 INFO:tasks.workunit.client.0.smithi090.stdout:1/664: sync 2024-03-11T19:42:20.372 INFO:tasks.workunit.client.0.smithi090.stdout:3/625: dwrite d2/db/d15/d1a/d3c/f76 [0,4194304] 0 2024-03-11T19:42:20.456 INFO:tasks.workunit.client.0.smithi090.stdout:7/656: sync 2024-03-11T19:42:20.696 INFO:tasks.workunit.client.0.smithi090.stdout:8/721: creat d0/d43/dc/d90/fec x:0 0 0 2024-03-11T19:42:20.711 INFO:tasks.workunit.client.0.smithi090.stdout:4/707: mknod d4/dd8/d66/da1/ced 0 2024-03-11T19:42:20.960 INFO:tasks.workunit.client.0.smithi090.stdout:6/717: dwrite d4/d13/d22/d3a/f65 [0,4194304] 0 2024-03-11T19:42:21.092 INFO:tasks.workunit.client.0.smithi090.stdout:2/666: write d0/de/d12/d46/fa8 [221293,82048] 0 2024-03-11T19:42:21.132 INFO:tasks.workunit.client.0.smithi090.stdout:9/654: dwrite d1/dd/d18/d3a/fbe [0,4194304] 0 2024-03-11T19:42:21.301 INFO:tasks.workunit.client.0.smithi090.stdout:0/688: rename d2/d11/d3a/d64/db7 to d2/d11/d3a/d64/dd5 0 2024-03-11T19:42:21.576 INFO:tasks.workunit.client.0.smithi090.stdout:1/665: rename d5/d4e/l61 to d5/d4e/d79/le1 0 2024-03-11T19:42:21.756 INFO:tasks.workunit.client.0.smithi090.stdout:3/626: dread d2/d1d/f27 [0,4194304] 0 2024-03-11T19:42:21.792 INFO:tasks.workunit.client.0.smithi090.stdout:6/718: unlink d4/d14/d8a/fb8 0 2024-03-11T19:42:21.949 INFO:tasks.workunit.client.0.smithi090.stdout:9/655: dwrite d1/dd/d18/d3a/fbe [0,4194304] 0 2024-03-11T19:42:22.099 INFO:tasks.workunit.client.0.smithi090.stdout:0/689: fsync d2/d11/d3f/f83 0 2024-03-11T19:42:22.153 INFO:tasks.workunit.client.0.smithi090.stdout:7/657: fdatasync d3/dc/d1e/d7c/d96/d32/d71/fdf 0 2024-03-11T19:42:22.183 INFO:tasks.workunit.client.0.smithi090.stdout:7/658: chown d3/dc/d29/f4e 237698 1 2024-03-11T19:42:22.289 INFO:tasks.workunit.client.0.smithi090.stdout:5/644: rename d1/d17/d39/d51/d53/d64/f84 to d1/d17/d39/d51/d53/d7b/d3b/fdb 0 2024-03-11T19:42:22.646 INFO:tasks.workunit.client.0.smithi090.stdout:6/719: creat d4/d13/d22/d42/d48/dd2/ff0 x:0 0 0 2024-03-11T19:42:23.429 INFO:tasks.workunit.client.0.smithi090.stdout:5/645: truncate d1/d17/d39/d51/d73/da6/fd8 782423 0 2024-03-11T19:42:24.193 INFO:tasks.workunit.client.0.smithi090.stdout:5/646: mknod d1/d17/d39/d51/d53/cdc 0 2024-03-11T19:42:24.275 INFO:tasks.workunit.client.0.smithi090.stdout:3/627: creat d2/d1d/fc5 x:0 0 0 2024-03-11T19:42:24.817 INFO:tasks.workunit.client.0.smithi090.stdout:4/708: rename d4/d51/fc3 to d4/fee 0 2024-03-11T19:42:25.153 INFO:tasks.workunit.client.0.smithi090.stdout:1/666: rename d5/d31/d76/fa2 to d5/d31/d76/fe2 0 2024-03-11T19:42:25.400 INFO:tasks.workunit.client.0.smithi090.stdout:3/628: dread d2/db/d15/f17 [0,4194304] 0 2024-03-11T19:42:25.566 INFO:tasks.workunit.client.0.smithi090.stdout:8/722: rename d0/d3/d3e/c6a to d0/d3/d3e/d85/ced 0 2024-03-11T19:42:25.670 INFO:tasks.workunit.client.0.smithi090.stdout:4/709: truncate d4/dd8/f65 3568763 0 2024-03-11T19:42:25.712 INFO:tasks.workunit.client.0.smithi090.stdout:4/710: read d4/d18/fab [479033,46877] 0 2024-03-11T19:42:25.905 INFO:tasks.workunit.client.0.smithi090.stdout:1/667: truncate d5/da/d1c/f39 4011871 0 2024-03-11T19:42:25.918 INFO:tasks.workunit.client.0.smithi090.stdout:4/711: sync 2024-03-11T19:42:25.937 INFO:tasks.workunit.client.0.smithi090.stdout:6/720: getdents d4/d13/d22/d42/d48/dd2 0 2024-03-11T19:42:25.996 INFO:tasks.workunit.client.0.smithi090.stdout:4/712: chown d4/d51/d8c 692277 1 2024-03-11T19:42:26.048 INFO:tasks.workunit.client.0.smithi090.stdout:3/629: readlink d2/db/d15/l39 0 2024-03-11T19:42:26.066 INFO:tasks.workunit.client.0.smithi090.stdout:9/656: write d1/dd/df/d1c/f81 [1112684,111615] 0 2024-03-11T19:42:26.150 INFO:tasks.workunit.client.0.smithi090.stdout:9/657: chown d1/dd/d18/d3a/l43 24666965 1 2024-03-11T19:42:26.219 INFO:tasks.workunit.client.0.smithi090.stdout:8/723: symlink d0/d53/lee 0 2024-03-11T19:42:26.495 INFO:tasks.workunit.client.0.smithi090.stdout:7/659: dwrite d3/dc/d1e/d7c/d96/d32/d45/de0/f5a [0,4194304] 0 2024-03-11T19:42:26.729 INFO:tasks.workunit.client.0.smithi090.stdout:7/660: stat d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/db5/de1/dc2/dd0/le5 0 2024-03-11T19:42:27.267 INFO:tasks.workunit.client.0.smithi090.stdout:2/667: rename d0/de/cf to d0/dad/ccf 0 2024-03-11T19:42:27.464 INFO:tasks.workunit.client.0.smithi090.stdout:1/668: truncate d5/f37 1575028 0 2024-03-11T19:42:27.934 INFO:tasks.workunit.client.0.smithi090.stdout:9/658: dread - d1/dd/d18/d3a/da2/fc0 zero size 2024-03-11T19:42:27.956 INFO:tasks.workunit.client.0.smithi090.stdout:9/659: readlink d1/d49/l5a 0 2024-03-11T19:42:28.389 INFO:tasks.workunit.client.0.smithi090.stdout:5/647: link d1/d17/f30 d1/d17/d39/d51/d53/d7b/fdd 0 2024-03-11T19:42:28.415 INFO:tasks.workunit.client.0.smithi090.stdout:7/661: read d3/dc/d1e/d7c/d96/d32/d71/fa9 [666521,101008] 0 2024-03-11T19:42:28.517 INFO:tasks.workunit.client.0.smithi090.stdout:7/662: dread - d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/fcb zero size 2024-03-11T19:42:28.946 INFO:tasks.workunit.client.0.smithi090.stdout:2/668: dread d0/de/d12/f1d [0,4194304] 0 2024-03-11T19:42:29.646 INFO:tasks.workunit.client.0.smithi090.stdout:6/721: dwrite d4/d14/d25/fe6 [0,4194304] 0 2024-03-11T19:42:29.929 INFO:tasks.workunit.client.0.smithi090.stdout:7/663: mknod d3/dc/d1e/d7c/d96/d32/da0/ce8 0 2024-03-11T19:42:30.101 INFO:tasks.workunit.client.0.smithi090.stdout:8/724: link d0/f58 d0/d3/d4c/d8d/fef 0 2024-03-11T19:42:31.181 INFO:tasks.workunit.client.0.smithi090.stdout:1/669: dwrite d5/da/f93 [0,4194304] 0 2024-03-11T19:42:31.396 INFO:tasks.workunit.client.0.smithi090.stdout:8/725: rmdir d0/d53 39 2024-03-11T19:42:31.465 INFO:tasks.workunit.client.0.smithi090.stdout:9/660: write d1/f58 [1059916,15208] 0 2024-03-11T19:42:31.474 INFO:tasks.workunit.client.0.smithi090.stdout:4/713: dwrite d4/dd8/fa7 [0,4194304] 0 2024-03-11T19:42:31.821 INFO:tasks.workunit.client.0.smithi090.stdout:7/664: getdents d3/dc/d1e/d7c/d96/d32/d67 0 2024-03-11T19:42:32.373 INFO:tasks.workunit.client.0.smithi090.stdout:6/722: link d4/d14/d8a/d74/cd9 d4/d13/d22/d34/cf1 0 2024-03-11T19:42:32.597 INFO:tasks.workunit.client.0.smithi090.stdout:4/714: truncate d4/dd8/d66/da1/fde 119217 0 2024-03-11T19:42:32.860 INFO:tasks.workunit.client.0.smithi090.stdout:7/665: rmdir d3/dc/d1e/d7c/d96/d32/d45/d84 39 2024-03-11T19:42:33.022 INFO:tasks.workunit.client.0.smithi090.stdout:8/726: mknod d0/cf0 0 2024-03-11T19:42:33.043 INFO:tasks.workunit.client.0.smithi090.stdout:1/670: symlink d5/da/d1c/d3b/le3 0 2024-03-11T19:42:33.240 INFO:tasks.workunit.client.0.smithi090.stdout:5/648: dwrite d1/f4f [0,4194304] 0 2024-03-11T19:42:33.379 INFO:tasks.workunit.client.0.smithi090.stdout:9/661: truncate d1/f17 2068677 0 2024-03-11T19:42:33.383 INFO:tasks.workunit.client.0.smithi090.stdout:6/723: fdatasync d4/d13/d22/d3a/f65 0 2024-03-11T19:42:33.413 INFO:tasks.workunit.client.0.smithi090.stdout:2/669: creat d0/de/d12/fd0 x:0 0 0 2024-03-11T19:42:33.673 INFO:tasks.workunit.client.0.smithi090.stdout:4/715: dread d4/dd8/fa4 [0,4194304] 0 2024-03-11T19:42:33.900 INFO:tasks.workunit.client.0.smithi090.stdout:1/671: chown d5/d65/d6b/l6f 147244084 1 2024-03-11T19:42:33.949 INFO:tasks.workunit.client.0.smithi090.stdout:1/672: write d5/da/f93 [296965,93816] 0 2024-03-11T19:42:34.495 INFO:tasks.workunit.client.0.smithi090.stdout:0/690: rename d2/d11/f18 to d2/fd6 0 2024-03-11T19:42:34.589 INFO:tasks.workunit.client.0.smithi090.stdout:4/716: dread d4/d18/fb0 [0,4194304] 0 2024-03-11T19:42:34.615 INFO:tasks.workunit.client.0.smithi090.stdout:1/673: mknod d5/da/d1c/d6d/d80/ce4 0 2024-03-11T19:42:34.665 INFO:tasks.workunit.client.0.smithi090.stdout:4/717: read d4/d27/d28/fbe [2516807,55116] 0 2024-03-11T19:42:35.134 INFO:tasks.workunit.client.0.smithi090.stdout:7/666: mknod d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/db5/de1/dc6/ce9 0 2024-03-11T19:42:35.479 INFO:tasks.workunit.client.0.smithi090.stdout:7/667: dread d3/dc/d1e/d7c/d96/d32/d71/fa9 [0,4194304] 0 2024-03-11T19:42:35.509 INFO:tasks.workunit.client.0.smithi090.stdout:7/668: chown d3/dc/fa7 7920 1 2024-03-11T19:42:35.869 INFO:tasks.workunit.client.0.smithi090.stdout:0/691: dread d2/d11/d3a/f75 [0,4194304] 0 2024-03-11T19:42:36.352 INFO:tasks.workunit.client.0.smithi090.stdout:7/669: truncate d3/dc/d1e/d7c/d96/d32/d45/f9d 523524 0 2024-03-11T19:42:36.856 INFO:tasks.workunit.client.0.smithi090.stdout:9/662: link d1/d1d/d6a/lc5 d1/da9/ld3 0 2024-03-11T19:42:36.967 INFO:tasks.workunit.client.0.smithi090.stdout:6/724: dwrite d4/d14/d8a/d74/fba [0,4194304] 0 2024-03-11T19:42:37.128 INFO:tasks.workunit.client.0.smithi090.stdout:7/670: read d3/dc/d1e/d7c/d96/d32/d45/de0/f42 [4005,127465] 0 2024-03-11T19:42:37.205 INFO:tasks.workunit.client.0.smithi090.stdout:3/630: rename d2/d1d to d2/db/d15/d1a/d75/dc6 0 2024-03-11T19:42:37.907 INFO:tasks.workunit.client.0.smithi090.stdout:8/727: write d0/d53/f5d [2779605,21343] 0 2024-03-11T19:42:38.035 INFO:tasks.workunit.client.0.smithi090.stdout:8/728: readlink d0/d60/l83 0 2024-03-11T19:42:38.131 INFO:tasks.workunit.client.0.smithi090.stdout:6/725: dread d4/d13/d22/d3a/f56 [0,4194304] 0 2024-03-11T19:42:38.430 INFO:tasks.workunit.client.0.smithi090.stdout:5/649: rename d1/d17/f6a to d1/d26/dc8/fde 0 2024-03-11T19:42:38.430 INFO:tasks.workunit.client.0.smithi090.stdout:7/671: creat d3/dc/d1e/d7c/d96/fea x:0 0 0 2024-03-11T19:42:38.435 INFO:tasks.workunit.client.0.smithi090.stdout:6/726: sync 2024-03-11T19:42:38.634 INFO:tasks.workunit.client.0.smithi090.stdout:4/718: link d4/c19 d4/d18/d6f/cef 0 2024-03-11T19:42:38.830 INFO:tasks.workunit.client.0.smithi090.stdout:8/729: rmdir d0/d43/dc 39 2024-03-11T19:42:38.936 INFO:tasks.workunit.client.0.smithi090.stdout:3/631: mkdir d2/db/d15/d69/da2/dc7 0 2024-03-11T19:42:38.943 INFO:tasks.workunit.client.0.smithi090.stdout:8/730: sync 2024-03-11T19:42:39.037 INFO:tasks.workunit.client.0.smithi090.stdout:3/632: read d2/db/d15/d1a/d75/dc6/f27 [3549827,47253] 0 2024-03-11T19:42:39.121 INFO:tasks.workunit.client.0.smithi090.stdout:7/672: unlink d3/dc/d29/l7e 0 2024-03-11T19:42:39.137 INFO:tasks.workunit.client.0.smithi090.stdout:1/674: write d5/f37 [1499188,74493] 0 2024-03-11T19:42:39.731 INFO:tasks.workunit.client.0.smithi090.stdout:4/719: dread d4/d27/d28/d7a/fc0 [0,4194304] 0 2024-03-11T19:42:40.384 INFO:tasks.workunit.client.0.smithi090.stdout:1/675: dread d5/da/d1c/d3b/d8c/fc4 [0,4194304] 0 2024-03-11T19:42:40.629 INFO:tasks.workunit.client.0.smithi090.stdout:9/663: dwrite d1/dd/f69 [0,4194304] 0 2024-03-11T19:42:40.939 INFO:tasks.workunit.client.0.smithi090.stdout:2/670: rename d0/de/db6/l74 to d0/de/d12/d46/d4c/ld1 0 2024-03-11T19:42:41.045 INFO:tasks.workunit.client.0.smithi090.stdout:6/727: link d4/d13/d22/d42/d48/d4e/l5b d4/d9b/lf2 0 2024-03-11T19:42:41.344 INFO:tasks.workunit.client.0.smithi090.stdout:1/676: symlink d5/d31/d4c/le5 0 2024-03-11T19:42:41.658 INFO:tasks.workunit.client.0.smithi090.stdout:0/692: rename d2/d11/d3f/f83 to d2/d11/d15/d29/d3d/d7b/fd7 0 2024-03-11T19:42:41.722 INFO:tasks.workunit.client.0.smithi090.stdout:5/650: dwrite d1/d17/d39/d51/d53/d7b/d3b/d76/f77 [0,4194304] 0 2024-03-11T19:42:41.835 INFO:tasks.workunit.client.0.smithi090.stdout:2/671: symlink d0/dad/ld2 0 2024-03-11T19:42:41.950 INFO:tasks.workunit.client.0.smithi090.stdout:5/651: dread d1/d26/d4d/fd1 [0,4194304] 0 2024-03-11T19:42:41.970 INFO:tasks.workunit.client.0.smithi090.stdout:5/652: chown d1/f4f 58 1 2024-03-11T19:42:43.147 INFO:tasks.workunit.client.0.smithi090.stdout:7/673: dwrite d3/fd1 [0,4194304] 0 2024-03-11T19:42:43.326 INFO:tasks.workunit.client.0.smithi090.stdout:5/653: mkdir d1/d17/d39/d51/d53/d7b/d3b/d76/db7/ddf 0 2024-03-11T19:42:43.504 INFO:tasks.workunit.client.0.smithi090.stdout:3/633: link d2/db/d15/d1a/d75/dc6/d47/d6a/d9b/fae d2/db/d15/d1a/d75/fc8 0 2024-03-11T19:42:43.539 INFO:tasks.workunit.client.0.smithi090.stdout:4/720: dwrite d4/d18/f41 [4194304,4194304] 0 2024-03-11T19:42:43.706 INFO:tasks.workunit.client.0.smithi090.stdout:8/731: dwrite d0/d43/dc/f28 [4194304,4194304] 0 2024-03-11T19:42:44.061 INFO:tasks.workunit.client.0.smithi090.stdout:1/677: link d5/da/cbe d5/d31/d86/ce6 0 2024-03-11T19:42:44.296 INFO:tasks.workunit.client.0.smithi090.stdout:6/728: dwrite d4/d13/d22/d3a/f56 [0,4194304] 0 2024-03-11T19:42:44.511 INFO:tasks.workunit.client.0.smithi090.stdout:9/664: link d1/dd/df/d1c/fc9 d1/dd/d18/d3a/fd4 0 2024-03-11T19:42:44.868 INFO:tasks.workunit.client.0.smithi090.stdout:8/732: read d0/d3/d86/f9c [725836,49859] 0 2024-03-11T19:42:44.907 INFO:tasks.workunit.client.0.smithi090.stdout:8/733: chown d0/d3/d86/d93/f9b 1 1 2024-03-11T19:42:45.274 INFO:tasks.workunit.client.0.smithi090.stdout:7/674: mkdir d3/dc/da6/deb 0 2024-03-11T19:42:45.701 INFO:tasks.workunit.client.0.smithi090.stdout:6/729: rename d4/d13/d22/d3a/dd7/cb4 to d4/d13/d22/d34/d83/d8d/cf3 0 2024-03-11T19:42:45.938 INFO:tasks.workunit.client.0.smithi090.stdout:4/721: truncate d4/f46 485761 0 2024-03-11T19:42:46.059 INFO:tasks.workunit.client.0.smithi090.stdout:0/693: dwrite d2/d11/d3f/f76 [0,4194304] 0 2024-03-11T19:42:46.208 INFO:tasks.workunit.client.0.smithi090.stdout:3/634: symlink d2/d2f/dc4/lc9 0 2024-03-11T19:42:46.240 INFO:tasks.workunit.client.0.smithi090.stdout:2/672: getdents d0/de/d12/d81 0 2024-03-11T19:42:46.250 INFO:tasks.workunit.client.0.smithi090.stdout:8/734: symlink d0/d3/d98/lf1 0 2024-03-11T19:42:46.414 INFO:tasks.workunit.client.0.smithi090.stdout:7/675: fsync d3/dc/d1e/d7c/d96/d32/d71/d80/f95 0 2024-03-11T19:42:46.535 INFO:tasks.workunit.client.0.smithi090.stdout:5/654: dwrite d1/f2c [0,4194304] 0 2024-03-11T19:42:46.561 INFO:tasks.workunit.client.0.smithi090.stdout:9/665: rename d1/dd/d18/d3a/l43 to d1/d1d/ld5 0 2024-03-11T19:42:47.274 INFO:tasks.workunit.client.0.smithi090.stdout:1/678: write d5/d3c/fa5 [730049,96237] 0 2024-03-11T19:42:47.377 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:42:47.382 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:42:47.569 INFO:tasks.workunit.client.0.smithi090.stdout:8/735: dread d0/d43/dc/f12 [0,4194304] 0 2024-03-11T19:42:47.708 INFO:tasks.workunit.client.0.smithi090.stdout:7/676: rename d3/dc/d29/f4e to d3/dc/d1e/d2a/fec 0 2024-03-11T19:42:47.749 INFO:tasks.workunit.client.0.smithi090.stdout:7/677: chown d3/dc/d1e/d7c/d96/d32/c38 1927 1 2024-03-11T19:42:47.794 INFO:tasks.workunit.client.0.smithi090.stdout:6/730: unlink d4/d13/d22/d34/d40/d7d/dc2/ce0 0 2024-03-11T19:42:47.913 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:08:07.671 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:42:47.921 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:08:07.680 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:42:48.009 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:08:07.767 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:42:48.038 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:08:07.738 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:42:48.041 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:08:07.732 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:42:48.042 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:08:07.742 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:42:48.046 INFO:tasks.workunit.client.0.smithi090.stdout:5/655: mkdir d1/d17/d39/d51/d53/d7b/d3b/d76/db7/de0 0 2024-03-11T19:42:48.056 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:08:07.747 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:42:48.117 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:08:07.816 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:42:48.129 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:08:07.820 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:42:48.741 INFO:tasks.workunit.client.0.smithi090.stdout:2/673: mkdir d0/de/dd3 0 2024-03-11T19:42:48.766 INFO:tasks.workunit.client.0.smithi090.stdout:8/736: mknod d0/d60/db6/db9/cf2 0 2024-03-11T19:42:48.780 INFO:tasks.workunit.client.0.smithi090.stdout:7/678: fdatasync d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/fcb 0 2024-03-11T19:42:48.834 INFO:tasks.workunit.client.0.smithi090.stdout:7/679: chown d3/f4b 40 1 2024-03-11T19:42:48.835 INFO:tasks.workunit.client.0.smithi090.stdout:8/737: stat d0/d43/dc/l4f 0 2024-03-11T19:42:48.931 INFO:tasks.workunit.client.0.smithi090.stdout:7/680: chown d3/dc/f33 307 1 2024-03-11T19:42:48.990 INFO:tasks.workunit.client.0.smithi090.stdout:5/656: rmdir d1/d17/d39/d51 39 2024-03-11T19:42:49.020 INFO:tasks.workunit.client.0.smithi090.stdout:0/694: write f0 [4779529,68174] 0 2024-03-11T19:42:49.103 INFO:tasks.workunit.client.0.smithi090.stdout:4/722: dwrite d4/d18/f2a [0,4194304] 0 2024-03-11T19:42:49.704 INFO:tasks.workunit.client.0.smithi090.stdout:8/738: rename d0/d43/l9d to d0/d53/d59/d96/de5/de7/lf3 0 2024-03-11T19:42:50.352 INFO:tasks.workunit.client.0.smithi090.stdout:3/635: dwrite d2/db/f84 [0,4194304] 0 2024-03-11T19:42:50.537 INFO:tasks.workunit.client.0.smithi090.stdout:7/681: creat d3/dc/d1e/d7c/d96/d32/da0/fed x:0 0 0 2024-03-11T19:42:50.867 INFO:tasks.workunit.client.0.smithi090.stdout:4/723: symlink d4/dd8/d66/da1/lf0 0 2024-03-11T19:42:50.919 INFO:tasks.workunit.client.0.smithi090.stdout:7/682: dread d3/dc/d1e/f46 [0,4194304] 0 2024-03-11T19:42:51.069 INFO:tasks.workunit.client.0.smithi090.stdout:9/666: link d1/dd/df/d1c/d88/la6 d1/d47/d7c/ld6 0 2024-03-11T19:42:51.141 INFO:tasks.workunit.client.0.smithi090.stdout:2/674: write d0/f2 [7144215,83788] 0 2024-03-11T19:42:51.279 INFO:tasks.workunit.client.0.smithi090.stdout:1/679: dwrite d5/d31/d76/fe2 [4194304,4194304] 0 2024-03-11T19:42:51.709 INFO:tasks.workunit.client.0.smithi090.stdout:1/680: sync 2024-03-11T19:42:52.317 INFO:tasks.workunit.client.0.smithi090.stdout:6/731: getdents d4/d14 0 2024-03-11T19:42:52.844 INFO:tasks.workunit.client.0.smithi090.stdout:1/681: mkdir d5/d65/d6b/de7 0 2024-03-11T19:42:53.340 INFO:tasks.workunit.client.0.smithi090.stdout:4/724: unlink d4/dd8/d66/da1/fde 0 2024-03-11T19:42:53.661 INFO:tasks.workunit.client.0.smithi090.stdout:6/732: truncate d4/d13/d22/d34/d37/f8e 179873 0 2024-03-11T19:42:54.137 INFO:tasks.workunit.client.0.smithi090.stdout:8/739: link d0/d43/f2e d0/d3/d4c/d8d/ff4 0 2024-03-11T19:42:54.383 INFO:tasks.workunit.client.0.smithi090.stdout:5/657: link d1/d26/f54 d1/d17/d39/d51/d53/d7b/d3b/fe1 0 2024-03-11T19:42:54.458 INFO:tasks.workunit.client.0.smithi090.stdout:7/683: dwrite d3/f3b [0,4194304] 0 2024-03-11T19:42:54.586 INFO:tasks.workunit.client.0.smithi090.stdout:2/675: write d0/de/d12/d46/d5d/d8d/fa6 [494795,34651] 0 2024-03-11T19:42:54.596 INFO:tasks.workunit.client.0.smithi090.stdout:0/695: getdents d2/d11/d3a/d64/dac 0 2024-03-11T19:42:54.709 INFO:tasks.workunit.client.0.smithi090.stdout:3/636: write d2/db/f24 [2907091,57834] 0 2024-03-11T19:42:54.839 INFO:tasks.workunit.client.0.smithi090.stdout:6/733: read d4/d13/f1f [848921,97160] 0 2024-03-11T19:42:54.971 INFO:tasks.workunit.client.0.smithi090.stdout:8/740: mknod d0/d3/d86/d93/dba/cf5 0 2024-03-11T19:42:55.012 INFO:tasks.workunit.client.0.smithi090.stdout:6/734: sync 2024-03-11T19:42:55.175 INFO:tasks.workunit.client.0.smithi090.stdout:4/725: mknod d4/d18/d6f/cf1 0 2024-03-11T19:42:55.217 INFO:tasks.workunit.client.0.smithi090.stdout:4/726: dread - d4/d51/d8c/d6a/d8d/f8e zero size 2024-03-11T19:42:55.339 INFO:tasks.workunit.client.0.smithi090.stdout:4/727: sync 2024-03-11T19:42:55.391 INFO:tasks.workunit.client.0.smithi090.stdout:4/728: chown d4/d27/d28/d7a 0 1 2024-03-11T19:42:55.841 INFO:tasks.workunit.client.0.smithi090.stdout:2/676: mkdir d0/de/d12/d46/d4c/d64/dd4 0 2024-03-11T19:42:56.202 INFO:tasks.workunit.client.0.smithi090.stdout:9/667: dwrite d1/dd/df/d1c/d21/d2a/fa3 [0,4194304] 0 2024-03-11T19:42:56.315 INFO:tasks.workunit.client.0.smithi090.stdout:7/684: truncate d3/dc/f33 780006 0 2024-03-11T19:42:56.330 INFO:tasks.workunit.client.0.smithi090.stdout:0/696: truncate d2/d11/d3a/f75 2413388 0 2024-03-11T19:42:56.356 INFO:tasks.workunit.client.0.smithi090.stdout:6/735: mknod d4/d13/d22/d42/cf4 0 2024-03-11T19:42:56.403 INFO:tasks.workunit.client.0.smithi090.stdout:4/729: rename d4/d18/f2a to d4/d51/d8c/d6a/d8d/ff2 0 2024-03-11T19:42:56.482 INFO:tasks.workunit.client.0.smithi090.stdout:8/741: chown d0/d53/d59/d96/de5/de7/lf3 9573 1 2024-03-11T19:42:56.553 INFO:tasks.workunit.client.0.smithi090.stdout:0/697: readlink d2/d11/d3a/da6/ld1 0 2024-03-11T19:42:56.671 INFO:tasks.workunit.client.0.smithi090.stdout:5/658: creat d1/d17/d39/d51/d53/d7b/d3b/d76/db7/de0/fe2 x:0 0 0 2024-03-11T19:42:56.720 INFO:tasks.workunit.client.0.smithi090.stdout:5/659: dread - d1/d17/d39/d51/d53/d64/f93 zero size 2024-03-11T19:42:57.128 INFO:tasks.workunit.client.0.smithi090.stdout:1/682: dwrite d5/d4e/d54/d94/f96 [4194304,4194304] 0 2024-03-11T19:42:57.502 INFO:tasks.workunit.client.0.smithi090.stdout:0/698: readlink d2/d11/d15/lcf 0 2024-03-11T19:42:57.524 INFO:tasks.workunit.client.0.smithi090.stdout:0/699: chown d2/d11/c92 24 1 2024-03-11T19:42:57.834 INFO:tasks.workunit.client.0.smithi090.stdout:7/685: mkdir d3/dc/d1e/d7c/d96/d32/d67/dee 0 2024-03-11T19:42:57.848 INFO:tasks.workunit.client.0.smithi090.stdout:8/742: creat d0/d3/d86/d93/dda/ff6 x:0 0 0 2024-03-11T19:42:58.005 INFO:tasks.workunit.client.0.smithi090.stdout:9/668: rename d1/d1d/f72 to d1/dd/d18/d3a/d70/fd7 0 2024-03-11T19:42:58.230 INFO:tasks.workunit.client.0.smithi090.stdout:3/637: dwrite d2/f6 [4194304,4194304] 0 2024-03-11T19:42:58.561 INFO:tasks.workunit.client.0.smithi090.stdout:8/743: read d0/d3/d3e/d85/fcd [2038248,124672] 0 2024-03-11T19:42:58.797 INFO:tasks.workunit.client.0.smithi090.stdout:7/686: symlink d3/dc/d1e/d7c/d96/lef 0 2024-03-11T19:42:59.251 INFO:tasks.workunit.client.0.smithi090.stdout:4/730: getdents d4/dd8/d66/dcf/dd7 0 2024-03-11T19:42:59.431 INFO:tasks.workunit.client.0.smithi090.stdout:9/669: dread d1/dd/d18/d32/f45 [0,4194304] 0 2024-03-11T19:42:59.595 INFO:tasks.workunit.client.0.smithi090.stdout:8/744: mknod d0/d3/d86/cf7 0 2024-03-11T19:42:59.650 INFO:tasks.workunit.client.0.smithi090.stdout:2/677: dwrite d0/d21/f6e [4194304,4194304] 0 2024-03-11T19:42:59.779 INFO:tasks.workunit.client.0.smithi090.stdout:7/687: truncate d3/dc/d1e/d2a/fcc 186215 0 2024-03-11T19:42:59.781 INFO:tasks.workunit.client.0.smithi090.stdout:4/731: dread d4/dd8/d66/f76 [0,4194304] 0 2024-03-11T19:42:59.834 INFO:tasks.workunit.client.0.smithi090.stdout:5/660: creat d1/d17/d39/fe3 x:0 0 0 2024-03-11T19:43:00.278 INFO:tasks.workunit.client.0.smithi090.stdout:1/683: creat d5/da/d1c/d6d/fe8 x:0 0 0 2024-03-11T19:43:00.742 INFO:tasks.workunit.client.0.smithi090.stdout:7/688: rmdir d3/dc/d1e/d7c/d96/d32/d7b 39 2024-03-11T19:43:01.047 INFO:tasks.workunit.client.0.smithi090.stdout:5/661: unlink d1/d17/d39/d51/d9a/fd4 0 2024-03-11T19:43:01.061 INFO:tasks.workunit.client.0.smithi090.stdout:3/638: link d2/f9a d2/db/d15/d1a/d75/dc6/d47/d6a/d9b/fca 0 2024-03-11T19:43:01.269 INFO:tasks.workunit.client.0.smithi090.stdout:9/670: mkdir d1/dd/d18/d3a/dd8 0 2024-03-11T19:43:01.599 INFO:tasks.workunit.client.0.smithi090.stdout:6/736: dwrite d4/d13/d22/d3a/fec [0,4194304] 0 2024-03-11T19:43:01.828 INFO:tasks.workunit.client.0.smithi090.stdout:8/745: unlink d0/d53/d59/l62 0 2024-03-11T19:43:01.952 INFO:tasks.workunit.client.0.smithi090.stdout:2/678: unlink d0/de/d12/d46/d5d/d63/c73 0 2024-03-11T19:43:01.981 INFO:tasks.workunit.client.0.smithi090.stdout:2/679: stat d0/de/d12/d46/cbf 0 2024-03-11T19:43:01.999 INFO:tasks.workunit.client.0.smithi090.stdout:2/680: chown d0/l8 743 1 2024-03-11T19:43:02.354 INFO:tasks.workunit.client.0.smithi090.stdout:3/639: rename d2/d1f to d2/db/d15/d1a/d75/dc6/d47/d6a/dcb 0 2024-03-11T19:43:02.363 INFO:tasks.workunit.client.0.smithi090.stdout:5/662: truncate d1/d17/d39/d51/d53/d7b/d6f/f8e 938708 0 2024-03-11T19:43:02.454 INFO:tasks.workunit.client.0.smithi090.stdout:3/640: chown d2/db/d15/d69/dad/lb4 1022197 1 2024-03-11T19:43:02.818 INFO:tasks.workunit.client.0.smithi090.stdout:0/700: dwrite d2/d11/d15/d29/fbc [0,4194304] 0 2024-03-11T19:43:02.972 INFO:tasks.workunit.client.0.smithi090.stdout:6/737: rmdir d4/d13/d22/d3a/dd7 39 2024-03-11T19:43:03.462 INFO:tasks.workunit.client.0.smithi090.stdout:8/746: mkdir d0/d3/d4c/d8d/df8 0 2024-03-11T19:43:04.041 INFO:tasks.workunit.client.0.smithi090.stdout:5/663: truncate d1/d17/d39/d51/d73/da6/fbf 560081 0 2024-03-11T19:43:04.260 INFO:tasks.workunit.client.0.smithi090.stdout:1/684: dwrite d5/da/d23/f2a [0,4194304] 0 2024-03-11T19:43:04.405 INFO:tasks.workunit.client.0.smithi090.stdout:6/738: truncate d4/d13/d22/d42/d48/d4e/fae 514252 0 2024-03-11T19:43:05.077 INFO:tasks.workunit.client.0.smithi090.stdout:3/641: dread d2/db/d15/d1a/d75/dc6/d47/d6a/f97 [0,4194304] 0 2024-03-11T19:43:05.101 INFO:tasks.workunit.client.0.smithi090.stdout:5/664: dread d1/d17/d39/f82 [0,4194304] 0 2024-03-11T19:43:05.271 INFO:tasks.workunit.client.0.smithi090.stdout:7/689: dwrite d3/f59 [0,4194304] 0 2024-03-11T19:43:05.495 INFO:tasks.workunit.client.0.smithi090.stdout:9/671: dwrite d1/dd/f1e [0,4194304] 0 2024-03-11T19:43:05.697 INFO:tasks.workunit.client.0.smithi090.stdout:8/747: rename d0/d53/d59/ca6 to d0/d3/d4c/d8d/df8/cf9 0 2024-03-11T19:43:05.697 INFO:tasks.workunit.client.0.smithi090.stdout:1/685: unlink d5/da/d1c/d6d/d80/fa0 0 2024-03-11T19:43:06.039 INFO:tasks.workunit.client.0.smithi090.stdout:0/701: creat d2/fd8 x:0 0 0 2024-03-11T19:43:06.481 INFO:tasks.workunit.client.0.smithi090.stdout:4/732: creat d4/ff3 x:0 0 0 2024-03-11T19:43:07.229 INFO:tasks.workunit.client.0.smithi090.stdout:9/672: fsync d1/dd/df/d1c/d21/f54 0 2024-03-11T19:43:07.446 INFO:tasks.workunit.client.0.smithi090.stdout:2/681: dwrite d0/de/d12/d46/d5d/d8d/fcb [0,4194304] 0 2024-03-11T19:43:07.459 INFO:tasks.workunit.client.0.smithi090.stdout:8/748: mknod d0/d53/d59/d96/cfa 0 2024-03-11T19:43:07.465 INFO:tasks.workunit.client.0.smithi090.stdout:1/686: rmdir d5/da/dd 39 2024-03-11T19:43:07.615 INFO:tasks.workunit.client.0.smithi090.stdout:2/682: stat d0/de/l26 0 2024-03-11T19:43:07.777 INFO:tasks.workunit.client.0.smithi090.stdout:4/733: dread - d4/d27/d28/fe6 zero size 2024-03-11T19:43:08.192 INFO:tasks.workunit.client.0.smithi090.stdout:6/739: rename d4/d13/d22/d34/d37/c49 to d4/d13/d22/d34/d40/dd5/d80/ddb/cf5 0 2024-03-11T19:43:08.775 INFO:tasks.workunit.client.0.smithi090.stdout:5/665: creat d1/d17/fe4 x:0 0 0 2024-03-11T19:43:08.781 INFO:tasks.workunit.client.0.smithi090.stdout:9/673: chown d1/dd/df/f4a 57543386 1 2024-03-11T19:43:08.810 INFO:tasks.workunit.client.0.smithi090.stdout:8/749: dread d0/f58 [0,4194304] 0 2024-03-11T19:43:08.860 INFO:tasks.workunit.client.0.smithi090.stdout:5/666: chown d1/d26/f83 51 1 2024-03-11T19:43:08.926 INFO:tasks.workunit.client.0.smithi090.stdout:0/702: mknod d2/d94/cd9 0 2024-03-11T19:43:09.385 INFO:tasks.workunit.client.0.smithi090.stdout:6/740: mknod d4/d13/d22/d42/d48/d4e/cf6 0 2024-03-11T19:43:09.670 INFO:tasks.workunit.client.0.smithi090.stdout:7/690: dwrite d3/dc/d1e/d7c/d96/fdc [0,4194304] 0 2024-03-11T19:43:09.884 INFO:tasks.workunit.client.0.smithi090.stdout:5/667: mkdir d1/d17/d39/d51/d53/d7b/d3b/de5 0 2024-03-11T19:43:09.937 INFO:tasks.workunit.client.0.smithi090.stdout:4/734: dread d4/d18/f41 [4194304,4194304] 0 2024-03-11T19:43:09.943 INFO:tasks.workunit.client.0.smithi090.stdout:0/703: fsync d2/d11/d15/f28 0 2024-03-11T19:43:10.038 INFO:tasks.workunit.client.0.smithi090.stdout:3/642: dwrite d2/db/d15/d1a/d75/dc6/d47/d6a/dcb/fac [0,4194304] 0 2024-03-11T19:43:10.192 INFO:tasks.workunit.client.0.smithi090.stdout:5/668: dread - d1/d17/d39/d51/d73/f95 zero size 2024-03-11T19:43:10.269 INFO:tasks.workunit.client.0.smithi090.stdout:5/669: write d1/d17/d39/d51/d53/fb2 [170161,73441] 0 2024-03-11T19:43:11.580 INFO:tasks.workunit.client.0.smithi090.stdout:4/735: truncate d4/d51/d8c/d89/fc7 6643687 0 2024-03-11T19:43:11.783 INFO:tasks.workunit.client.0.smithi090.stdout:5/670: creat d1/d17/d39/d51/d53/d7b/d3b/d76/d92/dbd/fe6 x:0 0 0 2024-03-11T19:43:11.797 INFO:tasks.workunit.client.0.smithi090.stdout:9/674: creat d1/dd/df/fd9 x:0 0 0 2024-03-11T19:43:11.901 INFO:tasks.workunit.client.0.smithi090.stdout:1/687: dwrite d5/d4e/d54/f6e [4194304,4194304] 0 2024-03-11T19:43:12.037 INFO:tasks.workunit.client.0.smithi090.stdout:5/671: chown d1/d26/l47 5 1 2024-03-11T19:43:12.415 INFO:tasks.workunit.client.0.smithi090.stdout:2/683: dwrite d0/f2 [4194304,4194304] 0 2024-03-11T19:43:12.540 INFO:tasks.workunit.client.0.smithi090.stdout:0/704: creat d2/d11/d3a/d64/dac/dba/dd3/fda x:0 0 0 2024-03-11T19:43:12.582 INFO:tasks.workunit.client.0.smithi090.stdout:0/705: chown d2/d11/d3f 4282 1 2024-03-11T19:43:12.621 INFO:tasks.workunit.client.0.smithi090.stdout:0/706: chown d2/d11/d15/d29/d2e/db4/daf/fb3 122281 1 2024-03-11T19:43:12.987 INFO:tasks.workunit.client.0.smithi090.stdout:8/750: dwrite d0/d43/dc/d90/fec [0,4194304] 0 2024-03-11T19:43:13.086 INFO:tasks.workunit.client.0.smithi090.stdout:5/672: symlink d1/d17/d39/d51/d73/da6/le7 0 2024-03-11T19:43:13.348 INFO:tasks.workunit.client.0.smithi090.stdout:2/684: unlink d0/f3a 0 2024-03-11T19:43:13.701 INFO:tasks.workunit.client.0.smithi090.stdout:1/688: mknod d5/ce9 0 2024-03-11T19:43:13.894 INFO:tasks.workunit.client.0.smithi090.stdout:6/741: dwrite d4/d13/d22/d34/d83/d9c/fa5 [0,4194304] 0 2024-03-11T19:43:14.030 INFO:tasks.workunit.client.0.smithi090.stdout:9/675: creat d1/dd/d18/d3a/dd8/fda x:0 0 0 2024-03-11T19:43:14.041 INFO:tasks.workunit.client.0.smithi090.stdout:8/751: mknod d0/d3/d86/cfb 0 2024-03-11T19:43:14.271 INFO:tasks.workunit.client.0.smithi090.stdout:7/691: write d3/dc/d1e/d7c/d96/d32/d71/fdf [2698997,129543] 0 2024-03-11T19:43:14.305 INFO:tasks.workunit.client.0.smithi090.stdout:3/643: write d2/db/f62 [359014,95608] 0 2024-03-11T19:43:14.981 INFO:tasks.workunit.client.0.smithi090.stdout:4/736: link d4/d51/f72 d4/d18/d6f/ff4 0 2024-03-11T19:43:15.051 INFO:tasks.workunit.client.0.smithi090.stdout:5/673: mkdir d1/d17/d39/de8 0 2024-03-11T19:43:15.215 INFO:tasks.workunit.client.0.smithi090.stdout:2/685: creat d0/de/d12/dc8/fd5 x:0 0 0 2024-03-11T19:43:15.546 INFO:tasks.workunit.client.0.smithi090.stdout:1/689: dread - d5/d65/d6b/d7f/fab zero size 2024-03-11T19:43:15.633 INFO:tasks.workunit.client.0.smithi090.stdout:8/752: symlink d0/d43/lfc 0 2024-03-11T19:43:16.137 INFO:tasks.workunit.client.0.smithi090.stdout:4/737: chown d4/d27/l6c 40711863 1 2024-03-11T19:43:17.001 INFO:tasks.workunit.client.0.smithi090.stdout:0/707: write d2/d11/d3a/d64/fc6 [454944,16373] 0 2024-03-11T19:43:17.050 INFO:tasks.workunit.client.0.smithi090.stdout:0/708: chown d2/d11/d15/d29/d2e/c95 25345 1 2024-03-11T19:43:17.080 INFO:tasks.workunit.client.0.smithi090.stdout:8/753: creat d0/d53/d59/ffd x:0 0 0 2024-03-11T19:43:17.263 INFO:tasks.workunit.client.0.smithi090.stdout:4/738: mknod d4/d51/d8c/d6a/d8d/cf5 0 2024-03-11T19:43:17.472 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:43:17.475 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:43:17.529 INFO:tasks.workunit.client.0.smithi090.stdout:2/686: creat d0/de/d12/d46/d75/dc5/fd6 x:0 0 0 2024-03-11T19:43:17.584 INFO:tasks.workunit.client.0.smithi090.stdout:4/739: dread d4/d18/f41 [0,4194304] 0 2024-03-11T19:43:17.692 INFO:tasks.workunit.client.0.smithi090.stdout:0/709: creat d2/d11/d15/d29/d2e/db4/daf/fdb x:0 0 0 2024-03-11T19:43:17.702 INFO:tasks.workunit.client.0.smithi090.stdout:8/754: unlink d0/d3/d98/da2/dcc/ldb 0 2024-03-11T19:43:17.741 INFO:tasks.workunit.client.0.smithi090.stdout:9/676: getdents d1/dd/df/d1c 0 2024-03-11T19:43:18.298 INFO:tasks.workunit.client.0.smithi090.stdout:8/755: creat d0/d3/d86/d93/dba/ffe x:0 0 0 2024-03-11T19:43:18.977 INFO:tasks.workunit.client.0.smithi090.stdout:9/677: dread d1/dd/df/d1c/d21/d2a/f50 [0,4194304] 0 2024-03-11T19:43:18.983 INFO:tasks.workunit.client.0.smithi090.stdout:6/742: dwrite d4/d14/d25/f41 [0,4194304] 0 2024-03-11T19:43:19.024 INFO:tasks.workunit.client.0.smithi090.stdout:7/692: write d3/dc/f30 [3857532,48076] 0 2024-03-11T19:43:19.188 INFO:tasks.workunit.client.0.smithi090.stdout:9/678: dread - d1/dd/d18/d3a/dd8/fda zero size 2024-03-11T19:43:19.378 INFO:tasks.workunit.client.0.smithi090.stdout:3/644: dwrite d2/f5f [0,4194304] 0 2024-03-11T19:43:19.527 INFO:tasks.workunit.client.0.smithi090.stdout:5/674: write d1/d17/d39/d51/d53/d7b/f5b [2048957,38307] 0 2024-03-11T19:43:19.953 INFO:tasks.workunit.client.0.smithi090.stdout:8/756: mknod d0/d43/cff 0 2024-03-11T19:43:20.133 INFO:tasks.workunit.client.0.smithi090.stdout:0/710: write d2/d11/d15/d29/fbc [1102348,40896] 0 2024-03-11T19:43:21.753 INFO:tasks.workunit.client.0.smithi090.stdout:1/690: rename d5/f4f to d5/d31/d86/fea 0 2024-03-11T19:43:22.221 INFO:tasks.workunit.client.0.smithi090.stdout:0/711: getdents d2/d11/d15/dcc 0 2024-03-11T19:43:22.277 INFO:tasks.workunit.client.0.smithi090.stdout:4/740: dread d4/ff [4194304,4194304] 0 2024-03-11T19:43:22.808 INFO:tasks.workunit.client.0.smithi090.stdout:0/712: mknod d2/d11/d15/d36/cdc 0 2024-03-11T19:43:22.898 INFO:tasks.workunit.client.0.smithi090.stdout:4/741: mknod d4/d18/d6f/db7/cf6 0 2024-03-11T19:43:22.944 INFO:tasks.workunit.client.0.smithi090.stdout:5/675: write d1/d26/d19/f21 [2739777,42252] 0 2024-03-11T19:43:22.986 INFO:tasks.workunit.client.0.smithi090.stdout:9/679: link d1/dd/d18/c2e d1/dd/cdb 0 2024-03-11T19:43:23.043 INFO:tasks.workunit.client.0.smithi090.stdout:6/743: dwrite d4/d14/d8a/d74/fba [4194304,4194304] 0 2024-03-11T19:43:23.279 INFO:tasks.workunit.client.0.smithi090.stdout:7/693: dwrite d3/dc/d1e/d7c/d96/d32/d45/de0/f18 [0,4194304] 0 2024-03-11T19:43:23.404 INFO:tasks.workunit.client.0.smithi090.stdout:1/691: creat d5/d65/d6b/dcf/feb x:0 0 0 2024-03-11T19:43:23.473 INFO:tasks.workunit.client.0.smithi090.stdout:8/757: write d0/d3/d4c/f69 [801951,87225] 0 2024-03-11T19:43:23.585 INFO:tasks.workunit.client.0.smithi090.stdout:7/694: chown d3/c16 13 1 2024-03-11T19:43:23.669 INFO:tasks.workunit.client.0.smithi090.stdout:0/713: dread d2/d11/d15/d29/fbc [0,4194304] 0 2024-03-11T19:43:24.231 INFO:tasks.workunit.client.0.smithi090.stdout:6/744: truncate d4/f11 4082429 0 2024-03-11T19:43:24.232 INFO:tasks.workunit.client.0.smithi090.stdout:4/742: creat d4/d51/ff7 x:0 0 0 2024-03-11T19:43:24.293 INFO:tasks.workunit.client.0.smithi090.stdout:1/692: chown d5/d65/d6b/lc2 1808661 1 2024-03-11T19:43:24.320 INFO:tasks.workunit.client.0.smithi090.stdout:9/680: truncate d1/d47/f90 564135 0 2024-03-11T19:43:24.492 INFO:tasks.workunit.client.0.smithi090.stdout:5/676: creat d1/d17/d39/d51/d53/d7b/d3b/d76/db7/fe9 x:0 0 0 2024-03-11T19:43:24.819 INFO:tasks.workunit.client.0.smithi090.stdout:8/758: creat d0/d53/d59/d82/f100 x:0 0 0 2024-03-11T19:43:24.837 INFO:tasks.workunit.client.0.smithi090.stdout:0/714: mknod d2/d11/d3a/cdd 0 2024-03-11T19:43:24.845 INFO:tasks.workunit.client.0.smithi090.stdout:3/645: dwrite d2/db/d15/d1a/d75/fc8 [0,4194304] 0 2024-03-11T19:43:25.040 INFO:tasks.workunit.client.0.smithi090.stdout:7/695: fsync d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/fc0 0 2024-03-11T19:43:25.133 INFO:tasks.workunit.client.0.smithi090.stdout:9/681: dread d1/d1d/f67 [4194304,4194304] 0 2024-03-11T19:43:25.194 INFO:tasks.workunit.client.0.smithi090.stdout:5/677: sync 2024-03-11T19:43:25.262 INFO:tasks.workunit.client.0.smithi090.stdout:5/678: chown d1/d17/f66 11800468 1 2024-03-11T19:43:25.313 INFO:tasks.workunit.client.0.smithi090.stdout:5/679: stat d1/d17/d39/d51/d53/d7b/d3b/d76/f77 0 2024-03-11T19:43:25.632 INFO:tasks.workunit.client.0.smithi090.stdout:6/745: creat d4/d9b/ff7 x:0 0 0 2024-03-11T19:43:26.056 INFO:tasks.workunit.client.0.smithi090.stdout:3/646: write d2/d45/faa [2901488,96930] 0 2024-03-11T19:43:26.480 INFO:tasks.workunit.client.0.smithi090.stdout:9/682: chown d1/dd/d18/c2e 142 1 2024-03-11T19:43:26.597 INFO:tasks.workunit.client.0.smithi090.stdout:2/687: rename d0/f66 to d0/de/d12/d46/d75/dbb/fd7 0 2024-03-11T19:43:27.140 INFO:tasks.workunit.client.0.smithi090.stdout:5/680: creat d1/d17/d39/d51/d53/d7b/d3b/d76/db7/fea x:0 0 0 2024-03-11T19:43:27.494 INFO:tasks.workunit.client.0.smithi090.stdout:5/681: dread d1/d26/d19/f21 [0,4194304] 0 2024-03-11T19:43:27.573 INFO:tasks.workunit.client.0.smithi090.stdout:5/682: write d1/d17/d39/d51/d53/d7b/d3b/d76/db7/fe9 [522786,70508] 0 2024-03-11T19:43:28.071 INFO:tasks.workunit.client.0.smithi090.stdout:9/683: mknod d1/dd/df/d1c/d21/d5c/cdc 0 2024-03-11T19:43:29.115 INFO:tasks.workunit.client.0.smithi090.stdout:4/743: link d4/dd8/f7e d4/dd8/ff8 0 2024-03-11T19:43:29.217 INFO:tasks.workunit.client.0.smithi090.stdout:1/693: dwrite d5/d65/fa6 [0,4194304] 0 2024-03-11T19:43:29.351 INFO:tasks.workunit.client.0.smithi090.stdout:8/759: rename d0/d43/c77 to d0/d43/dc/da1/c101 0 2024-03-11T19:43:29.670 INFO:tasks.workunit.client.0.smithi090.stdout:0/715: dwrite d2/fd4 [0,4194304] 0 2024-03-11T19:43:29.870 INFO:tasks.workunit.client.0.smithi090.stdout:5/683: mknod d1/d17/d39/d51/d53/d7b/d6f/dd2/ceb 0 2024-03-11T19:43:29.908 INFO:tasks.workunit.client.0.smithi090.stdout:7/696: write d3/f3b [3397432,2758] 0 2024-03-11T19:43:29.910 INFO:tasks.workunit.client.0.smithi090.stdout:9/684: mkdir d1/dd/d18/ddd 0 2024-03-11T19:43:30.181 INFO:tasks.workunit.client.0.smithi090.stdout:4/744: unlink d4/d27/d28/c33 0 2024-03-11T19:43:30.691 INFO:tasks.workunit.client.0.smithi090.stdout:8/760: creat d0/d3/d86/d93/dda/f102 x:0 0 0 2024-03-11T19:43:30.817 INFO:tasks.workunit.client.0.smithi090.stdout:0/716: truncate d2/d11/d15/d29/fbd 3641132 0 2024-03-11T19:43:31.043 INFO:tasks.workunit.client.0.smithi090.stdout:0/717: dread d2/d11/d15/f19 [0,4194304] 0 2024-03-11T19:43:31.166 INFO:tasks.workunit.client.0.smithi090.stdout:5/684: symlink d1/d17/d39/d51/d53/d7b/d3b/d76/d92/lec 0 2024-03-11T19:43:31.391 INFO:tasks.workunit.client.0.smithi090.stdout:7/697: fsync d3/dc/d1e/d7c/d96/d32/d45/de0/fb9 0 2024-03-11T19:43:31.820 INFO:tasks.workunit.client.0.smithi090.stdout:1/694: read - d5/da/d1c/d3b/d40/fda zero size 2024-03-11T19:43:32.056 INFO:tasks.workunit.client.0.smithi090.stdout:6/746: write d4/d13/d22/d34/d40/da4/f9a [1946267,103315] 0 2024-03-11T19:43:32.298 INFO:tasks.workunit.client.0.smithi090.stdout:9/685: unlink d1/dd/d18/d3a/d48/f9a 0 2024-03-11T19:43:32.331 INFO:tasks.workunit.client.0.smithi090.stdout:5/685: readlink d1/l15 0 2024-03-11T19:43:32.467 INFO:tasks.workunit.client.0.smithi090.stdout:3/647: write d2/db/d15/d1a/d75/dc6/d47/d6a/dcb/f78 [58816,3808] 0 2024-03-11T19:43:33.252 INFO:tasks.workunit.client.0.smithi090.stdout:6/747: rmdir d4/d13/d22/d42/d48/d4e 39 2024-03-11T19:43:33.383 INFO:tasks.workunit.client.0.smithi090.stdout:0/718: mkdir d2/d11/d15/d29/dde 0 2024-03-11T19:43:33.665 INFO:tasks.workunit.client.0.smithi090.stdout:9/686: creat d1/dd/d18/d32/fde x:0 0 0 2024-03-11T19:43:33.671 INFO:tasks.workunit.client.0.smithi090.stdout:5/686: fdatasync d1/d17/d39/d51/d53/d7b/d3b/d76/d92/fa2 0 2024-03-11T19:43:33.788 INFO:tasks.workunit.client.0.smithi090.stdout:3/648: symlink d2/d4f/d59/da1/lcc 0 2024-03-11T19:43:33.893 INFO:tasks.workunit.client.0.smithi090.stdout:4/745: link d4/l13 d4/d51/d8c/d6a/lf9 0 2024-03-11T19:43:34.070 INFO:tasks.workunit.client.0.smithi090.stdout:2/688: dwrite d0/de/db6/f22 [0,4194304] 0 2024-03-11T19:43:34.220 INFO:tasks.workunit.client.0.smithi090.stdout:7/698: creat d3/dc/da6/deb/ff0 x:0 0 0 2024-03-11T19:43:34.294 INFO:tasks.workunit.client.0.smithi090.stdout:2/689: chown d0/de/d12/d46/cbf 3427426 1 2024-03-11T19:43:34.950 INFO:tasks.workunit.client.0.smithi090.stdout:4/746: rename d4/d18/d6f to d4/dd8/dfa 0 2024-03-11T19:43:35.008 INFO:tasks.workunit.client.0.smithi090.stdout:6/748: symlink d4/d13/d22/d42/dee/lf8 0 2024-03-11T19:43:35.801 INFO:tasks.workunit.client.0.smithi090.stdout:5/687: mkdir d1/d17/d39/d51/d53/d7b/d3b/d76/db7/ddf/ded 0 2024-03-11T19:43:36.547 INFO:tasks.workunit.client.0.smithi090.stdout:8/761: dwrite d0/d3/d4c/f6f [0,4194304] 0 2024-03-11T19:43:36.719 INFO:tasks.workunit.client.0.smithi090.stdout:3/649: link d2/db/d15/d49/l82 d2/d2f/lcd 0 2024-03-11T19:43:37.091 INFO:tasks.workunit.client.0.smithi090.stdout:5/688: dread d1/d17/f41 [0,4194304] 0 2024-03-11T19:43:37.219 INFO:tasks.workunit.client.0.smithi090.stdout:6/749: truncate d4/d13/d22/d3a/d67/f9d 502988 0 2024-03-11T19:43:37.378 INFO:tasks.workunit.client.0.smithi090.stdout:1/695: write d5/da/d1c/d3b/fcd [3847699,73445] 0 2024-03-11T19:43:37.768 INFO:tasks.workunit.client.0.smithi090.stdout:9/687: getdents d1/d1d 0 2024-03-11T19:43:37.857 INFO:tasks.workunit.client.0.smithi090.stdout:3/650: fsync d2/d2f/f30 0 2024-03-11T19:43:38.221 INFO:tasks.workunit.client.0.smithi090.stdout:4/747: rename d4/dd8/d66/dcf/dd7/dd3/f84 to d4/d18/ffb 0 2024-03-11T19:43:38.728 INFO:tasks.workunit.client.0.smithi090.stdout:6/750: dread d4/d14/d8a/d74/fba [0,4194304] 0 2024-03-11T19:43:39.330 INFO:tasks.workunit.client.0.smithi090.stdout:4/748: symlink d4/d51/d8c/d6a/d8d/lfc 0 2024-03-11T19:43:39.350 INFO:tasks.workunit.client.0.smithi090.stdout:4/749: stat d4/d18/fab 0 2024-03-11T19:43:39.378 INFO:tasks.workunit.client.0.smithi090.stdout:4/750: stat d4/dd8/d66/de5 0 2024-03-11T19:43:39.604 INFO:tasks.workunit.client.0.smithi090.stdout:1/696: read d5/d4e/d79/f98 [55624,24263] 0 2024-03-11T19:43:39.713 INFO:tasks.workunit.client.0.smithi090.stdout:6/751: rmdir d4/d13/d22/d42/dee 39 2024-03-11T19:43:39.731 INFO:tasks.workunit.client.0.smithi090.stdout:9/688: creat d1/dd/d18/ddd/fdf x:0 0 0 2024-03-11T19:43:39.734 INFO:tasks.workunit.client.0.smithi090.stdout:4/751: sync 2024-03-11T19:43:40.114 INFO:tasks.workunit.client.0.smithi090.stdout:0/719: dwrite d2/d11/d15/d29/d3d/d7b/fd7 [0,4194304] 0 2024-03-11T19:43:40.368 INFO:tasks.workunit.client.0.smithi090.stdout:2/690: write d0/de/d12/d46/d75/dbb/fd7 [5382620,16677] 0 2024-03-11T19:43:40.436 INFO:tasks.workunit.client.0.smithi090.stdout:1/697: dread - d5/d4e/d79/fc3 zero size 2024-03-11T19:43:40.547 INFO:tasks.workunit.client.0.smithi090.stdout:8/762: getdents d0/d43 0 2024-03-11T19:43:40.589 INFO:tasks.workunit.client.0.smithi090.stdout:9/689: dread - d1/dd/d18/d3a/fd4 zero size 2024-03-11T19:43:40.889 INFO:tasks.workunit.client.0.smithi090.stdout:4/752: rename d4/d27/d28/d38/f91 to d4/d51/dbf/ffd 0 2024-03-11T19:43:40.909 INFO:tasks.workunit.client.0.smithi090.stdout:4/753: chown d4 852862836 1 2024-03-11T19:43:41.026 INFO:tasks.workunit.client.0.smithi090.stdout:0/720: creat d2/d11/d6e/fdf x:0 0 0 2024-03-11T19:43:41.067 INFO:tasks.workunit.client.0.smithi090.stdout:4/754: sync 2024-03-11T19:43:41.466 INFO:tasks.workunit.client.0.smithi090.stdout:9/690: rename d1/dd/df/d1c/f81 to d1/d1d/d6a/fe0 0 2024-03-11T19:43:41.492 INFO:tasks.workunit.client.0.smithi090.stdout:9/691: readlink d1/d49/d9c/daf/ld2 0 2024-03-11T19:43:41.603 INFO:tasks.workunit.client.0.smithi090.stdout:0/721: creat d2/d11/d3f/fe0 x:0 0 0 2024-03-11T19:43:41.719 INFO:tasks.workunit.client.0.smithi090.stdout:2/691: chown d0/de/db6/d29/c7e 1053 1 2024-03-11T19:43:42.038 INFO:tasks.workunit.client.0.smithi090.stdout:4/755: creat d4/dd8/d66/dcf/dd7/dd3/ffe x:0 0 0 2024-03-11T19:43:42.204 INFO:tasks.workunit.client.0.smithi090.stdout:1/698: unlink d5/d4e/d79/le1 0 2024-03-11T19:43:42.291 INFO:tasks.workunit.client.0.smithi090.stdout:7/699: unlink d3/f22 0 2024-03-11T19:43:42.388 INFO:tasks.workunit.client.0.smithi090.stdout:9/692: fsync d1/dd/f69 0 2024-03-11T19:43:42.421 INFO:tasks.workunit.client.0.smithi090.stdout:9/693: chown d1/d1d/ld5 3 1 2024-03-11T19:43:42.753 INFO:tasks.workunit.client.0.smithi090.stdout:5/689: write d1/d17/f41 [2264396,124671] 0 2024-03-11T19:43:42.884 INFO:tasks.workunit.client.0.smithi090.stdout:5/690: sync 2024-03-11T19:43:43.556 INFO:tasks.workunit.client.0.smithi090.stdout:7/700: dread d3/dc/d1e/d7c/d96/d32/d45/de0/f21 [0,4194304] 0 2024-03-11T19:43:43.588 INFO:tasks.workunit.client.0.smithi090.stdout:9/694: dread d1/dd/d18/d3a/d48/f9b [0,4194304] 0 2024-03-11T19:43:43.730 INFO:tasks.workunit.client.0.smithi090.stdout:2/692: rename d0/de/d12/d46/d75/dc5/fd6 to d0/de/db6/d25/d8c/fd8 0 2024-03-11T19:43:43.846 INFO:tasks.workunit.client.0.smithi090.stdout:3/651: dwrite d2/d45/faa [4194304,4194304] 0 2024-03-11T19:43:43.964 INFO:tasks.workunit.client.0.smithi090.stdout:1/699: unlink d5/d4e/c83 0 2024-03-11T19:43:44.154 INFO:tasks.workunit.client.0.smithi090.stdout:2/693: read d0/de/d12/d81/f87 [1163863,52185] 0 2024-03-11T19:43:44.274 INFO:tasks.workunit.client.0.smithi090.stdout:5/691: dread d1/d26/dc8/fde [0,4194304] 0 2024-03-11T19:43:44.293 INFO:tasks.workunit.client.0.smithi090.stdout:2/694: sync 2024-03-11T19:43:44.467 INFO:tasks.workunit.client.0.smithi090.stdout:3/652: symlink d2/db/d15/d69/da2/lce 0 2024-03-11T19:43:44.543 INFO:tasks.workunit.client.0.smithi090.stdout:3/653: sync 2024-03-11T19:43:44.734 INFO:tasks.workunit.client.0.smithi090.stdout:9/695: mkdir d1/dd/d18/d3a/db6/de1 0 2024-03-11T19:43:44.758 INFO:tasks.workunit.client.0.smithi090.stdout:7/701: creat d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/db5/de1/dc2/dd0/ff1 x:0 0 0 2024-03-11T19:43:44.794 INFO:tasks.workunit.client.0.smithi090.stdout:9/696: stat d1/dd/df/l4e 0 2024-03-11T19:43:44.812 INFO:tasks.workunit.client.0.smithi090.stdout:7/702: chown d3/dc/d1e/l39 779826 1 2024-03-11T19:43:44.850 INFO:tasks.workunit.client.0.smithi090.stdout:9/697: chown d1/dd/f80 161 1 2024-03-11T19:43:45.128 INFO:tasks.workunit.client.0.smithi090.stdout:5/692: mkdir d1/d17/d39/d51/d53/d7b/d3b/d76/db7/ddf/dee 0 2024-03-11T19:43:45.198 INFO:tasks.workunit.client.0.smithi090.stdout:6/752: dwrite d4/d13/d22/d3a/f65 [4194304,4194304] 0 2024-03-11T19:43:45.515 INFO:tasks.workunit.client.0.smithi090.stdout:9/698: readlink d1/da9/ld3 0 2024-03-11T19:43:45.584 INFO:tasks.workunit.client.0.smithi090.stdout:7/703: dread d3/f3b [0,4194304] 0 2024-03-11T19:43:46.157 INFO:tasks.workunit.client.0.smithi090.stdout:6/753: mknod d4/d9b/cf9 0 2024-03-11T19:43:46.183 INFO:tasks.workunit.client.0.smithi090.stdout:9/699: mknod d1/d49/d9c/ce2 0 2024-03-11T19:43:46.483 INFO:tasks.workunit.client.0.smithi090.stdout:7/704: unlink d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/fc0 0 2024-03-11T19:43:46.580 INFO:tasks.workunit.client.0.smithi090.stdout:2/695: getdents d0 0 2024-03-11T19:43:46.913 INFO:tasks.workunit.client.0.smithi090.stdout:8/763: write d0/d3/f80 [1074872,119140] 0 2024-03-11T19:43:47.264 INFO:tasks.workunit.client.0.smithi090.stdout:6/754: unlink d4/d13/fb6 0 2024-03-11T19:43:47.277 INFO:tasks.workunit.client.0.smithi090.stdout:4/756: write d4/dd8/f94 [584127,21202] 0 2024-03-11T19:43:47.583 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:43:47.586 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:43:47.631 INFO:tasks.workunit.client.0.smithi090.stdout:9/700: creat d1/dd/d18/d32/fe3 x:0 0 0 2024-03-11T19:43:47.654 INFO:tasks.workunit.client.0.smithi090.stdout:2/696: fsync d0/d21/f6e 0 2024-03-11T19:43:47.656 INFO:tasks.workunit.client.0.smithi090.stdout:7/705: mknod d3/dc/d87/cf2 0 2024-03-11T19:43:47.714 INFO:tasks.workunit.client.0.smithi090.stdout:9/701: stat d1/dd/d18/d25/ld1 0 2024-03-11T19:43:48.441 INFO:tasks.workunit.client.0.smithi090.stdout:6/755: mkdir d4/d13/d22/d42/d48/dd2/dfa 0 2024-03-11T19:43:48.534 INFO:tasks.workunit.client.0.smithi090.stdout:1/700: write d5/d65/fa6 [3999441,67299] 0 2024-03-11T19:43:49.608 INFO:tasks.workunit.client.0.smithi090.stdout:3/654: write d2/db/f62 [5949045,115482] 0 2024-03-11T19:43:50.262 INFO:tasks.workunit.client.0.smithi090.stdout:3/655: creat d2/de/fcf x:0 0 0 2024-03-11T19:43:50.511 INFO:tasks.workunit.client.0.smithi090.stdout:5/693: dwrite d1/d17/d39/d51/d53/d7b/f5b [0,4194304] 0 2024-03-11T19:43:50.688 INFO:tasks.workunit.client.0.smithi090.stdout:8/764: getdents d0/d3/d4c/d8d/df8 0 2024-03-11T19:43:50.882 INFO:tasks.workunit.client.0.smithi090.stdout:1/701: creat d5/da/d1c/d6d/fec x:0 0 0 2024-03-11T19:43:51.355 INFO:tasks.workunit.client.0.smithi090.stdout:5/694: truncate d1/d17/f79 2546680 0 2024-03-11T19:43:51.406 INFO:tasks.workunit.client.0.smithi090.stdout:0/722: creat d2/d11/d15/fe1 x:0 0 0 2024-03-11T19:43:51.492 INFO:tasks.workunit.client.0.smithi090.stdout:9/702: write d1/dd/d18/d3a/f86 [1974220,42553] 0 2024-03-11T19:43:51.742 INFO:tasks.workunit.client.0.smithi090.stdout:0/723: chown d2/d11/d3a/f42 14 1 2024-03-11T19:43:51.875 INFO:tasks.workunit.client.0.smithi090.stdout:4/757: dwrite d4/ff3 [0,4194304] 0 2024-03-11T19:43:52.016 INFO:tasks.workunit.client.0.smithi090.stdout:1/702: fdatasync d5/da/d1c/d3b/fd0 0 2024-03-11T19:43:52.078 INFO:tasks.workunit.client.0.smithi090.stdout:3/656: creat d2/d45/fd0 x:0 0 0 2024-03-11T19:43:52.342 INFO:tasks.workunit.client.0.smithi090.stdout:5/695: dread - d1/d17/d39/d51/d53/d7b/da0/dc0/fcc zero size 2024-03-11T19:43:52.812 INFO:tasks.workunit.client.0.smithi090.stdout:7/706: dread - d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/d9c/fdb zero size 2024-03-11T19:43:52.948 INFO:tasks.workunit.client.0.smithi090.stdout:4/758: fdatasync d4/dd8/d66/fce 0 2024-03-11T19:43:53.379 INFO:tasks.workunit.client.0.smithi090.stdout:1/703: fsync d5/da/d1c/fdb 0 2024-03-11T19:43:53.798 INFO:tasks.workunit.client.0.smithi090.stdout:0/724: unlink d2/d11/d3a/d64/dac/cc4 0 2024-03-11T19:43:53.808 INFO:tasks.workunit.client.0.smithi090.stdout:8/765: getdents d0 0 2024-03-11T19:43:53.908 INFO:tasks.workunit.client.0.smithi090.stdout:6/756: write d4/d13/d22/d42/d48/d4e/fae [699386,71648] 0 2024-03-11T19:43:54.519 INFO:tasks.workunit.client.0.smithi090.stdout:5/696: symlink d1/d17/d39/d51/lef 0 2024-03-11T19:43:54.725 INFO:tasks.workunit.client.0.smithi090.stdout:4/759: dread d4/f4a [0,4194304] 0 2024-03-11T19:43:54.732 INFO:tasks.workunit.client.0.smithi090.stdout:1/704: symlink d5/d4e/d79/led 0 2024-03-11T19:43:54.926 INFO:tasks.workunit.client.0.smithi090.stdout:6/757: readlink d4/d13/d22/lb9 0 2024-03-11T19:43:55.071 INFO:tasks.workunit.client.0.smithi090.stdout:0/725: truncate d2/d11/d15/d29/d43/d58/fc7 1772322 0 2024-03-11T19:43:55.209 INFO:tasks.workunit.client.0.smithi090.stdout:9/703: rename d1/d1d/d6a/f8c to d1/dd/df/d1c/d9f/fe4 0 2024-03-11T19:43:55.712 INFO:tasks.workunit.client.0.smithi090.stdout:0/726: creat d2/d11/d3a/d64/fe2 x:0 0 0 2024-03-11T19:43:55.723 INFO:tasks.workunit.client.0.smithi090.stdout:6/758: read d4/d13/d22/d34/d40/da4/fd8 [3669986,26585] 0 2024-03-11T19:43:55.795 INFO:tasks.workunit.client.0.smithi090.stdout:0/727: dread - d2/d11/d15/d36/f7c zero size 2024-03-11T19:43:56.456 INFO:tasks.workunit.client.0.smithi090.stdout:2/697: unlink d0/f5f 0 2024-03-11T19:43:56.507 INFO:tasks.workunit.client.0.smithi090.stdout:1/705: symlink d5/da/d1c/d3b/lee 0 2024-03-11T19:43:56.543 INFO:tasks.workunit.client.0.smithi090.stdout:3/657: write d2/db/d15/d1a/d75/dc6/d47/d6a/fa0 [386382,94740] 0 2024-03-11T19:43:56.577 INFO:tasks.workunit.client.0.smithi090.stdout:9/704: dread - d1/dd/df/d1c/fad zero size 2024-03-11T19:43:56.734 INFO:tasks.workunit.client.0.smithi090.stdout:0/728: mkdir d2/d11/d3a/d64/dd5/de3 0 2024-03-11T19:43:56.779 INFO:tasks.workunit.client.0.smithi090.stdout:3/658: write d2/f2c [7767228,102043] 0 2024-03-11T19:43:57.087 INFO:tasks.workunit.client.0.smithi090.stdout:7/707: dwrite d3/d55/fb7 [0,4194304] 0 2024-03-11T19:43:57.196 INFO:tasks.workunit.client.0.smithi090.stdout:4/760: symlink d4/d27/d28/d38/d64/lff 0 2024-03-11T19:43:57.843 INFO:tasks.workunit.client.0.smithi090.stdout:8/766: mkdir d0/d103 0 2024-03-11T19:43:57.863 INFO:tasks.workunit.client.0.smithi090.stdout:9/705: dread d1/d1d/d6a/fe0 [0,4194304] 0 2024-03-11T19:43:57.898 INFO:tasks.workunit.client.0.smithi090.stdout:9/706: chown d1/f58 88 1 2024-03-11T19:43:58.361 INFO:tasks.workunit.client.0.smithi090.stdout:1/706: fdatasync d5/da/d1c/d6d/d80/fb9 0 2024-03-11T19:43:58.606 INFO:tasks.workunit.client.0.smithi090.stdout:5/697: dwrite d1/d17/d39/d51/d53/d7b/d3b/fe1 [4194304,4194304] 0 2024-03-11T19:43:58.856 INFO:tasks.workunit.client.0.smithi090.stdout:7/708: rename d3/c16 to d3/dc/d29/cf3 0 2024-03-11T19:43:58.948 INFO:tasks.workunit.client.0.smithi090.stdout:4/761: fsync d4/d51/d8c/d89/fc7 0 2024-03-11T19:43:59.141 INFO:tasks.workunit.client.0.smithi090.stdout:6/759: dwrite d4/d13/d22/d42/d48/fc5 [0,4194304] 0 2024-03-11T19:43:59.823 INFO:tasks.workunit.client.0.smithi090.stdout:1/707: symlink d5/d4e/d54/d94/lef 0 2024-03-11T19:44:00.174 INFO:tasks.workunit.client.0.smithi090.stdout:5/698: dread d1/d17/d39/d51/d53/d7b/d3b/f5a [0,4194304] 0 2024-03-11T19:44:00.269 INFO:tasks.workunit.client.0.smithi090.stdout:1/708: sync 2024-03-11T19:44:00.400 INFO:tasks.workunit.client.0.smithi090.stdout:9/707: fdatasync d1/dd/df/d1c/d21/d2a/f50 0 2024-03-11T19:44:00.415 INFO:tasks.workunit.client.0.smithi090.stdout:4/762: dread d4/ff3 [0,4194304] 0 2024-03-11T19:44:00.517 INFO:tasks.workunit.client.0.smithi090.stdout:2/698: dread - d0/de/d12/d46/d5d/fcd zero size 2024-03-11T19:44:00.624 INFO:tasks.workunit.client.0.smithi090.stdout:4/763: dread d4/d18/f41 [8388608,4194304] 0 2024-03-11T19:44:00.871 INFO:tasks.workunit.client.0.smithi090.stdout:3/659: dwrite d2/db/d15/d69/dad/fbf [0,4194304] 0 2024-03-11T19:44:01.133 INFO:tasks.workunit.client.0.smithi090.stdout:8/767: chown d0/d3/d98/da2/lc5 20534976 1 2024-03-11T19:44:01.312 INFO:tasks.workunit.client.0.smithi090.stdout:2/699: creat d0/de/d12/d46/d5d/d63/fd9 x:0 0 0 2024-03-11T19:44:01.458 INFO:tasks.workunit.client.0.smithi090.stdout:0/729: getdents d2/d11/d15/d29/d3d/d7b 0 2024-03-11T19:44:01.466 INFO:tasks.workunit.client.0.smithi090.stdout:4/764: mkdir d4/d51/d8c/d6a/d100 0 2024-03-11T19:44:01.610 INFO:tasks.workunit.client.0.smithi090.stdout:5/699: dread d1/d26/f43 [0,4194304] 0 2024-03-11T19:44:01.784 INFO:tasks.workunit.client.0.smithi090.stdout:3/660: dread d2/d2f/f31 [0,4194304] 0 2024-03-11T19:44:01.935 INFO:tasks.workunit.client.0.smithi090.stdout:5/700: read d1/d17/d39/d51/d53/d7b/f37 [2149450,4917] 0 2024-03-11T19:44:01.965 INFO:tasks.workunit.client.0.smithi090.stdout:2/700: read d0/f18 [4286588,131025] 0 2024-03-11T19:44:01.991 INFO:tasks.workunit.client.0.smithi090.stdout:7/709: link d3/dc/d1e/d7c/d96/lef d3/dc/d1e/d7c/d96/d32/lf4 0 2024-03-11T19:44:02.228 INFO:tasks.workunit.client.0.smithi090.stdout:0/730: creat d2/d11/d15/d36/fe4 x:0 0 0 2024-03-11T19:44:02.410 INFO:tasks.workunit.client.0.smithi090.stdout:8/768: truncate d0/d3/d86/d93/dda/ff6 33501 0 2024-03-11T19:44:02.417 INFO:tasks.workunit.client.0.smithi090.stdout:4/765: unlink d4/d18/c88 0 2024-03-11T19:44:02.483 INFO:tasks.workunit.client.0.smithi090.stdout:4/766: read d4/dd8/fa4 [3850835,100579] 0 2024-03-11T19:44:02.695 INFO:tasks.workunit.client.0.smithi090.stdout:3/661: chown d2/db/d15/d1a/d3c/c79 601 1 2024-03-11T19:44:02.759 INFO:tasks.workunit.client.0.smithi090.stdout:3/662: dread d2/d45/f4a [0,4194304] 0 2024-03-11T19:44:02.820 INFO:tasks.workunit.client.0.smithi090.stdout:3/663: dread d2/d45/f4a [0,4194304] 0 2024-03-11T19:44:03.178 INFO:tasks.workunit.client.0.smithi090.stdout:6/760: dwrite d4/d13/d22/d34/d40/da4/fe7 [0,4194304] 0 2024-03-11T19:44:03.321 INFO:tasks.workunit.client.0.smithi090.stdout:9/708: link d1/dd/df/d1c/d77/fc4 d1/dd/d18/d3a/d70/fe5 0 2024-03-11T19:44:03.333 INFO:tasks.workunit.client.0.smithi090.stdout:1/709: write d5/d65/d6b/fa4 [172041,118659] 0 2024-03-11T19:44:03.351 INFO:tasks.workunit.client.0.smithi090.stdout:0/731: fsync d2/f14 0 2024-03-11T19:44:03.799 INFO:tasks.workunit.client.0.smithi090.stdout:3/664: unlink d2/de/f58 0 2024-03-11T19:44:03.809 INFO:tasks.workunit.client.0.smithi090.stdout:8/769: dread - d0/d3/fdd zero size 2024-03-11T19:44:04.229 INFO:tasks.workunit.client.0.smithi090.stdout:6/761: rmdir d4/d13/d22/d42/d48 39 2024-03-11T19:44:05.081 INFO:tasks.workunit.client.0.smithi090.stdout:3/665: mknod d2/db/d15/d49/cd1 0 2024-03-11T19:44:05.349 INFO:tasks.workunit.client.0.smithi090.stdout:7/710: fdatasync d3/dc/d1e/d7c/d96/d32/d45/f9d 0 2024-03-11T19:44:05.809 INFO:tasks.workunit.client.0.smithi090.stdout:6/762: rename d4/d13/d22/d34/d83/d9c/db7/ld1 to d4/d13/d22/d34/d40/dd5/d80/dd4/lfb 0 2024-03-11T19:44:06.203 INFO:tasks.workunit.client.0.smithi090.stdout:2/701: write d0/de/d12/d46/d4c/d64/f82 [255285,13942] 0 2024-03-11T19:44:06.253 INFO:tasks.workunit.client.0.smithi090.stdout:2/702: dread - d0/de/d12/d46/fb1 zero size 2024-03-11T19:44:06.709 INFO:tasks.workunit.client.0.smithi090.stdout:5/701: link d1/d17/fe4 d1/d17/d39/d51/d53/d7b/d6f/ff0 0 2024-03-11T19:44:06.820 INFO:tasks.workunit.client.0.smithi090.stdout:6/763: mknod d4/d9b/cfc 0 2024-03-11T19:44:07.298 INFO:tasks.workunit.client.0.smithi090.stdout:2/703: dread d0/de/f52 [0,4194304] 0 2024-03-11T19:44:07.368 INFO:tasks.workunit.client.0.smithi090.stdout:0/732: write d2/d11/d6e/fdf [882926,4253] 0 2024-03-11T19:44:07.498 INFO:tasks.workunit.client.0.smithi090.stdout:1/710: write d5/da/d1c/fdb [607889,48634] 0 2024-03-11T19:44:07.951 INFO:tasks.workunit.client.0.smithi090.stdout:7/711: creat d3/dc/d1e/d7c/d96/d32/d45/ff5 x:0 0 0 2024-03-11T19:44:07.955 INFO:tasks.workunit.client.0.smithi090.stdout:9/709: write d1/dd/df/d1c/d21/f9d [263110,128199] 0 2024-03-11T19:44:08.518 INFO:tasks.workunit.client.0.smithi090.stdout:4/767: dwrite d4/f46 [0,4194304] 0 2024-03-11T19:44:08.725 INFO:tasks.workunit.client.0.smithi090.stdout:4/768: readlink d4/d27/l6c 0 2024-03-11T19:44:08.729 INFO:tasks.workunit.client.0.smithi090.stdout:1/711: fsync d5/da/f4a 0 2024-03-11T19:44:08.857 INFO:tasks.workunit.client.0.smithi090.stdout:1/712: stat d5/da/d1c/d3b/lee 0 2024-03-11T19:44:08.861 INFO:tasks.workunit.client.0.smithi090.stdout:8/770: dwrite d0/d3/d3e/d85/fcd [0,4194304] 0 2024-03-11T19:44:08.989 INFO:tasks.workunit.client.0.smithi090.stdout:4/769: sync 2024-03-11T19:44:08.992 INFO:tasks.workunit.client.0.smithi090.stdout:7/712: symlink d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/db5/de1/dd2/lf6 0 2024-03-11T19:44:09.577 INFO:tasks.workunit.client.0.smithi090.stdout:0/733: mkdir d2/d11/d15/d29/de5 0 2024-03-11T19:44:09.856 INFO:tasks.workunit.client.0.smithi090.stdout:3/666: dwrite d2/db/d15/d1a/d75/fc8 [0,4194304] 0 2024-03-11T19:44:10.024 INFO:tasks.workunit.client.0.smithi090.stdout:4/770: fsync d4/d51/d8c/d89/f98 0 2024-03-11T19:44:10.233 INFO:tasks.workunit.client.0.smithi090.stdout:6/764: write d4/d13/f3b [859934,112091] 0 2024-03-11T19:44:10.346 INFO:tasks.workunit.client.0.smithi090.stdout:0/734: mkdir d2/d11/d15/d29/d3d/d7b/de6 0 2024-03-11T19:44:10.466 INFO:tasks.workunit.client.0.smithi090.stdout:5/702: dwrite d1/d26/f20 [0,4194304] 0 2024-03-11T19:44:10.640 INFO:tasks.workunit.client.0.smithi090.stdout:1/713: rename d5/da/d1c/d6d/d80/l9f to d5/d4e/lf0 0 2024-03-11T19:44:10.804 INFO:tasks.workunit.client.0.smithi090.stdout:8/771: rmdir d0/d60/db6 39 2024-03-11T19:44:11.228 INFO:tasks.workunit.client.0.smithi090.stdout:0/735: dread f0 [4194304,4194304] 0 2024-03-11T19:44:11.237 INFO:tasks.workunit.client.0.smithi090.stdout:6/765: truncate d4/ff 2060965 0 2024-03-11T19:44:11.442 INFO:tasks.workunit.client.0.smithi090.stdout:1/714: read d5/d31/d4c/f73 [140563,57322] 0 2024-03-11T19:44:11.500 INFO:tasks.workunit.client.0.smithi090.stdout:4/771: mknod d4/dd8/dfa/c101 0 2024-03-11T19:44:11.602 INFO:tasks.workunit.client.0.smithi090.stdout:0/736: truncate d2/d11/d15/d29/d3d/d7b/fb2 4125 0 2024-03-11T19:44:11.690 INFO:tasks.workunit.client.0.smithi090.stdout:4/772: chown d4/d51/l6d 3713703 1 2024-03-11T19:44:11.853 INFO:tasks.workunit.client.0.smithi090.stdout:5/703: unlink d1/d17/l74 0 2024-03-11T19:44:11.872 INFO:tasks.workunit.client.0.smithi090.stdout:9/710: dwrite d1/dd/df/d1c/d21/dab/fca [4194304,4194304] 0 2024-03-11T19:44:12.057 INFO:tasks.workunit.client.0.smithi090.stdout:3/667: rename d2/db/d15/d1a/cb6 to d2/db/d15/d49/cd2 0 2024-03-11T19:44:12.121 INFO:tasks.workunit.client.0.smithi090.stdout:6/766: creat d4/d13/d22/d34/d83/ffd x:0 0 0 2024-03-11T19:44:12.378 INFO:tasks.workunit.client.0.smithi090.stdout:4/773: creat d4/d27/d28/d38/d64/f102 x:0 0 0 2024-03-11T19:44:12.521 INFO:tasks.workunit.client.0.smithi090.stdout:2/704: dwrite d0/f2 [4194304,4194304] 0 2024-03-11T19:44:12.536 INFO:tasks.workunit.client.0.smithi090.stdout:0/737: truncate d2/d11/d6e/f9b 3890573 0 2024-03-11T19:44:12.675 INFO:tasks.workunit.client.0.smithi090.stdout:9/711: mkdir d1/dd/d18/d3a/db6/de6 0 2024-03-11T19:44:12.686 INFO:tasks.workunit.client.0.smithi090.stdout:5/704: stat d1/d17/d39/d51/d53/d7b/d3b/fdb 0 2024-03-11T19:44:12.718 INFO:tasks.workunit.client.0.smithi090.stdout:7/713: write d3/f4 [254540,98730] 0 2024-03-11T19:44:12.764 INFO:tasks.workunit.client.0.smithi090.stdout:3/668: truncate d2/db/f92 3875812 0 2024-03-11T19:44:12.872 INFO:tasks.workunit.client.0.smithi090.stdout:5/705: sync 2024-03-11T19:44:13.197 INFO:tasks.workunit.client.0.smithi090.stdout:2/705: chown d0/de/db6/l69 1058355 1 2024-03-11T19:44:13.424 INFO:tasks.workunit.client.0.smithi090.stdout:9/712: symlink d1/dd/d18/d3a/db6/le7 0 2024-03-11T19:44:13.528 INFO:tasks.workunit.client.0.smithi090.stdout:7/714: mknod d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/cf7 0 2024-03-11T19:44:14.052 INFO:tasks.workunit.client.0.smithi090.stdout:4/774: mkdir d4/dd8/d66/de5/d103 0 2024-03-11T19:44:14.195 INFO:tasks.workunit.client.0.smithi090.stdout:0/738: symlink d2/d11/d3a/dc8/dd0/le7 0 2024-03-11T19:44:14.516 INFO:tasks.workunit.client.0.smithi090.stdout:2/706: dread d0/f7 [0,4194304] 0 2024-03-11T19:44:15.092 INFO:tasks.workunit.client.0.smithi090.stdout:1/715: write d5/d31/d36/fca [4079411,118462] 0 2024-03-11T19:44:15.360 INFO:tasks.workunit.client.0.smithi090.stdout:4/775: mkdir d4/d27/d28/d7a/d104 0 2024-03-11T19:44:15.455 INFO:tasks.workunit.client.0.smithi090.stdout:4/776: chown d4/dd8/d66/c7c 71892 1 2024-03-11T19:44:15.632 INFO:tasks.workunit.client.0.smithi090.stdout:0/739: dread d2/fcb [0,4194304] 0 2024-03-11T19:44:15.997 INFO:tasks.workunit.client.0.smithi090.stdout:2/707: mknod d0/de/d12/d46/d5d/d8d/cda 0 2024-03-11T19:44:16.423 INFO:tasks.workunit.client.0.smithi090.stdout:6/767: dwrite d4/d13/d22/d34/d40/da4/fe7 [0,4194304] 0 2024-03-11T19:44:16.464 INFO:tasks.workunit.client.0.smithi090.stdout:1/716: symlink d5/da/d1c/d6d/d80/lf1 0 2024-03-11T19:44:16.594 INFO:tasks.workunit.client.0.smithi090.stdout:8/772: dwrite d0/d53/f7a [0,4194304] 0 2024-03-11T19:44:16.710 INFO:tasks.workunit.client.0.smithi090.stdout:5/706: write d1/d17/d39/d51/d53/d7b/d3b/d76/db7/fe9 [644128,58387] 0 2024-03-11T19:44:16.802 INFO:tasks.workunit.client.0.smithi090.stdout:4/777: dread d4/d18/f90 [0,4194304] 0 2024-03-11T19:44:16.842 INFO:tasks.workunit.client.0.smithi090.stdout:7/715: creat d3/dc/d1e/ff8 x:0 0 0 2024-03-11T19:44:16.883 INFO:tasks.workunit.client.0.smithi090.stdout:3/669: link d2/d2f/c8f d2/db/d15/d1a/d75/d8c/cd3 0 2024-03-11T19:44:17.305 INFO:tasks.workunit.client.0.smithi090.stdout:8/773: creat d0/d3/d3e/f104 x:0 0 0 2024-03-11T19:44:17.870 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:44:17.873 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:44:17.903 INFO:tasks.workunit.client.0.smithi090.stdout:2/708: fsync d0/fae 0 2024-03-11T19:44:18.581 INFO:tasks.workunit.client.0.smithi090.stdout:8/774: symlink d0/d53/d59/d96/l105 0 2024-03-11T19:44:18.629 INFO:tasks.workunit.client.0.smithi090.stdout:8/775: dread d0/d3/d4c/d8d/fef [0,4194304] 0 2024-03-11T19:44:18.931 INFO:tasks.workunit.client.0.smithi090.stdout:9/713: write d1/dd/df/d1c/d21/d2a/fba [330795,89445] 0 2024-03-11T19:44:19.065 INFO:tasks.workunit.client.0.smithi090.stdout:6/768: rename d4/d13/d22/d34/d40/l5a to d4/d9b/def/lfe 0 2024-03-11T19:44:19.276 INFO:tasks.workunit.client.0.smithi090.stdout:9/714: dread d1/dd/d18/d32/f34 [0,4194304] 0 2024-03-11T19:44:19.926 INFO:tasks.workunit.client.0.smithi090.stdout:3/670: rename d2/db/d15/d1a/d3c/l93 to d2/db/d15/d1a/d75/dc6/d47/d6a/dcb/ld4 0 2024-03-11T19:44:20.174 INFO:tasks.workunit.client.0.smithi090.stdout:5/707: creat d1/d17/d39/d51/ff1 x:0 0 0 2024-03-11T19:44:20.191 INFO:tasks.workunit.client.0.smithi090.stdout:1/717: dwrite d5/da/d1c/d3b/fcd [0,4194304] 0 2024-03-11T19:44:20.326 INFO:tasks.workunit.client.0.smithi090.stdout:9/715: creat d1/dd/df/d1c/d21/d2a/fe8 x:0 0 0 2024-03-11T19:44:20.372 INFO:tasks.workunit.client.0.smithi090.stdout:2/709: mknod d0/de/d12/cdb 0 2024-03-11T19:44:20.374 INFO:tasks.workunit.client.0.smithi090.stdout:1/718: chown d5/d4e/d54 5798348 1 2024-03-11T19:44:20.668 INFO:tasks.workunit.client.0.smithi090.stdout:8/776: mknod d0/c106 0 2024-03-11T19:44:20.696 INFO:tasks.workunit.client.0.smithi090.stdout:6/769: rename d4/d13/f1f to d4/d13/d22/d34/d40/dd5/fff 0 2024-03-11T19:44:20.780 INFO:tasks.workunit.client.0.smithi090.stdout:2/710: dread d0/de/d12/d46/f5b [0,4194304] 0 2024-03-11T19:44:20.782 INFO:tasks.workunit.client.0.smithi090.stdout:3/671: mkdir d2/d2f/dd5 0 2024-03-11T19:44:20.990 INFO:tasks.workunit.client.0.smithi090.stdout:2/711: stat d0/de/db6/d25/l7b 0 2024-03-11T19:44:21.033 INFO:tasks.workunit.client.0.smithi090.stdout:4/778: dwrite d4/d27/d28/f63 [0,4194304] 0 2024-03-11T19:44:21.121 INFO:tasks.workunit.client.0.smithi090.stdout:2/712: stat d0/de/d12 0 2024-03-11T19:44:21.150 INFO:tasks.workunit.client.0.smithi090.stdout:1/719: mknod d5/da/d1c/d6d/d80/db2/cf2 0 2024-03-11T19:44:21.577 INFO:tasks.workunit.client.0.smithi090.stdout:7/716: dwrite d3/d55/f64 [4194304,4194304] 0 2024-03-11T19:44:21.684 INFO:tasks.workunit.client.0.smithi090.stdout:5/708: mkdir d1/d17/d39/df2 0 2024-03-11T19:44:22.108 INFO:tasks.workunit.client.0.smithi090.stdout:9/716: unlink d1/d47/fb5 0 2024-03-11T19:44:22.401 INFO:tasks.workunit.client.0.smithi090.stdout:4/779: mkdir d4/d18/d105 0 2024-03-11T19:44:22.418 INFO:tasks.workunit.client.0.smithi090.stdout:4/780: readlink d4/d51/l6d 0 2024-03-11T19:44:22.618 INFO:tasks.workunit.client.0.smithi090.stdout:1/720: unlink d5/da/dd/cb7 0 2024-03-11T19:44:23.078 INFO:tasks.workunit.client.0.smithi090.stdout:2/713: fsync d0/de/d12/d46/d4c/d64/fc6 0 2024-03-11T19:44:24.702 INFO:tasks.workunit.client.0.smithi090.stdout:7/717: creat d3/dc/d1e/d7c/d96/d32/d71/d80/ff9 x:0 0 0 2024-03-11T19:44:24.949 INFO:tasks.workunit.client.0.smithi090.stdout:1/721: creat d5/da/d97/dd1/ff3 x:0 0 0 2024-03-11T19:44:25.004 INFO:tasks.workunit.client.0.smithi090.stdout:8/777: getdents d0/d3/d3e/d85 0 2024-03-11T19:44:25.365 INFO:tasks.workunit.client.0.smithi090.stdout:5/709: link d1/d17/d39/d51/c87 d1/d17/d39/d51/d9a/d80/cf3 0 2024-03-11T19:44:25.405 INFO:tasks.workunit.client.0.smithi090.stdout:5/710: chown d1/d26/dc8/fde 3467288 1 2024-03-11T19:44:25.456 INFO:tasks.workunit.client.0.smithi090.stdout:4/781: symlink d4/d27/d28/d38/d75/l106 0 2024-03-11T19:44:25.463 INFO:tasks.workunit.client.0.smithi090.stdout:6/770: write d4/d13/d22/d34/d83/fbc [591016,52982] 0 2024-03-11T19:44:25.588 INFO:tasks.workunit.client.0.smithi090.stdout:3/672: write d2/db/d15/d69/dad/fa4 [860071,21441] 0 2024-03-11T19:44:25.606 INFO:tasks.workunit.client.0.smithi090.stdout:8/778: chown d0/d43 1681 1 2024-03-11T19:44:25.910 INFO:tasks.workunit.client.0.smithi090.stdout:4/782: sync 2024-03-11T19:44:25.933 INFO:tasks.workunit.client.0.smithi090.stdout:5/711: dread - d1/d26/fd6 zero size 2024-03-11T19:44:26.009 INFO:tasks.workunit.client.0.smithi090.stdout:9/717: rename d1/dd/f1e to d1/dd/d18/d3a/d57/fe9 0 2024-03-11T19:44:26.769 INFO:tasks.workunit.client.0.smithi090.stdout:8/779: rmdir d0/d3/d4c/d8d 39 2024-03-11T19:44:27.066 INFO:tasks.workunit.client.0.smithi090.stdout:0/740: dwrite d2/d11/d6e/f9b [4194304,4194304] 0 2024-03-11T19:44:27.302 INFO:tasks.workunit.client.0.smithi090.stdout:2/714: link d0/de/d12/d46/d5d/d63/faa d0/de/db6/fdc 0 2024-03-11T19:44:27.424 INFO:tasks.workunit.client.0.smithi090.stdout:9/718: unlink d1/f9 0 2024-03-11T19:44:28.029 INFO:tasks.workunit.client.0.smithi090.stdout:8/780: fsync d0/d3/d86/d93/fe3 0 2024-03-11T19:44:28.110 INFO:tasks.workunit.client.0.smithi090.stdout:0/741: creat d2/d11/d15/d29/d3d/d7e/fe8 x:0 0 0 2024-03-11T19:44:28.131 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:09:47.889 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:44:28.147 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:09:47.905 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:44:28.172 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:09:47.872 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:44:28.189 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:09:47.889 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:44:28.198 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:09:47.890 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:44:28.209 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:09:47.901 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:44:28.255 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:09:48.014 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:44:28.334 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:09:48.034 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:44:28.337 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:09:48.029 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:44:28.538 INFO:tasks.workunit.client.0.smithi090.stdout:2/715: dread - d0/de/d12/dc8/fd5 zero size 2024-03-11T19:44:28.573 INFO:tasks.workunit.client.0.smithi090.stdout:2/716: stat d0/de/d12/d46/d49 0 2024-03-11T19:44:28.913 INFO:tasks.workunit.client.0.smithi090.stdout:7/718: write d3/dc/d1e/d7c/d96/fb8 [2075519,28256] 0 2024-03-11T19:44:28.922 INFO:tasks.workunit.client.0.smithi090.stdout:1/722: getdents d5 0 2024-03-11T19:44:29.362 INFO:tasks.workunit.client.0.smithi090.stdout:0/742: symlink d2/d11/d15/d29/d2e/db4/le9 0 2024-03-11T19:44:29.397 INFO:tasks.workunit.client.0.smithi090.stdout:0/743: chown d2/fcb 113125 1 2024-03-11T19:44:29.920 INFO:tasks.workunit.client.0.smithi090.stdout:6/771: dwrite d4/f62 [8388608,4194304] 0 2024-03-11T19:44:30.099 INFO:tasks.workunit.client.0.smithi090.stdout:4/783: getdents d4/dd8 0 2024-03-11T19:44:30.118 INFO:tasks.workunit.client.0.smithi090.stdout:6/772: sync 2024-03-11T19:44:30.236 INFO:tasks.workunit.client.0.smithi090.stdout:2/717: dread d0/de/d12/d46/d75/dbb/fd7 [0,4194304] 0 2024-03-11T19:44:30.295 INFO:tasks.workunit.client.0.smithi090.stdout:8/781: chown d0/d60/db6/fca 0 1 2024-03-11T19:44:30.297 INFO:tasks.workunit.client.0.smithi090.stdout:2/718: dread - d0/fae zero size 2024-03-11T19:44:30.501 INFO:tasks.workunit.client.0.smithi090.stdout:1/723: mknod d5/d31/cf4 0 2024-03-11T19:44:30.661 INFO:tasks.workunit.client.0.smithi090.stdout:5/712: dwrite d1/d17/d39/d51/d53/f67 [0,4194304] 0 2024-03-11T19:44:31.083 INFO:tasks.workunit.client.0.smithi090.stdout:6/773: mknod d4/d13/d22/d34/d40/dd5/dc1/c100 0 2024-03-11T19:44:31.362 INFO:tasks.workunit.client.0.smithi090.stdout:3/673: write d2/db/d15/d1a/d75/dc6/f3d [659732,30902] 0 2024-03-11T19:44:31.372 INFO:tasks.workunit.client.0.smithi090.stdout:2/719: creat d0/de/d12/d46/d75/dbb/fdd x:0 0 0 2024-03-11T19:44:32.122 INFO:tasks.workunit.client.0.smithi090.stdout:9/719: dwrite d1/dd/df/d1c/d21/d5c/f8b [4194304,4194304] 0 2024-03-11T19:44:32.294 INFO:tasks.workunit.client.0.smithi090.stdout:8/782: symlink d0/d3/d4c/l107 0 2024-03-11T19:44:32.566 INFO:tasks.workunit.client.0.smithi090.stdout:3/674: fdatasync d2/d45/f7a 0 2024-03-11T19:44:32.839 INFO:tasks.workunit.client.0.smithi090.stdout:8/783: symlink d0/d43/dc/d90/l108 0 2024-03-11T19:44:32.960 INFO:tasks.workunit.client.0.smithi090.stdout:2/720: symlink d0/dad/lde 0 2024-03-11T19:44:33.359 INFO:tasks.workunit.client.0.smithi090.stdout:9/720: symlink d1/dd/df/d1c/lea 0 2024-03-11T19:44:33.762 INFO:tasks.workunit.client.0.smithi090.stdout:0/744: write d2/d11/d15/d29/d2e/f52 [36729,702] 0 2024-03-11T19:44:34.025 INFO:tasks.workunit.client.0.smithi090.stdout:6/774: getdents d4/d13/d22/d34/d83 0 2024-03-11T19:44:34.050 INFO:tasks.workunit.client.0.smithi090.stdout:8/784: dread d0/d3/f68 [0,4194304] 0 2024-03-11T19:44:34.083 INFO:tasks.workunit.client.0.smithi090.stdout:9/721: chown d1/dd/df/f10 166031773 1 2024-03-11T19:44:34.217 INFO:tasks.workunit.client.0.smithi090.stdout:8/785: sync 2024-03-11T19:44:34.460 INFO:tasks.workunit.client.0.smithi090.stdout:2/721: truncate d0/de/d12/f24 886534 0 2024-03-11T19:44:34.503 INFO:tasks.workunit.client.0.smithi090.stdout:7/719: dwrite d3/dc/d1e/d7c/d96/d7f/fbb [0,4194304] 0 2024-03-11T19:44:34.679 INFO:tasks.workunit.client.0.smithi090.stdout:0/745: rename d2/d11/d3a/l9c to d2/d11/d15/d29/d3d/d7b/de6/lea 0 2024-03-11T19:44:34.719 INFO:tasks.workunit.client.0.smithi090.stdout:9/722: mkdir d1/dd/df/d1c/d77/deb 0 2024-03-11T19:44:34.835 INFO:tasks.workunit.client.0.smithi090.stdout:1/724: write d5/d4e/d79/f98 [360087,59662] 0 2024-03-11T19:44:34.841 INFO:tasks.workunit.client.0.smithi090.stdout:4/784: dwrite d4/d27/d28/f30 [0,4194304] 0 2024-03-11T19:44:35.041 INFO:tasks.workunit.client.0.smithi090.stdout:1/725: read d5/d31/d4c/f73 [239202,100659] 0 2024-03-11T19:44:35.110 INFO:tasks.workunit.client.0.smithi090.stdout:0/746: fdatasync d2/f37 0 2024-03-11T19:44:35.133 INFO:tasks.workunit.client.0.smithi090.stdout:5/713: write d1/d17/d39/f68 [3278137,129249] 0 2024-03-11T19:44:35.192 INFO:tasks.workunit.client.0.smithi090.stdout:1/726: write d5/d4e/d79/f98 [1176540,123998] 0 2024-03-11T19:44:35.259 INFO:tasks.workunit.client.0.smithi090.stdout:8/786: truncate d0/d3/d86/fd6 326577 0 2024-03-11T19:44:35.296 INFO:tasks.workunit.client.0.smithi090.stdout:2/722: dread d0/d21/f6e [4194304,4194304] 0 2024-03-11T19:44:35.392 INFO:tasks.workunit.client.0.smithi090.stdout:7/720: symlink d3/dc/da6/deb/lfa 0 2024-03-11T19:44:35.615 INFO:tasks.workunit.client.0.smithi090.stdout:4/785: creat d4/dd8/d66/dcf/dd7/f107 x:0 0 0 2024-03-11T19:44:35.743 INFO:tasks.workunit.client.0.smithi090.stdout:9/723: dread d1/dd/d18/d25/f97 [0,4194304] 0 2024-03-11T19:44:35.830 INFO:tasks.workunit.client.0.smithi090.stdout:1/727: dread d5/da/dd/f1f [0,4194304] 0 2024-03-11T19:44:35.864 INFO:tasks.workunit.client.0.smithi090.stdout:5/714: fsync d1/d26/f8f 0 2024-03-11T19:44:35.902 INFO:tasks.workunit.client.0.smithi090.stdout:9/724: dread - d1/dd/d18/d3a/d48/f92 zero size 2024-03-11T19:44:36.048 INFO:tasks.workunit.client.0.smithi090.stdout:8/787: dread - d0/d3/d86/d93/fc1 zero size 2024-03-11T19:44:36.096 INFO:tasks.workunit.client.0.smithi090.stdout:2/723: mkdir d0/de/d12/d46/d5d/d8d/ddf 0 2024-03-11T19:44:36.154 INFO:tasks.workunit.client.0.smithi090.stdout:3/675: write d2/db/f62 [441563,122024] 0 2024-03-11T19:44:36.196 INFO:tasks.workunit.client.0.smithi090.stdout:7/721: unlink d3/de6/le7 0 2024-03-11T19:44:36.547 INFO:tasks.workunit.client.0.smithi090.stdout:4/786: dread - d4/d51/d8c/fd1 zero size 2024-03-11T19:44:36.838 INFO:tasks.workunit.client.0.smithi090.stdout:1/728: truncate d5/d31/d4c/f73 443588 0 2024-03-11T19:44:37.551 INFO:tasks.workunit.client.0.smithi090.stdout:6/775: dwrite d4/d13/d22/d34/d40/da4/fd8 [4194304,4194304] 0 2024-03-11T19:44:37.768 INFO:tasks.workunit.client.0.smithi090.stdout:3/676: rmdir d2/d2f 39 2024-03-11T19:44:37.883 INFO:tasks.workunit.client.0.smithi090.stdout:3/677: dread d2/d45/f4a [0,4194304] 0 2024-03-11T19:44:38.205 INFO:tasks.workunit.client.0.smithi090.stdout:7/722: dread d3/d55/fb4 [0,4194304] 0 2024-03-11T19:44:38.207 INFO:tasks.workunit.client.0.smithi090.stdout:3/678: chown d2/db/d15/d1a/d75/dc6/c40 5785 1 2024-03-11T19:44:38.384 INFO:tasks.workunit.client.0.smithi090.stdout:3/679: sync 2024-03-11T19:44:38.560 INFO:tasks.workunit.client.0.smithi090.stdout:0/747: write d2/d11/d15/d29/d2e/db4/daf/fdb [56263,83050] 0 2024-03-11T19:44:38.862 INFO:tasks.workunit.client.0.smithi090.stdout:8/788: stat d0/d53/d59/c56 0 2024-03-11T19:44:39.453 INFO:tasks.workunit.client.0.smithi090.stdout:1/729: unlink d5/d31/d4c/f88 0 2024-03-11T19:44:39.587 INFO:tasks.workunit.client.0.smithi090.stdout:7/723: creat d3/dc/d1e/d7c/d96/d32/d71/d80/ffb x:0 0 0 2024-03-11T19:44:39.742 INFO:tasks.workunit.client.0.smithi090.stdout:9/725: link d1/dd/d18/d32/l56 d1/dd/df/d1c/d9f/lec 0 2024-03-11T19:44:39.863 INFO:tasks.workunit.client.0.smithi090.stdout:7/724: dwrite d3/dc/d1e/d7c/d96/d7f/fbb [0,4194304] 0 2024-03-11T19:44:39.987 INFO:tasks.workunit.client.0.smithi090.stdout:0/748: symlink d2/d11/d3f/leb 0 2024-03-11T19:44:40.306 INFO:tasks.workunit.client.0.smithi090.stdout:5/715: link d1/d26/fc d1/d17/d39/d51/d53/d7b/d3b/d76/d92/dbd/ff4 0 2024-03-11T19:44:40.460 INFO:tasks.workunit.client.0.smithi090.stdout:1/730: read d5/da/f47 [4272509,108138] 0 2024-03-11T19:44:40.605 INFO:tasks.workunit.client.0.smithi090.stdout:4/787: write d4/d51/d8c/d89/f98 [562729,35404] 0 2024-03-11T19:44:40.743 INFO:tasks.workunit.client.0.smithi090.stdout:4/788: sync 2024-03-11T19:44:40.757 INFO:tasks.workunit.client.0.smithi090.stdout:4/789: stat d4/d51/l6d 0 2024-03-11T19:44:40.821 INFO:tasks.workunit.client.0.smithi090.stdout:6/776: creat d4/f101 x:0 0 0 2024-03-11T19:44:40.875 INFO:tasks.workunit.client.0.smithi090.stdout:3/680: mknod d2/db/d15/d1a/d75/cd6 0 2024-03-11T19:44:40.955 INFO:tasks.workunit.client.0.smithi090.stdout:2/724: getdents d0/de/d12 0 2024-03-11T19:44:40.988 INFO:tasks.workunit.client.0.smithi090.stdout:0/749: mkdir d2/d11/d3a/d64/dac/dba/dec 0 2024-03-11T19:44:41.107 INFO:tasks.workunit.client.0.smithi090.stdout:2/725: chown d0/de/db6/d29/c8b 11689843 1 2024-03-11T19:44:41.223 INFO:tasks.workunit.client.0.smithi090.stdout:5/716: mknod d1/d17/d39/d51/d73/da6/cf5 0 2024-03-11T19:44:42.782 INFO:tasks.workunit.client.0.smithi090.stdout:3/681: dread d2/db/d15/d1a/d75/dc6/d47/d6a/dcb/f78 [0,4194304] 0 2024-03-11T19:44:42.821 INFO:tasks.workunit.client.0.smithi090.stdout:0/750: dread d2/d11/d3a/f69 [0,4194304] 0 2024-03-11T19:44:42.903 INFO:tasks.workunit.client.0.smithi090.stdout:0/751: chown d2/d11/d15/d29/d43/fbe 719526663 1 2024-03-11T19:44:43.160 INFO:tasks.workunit.client.0.smithi090.stdout:2/726: truncate d0/f3 3305922 0 2024-03-11T19:44:43.367 INFO:tasks.workunit.client.0.smithi090.stdout:8/789: dwrite d0/d43/dc/f3b [4194304,4194304] 0 2024-03-11T19:44:43.503 INFO:tasks.workunit.client.0.smithi090.stdout:0/752: creat d2/d11/d15/d29/d2e/d6d/fed x:0 0 0 2024-03-11T19:44:43.521 INFO:tasks.workunit.client.0.smithi090.stdout:3/682: read d2/db/d15/d69/dad/fa4 [747958,36877] 0 2024-03-11T19:44:43.866 INFO:tasks.workunit.client.0.smithi090.stdout:7/725: dwrite d3/f70 [0,4194304] 0 2024-03-11T19:44:44.015 INFO:tasks.workunit.client.0.smithi090.stdout:1/731: getdents d5/d4e/d54 0 2024-03-11T19:44:44.032 INFO:tasks.workunit.client.0.smithi090.stdout:1/732: chown d5/da 423755 1 2024-03-11T19:44:44.055 INFO:tasks.workunit.client.0.smithi090.stdout:1/733: chown d5/da/d1c/c25 5 1 2024-03-11T19:44:44.075 INFO:tasks.workunit.client.0.smithi090.stdout:1/734: stat d5/da/dd/l8b 0 2024-03-11T19:44:44.270 INFO:tasks.workunit.client.0.smithi090.stdout:8/790: dread d0/d53/d59/f72 [0,4194304] 0 2024-03-11T19:44:44.382 INFO:tasks.workunit.client.0.smithi090.stdout:3/683: creat d2/db/d15/d49/fd7 x:0 0 0 2024-03-11T19:44:44.485 INFO:tasks.workunit.client.0.smithi090.stdout:0/753: creat d2/d11/d15/d29/d2e/d6d/da4/fee x:0 0 0 2024-03-11T19:44:44.708 INFO:tasks.workunit.client.0.smithi090.stdout:2/727: unlink d0/de/db6/d25/d8c/la0 0 2024-03-11T19:44:45.059 INFO:tasks.workunit.client.0.smithi090.stdout:7/726: creat d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/ffc x:0 0 0 2024-03-11T19:44:45.898 INFO:tasks.workunit.client.0.smithi090.stdout:7/727: creat d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/ffd x:0 0 0 2024-03-11T19:44:46.001 INFO:tasks.workunit.client.0.smithi090.stdout:4/790: dwrite d4/d27/d28/d38/d64/f102 [0,4194304] 0 2024-03-11T19:44:46.272 INFO:tasks.workunit.client.0.smithi090.stdout:0/754: mkdir d2/d11/d15/d29/dde/def 0 2024-03-11T19:44:46.280 INFO:tasks.workunit.client.0.smithi090.stdout:9/726: dwrite d1/f17 [0,4194304] 0 2024-03-11T19:44:46.413 INFO:tasks.workunit.client.0.smithi090.stdout:5/717: write d1/d17/d39/d51/d53/d7b/d3b/d76/db7/fb8 [754450,122713] 0 2024-03-11T19:44:46.468 INFO:tasks.workunit.client.0.smithi090.stdout:6/777: write d4/d13/fa2 [208249,113801] 0 2024-03-11T19:44:46.734 INFO:tasks.workunit.client.0.smithi090.stdout:2/728: creat d0/de/d12/d46/d4c/fe0 x:0 0 0 2024-03-11T19:44:46.759 INFO:tasks.workunit.client.0.smithi090.stdout:4/791: creat d4/dd8/d66/dcf/f108 x:0 0 0 2024-03-11T19:44:46.949 INFO:tasks.workunit.client.0.smithi090.stdout:9/727: rmdir d1/d47 39 2024-03-11T19:44:47.172 INFO:tasks.workunit.client.0.smithi090.stdout:7/728: unlink d3/dc/d1e/d7c/d96/d32/d45/de0/f3e 0 2024-03-11T19:44:47.278 INFO:tasks.workunit.client.0.smithi090.stdout:7/729: readlink d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/lc7 0 2024-03-11T19:44:47.346 INFO:tasks.workunit.client.0.smithi090.stdout:0/755: creat d2/d11/d15/d29/d3d/d8c/ff0 x:0 0 0 2024-03-11T19:44:47.519 INFO:tasks.workunit.client.0.smithi090.stdout:5/718: mkdir d1/d17/d39/d51/d73/df6 0 2024-03-11T19:44:47.616 INFO:tasks.workunit.client.0.smithi090.stdout:5/719: truncate d1/d17/d39/d51/d53/d7b/d3b/d76/db7/fb8 888790 0 2024-03-11T19:44:47.657 INFO:tasks.workunit.client.0.smithi090.stdout:5/720: stat d1/d17/d39/d51/d53/d7b/d3b/d76/d92/fa2 0 2024-03-11T19:44:47.818 INFO:tasks.workunit.client.0.smithi090.stdout:2/729: dread - d0/de/d12/d46/d75/dbb/fdd zero size 2024-03-11T19:44:47.898 INFO:tasks.workunit.client.0.smithi090.stdout:9/728: mknod d1/dd/df/d1c/d21/d2a/ced 0 2024-03-11T19:44:47.922 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:44:47.925 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:44:47.932 INFO:tasks.workunit.client.0.smithi090.stdout:9/729: chown d1/dd/df/f10 36 1 2024-03-11T19:44:48.025 INFO:tasks.workunit.client.0.smithi090.stdout:7/730: rename d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/db5/de1/fac to d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/ffe 0 2024-03-11T19:44:48.155 INFO:tasks.workunit.client.0.smithi090.stdout:0/756: rename d2/d11/d15/d29/d43/d58/f88 to d2/d11/d15/d29/d43/d58/ff1 0 2024-03-11T19:44:48.205 INFO:tasks.workunit.client.0.smithi090.stdout:6/778: creat d4/d13/d22/d34/d40/dd5/d80/ddb/ddc/f102 x:0 0 0 2024-03-11T19:44:48.286 INFO:tasks.workunit.client.0.smithi090.stdout:5/721: chown d1/d75/f8c 35 1 2024-03-11T19:44:48.675 INFO:tasks.workunit.client.0.smithi090.stdout:9/730: dread - d1/dd/df/d1c/fc9 zero size 2024-03-11T19:44:49.075 INFO:tasks.workunit.client.0.smithi090.stdout:6/779: symlink d4/d14/d8a/d74/l103 0 2024-03-11T19:44:49.362 INFO:tasks.workunit.client.0.smithi090.stdout:1/735: dwrite d5/d4e/d54/d94/fb0 [0,4194304] 0 2024-03-11T19:44:49.714 INFO:tasks.workunit.client.0.smithi090.stdout:2/730: rename d0/de/db6/d29/c9e to d0/de/d12/d46/d4c/d71/ce1 0 2024-03-11T19:44:49.749 INFO:tasks.workunit.client.0.smithi090.stdout:6/780: unlink d4/d13/d22/d34/d40/dd5/fa8 0 2024-03-11T19:44:50.102 INFO:tasks.workunit.client.0.smithi090.stdout:2/731: chown d0/de/db6/d29/f6d 966185989 1 2024-03-11T19:44:50.153 INFO:tasks.workunit.client.0.smithi090.stdout:1/736: mkdir d5/df5 0 2024-03-11T19:44:50.288 INFO:tasks.workunit.client.0.smithi090.stdout:8/791: dwrite d0/d43/dc/f7e [0,4194304] 0 2024-03-11T19:44:50.291 INFO:tasks.workunit.client.0.smithi090.stdout:3/684: dwrite d2/f9 [0,4194304] 0 2024-03-11T19:44:51.156 INFO:tasks.workunit.client.0.smithi090.stdout:2/732: dread d0/de/d1f/f1c [0,4194304] 0 2024-03-11T19:44:51.264 INFO:tasks.workunit.client.0.smithi090.stdout:6/781: dread d4/d13/d22/d34/d37/d7f/d90/fad [0,4194304] 0 2024-03-11T19:44:51.317 INFO:tasks.workunit.client.0.smithi090.stdout:0/757: link d2/d11/d15/d29/c55 d2/d11/d15/d29/d2e/d6d/da4/d78/cf2 0 2024-03-11T19:44:51.502 INFO:tasks.workunit.client.0.smithi090.stdout:9/731: mkdir d1/d47/d7c/dee 0 2024-03-11T19:44:51.723 INFO:tasks.workunit.client.0.smithi090.stdout:8/792: symlink d0/d43/dc/d90/l109 0 2024-03-11T19:44:51.800 INFO:tasks.workunit.client.0.smithi090.stdout:3/685: mkdir d2/d4f/d59/da1/dd8 0 2024-03-11T19:44:51.862 INFO:tasks.workunit.client.0.smithi090.stdout:4/792: write d4/d51/d8c/d89/fc7 [3225436,100595] 0 2024-03-11T19:44:52.795 INFO:tasks.workunit.client.0.smithi090.stdout:2/733: rmdir d0/de/d12/dc8 39 2024-03-11T19:44:52.895 INFO:tasks.workunit.client.0.smithi090.stdout:4/793: chown d4/d27/d28/d38/d75/c9d 0 1 2024-03-11T19:44:53.022 INFO:tasks.workunit.client.0.smithi090.stdout:6/782: creat d4/d13/d22/d42/dee/f104 x:0 0 0 2024-03-11T19:44:53.026 INFO:tasks.workunit.client.0.smithi090.stdout:7/731: write d3/dc/d1e/d7c/d96/d32/d45/de0/f5a [3292649,117456] 0 2024-03-11T19:44:53.090 INFO:tasks.workunit.client.0.smithi090.stdout:0/758: mkdir d2/d11/d15/d29/df3 0 2024-03-11T19:44:53.426 INFO:tasks.workunit.client.0.smithi090.stdout:7/732: dread d3/dc/d1e/d7c/d96/d32/d45/de0/f21 [0,4194304] 0 2024-03-11T19:44:53.970 INFO:tasks.workunit.client.0.smithi090.stdout:4/794: creat d4/d51/dbf/f109 x:0 0 0 2024-03-11T19:44:54.007 INFO:tasks.workunit.client.0.smithi090.stdout:4/795: read d4/ff3 [2808591,98020] 0 2024-03-11T19:44:54.058 INFO:tasks.workunit.client.0.smithi090.stdout:6/783: mknod d4/d9b/def/c105 0 2024-03-11T19:44:54.355 INFO:tasks.workunit.client.0.smithi090.stdout:5/722: dwrite d1/d17/fb4 [0,4194304] 0 2024-03-11T19:44:54.747 INFO:tasks.workunit.client.0.smithi090.stdout:7/733: dread - d3/dc/fc4 zero size 2024-03-11T19:44:54.821 INFO:tasks.workunit.client.0.smithi090.stdout:5/723: sync 2024-03-11T19:44:54.840 INFO:tasks.workunit.client.0.smithi090.stdout:5/724: chown d1/d17/d39/f82 4 1 2024-03-11T19:44:54.851 INFO:tasks.workunit.client.0.smithi090.stdout:5/725: stat d1/d17/f66 0 2024-03-11T19:44:54.917 INFO:tasks.workunit.client.0.smithi090.stdout:9/732: mknod d1/dd/df/d1c/d88/d96/cef 0 2024-03-11T19:44:55.204 INFO:tasks.workunit.client.0.smithi090.stdout:4/796: symlink d4/dd8/d66/l10a 0 2024-03-11T19:44:55.766 INFO:tasks.workunit.client.0.smithi090.stdout:1/737: chown d5/l9 1343746433 1 2024-03-11T19:44:56.276 INFO:tasks.workunit.client.0.smithi090.stdout:4/797: fsync d4/d18/f6e 0 2024-03-11T19:44:56.349 INFO:tasks.workunit.client.0.smithi090.stdout:9/733: dwrite d1/f17 [4194304,4194304] 0 2024-03-11T19:44:56.994 INFO:tasks.workunit.client.0.smithi090.stdout:4/798: symlink d4/d27/d28/d7a/l10b 0 2024-03-11T19:44:57.123 INFO:tasks.workunit.client.0.smithi090.stdout:8/793: write d0/d53/d59/d96/fe0 [151551,77052] 0 2024-03-11T19:44:57.179 INFO:tasks.workunit.client.0.smithi090.stdout:1/738: mkdir d5/da/d97/df6 0 2024-03-11T19:44:57.209 INFO:tasks.workunit.client.0.smithi090.stdout:1/739: chown d5/da/d1c/d3b/d40/d89 0 1 2024-03-11T19:44:57.411 INFO:tasks.workunit.client.0.smithi090.stdout:3/686: write d2/d45/f7a [332557,13838] 0 2024-03-11T19:44:57.779 INFO:tasks.workunit.client.0.smithi090.stdout:8/794: chown d0/d3/db0 1244017435 1 2024-03-11T19:44:57.897 INFO:tasks.workunit.client.0.smithi090.stdout:1/740: dread - d5/da/d1c/d6d/d80/fdd zero size 2024-03-11T19:44:58.029 INFO:tasks.workunit.client.0.smithi090.stdout:3/687: truncate d2/db/d15/d1a/d75/dc6/d47/d6a/d9b/fae 761720 0 2024-03-11T19:44:58.053 INFO:tasks.workunit.client.0.smithi090.stdout:0/759: write d2/d11/d15/f28 [976166,41210] 0 2024-03-11T19:44:58.600 INFO:tasks.workunit.client.0.smithi090.stdout:3/688: rename d2/d4f/d59/da1/dd8 to d2/db/d15/d1a/d3c/dd9 0 2024-03-11T19:44:58.655 INFO:tasks.workunit.client.0.smithi090.stdout:4/799: creat d4/d27/d28/d38/d75/f10c x:0 0 0 2024-03-11T19:44:58.673 INFO:tasks.workunit.client.0.smithi090.stdout:3/689: readlink d2/db/d15/d1a/d75/dc6/l23 0 2024-03-11T19:44:58.725 INFO:tasks.workunit.client.0.smithi090.stdout:3/690: chown d2/db/d15/d1a/d75/cd6 2 1 2024-03-11T19:44:58.965 INFO:tasks.workunit.client.0.smithi090.stdout:1/741: write d5/d4e/d54/d94/fb0 [2977516,26430] 0 2024-03-11T19:44:59.488 INFO:tasks.workunit.client.0.smithi090.stdout:3/691: truncate d2/db/f1c 2099458 0 2024-03-11T19:44:59.611 INFO:tasks.workunit.client.0.smithi090.stdout:8/795: mknod d0/d103/c10a 0 2024-03-11T19:44:59.941 INFO:tasks.workunit.client.0.smithi090.stdout:6/784: dwrite d4/d13/d22/d3a/fec [4194304,4194304] 0 2024-03-11T19:45:00.207 INFO:tasks.workunit.client.0.smithi090.stdout:5/726: dwrite d1/d26/fd6 [0,4194304] 0 2024-03-11T19:45:00.273 INFO:tasks.workunit.client.0.smithi090.stdout:6/785: dread - d4/d14/d8a/f96 zero size 2024-03-11T19:45:00.278 INFO:tasks.workunit.client.0.smithi090.stdout:3/692: read - d2/db/d15/d1a/d75/dc6/d47/d6a/fb5 zero size 2024-03-11T19:45:00.324 INFO:tasks.workunit.client.0.smithi090.stdout:8/796: mkdir d0/d53/d59/d96/de5/d10b 0 2024-03-11T19:45:00.338 INFO:tasks.workunit.client.0.smithi090.stdout:2/734: write d0/de/d12/d46/fa8 [627756,28447] 0 2024-03-11T19:45:00.359 INFO:tasks.workunit.client.0.smithi090.stdout:8/797: stat d0 0 2024-03-11T19:45:00.842 INFO:tasks.workunit.client.0.smithi090.stdout:0/760: mkdir d2/d11/df4 0 2024-03-11T19:45:00.857 INFO:tasks.workunit.client.0.smithi090.stdout:1/742: truncate d5/d31/d4c/f91 741209 0 2024-03-11T19:45:01.134 INFO:tasks.workunit.client.0.smithi090.stdout:9/734: dwrite d1/dd/d18/d3a/d57/f5b [0,4194304] 0 2024-03-11T19:45:01.281 INFO:tasks.workunit.client.0.smithi090.stdout:4/800: link d4/dd8/db5/feb d4/d18/d105/f10d 0 2024-03-11T19:45:01.730 INFO:tasks.workunit.client.0.smithi090.stdout:3/693: fdatasync d2/f5a 0 2024-03-11T19:45:01.801 INFO:tasks.workunit.client.0.smithi090.stdout:7/734: mkdir d3/dff 0 2024-03-11T19:45:02.050 INFO:tasks.workunit.client.0.smithi090.stdout:5/727: creat d1/d26/db5/ff7 x:0 0 0 2024-03-11T19:45:02.067 INFO:tasks.workunit.client.0.smithi090.stdout:9/735: creat d1/dbb/ff0 x:0 0 0 2024-03-11T19:45:02.166 INFO:tasks.workunit.client.0.smithi090.stdout:5/728: readlink d1/d17/d39/d51/d53/d7b/da0/db9/dc6/lcb 0 2024-03-11T19:45:02.877 INFO:tasks.workunit.client.0.smithi090.stdout:9/736: creat d1/dd/d18/d3a/d57/ff1 x:0 0 0 2024-03-11T19:45:03.724 INFO:tasks.workunit.client.0.smithi090.stdout:6/786: rename d4/d14/d8a/d74/cd9 to d4/d13/d22/d3a/c106 0 2024-03-11T19:45:03.768 INFO:tasks.workunit.client.0.smithi090.stdout:9/737: read - d1/dd/df/d1c/d21/d2a/f75 zero size 2024-03-11T19:45:03.840 INFO:tasks.workunit.client.0.smithi090.stdout:6/787: read d4/d13/d22/d34/d40/da4/fd8 [3645790,110102] 0 2024-03-11T19:45:03.951 INFO:tasks.workunit.client.0.smithi090.stdout:5/729: creat d1/d17/d39/d51/d53/d7b/d3b/d76/db7/ddf/dee/ff8 x:0 0 0 2024-03-11T19:45:04.613 INFO:tasks.workunit.client.0.smithi090.stdout:6/788: mkdir d4/d14/d107 0 2024-03-11T19:45:04.826 INFO:tasks.workunit.client.0.smithi090.stdout:9/738: mknod d1/d47/cf2 0 2024-03-11T19:45:05.078 INFO:tasks.workunit.client.0.smithi090.stdout:8/798: rename d0/d53/f5d to d0/d3/d4c/f10c 0 2024-03-11T19:45:05.478 INFO:tasks.workunit.client.0.smithi090.stdout:6/789: creat d4/d13/d22/d34/d40/d7d/dc2/f108 x:0 0 0 2024-03-11T19:45:05.641 INFO:tasks.workunit.client.0.smithi090.stdout:9/739: dread d1/f17 [4194304,4194304] 0 2024-03-11T19:45:05.670 INFO:tasks.workunit.client.0.smithi090.stdout:2/735: rename d0/de/d12/d46/d4c/d71/l9a to d0/de/d1f/le2 0 2024-03-11T19:45:05.730 INFO:tasks.workunit.client.0.smithi090.stdout:9/740: chown d1/dd/d18/d3a/d70/ca1 107414699 1 2024-03-11T19:45:05.920 INFO:tasks.workunit.client.0.smithi090.stdout:8/799: unlink d0/d3/d3e/fa5 0 2024-03-11T19:45:06.256 INFO:tasks.workunit.client.0.smithi090.stdout:5/730: rename d1/d17/d39/d51/d53/d7b/d3b/d76/d92/dbd to d1/d17/d39/df2/df9 0 2024-03-11T19:45:06.919 INFO:tasks.workunit.client.0.smithi090.stdout:1/743: dwrite d5/da/f68 [0,4194304] 0 2024-03-11T19:45:07.025 INFO:tasks.workunit.client.0.smithi090.stdout:2/736: dread d0/f2 [0,4194304] 0 2024-03-11T19:45:07.156 INFO:tasks.workunit.client.0.smithi090.stdout:3/694: write d2/db/d15/d1a/d75/f80 [324142,27627] 0 2024-03-11T19:45:07.169 INFO:tasks.workunit.client.0.smithi090.stdout:7/735: write d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/ffe [517110,28154] 0 2024-03-11T19:45:07.587 INFO:tasks.workunit.client.0.smithi090.stdout:2/737: unlink d0/de/l26 0 2024-03-11T19:45:07.631 INFO:tasks.workunit.client.0.smithi090.stdout:1/744: chown d5/f9a 1114628469 1 2024-03-11T19:45:07.794 INFO:tasks.workunit.client.0.smithi090.stdout:4/801: dwrite d4/d51/d8c/d89/f98 [0,4194304] 0 2024-03-11T19:45:07.943 INFO:tasks.workunit.client.0.smithi090.stdout:0/761: dwrite d2/d11/d15/d29/d3d/d7b/fb2 [0,4194304] 0 2024-03-11T19:45:08.390 INFO:tasks.workunit.client.0.smithi090.stdout:0/762: sync 2024-03-11T19:45:08.637 INFO:tasks.workunit.client.0.smithi090.stdout:8/800: write d0/d3/d3e/f104 [96894,94251] 0 2024-03-11T19:45:08.643 INFO:tasks.workunit.client.0.smithi090.stdout:9/741: write d1/dd/d18/d3a/d48/fbc [339384,12252] 0 2024-03-11T19:45:09.263 INFO:tasks.workunit.client.0.smithi090.stdout:3/695: truncate d2/db/d15/f17 252092 0 2024-03-11T19:45:09.970 INFO:tasks.workunit.client.0.smithi090.stdout:9/742: mkdir d1/dd/df3 0 2024-03-11T19:45:10.255 INFO:tasks.workunit.client.0.smithi090.stdout:8/801: truncate d0/d53/d59/d96/fe0 4222 0 2024-03-11T19:45:10.393 INFO:tasks.workunit.client.0.smithi090.stdout:6/790: rename d4/d13/d22/d34/f3c to d4/d13/d22/d34/d40/d7d/f109 0 2024-03-11T19:45:10.562 INFO:tasks.workunit.client.0.smithi090.stdout:1/745: mkdir d5/d65/d6b/d7f/df7 0 2024-03-11T19:45:10.651 INFO:tasks.workunit.client.0.smithi090.stdout:2/738: truncate d0/d21/f2a 2049100 0 2024-03-11T19:45:10.732 INFO:tasks.workunit.client.0.smithi090.stdout:3/696: symlink d2/db/d15/d69/da2/lda 0 2024-03-11T19:45:11.087 INFO:tasks.workunit.client.0.smithi090.stdout:9/743: symlink d1/dd/d18/ddd/lf4 0 2024-03-11T19:45:11.105 INFO:tasks.workunit.client.0.smithi090.stdout:7/736: write d3/d55/f64 [1350537,66608] 0 2024-03-11T19:45:11.208 INFO:tasks.workunit.client.0.smithi090.stdout:7/737: stat d3/dc/d1e/d7c/d96/d32/d71 0 2024-03-11T19:45:11.537 INFO:tasks.workunit.client.0.smithi090.stdout:1/746: dread d5/da/d1c/d3b/fcd [0,4194304] 0 2024-03-11T19:45:11.576 INFO:tasks.workunit.client.0.smithi090.stdout:2/739: mknod d0/de/d12/d46/d4c/d71/ce3 0 2024-03-11T19:45:11.637 INFO:tasks.workunit.client.0.smithi090.stdout:3/697: dread - d2/d45/fb8 zero size 2024-03-11T19:45:11.664 INFO:tasks.workunit.client.0.smithi090.stdout:3/698: readlink d2/db/d15/l39 0 2024-03-11T19:45:12.088 INFO:tasks.workunit.client.0.smithi090.stdout:9/744: fsync d1/dd/df/f10 0 2024-03-11T19:45:12.245 INFO:tasks.workunit.client.0.smithi090.stdout:5/731: rename d1/d17/d39/df2/df9/ff4 to d1/d26/db5/ffa 0 2024-03-11T19:45:12.278 INFO:tasks.workunit.client.0.smithi090.stdout:5/732: chown d1/d17/d39 451042241 1 2024-03-11T19:45:12.313 INFO:tasks.workunit.client.0.smithi090.stdout:5/733: chown d1/d17/d39/d51/d73/l7f 3 1 2024-03-11T19:45:12.412 INFO:tasks.workunit.client.0.smithi090.stdout:5/734: dread d1/d17/d39/d51/d53/d64/f6d [0,4194304] 0 2024-03-11T19:45:12.480 INFO:tasks.workunit.client.0.smithi090.stdout:6/791: mknod d4/c10a 0 2024-03-11T19:45:12.499 INFO:tasks.workunit.client.0.smithi090.stdout:0/763: write d2/d11/f41 [3675811,70421] 0 2024-03-11T19:45:12.505 INFO:tasks.workunit.client.0.smithi090.stdout:4/802: write d4/dd8/d66/dcf/f108 [120290,21681] 0 2024-03-11T19:45:12.590 INFO:tasks.workunit.client.0.smithi090.stdout:1/747: symlink d5/d65/d6b/d7f/lf8 0 2024-03-11T19:45:12.637 INFO:tasks.workunit.client.0.smithi090.stdout:6/792: chown d4/d13/d22/l9e 231178 1 2024-03-11T19:45:12.692 INFO:tasks.workunit.client.0.smithi090.stdout:6/793: read - d4/d13/d22/d34/d83/ffd zero size 2024-03-11T19:45:13.132 INFO:tasks.workunit.client.0.smithi090.stdout:7/738: creat d3/dc/d1e/d7c/d96/d32/d7b/d99/f100 x:0 0 0 2024-03-11T19:45:13.307 INFO:tasks.workunit.client.0.smithi090.stdout:9/745: creat d1/d49/d9c/daf/ff5 x:0 0 0 2024-03-11T19:45:13.659 INFO:tasks.workunit.client.0.smithi090.stdout:5/735: creat d1/d17/d39/d51/d53/ffb x:0 0 0 2024-03-11T19:45:13.818 INFO:tasks.workunit.client.0.smithi090.stdout:4/803: stat d4/d27/d28/c4c 0 2024-03-11T19:45:13.989 INFO:tasks.workunit.client.0.smithi090.stdout:0/764: rmdir d2/d11/d15/d29/d2e/db4 39 2024-03-11T19:45:14.094 INFO:tasks.workunit.client.0.smithi090.stdout:1/748: chown d5/d31/d76/f84 19943679 1 2024-03-11T19:45:15.607 INFO:tasks.workunit.client.0.smithi090.stdout:9/746: dread d1/dd/df/d1c/d21/d2a/da4/fa8 [0,4194304] 0 2024-03-11T19:45:15.647 INFO:tasks.workunit.client.0.smithi090.stdout:9/747: stat d1/dd/d18/d3a/d57/caa 0 2024-03-11T19:45:16.059 INFO:tasks.workunit.client.0.smithi090.stdout:1/749: symlink d5/da/d97/lf9 0 2024-03-11T19:45:16.748 INFO:tasks.workunit.client.0.smithi090.stdout:8/802: rename d0/d43/dc/l8a to d0/d3/d4c/l10d 0 2024-03-11T19:45:17.379 INFO:tasks.workunit.client.0.smithi090.stdout:2/740: getdents d0/de 0 2024-03-11T19:45:17.399 INFO:tasks.workunit.client.0.smithi090.stdout:0/765: mkdir d2/d11/d15/d29/df5 0 2024-03-11T19:45:17.400 INFO:tasks.workunit.client.0.smithi090.stdout:9/748: creat d1/dd/df/d1c/d77/ff6 x:0 0 0 2024-03-11T19:45:17.489 INFO:tasks.workunit.client.0.smithi090.stdout:9/749: chown d1/d49/d9c/daf 345284885 1 2024-03-11T19:45:17.625 INFO:tasks.workunit.client.0.smithi090.stdout:6/794: fdatasync d4/d14/d25/fe6 0 2024-03-11T19:45:17.668 INFO:tasks.workunit.client.0.smithi090.stdout:6/795: readlink d4/d13/d22/d34/d37/l57 0 2024-03-11T19:45:17.758 INFO:tasks.workunit.client.0.smithi090.stdout:3/699: getdents d2/db/d15 0 2024-03-11T19:45:17.991 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:45:17.994 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:45:18.002 INFO:tasks.workunit.client.0.smithi090.stdout:8/803: creat d0/d43/dc/d90/f10e x:0 0 0 2024-03-11T19:45:18.318 INFO:tasks.workunit.client.0.smithi090.stdout:2/741: stat d0/de/d12/d46/d5d/lc9 0 2024-03-11T19:45:18.346 INFO:tasks.workunit.client.0.smithi090.stdout:2/742: stat d0/de/db6/d29/fca 0 2024-03-11T19:45:18.767 INFO:tasks.workunit.client.0.smithi090.stdout:9/750: dread d1/dd/df/d1c/d21/d2a/fa3 [0,4194304] 0 2024-03-11T19:45:18.785 INFO:tasks.workunit.client.0.smithi090.stdout:6/796: chown d4/d13/d22/d34/d40/dd5/d80/ddb/cf5 5099 1 2024-03-11T19:45:19.392 INFO:tasks.workunit.client.0.smithi090.stdout:0/766: truncate d2/d11/d15/d29/d43/d58/ff1 5075489 0 2024-03-11T19:45:19.405 INFO:tasks.workunit.client.0.smithi090.stdout:9/751: dread - d1/dbb/ff0 zero size 2024-03-11T19:45:19.472 INFO:tasks.workunit.client.0.smithi090.stdout:0/767: read d2/f37 [1148515,105844] 0 2024-03-11T19:45:19.552 INFO:tasks.workunit.client.0.smithi090.stdout:6/797: mknod d4/d13/d22/d34/d40/dd5/d80/dd4/c10b 0 2024-03-11T19:45:19.704 INFO:tasks.workunit.client.0.smithi090.stdout:7/739: getdents d3/dc/d1e/d7c/d96/d32/d67 0 2024-03-11T19:45:19.784 INFO:tasks.workunit.client.0.smithi090.stdout:9/752: dread d1/dd/d18/d32/f45 [0,4194304] 0 2024-03-11T19:45:19.860 INFO:tasks.workunit.client.0.smithi090.stdout:0/768: sync 2024-03-11T19:45:20.276 INFO:tasks.workunit.client.0.smithi090.stdout:8/804: fdatasync d0/d43/dc/f8e 0 2024-03-11T19:45:20.322 INFO:tasks.workunit.client.0.smithi090.stdout:8/805: rename d0/d60/db6 to d0/d60/db6/db9/d10f 22 2024-03-11T19:45:20.872 INFO:tasks.workunit.client.0.smithi090.stdout:9/753: truncate d1/d1d/f67 8253992 0 2024-03-11T19:45:21.146 INFO:tasks.workunit.client.0.smithi090.stdout:0/769: mknod d2/d11/d15/d29/d2e/d6d/cf6 0 2024-03-11T19:45:21.296 INFO:tasks.workunit.client.0.smithi090.stdout:1/750: write d5/da/d1c/d6d/fe8 [152873,18251] 0 2024-03-11T19:45:21.645 INFO:tasks.workunit.client.0.smithi090.stdout:9/754: mkdir d1/d49/d9c/df7 0 2024-03-11T19:45:21.683 INFO:tasks.workunit.client.0.smithi090.stdout:9/755: dread - d1/dd/df/d1c/fad zero size 2024-03-11T19:45:22.076 INFO:tasks.workunit.client.0.smithi090.stdout:8/806: fdatasync d0/f5a 0 2024-03-11T19:45:22.187 INFO:tasks.workunit.client.0.smithi090.stdout:7/740: link d3/ca d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/db5/de1/dc2/dd0/c101 0 2024-03-11T19:45:22.286 INFO:tasks.workunit.client.0.smithi090.stdout:4/804: mkdir d4/d10e 0 2024-03-11T19:45:22.361 INFO:tasks.workunit.client.0.smithi090.stdout:1/751: symlink d5/d31/d4c/daf/lfa 0 2024-03-11T19:45:22.502 INFO:tasks.workunit.client.0.smithi090.stdout:8/807: mkdir d0/d60/db6/d110 0 2024-03-11T19:45:22.628 INFO:tasks.workunit.client.0.smithi090.stdout:3/700: dwrite d2/d45/f70 [0,4194304] 0 2024-03-11T19:45:22.640 INFO:tasks.workunit.client.0.smithi090.stdout:9/756: creat d1/dd/df3/ff8 x:0 0 0 2024-03-11T19:45:22.820 INFO:tasks.workunit.client.0.smithi090.stdout:3/701: readlink d2/d4f/d59/da1/lcc 0 2024-03-11T19:45:22.901 INFO:tasks.workunit.client.0.smithi090.stdout:3/702: dread - d2/db/d15/d1a/d75/dc6/d47/d6a/fb5 zero size 2024-03-11T19:45:22.938 INFO:tasks.workunit.client.0.smithi090.stdout:3/703: write d2/d45/f70 [3628589,39198] 0 2024-03-11T19:45:22.994 INFO:tasks.workunit.client.0.smithi090.stdout:7/741: creat d3/dc/da6/f102 x:0 0 0 2024-03-11T19:45:23.039 INFO:tasks.workunit.client.0.smithi090.stdout:4/805: read d4/d27/d28/f68 [1186354,96753] 0 2024-03-11T19:45:23.260 INFO:tasks.workunit.client.0.smithi090.stdout:1/752: fdatasync d5/d31/d36/f49 0 2024-03-11T19:45:23.450 INFO:tasks.workunit.client.0.smithi090.stdout:8/808: truncate d0/d3/d86/fce 480286 0 2024-03-11T19:45:23.470 INFO:tasks.workunit.client.0.smithi090.stdout:1/753: dread d5/da/f47 [4194304,4194304] 0 2024-03-11T19:45:23.741 INFO:tasks.workunit.client.0.smithi090.stdout:3/704: mknod d2/db/d15/d69/dad/cdb 0 2024-03-11T19:45:24.085 INFO:tasks.workunit.client.0.smithi090.stdout:2/743: write d0/de/d12/f32 [1082052,107696] 0 2024-03-11T19:45:24.254 INFO:tasks.workunit.client.0.smithi090.stdout:7/742: mkdir d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/db5/de1/dc6/d103 0 2024-03-11T19:45:24.268 INFO:tasks.workunit.client.0.smithi090.stdout:6/798: write d4/d13/d22/d34/d83/d9c/fa5 [1572807,84095] 0 2024-03-11T19:45:24.666 INFO:tasks.workunit.client.0.smithi090.stdout:8/809: readlink d0/d60/l9f 0 2024-03-11T19:45:24.707 INFO:tasks.workunit.client.0.smithi090.stdout:5/736: symlink d1/lfc 0 2024-03-11T19:45:25.233 INFO:tasks.workunit.client.0.smithi090.stdout:2/744: creat d0/de/d1f/fe4 x:0 0 0 2024-03-11T19:45:25.503 INFO:tasks.workunit.client.0.smithi090.stdout:6/799: creat d4/d13/d22/d34/d40/dd5/d80/f10c x:0 0 0 2024-03-11T19:45:25.512 INFO:tasks.workunit.client.0.smithi090.stdout:8/810: fdatasync d0/d3/d86/fe6 0 2024-03-11T19:45:25.759 INFO:tasks.workunit.client.0.smithi090.stdout:0/770: write d2/d11/d15/d29/fb9 [189830,25103] 0 2024-03-11T19:45:25.992 INFO:tasks.workunit.client.0.smithi090.stdout:9/757: rename d1/dd/d18/l24 to d1/dd/d18/d3a/da2/lf9 0 2024-03-11T19:45:26.130 INFO:tasks.workunit.client.0.smithi090.stdout:2/745: mkdir d0/de/d12/d46/d75/dc5/de5 0 2024-03-11T19:45:26.198 INFO:tasks.workunit.client.0.smithi090.stdout:9/758: sync 2024-03-11T19:45:26.330 INFO:tasks.workunit.client.0.smithi090.stdout:8/811: unlink d0/d43/c42 0 2024-03-11T19:45:26.720 INFO:tasks.workunit.client.0.smithi090.stdout:7/743: creat d3/dc/d1e/f104 x:0 0 0 2024-03-11T19:45:27.140 INFO:tasks.workunit.client.0.smithi090.stdout:0/771: getdents d2/d11/d3a/d64/dd5/de3 0 2024-03-11T19:45:27.208 INFO:tasks.workunit.client.0.smithi090.stdout:8/812: dread d0/d3/d86/d93/dda/ff6 [0,4194304] 0 2024-03-11T19:45:27.338 INFO:tasks.workunit.client.0.smithi090.stdout:2/746: mkdir d0/de/d12/d46/de6 0 2024-03-11T19:45:28.354 INFO:tasks.workunit.client.0.smithi090.stdout:7/744: mkdir d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/db5/de1/dc6/d103/d105 0 2024-03-11T19:45:29.200 INFO:tasks.workunit.client.0.smithi090.stdout:1/754: write d5/da/dd/f14 [2644862,16822] 0 2024-03-11T19:45:29.204 INFO:tasks.workunit.client.0.smithi090.stdout:3/705: write d2/db/d15/d69/dad/fbf [4077554,839] 0 2024-03-11T19:45:29.358 INFO:tasks.workunit.client.0.smithi090.stdout:7/745: chown d3/dc/d1e/d7c/d96/d32/da0/ce8 1 1 2024-03-11T19:45:29.592 INFO:tasks.workunit.client.0.smithi090.stdout:4/806: dwrite d4/d27/d28/d38/d64/fa3 [0,4194304] 0 2024-03-11T19:45:29.754 INFO:tasks.workunit.client.0.smithi090.stdout:5/737: write d1/d17/d39/d51/d53/d7b/fa3 [1683374,41909] 0 2024-03-11T19:45:29.802 INFO:tasks.workunit.client.0.smithi090.stdout:6/800: write d4/d13/d22/d3a/f54 [7400476,37963] 0 2024-03-11T19:45:29.974 INFO:tasks.workunit.client.0.smithi090.stdout:5/738: chown d1/d17/d39/d51/d73 107227 1 2024-03-11T19:45:30.058 INFO:tasks.workunit.client.0.smithi090.stdout:3/706: creat d2/db/d15/d1a/d75/d8c/fdc x:0 0 0 2024-03-11T19:45:30.165 INFO:tasks.workunit.client.0.smithi090.stdout:9/759: dwrite d1/dd/d18/f65 [0,4194304] 0 2024-03-11T19:45:30.331 INFO:tasks.workunit.client.0.smithi090.stdout:1/755: symlink d5/da/d23/lfb 0 2024-03-11T19:45:30.717 INFO:tasks.workunit.client.0.smithi090.stdout:0/772: write d2/d11/d15/d29/d43/f98 [1323484,10090] 0 2024-03-11T19:45:31.217 INFO:tasks.workunit.client.0.smithi090.stdout:4/807: truncate d4/d27/d28/d38/d64/f102 2111749 0 2024-03-11T19:45:31.261 INFO:tasks.workunit.client.0.smithi090.stdout:8/813: dwrite d0/d3/d86/d93/fe3 [0,4194304] 0 2024-03-11T19:45:31.478 INFO:tasks.workunit.client.0.smithi090.stdout:1/756: mkdir d5/d31/d4c/daf/dfc 0 2024-03-11T19:45:31.581 INFO:tasks.workunit.client.0.smithi090.stdout:2/747: dwrite d0/fae [0,4194304] 0 2024-03-11T19:45:31.922 INFO:tasks.workunit.client.0.smithi090.stdout:3/707: mkdir d2/db/d15/d69/da2/dc7/ddd 0 2024-03-11T19:45:32.185 INFO:tasks.workunit.client.0.smithi090.stdout:9/760: creat d1/da9/ffa x:0 0 0 2024-03-11T19:45:32.450 INFO:tasks.workunit.client.0.smithi090.stdout:1/757: unlink d5/d31/d4c/daf/lfa 0 2024-03-11T19:45:32.475 INFO:tasks.workunit.client.0.smithi090.stdout:1/758: readlink d5/da/d1c/l2f 0 2024-03-11T19:45:32.616 INFO:tasks.workunit.client.0.smithi090.stdout:2/748: unlink d0/de/db6/d29/f6d 0 2024-03-11T19:45:32.986 INFO:tasks.workunit.client.0.smithi090.stdout:8/814: symlink d0/d3/d4c/l111 0 2024-03-11T19:45:33.026 INFO:tasks.workunit.client.0.smithi090.stdout:8/815: dread - d0/d3/d86/d93/fc1 zero size 2024-03-11T19:45:33.341 INFO:tasks.workunit.client.0.smithi090.stdout:7/746: dwrite d3/dc/d1e/d7c/d96/f9e [0,4194304] 0 2024-03-11T19:45:33.454 INFO:tasks.workunit.client.0.smithi090.stdout:5/739: mknod d1/cfd 0 2024-03-11T19:45:33.513 INFO:tasks.workunit.client.0.smithi090.stdout:0/773: creat d2/d11/ff7 x:0 0 0 2024-03-11T19:45:33.551 INFO:tasks.workunit.client.0.smithi090.stdout:7/747: stat d3/dc/d1e/l8a 0 2024-03-11T19:45:33.599 INFO:tasks.workunit.client.0.smithi090.stdout:7/748: read d3/dc/d1e/d2a/f66 [2221783,38098] 0 2024-03-11T19:45:33.762 INFO:tasks.workunit.client.0.smithi090.stdout:9/761: symlink d1/dd/d18/d3a/db6/de6/lfb 0 2024-03-11T19:45:33.806 INFO:tasks.workunit.client.0.smithi090.stdout:0/774: sync 2024-03-11T19:45:33.843 INFO:tasks.workunit.client.0.smithi090.stdout:0/775: dread - d2/d11/d3a/d64/dac/fca zero size 2024-03-11T19:45:33.934 INFO:tasks.workunit.client.0.smithi090.stdout:8/816: fdatasync d0/d43/f61 0 2024-03-11T19:45:33.978 INFO:tasks.workunit.client.0.smithi090.stdout:8/817: chown d0/d43/dc/d90/c97 236855704 1 2024-03-11T19:45:34.097 INFO:tasks.workunit.client.0.smithi090.stdout:6/801: write d4/d13/d22/d34/d40/dd5/f5f [389980,79536] 0 2024-03-11T19:45:34.927 INFO:tasks.workunit.client.0.smithi090.stdout:5/740: fsync d1/d26/db5/ffa 0 2024-03-11T19:45:35.036 INFO:tasks.workunit.client.0.smithi090.stdout:6/802: symlink d4/d13/d64/l10d 0 2024-03-11T19:45:35.114 INFO:tasks.workunit.client.0.smithi090.stdout:8/818: dread d0/d3/d4c/d8d/f94 [0,4194304] 0 2024-03-11T19:45:35.130 INFO:tasks.workunit.client.0.smithi090.stdout:6/803: readlink d4/d13/d22/l4c 0 2024-03-11T19:45:35.161 INFO:tasks.workunit.client.0.smithi090.stdout:4/808: rename d4/l13 to d4/d10e/l10f 0 2024-03-11T19:45:35.198 INFO:tasks.workunit.client.0.smithi090.stdout:6/804: chown d4/c2c 10653421 1 2024-03-11T19:45:35.951 INFO:tasks.workunit.client.0.smithi090.stdout:8/819: fsync d0/d43/f64 0 2024-03-11T19:45:35.984 INFO:tasks.workunit.client.0.smithi090.stdout:5/741: read d1/d26/d4d/f55 [3515441,46235] 0 2024-03-11T19:45:36.044 INFO:tasks.workunit.client.0.smithi090.stdout:0/776: truncate d2/d11/d15/d29/fb9 1164644 0 2024-03-11T19:45:36.189 INFO:tasks.workunit.client.0.smithi090.stdout:2/749: getdents d0/de/d12/d46/d4c 0 2024-03-11T19:45:36.298 INFO:tasks.workunit.client.0.smithi090.stdout:6/805: readlink d4/d13/d22/lca 0 2024-03-11T19:45:37.057 INFO:tasks.workunit.client.0.smithi090.stdout:8/820: creat d0/d43/dc/da1/f112 x:0 0 0 2024-03-11T19:45:37.112 INFO:tasks.workunit.client.0.smithi090.stdout:8/821: dread d0/d3/d86/d93/dda/ff6 [0,4194304] 0 2024-03-11T19:45:37.189 INFO:tasks.workunit.client.0.smithi090.stdout:5/742: symlink d1/d17/d39/df2/df9/lfe 0 2024-03-11T19:45:37.201 INFO:tasks.workunit.client.0.smithi090.stdout:1/759: write d5/da/d23/f2a [3101681,66702] 0 2024-03-11T19:45:37.246 INFO:tasks.workunit.client.0.smithi090.stdout:5/743: chown d1/d26/l47 1219 1 2024-03-11T19:45:37.845 INFO:tasks.workunit.client.0.smithi090.stdout:3/708: dwrite d2/d45/faa [0,4194304] 0 2024-03-11T19:45:38.280 INFO:tasks.workunit.client.0.smithi090.stdout:5/744: rmdir d1/d75 39 2024-03-11T19:45:38.284 INFO:tasks.workunit.client.0.smithi090.stdout:1/760: fdatasync d5/da/d1c/d3b/d40/fd8 0 2024-03-11T19:45:38.393 INFO:tasks.workunit.client.0.smithi090.stdout:2/750: mknod d0/de/d12/dc8/ce7 0 2024-03-11T19:45:38.496 INFO:tasks.workunit.client.0.smithi090.stdout:9/762: write d1/dd/df/d1c/fad [646709,16149] 0 2024-03-11T19:45:38.722 INFO:tasks.workunit.client.0.smithi090.stdout:7/749: dwrite d3/dc/d1e/ff [4194304,4194304] 0 2024-03-11T19:45:39.286 INFO:tasks.workunit.client.0.smithi090.stdout:3/709: rename d2/db/d15/d1a/d75/dc6/d47 to d2/db/d15/d1a/dde 0 2024-03-11T19:45:40.319 INFO:tasks.workunit.client.0.smithi090.stdout:4/809: write d4/f12 [5810136,40990] 0 2024-03-11T19:45:40.386 INFO:tasks.workunit.client.0.smithi090.stdout:7/750: chown d3/dc/d1e/d7c/d96/d32/d45/ld4 23 1 2024-03-11T19:45:40.403 INFO:tasks.workunit.client.0.smithi090.stdout:4/810: chown d4/d51/d8c/d6a/d8d/ff2 623844214 1 2024-03-11T19:45:40.733 INFO:tasks.workunit.client.0.smithi090.stdout:3/710: creat d2/db/d15/d1a/dde/d6a/fdf x:0 0 0 2024-03-11T19:45:41.379 INFO:tasks.workunit.client.0.smithi090.stdout:6/806: getdents d4/d13/d58 0 2024-03-11T19:45:41.513 INFO:tasks.workunit.client.0.smithi090.stdout:0/777: truncate d2/d11/d15/d29/fb9 1951479 0 2024-03-11T19:45:41.862 INFO:tasks.workunit.client.0.smithi090.stdout:4/811: rename d4/dd8/d66/da1/lf0 to d4/d10e/l110 0 2024-03-11T19:45:42.455 INFO:tasks.workunit.client.0.smithi090.stdout:8/822: dwrite d0/d3/d86/d93/dba/fdf [0,4194304] 0 2024-03-11T19:45:42.562 INFO:tasks.workunit.client.0.smithi090.stdout:0/778: mkdir d2/d11/d15/d29/d43/df8 0 2024-03-11T19:45:42.751 INFO:tasks.workunit.client.0.smithi090.stdout:4/812: unlink d4/d18/l26 0 2024-03-11T19:45:43.220 INFO:tasks.workunit.client.0.smithi090.stdout:0/779: dread d2/d11/d6e/fb6 [0,4194304] 0 2024-03-11T19:45:43.295 INFO:tasks.workunit.client.0.smithi090.stdout:8/823: mkdir d0/d3/d4c/d8d/df8/d113 0 2024-03-11T19:45:43.335 INFO:tasks.workunit.client.0.smithi090.stdout:8/824: chown d0/d3/d4c/d8d/ff4 276501 1 2024-03-11T19:45:43.366 INFO:tasks.workunit.client.0.smithi090.stdout:8/825: read d0/d53/d59/f72 [56229,38203] 0 2024-03-11T19:45:43.659 INFO:tasks.workunit.client.0.smithi090.stdout:4/813: mkdir d4/d27/d28/d69/d111 0 2024-03-11T19:45:43.877 INFO:tasks.workunit.client.0.smithi090.stdout:6/807: symlink d4/d14/d107/l10e 0 2024-03-11T19:45:43.895 INFO:tasks.workunit.client.0.smithi090.stdout:5/745: write d1/d26/fd6 [124668,58322] 0 2024-03-11T19:45:44.086 INFO:tasks.workunit.client.0.smithi090.stdout:8/826: fdatasync d0/d3/f68 0 2024-03-11T19:45:44.282 INFO:tasks.workunit.client.0.smithi090.stdout:3/711: link d2/de/cab d2/d45/ce0 0 2024-03-11T19:45:44.699 INFO:tasks.workunit.client.0.smithi090.stdout:0/780: mkdir d2/d11/d15/d29/d2e/db4/daf/df9 0 2024-03-11T19:45:44.970 INFO:tasks.workunit.client.0.smithi090.stdout:6/808: mknod d4/d13/d22/d34/d40/da4/c10f 0 2024-03-11T19:45:44.993 INFO:tasks.workunit.client.0.smithi090.stdout:5/746: mkdir d1/d17/d39/d51/d53/d7b/d3b/d76/d92/dc1/dff 0 2024-03-11T19:45:45.026 INFO:tasks.workunit.client.0.smithi090.stdout:0/781: sync 2024-03-11T19:45:45.391 INFO:tasks.workunit.client.0.smithi090.stdout:6/809: unlink d4/d13/d22/d34/d83/ffd 0 2024-03-11T19:45:46.113 INFO:tasks.workunit.client.0.smithi090.stdout:1/761: dwrite d5/da/d1c/f39 [0,4194304] 0 2024-03-11T19:45:46.133 INFO:tasks.workunit.client.0.smithi090.stdout:5/747: mkdir d1/d17/d39/d51/d73/df6/d100 0 2024-03-11T19:45:46.423 INFO:tasks.workunit.client.0.smithi090.stdout:2/751: dwrite d0/de/d12/d81/fbc [0,4194304] 0 2024-03-11T19:45:46.558 INFO:tasks.workunit.client.0.smithi090.stdout:2/752: sync 2024-03-11T19:45:46.702 INFO:tasks.workunit.client.0.smithi090.stdout:1/762: fsync d5/da/d23/fbf 0 2024-03-11T19:45:46.728 INFO:tasks.workunit.client.0.smithi090.stdout:9/763: dwrite d1/dd/df/d1c/d77/fc4 [0,4194304] 0 2024-03-11T19:45:47.096 INFO:tasks.workunit.client.0.smithi090.stdout:7/751: dwrite d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/db5/de1/dc2/f65 [0,4194304] 0 2024-03-11T19:45:47.270 INFO:tasks.workunit.client.0.smithi090.stdout:3/712: getdents d2/d45 0 2024-03-11T19:45:47.303 INFO:tasks.workunit.client.0.smithi090.stdout:1/763: chown d5/da/d1c/d3b/d40/d89/l8f 562 1 2024-03-11T19:45:47.464 INFO:tasks.workunit.client.0.smithi090.stdout:4/814: write d4/d51/d8c/fd1 [538721,18456] 0 2024-03-11T19:45:47.631 INFO:tasks.workunit.client.0.smithi090.stdout:2/753: dread - d0/de/db6/d25/d8c/fd8 zero size 2024-03-11T19:45:47.738 INFO:tasks.workunit.client.0.smithi090.stdout:5/748: fsync d1/d17/d39/d51/d53/d7b/d3b/d76/db7/fe9 0 2024-03-11T19:45:47.751 INFO:tasks.workunit.client.0.smithi090.stdout:2/754: sync 2024-03-11T19:45:47.924 INFO:tasks.workunit.client.0.smithi090.stdout:9/764: symlink d1/dd/d18/d3a/d70/lfc 0 2024-03-11T19:45:48.047 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:45:48.050 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:45:48.113 INFO:tasks.workunit.client.0.smithi090.stdout:7/752: dread - d3/dc/d1e/ff8 zero size 2024-03-11T19:45:48.338 INFO:tasks.workunit.client.0.smithi090.stdout:4/815: mkdir d4/dd8/d66/da1/d112 0 2024-03-11T19:45:48.431 INFO:tasks.workunit.client.0.smithi090.stdout:1/764: dread d5/da/d1c/fdb [0,4194304] 0 2024-03-11T19:45:48.519 INFO:tasks.workunit.client.0.smithi090.stdout:0/782: write d2/d11/d15/d36/f57 [5064800,28154] 0 2024-03-11T19:45:48.586 INFO:tasks.workunit.client.0.smithi090.stdout:5/749: creat d1/d17/d39/d51/d53/d7b/d3b/d76/db7/ddf/f101 x:0 0 0 2024-03-11T19:45:49.025 INFO:tasks.workunit.client.0.smithi090.stdout:9/765: mknod d1/da9/cfd 0 2024-03-11T19:45:49.178 INFO:tasks.workunit.client.0.smithi090.stdout:6/810: write d4/d13/d22/d34/d40/dd5/f26 [3342881,99362] 0 2024-03-11T19:45:49.233 INFO:tasks.workunit.client.0.smithi090.stdout:4/816: dread d4/d27/d28/d38/d75/fa9 [0,4194304] 0 2024-03-11T19:45:49.286 INFO:tasks.workunit.client.0.smithi090.stdout:8/827: dwrite d0/f58 [0,4194304] 0 2024-03-11T19:45:49.423 INFO:tasks.workunit.client.0.smithi090.stdout:2/755: mkdir d0/de/d12/d46/de8 0 2024-03-11T19:45:49.445 INFO:tasks.workunit.client.0.smithi090.stdout:5/750: fsync d1/d26/f52 0 2024-03-11T19:45:49.492 INFO:tasks.workunit.client.0.smithi090.stdout:4/817: chown d4/dd8/dfa/cda 51 1 2024-03-11T19:45:49.893 INFO:tasks.workunit.client.0.smithi090.stdout:0/783: creat d2/d11/d15/d29/d2e/db4/daf/ffa x:0 0 0 2024-03-11T19:45:50.386 INFO:tasks.workunit.client.0.smithi090.stdout:1/765: symlink d5/da/d1c/d3b/d40/lfd 0 2024-03-11T19:45:50.455 INFO:tasks.workunit.client.0.smithi090.stdout:8/828: read d0/f17 [32140,121162] 0 2024-03-11T19:45:50.711 INFO:tasks.workunit.client.0.smithi090.stdout:5/751: dread d1/f4f [0,4194304] 0 2024-03-11T19:45:50.874 INFO:tasks.workunit.client.0.smithi090.stdout:7/753: rmdir d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/de4 0 2024-03-11T19:45:51.198 INFO:tasks.workunit.client.0.smithi090.stdout:2/756: dread d0/fae [0,4194304] 0 2024-03-11T19:45:51.324 INFO:tasks.workunit.client.0.smithi090.stdout:4/818: dread d4/dd8/db5/feb [0,4194304] 0 2024-03-11T19:45:51.780 INFO:tasks.workunit.client.0.smithi090.stdout:0/784: symlink d2/d11/d15/dcc/lfb 0 2024-03-11T19:45:52.051 INFO:tasks.workunit.client.0.smithi090.stdout:6/811: creat d4/d14/f110 x:0 0 0 2024-03-11T19:45:52.085 INFO:tasks.workunit.client.0.smithi090.stdout:2/757: read - d0/de/db6/d25/fc7 zero size 2024-03-11T19:45:52.325 INFO:tasks.workunit.client.0.smithi090.stdout:8/829: truncate d0/d3/d86/fd6 269493 0 2024-03-11T19:45:52.490 INFO:tasks.workunit.client.0.smithi090.stdout:3/713: dwrite d2/db/d15/d1a/dde/d6a/f73 [0,4194304] 0 2024-03-11T19:45:52.605 INFO:tasks.workunit.client.0.smithi090.stdout:5/752: fsync d1/d17/fe4 0 2024-03-11T19:45:52.656 INFO:tasks.workunit.client.0.smithi090.stdout:2/758: creat d0/de/d12/d46/d4c/fe9 x:0 0 0 2024-03-11T19:45:53.392 INFO:tasks.workunit.client.0.smithi090.stdout:6/812: creat d4/d14/d8a/d74/f111 x:0 0 0 2024-03-11T19:45:53.402 INFO:tasks.workunit.client.0.smithi090.stdout:3/714: truncate d2/db/d15/d1a/d75/dc6/f27 1851214 0 2024-03-11T19:45:53.492 INFO:tasks.workunit.client.0.smithi090.stdout:1/766: rename d5/da/d1c/d3b/d40 to d5/da/d23/dfe 0 2024-03-11T19:45:53.525 INFO:tasks.workunit.client.0.smithi090.stdout:2/759: truncate d0/de/d12/d46/fa8 468348 0 2024-03-11T19:45:53.761 INFO:tasks.workunit.client.0.smithi090.stdout:9/766: dwrite d1/dd/d18/d32/f34 [0,4194304] 0 2024-03-11T19:45:53.971 INFO:tasks.workunit.client.0.smithi090.stdout:8/830: truncate d0/d3/d4c/f10c 2936246 0 2024-03-11T19:45:54.220 INFO:tasks.workunit.client.0.smithi090.stdout:1/767: dread d5/d31/d36/f49 [0,4194304] 0 2024-03-11T19:45:54.406 INFO:tasks.workunit.client.0.smithi090.stdout:3/715: symlink d2/db/d15/d69/da2/le1 0 2024-03-11T19:45:54.933 INFO:tasks.workunit.client.0.smithi090.stdout:8/831: creat d0/d43/dc/f114 x:0 0 0 2024-03-11T19:45:55.302 INFO:tasks.workunit.client.0.smithi090.stdout:5/753: link d1/d17/d39/d51/d53/lda d1/d17/d39/d51/d53/d7b/d3b/d76/db7/ddf/dee/l102 0 2024-03-11T19:45:55.402 INFO:tasks.workunit.client.0.smithi090.stdout:3/716: read d2/db/f62 [551143,115345] 0 2024-03-11T19:45:55.479 INFO:tasks.workunit.client.0.smithi090.stdout:7/754: dwrite d3/f7 [4194304,4194304] 0 2024-03-11T19:45:55.715 INFO:tasks.workunit.client.0.smithi090.stdout:8/832: read d0/d53/d59/d96/fd7 [2282450,106700] 0 2024-03-11T19:45:55.797 INFO:tasks.workunit.client.0.smithi090.stdout:6/813: creat d4/d13/d22/d42/f112 x:0 0 0 2024-03-11T19:45:55.917 INFO:tasks.workunit.client.0.smithi090.stdout:4/819: dwrite d4/d27/d28/d38/d64/f93 [0,4194304] 0 2024-03-11T19:45:56.309 INFO:tasks.workunit.client.0.smithi090.stdout:5/754: dread d1/d17/f30 [0,4194304] 0 2024-03-11T19:45:56.376 INFO:tasks.workunit.client.0.smithi090.stdout:3/717: creat d2/d4f/fe2 x:0 0 0 2024-03-11T19:45:56.764 INFO:tasks.workunit.client.0.smithi090.stdout:7/755: truncate d3/dc/d1e/d7c/d96/d32/d45/de0/fb9 281917 0 2024-03-11T19:45:56.787 INFO:tasks.workunit.client.0.smithi090.stdout:6/814: dread d4/d13/d22/d34/d40/da4/f9a [0,4194304] 0 2024-03-11T19:45:57.654 INFO:tasks.workunit.client.0.smithi090.stdout:6/815: truncate d4/d13/d22/d34/d40/d7d/fa6 884117 0 2024-03-11T19:45:57.907 INFO:tasks.workunit.client.0.smithi090.stdout:3/718: unlink d2/db/d15/d1a/d75/dc6/fc 0 2024-03-11T19:45:58.824 INFO:tasks.workunit.client.0.smithi090.stdout:6/816: dread d4/d13/d22/d34/d40/dd5/f3f [0,4194304] 0 2024-03-11T19:45:59.034 INFO:tasks.workunit.client.0.smithi090.stdout:8/833: link d0/cf0 d0/d53/d59/d96/de5/c115 0 2024-03-11T19:45:59.062 INFO:tasks.workunit.client.0.smithi090.stdout:9/767: dwrite d1/dd/df/d1c/d9f/f8d [0,4194304] 0 2024-03-11T19:45:59.068 INFO:tasks.workunit.client.0.smithi090.stdout:1/768: write d5/da/d1c/d6d/fec [846017,84606] 0 2024-03-11T19:45:59.300 INFO:tasks.workunit.client.0.smithi090.stdout:8/834: chown d0/d3/d86/cf7 4 1 2024-03-11T19:45:59.520 INFO:tasks.workunit.client.0.smithi090.stdout:4/820: write d4/d51/dbf/fe9 [407855,123424] 0 2024-03-11T19:45:59.601 INFO:tasks.workunit.client.0.smithi090.stdout:2/760: dwrite d0/de/d12/d46/d75/f7c [0,4194304] 0 2024-03-11T19:45:59.917 INFO:tasks.workunit.client.0.smithi090.stdout:9/768: dread d1/dd/f69 [0,4194304] 0 2024-03-11T19:45:59.958 INFO:tasks.workunit.client.0.smithi090.stdout:4/821: chown d4/d51/d8c/d6a/d100 15 1 2024-03-11T19:46:00.032 INFO:tasks.workunit.client.0.smithi090.stdout:0/785: rename d2/d11/d15/d29/dde to d2/d11/d15/d29/d3d/d7e/dfc 0 2024-03-11T19:46:00.059 INFO:tasks.workunit.client.0.smithi090.stdout:1/769: unlink d5/da/d1c/d6d/d80/fdd 0 2024-03-11T19:46:00.255 INFO:tasks.workunit.client.0.smithi090.stdout:3/719: getdents d2/d4f/d59/da1 0 2024-03-11T19:46:00.271 INFO:tasks.workunit.client.0.smithi090.stdout:2/761: dread - d0/de/d12/d46/d5d/d63/f88 zero size 2024-03-11T19:46:00.322 INFO:tasks.workunit.client.0.smithi090.stdout:8/835: rmdir d0/d53/d59/d96/de5/de7 39 2024-03-11T19:46:00.585 INFO:tasks.workunit.client.0.smithi090.stdout:6/817: dread d4/d13/d22/d42/f72 [0,4194304] 0 2024-03-11T19:46:00.637 INFO:tasks.workunit.client.0.smithi090.stdout:4/822: mkdir d4/d10e/d113 0 2024-03-11T19:46:00.709 INFO:tasks.workunit.client.0.smithi090.stdout:6/818: chown d4 193911 1 2024-03-11T19:46:00.795 INFO:tasks.workunit.client.0.smithi090.stdout:5/755: dwrite d1/d17/d39/d51/d53/f67 [0,4194304] 0 2024-03-11T19:46:00.952 INFO:tasks.workunit.client.0.smithi090.stdout:1/770: creat d5/d31/d76/d77/fff x:0 0 0 2024-03-11T19:46:01.059 INFO:tasks.workunit.client.0.smithi090.stdout:3/720: rmdir d2/db/d15/d1a/d75/dc6 39 2024-03-11T19:46:01.095 INFO:tasks.workunit.client.0.smithi090.stdout:7/756: dwrite d3/dc/d29/f6b [0,4194304] 0 2024-03-11T19:46:01.608 INFO:tasks.workunit.client.0.smithi090.stdout:4/823: creat d4/d27/f114 x:0 0 0 2024-03-11T19:46:02.098 INFO:tasks.workunit.client.0.smithi090.stdout:8/836: rename d0/d103 to d0/d3/d3e/d85/d116 0 2024-03-11T19:46:02.110 INFO:tasks.workunit.client.0.smithi090.stdout:2/762: truncate d0/de/d1f/fe4 182437 0 2024-03-11T19:46:02.725 INFO:tasks.workunit.client.0.smithi090.stdout:1/771: mknod d5/d4e/d54/d94/c100 0 2024-03-11T19:46:03.119 INFO:tasks.workunit.client.0.smithi090.stdout:3/721: dread d2/db/d15/d1a/f3a [0,4194304] 0 2024-03-11T19:46:03.238 INFO:tasks.workunit.client.0.smithi090.stdout:4/824: unlink d4/d27/d28/d69/l71 0 2024-03-11T19:46:03.530 INFO:tasks.workunit.client.0.smithi090.stdout:2/763: truncate d0/de/db6/d25/fc7 1047754 0 2024-03-11T19:46:03.752 INFO:tasks.workunit.client.0.smithi090.stdout:1/772: truncate d5/da/d1c/d3b/fd0 542219 0 2024-03-11T19:46:03.996 INFO:tasks.workunit.client.0.smithi090.stdout:1/773: chown c4 15788 1 2024-03-11T19:46:04.041 INFO:tasks.workunit.client.0.smithi090.stdout:8/837: dread d0/d3/d4c/f5e [0,4194304] 0 2024-03-11T19:46:04.092 INFO:tasks.workunit.client.0.smithi090.stdout:9/769: dwrite d1/d1d/f2b [0,4194304] 0 2024-03-11T19:46:04.208 INFO:tasks.workunit.client.0.smithi090.stdout:4/825: rmdir d4/dd8/dfa/db7 39 2024-03-11T19:46:04.222 INFO:tasks.workunit.client.0.smithi090.stdout:3/722: read d2/db/d15/d1a/d75/dc6/f77 [1339951,64473] 0 2024-03-11T19:46:04.368 INFO:tasks.workunit.client.0.smithi090.stdout:2/764: dread - d0/de/d12/fd0 zero size 2024-03-11T19:46:04.707 INFO:tasks.workunit.client.0.smithi090.stdout:7/757: rename d3/dc/d1e/d7c/d96/d32/d45/l5c to d3/dc/l106 0 2024-03-11T19:46:04.736 INFO:tasks.workunit.client.0.smithi090.stdout:0/786: write d2/d11/d15/d29/d2e/d6d/da4/fa2 [5052163,116699] 0 2024-03-11T19:46:05.672 INFO:tasks.workunit.client.0.smithi090.stdout:4/826: dread d4/d51/d8c/d89/fe4 [0,4194304] 0 2024-03-11T19:46:05.817 INFO:tasks.workunit.client.0.smithi090.stdout:5/756: write d1/d17/d39/d51/d53/d7b/d3b/d76/db7/de0/fe2 [155717,127561] 0 2024-03-11T19:46:06.050 INFO:tasks.workunit.client.0.smithi090.stdout:1/774: getdents d5/d65/d6b/de7 0 2024-03-11T19:46:06.604 INFO:tasks.workunit.client.0.smithi090.stdout:6/819: dwrite d4/d13/d22/d34/d40/fa1 [0,4194304] 0 2024-03-11T19:46:06.755 INFO:tasks.workunit.client.0.smithi090.stdout:6/820: chown d4/d13/d22/d42/cf4 10186 1 2024-03-11T19:46:06.774 INFO:tasks.workunit.client.0.smithi090.stdout:6/821: chown d4/d13/d22/l4c 147 1 2024-03-11T19:46:06.901 INFO:tasks.workunit.client.0.smithi090.stdout:4/827: truncate d4/f2e 4231495 0 2024-03-11T19:46:07.413 INFO:tasks.workunit.client.0.smithi090.stdout:1/775: rename d5/d65/d6b/d7f/l85 to d5/d65/d6b/de7/l101 0 2024-03-11T19:46:07.494 INFO:tasks.workunit.client.0.smithi090.stdout:5/757: dread d1/d17/d39/d51/d73/fb0 [0,4194304] 0 2024-03-11T19:46:07.839 INFO:tasks.workunit.client.0.smithi090.stdout:9/770: write d1/dd/df/d1c/d21/d2a/f3d [1366626,65935] 0 2024-03-11T19:46:07.997 INFO:tasks.workunit.client.0.smithi090.stdout:3/723: write d2/db/d15/f17 [80984,79071] 0 2024-03-11T19:46:08.138 INFO:tasks.workunit.client.0.smithi090.stdout:8/838: dwrite d0/d3/d86/d93/dba/fcf [0,4194304] 0 2024-03-11T19:46:08.268 INFO:tasks.workunit.client.0.smithi090.stdout:0/787: write d2/d11/d6e/fb6 [692145,91053] 0 2024-03-11T19:46:08.338 INFO:tasks.workunit.client.0.smithi090.stdout:8/839: readlink d0/d3/d4c/d8d/lb2 0 2024-03-11T19:46:08.422 INFO:tasks.workunit.client.0.smithi090.stdout:7/758: write d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/ffd [727521,28537] 0 2024-03-11T19:46:08.901 INFO:tasks.workunit.client.0.smithi090.stdout:5/758: dread d1/fa7 [0,4194304] 0 2024-03-11T19:46:09.208 INFO:tasks.workunit.client.0.smithi090.stdout:4/828: mknod d4/d51/d8c/d6a/d100/c115 0 2024-03-11T19:46:09.246 INFO:tasks.workunit.client.0.smithi090.stdout:3/724: read d2/f67 [475141,127098] 0 2024-03-11T19:46:09.621 INFO:tasks.workunit.client.0.smithi090.stdout:1/776: fsync d5/da/d1c/d3b/fb3 0 2024-03-11T19:46:09.839 INFO:tasks.workunit.client.0.smithi090.stdout:5/759: creat d1/d26/d19/f103 x:0 0 0 2024-03-11T19:46:10.057 INFO:tasks.workunit.client.0.smithi090.stdout:2/765: dwrite d0/de/d12/f1d [0,4194304] 0 2024-03-11T19:46:10.171 INFO:tasks.workunit.client.0.smithi090.stdout:4/829: fdatasync d4/d27/f34 0 2024-03-11T19:46:10.213 INFO:tasks.workunit.client.0.smithi090.stdout:2/766: chown d0/de/db6/l69 157582850 1 2024-03-11T19:46:10.307 INFO:tasks.workunit.client.0.smithi090.stdout:3/725: truncate d2/db/d15/d1a/d75/dc6/f3d 903013 0 2024-03-11T19:46:10.649 INFO:tasks.workunit.client.0.smithi090.stdout:6/822: write d4/d13/d22/d34/d40/dd5/f3f [1636472,6331] 0 2024-03-11T19:46:10.950 INFO:tasks.workunit.client.0.smithi090.stdout:5/760: mknod d1/d17/d39/d51/d73/c104 0 2024-03-11T19:46:11.383 INFO:tasks.workunit.client.0.smithi090.stdout:6/823: mknod d4/d13/d58/c113 0 2024-03-11T19:46:11.650 INFO:tasks.workunit.client.0.smithi090.stdout:5/761: mkdir d1/d17/d39/d51/d53/d7b/d3b/d76/d92/d105 0 2024-03-11T19:46:11.784 INFO:tasks.workunit.client.0.smithi090.stdout:7/759: write d3/dc/d1e/d7c/d96/d32/d45/de0/f25 [1478665,118468] 0 2024-03-11T19:46:12.039 INFO:tasks.workunit.client.0.smithi090.stdout:2/767: mkdir d0/de/d12/d46/dea 0 2024-03-11T19:46:12.449 INFO:tasks.workunit.client.0.smithi090.stdout:6/824: creat d4/d13/d22/d34/d40/dd5/d80/dd4/f114 x:0 0 0 2024-03-11T19:46:12.638 INFO:tasks.workunit.client.0.smithi090.stdout:8/840: dwrite d0/d43/dc/d90/fac [0,4194304] 0 2024-03-11T19:46:12.643 INFO:tasks.workunit.client.0.smithi090.stdout:9/771: dwrite d1/dd/f13 [0,4194304] 0 2024-03-11T19:46:12.678 INFO:tasks.workunit.client.0.smithi090.stdout:0/788: dwrite d2/d11/d3a/d64/dac/fca [0,4194304] 0 2024-03-11T19:46:12.895 INFO:tasks.workunit.client.0.smithi090.stdout:5/762: mknod d1/d17/d39/d51/d53/d7b/d3b/c106 0 2024-03-11T19:46:12.966 INFO:tasks.workunit.client.0.smithi090.stdout:5/763: stat d1/d17/d39/df2/df9/lfe 0 2024-03-11T19:46:13.015 INFO:tasks.workunit.client.0.smithi090.stdout:1/777: dwrite d5/da/d23/f2d [0,4194304] 0 2024-03-11T19:46:13.092 INFO:tasks.workunit.client.0.smithi090.stdout:8/841: sync 2024-03-11T19:46:13.243 INFO:tasks.workunit.client.0.smithi090.stdout:3/726: rmdir d2/d45 39 2024-03-11T19:46:13.531 INFO:tasks.workunit.client.0.smithi090.stdout:7/760: creat d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/db5/de1/dc2/d8e/f107 x:0 0 0 2024-03-11T19:46:13.754 INFO:tasks.workunit.client.0.smithi090.stdout:2/768: mkdir d0/de/d12/d46/de8/deb 0 2024-03-11T19:46:14.026 INFO:tasks.workunit.client.0.smithi090.stdout:5/764: rename d1/d26/d4d/f55 to d1/d17/d39/d51/d53/d7b/d3b/d76/d92/dc1/f107 0 2024-03-11T19:46:14.030 INFO:tasks.workunit.client.0.smithi090.stdout:8/842: fsync d0/d43/f2f 0 2024-03-11T19:46:14.376 INFO:tasks.workunit.client.0.smithi090.stdout:7/761: mknod d3/dc/d1e/d7c/d96/d32/d71/d80/c108 0 2024-03-11T19:46:15.191 INFO:tasks.workunit.client.0.smithi090.stdout:0/789: mkdir d2/d11/d3a/d64/dac/dba/dec/dfd 0 2024-03-11T19:46:15.219 INFO:tasks.workunit.client.0.smithi090.stdout:0/790: chown d2/d11/d3a/lb0 4 1 2024-03-11T19:46:15.519 INFO:tasks.workunit.client.0.smithi090.stdout:1/778: symlink d5/da/d97/df6/l102 0 2024-03-11T19:46:15.753 INFO:tasks.workunit.client.0.smithi090.stdout:6/825: write d4/d14/d8a/d74/f111 [605003,91531] 0 2024-03-11T19:46:15.882 INFO:tasks.workunit.client.0.smithi090.stdout:2/769: dread d0/de/d12/d46/d5d/d8d/fa6 [0,4194304] 0 2024-03-11T19:46:16.094 INFO:tasks.workunit.client.0.smithi090.stdout:8/843: truncate d0/f17 897450 0 2024-03-11T19:46:16.134 INFO:tasks.workunit.client.0.smithi090.stdout:8/844: readlink d0/d3/d98/da2/la9 0 2024-03-11T19:46:16.237 INFO:tasks.workunit.client.0.smithi090.stdout:7/762: fdatasync d3/dc/d1e/f46 0 2024-03-11T19:46:16.571 INFO:tasks.workunit.client.0.smithi090.stdout:9/772: write d1/dd/d18/d3a/d70/fd7 [404683,117754] 0 2024-03-11T19:46:16.640 INFO:tasks.workunit.client.0.smithi090.stdout:9/773: write d1/dd/f13 [4359165,36561] 0 2024-03-11T19:46:17.839 INFO:tasks.workunit.client.0.smithi090.stdout:0/791: mkdir d2/d11/d15/d29/de5/dfe 0 2024-03-11T19:46:18.202 INFO:tasks.workunit.client.0.smithi090.stdout:9/774: symlink d1/dd/df/d1c/d9f/lfe 0 2024-03-11T19:46:18.326 INFO:tasks.workunit.client.0.smithi090.stdout:6/826: rename d4/d14/f78 to d4/d13/d22/d34/d83/d9c/db7/f115 0 2024-03-11T19:46:18.425 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:46:18.428 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:46:18.546 INFO:tasks.workunit.client.0.smithi090.stdout:6/827: sync 2024-03-11T19:46:18.622 INFO:tasks.workunit.client.0.smithi090.stdout:3/727: getdents d2/db/d15/d1a/dde/d6a 0 2024-03-11T19:46:19.567 INFO:tasks.workunit.client.0.smithi090.stdout:5/765: getdents d1/d17/d39/df2/df9 0 2024-03-11T19:46:19.786 INFO:tasks.workunit.client.0.smithi090.stdout:9/775: rename d1/dd/df/l4e to d1/dd/d18/d3a/db6/de6/lff 0 2024-03-11T19:46:19.818 INFO:tasks.workunit.client.0.smithi090.stdout:9/776: stat d1/dd/f61 0 2024-03-11T19:46:19.986 INFO:tasks.workunit.client.0.smithi090.stdout:3/728: truncate d2/db/d15/d1a/dde/d6a/dcb/fac 2389384 0 2024-03-11T19:46:20.003 INFO:tasks.workunit.client.0.smithi090.stdout:3/729: chown d2/de 522183099 1 2024-03-11T19:46:20.301 INFO:tasks.workunit.client.0.smithi090.stdout:3/730: sync 2024-03-11T19:46:20.433 INFO:tasks.workunit.client.0.smithi090.stdout:2/770: dwrite d0/de/db6/d25/d8c/fd8 [0,4194304] 0 2024-03-11T19:46:20.660 INFO:tasks.workunit.client.0.smithi090.stdout:1/779: getdents d5/da/dd 0 2024-03-11T19:46:20.699 INFO:tasks.workunit.client.0.smithi090.stdout:4/830: link d4/d18/cd4 d4/d27/c116 0 2024-03-11T19:46:20.899 INFO:tasks.workunit.client.0.smithi090.stdout:5/766: creat d1/d17/d39/d51/d53/d64/f108 x:0 0 0 2024-03-11T19:46:21.008 INFO:tasks.workunit.client.0.smithi090.stdout:8/845: dwrite d0/d53/d59/d96/fde [0,4194304] 0 2024-03-11T19:46:21.023 INFO:tasks.workunit.client.0.smithi090.stdout:9/777: truncate d1/dd/d18/d32/fe3 343901 0 2024-03-11T19:46:21.230 INFO:tasks.workunit.client.0.smithi090.stdout:2/771: chown d0/de/d12/d46/d75/c9f 951 1 2024-03-11T19:46:21.301 INFO:tasks.workunit.client.0.smithi090.stdout:7/763: dwrite d3/dc/d1e/d7c/d96/d32/d71/f77 [0,4194304] 0 2024-03-11T19:46:21.639 INFO:tasks.workunit.client.0.smithi090.stdout:3/731: creat d2/db/d15/d69/da2/dc7/fe3 x:0 0 0 2024-03-11T19:46:22.992 INFO:tasks.workunit.client.0.smithi090.stdout:4/831: mknod d4/d27/d28/d38/d64/c117 0 2024-03-11T19:46:23.375 INFO:tasks.workunit.client.0.smithi090.stdout:2/772: truncate d0/de/d12/d81/f87 1834429 0 2024-03-11T19:46:23.682 INFO:tasks.workunit.client.0.smithi090.stdout:0/792: getdents d2/d11/d15/d29/d43/d58 0 2024-03-11T19:46:23.788 INFO:tasks.workunit.client.0.smithi090.stdout:3/732: creat d2/db/d15/d69/dad/fe4 x:0 0 0 2024-03-11T19:46:24.164 INFO:tasks.workunit.client.0.smithi090.stdout:2/773: dwrite d0/de/db6/d25/d8c/fd8 [4194304,4194304] 0 2024-03-11T19:46:24.278 INFO:tasks.workunit.client.0.smithi090.stdout:6/828: link d4/d13/d22/d34/d40/dd5/d80/f10c d4/f116 0 2024-03-11T19:46:25.145 INFO:tasks.workunit.client.0.smithi090.stdout:3/733: stat d2/fb9 0 2024-03-11T19:46:25.589 INFO:tasks.workunit.client.0.smithi090.stdout:6/829: unlink d4/d14/d8a/d74/fba 0 2024-03-11T19:46:25.977 INFO:tasks.workunit.client.0.smithi090.stdout:5/767: dwrite d1/f2c [0,4194304] 0 2024-03-11T19:46:26.111 INFO:tasks.workunit.client.0.smithi090.stdout:6/830: truncate d4/d13/d22/d34/d40/da4/fe7 3831219 0 2024-03-11T19:46:26.148 INFO:tasks.workunit.client.0.smithi090.stdout:3/734: fsync d2/f9a 0 2024-03-11T19:46:26.155 INFO:tasks.workunit.client.0.smithi090.stdout:8/846: write d0/d3/d3e/d85/fcd [3509376,86119] 0 2024-03-11T19:46:26.458 INFO:tasks.workunit.client.0.smithi090.stdout:6/831: chown d4/d13/d22/d34/d40/dd5/d80/ddb 212475 1 2024-03-11T19:46:26.519 INFO:tasks.workunit.client.0.smithi090.stdout:9/778: dwrite d1/dd/d18/d3a/d48/fbc [0,4194304] 0 2024-03-11T19:46:26.538 INFO:tasks.workunit.client.0.smithi090.stdout:5/768: fdatasync d1/d17/d39/d51/d53/d64/f93 0 2024-03-11T19:46:26.708 INFO:tasks.workunit.client.0.smithi090.stdout:3/735: read - d2/d4f/fe2 zero size 2024-03-11T19:46:27.488 INFO:tasks.workunit.client.0.smithi090.stdout:3/736: symlink d2/db/d15/le5 0 2024-03-11T19:46:27.600 INFO:tasks.workunit.client.0.smithi090.stdout:5/769: truncate d1/d17/d39/d51/d53/d7b/d3b/d76/f77 733463 0 2024-03-11T19:46:27.809 INFO:tasks.workunit.client.0.smithi090.stdout:1/780: write d5/d31/d4c/f73 [427683,124309] 0 2024-03-11T19:46:27.861 INFO:tasks.workunit.client.0.smithi090.stdout:6/832: dread d4/d13/d22/d34/d40/da4/fd8 [4194304,4194304] 0 2024-03-11T19:46:28.150 INFO:tasks.workunit.client.0.smithi090.stdout:4/832: write d4/dd8/fa7 [2593619,77673] 0 2024-03-11T19:46:28.609 INFO:tasks.workunit.client.0.smithi090.stdout:0/793: write d2/f14 [885158,94889] 0 2024-03-11T19:46:28.699 INFO:tasks.workunit.client.0.smithi090.stdout:4/833: creat d4/dd8/db5/f118 x:0 0 0 2024-03-11T19:46:28.795 INFO:tasks.workunit.client.0.smithi090.stdout:7/764: dwrite d3/dc/d1e/d2a/fcc [0,4194304] 0 2024-03-11T19:46:28.899 INFO:tasks.workunit.client.0.smithi090.stdout:0/794: sync 2024-03-11T19:46:28.933 INFO:tasks.workunit.client.0.smithi090.stdout:2/774: write d0/de/db6/d29/fa4 [1479605,110881] 0 2024-03-11T19:46:29.083 INFO:tasks.workunit.client.0.smithi090.stdout:7/765: chown d3/dc/d1e/d7c/d96/d32/d45/cbf 217077677 1 2024-03-11T19:46:29.127 INFO:tasks.workunit.client.0.smithi090.stdout:6/833: mknod d4/d13/d22/d34/d37/d7f/c117 0 2024-03-11T19:46:29.539 INFO:tasks.workunit.client.0.smithi090.stdout:4/834: mkdir d4/d27/d28/d38/d64/d119 0 2024-03-11T19:46:29.653 INFO:tasks.workunit.client.0.smithi090.stdout:2/775: symlink d0/d21/lec 0 2024-03-11T19:46:29.694 INFO:tasks.workunit.client.0.smithi090.stdout:2/776: stat d0/de/db6/d25/d8c/fb8 0 2024-03-11T19:46:30.055 INFO:tasks.workunit.client.0.smithi090.stdout:5/770: getdents d1/d17/d39/d51/d53/d7b/d3b/d76/db7/de0 0 2024-03-11T19:46:30.129 INFO:tasks.workunit.client.0.smithi090.stdout:4/835: fsync d4/dd8/d66/dcf/f108 0 2024-03-11T19:46:30.175 INFO:tasks.workunit.client.0.smithi090.stdout:4/836: chown d4/d51/d8c/d6a/cc8 47 1 2024-03-11T19:46:30.442 INFO:tasks.workunit.client.0.smithi090.stdout:8/847: dwrite d0/d3/d98/da2/fa4 [0,4194304] 0 2024-03-11T19:46:30.603 INFO:tasks.workunit.client.0.smithi090.stdout:8/848: chown d0/d53/d59/d82/la0 48 1 2024-03-11T19:46:30.640 INFO:tasks.workunit.client.0.smithi090.stdout:5/771: chown d1/c9 10157 1 2024-03-11T19:46:30.658 INFO:tasks.workunit.client.0.smithi090.stdout:5/772: readlink d1/l15 0 2024-03-11T19:46:30.923 INFO:tasks.workunit.client.0.smithi090.stdout:2/777: symlink d0/led 0 2024-03-11T19:46:31.319 INFO:tasks.workunit.client.0.smithi090.stdout:5/773: symlink d1/d17/d39/d51/d53/d7b/d3b/d76/db7/l109 0 2024-03-11T19:46:31.799 INFO:tasks.workunit.client.0.smithi090.stdout:6/834: link d4/d13/d22/d42/d48/fc5 d4/d13/d22/d34/d83/d8d/f118 0 2024-03-11T19:46:31.997 INFO:tasks.workunit.client.0.smithi090.stdout:5/774: mkdir d1/d17/d39/d51/d73/da6/d10a 0 2024-03-11T19:46:32.032 INFO:tasks.workunit.client.0.smithi090.stdout:9/779: write d1/dd/d18/d32/fe3 [237441,70155] 0 2024-03-11T19:46:32.163 INFO:tasks.workunit.client.0.smithi090.stdout:3/737: mknod d2/db/d15/d1a/ce6 0 2024-03-11T19:46:32.208 INFO:tasks.workunit.client.0.smithi090.stdout:9/780: sync 2024-03-11T19:46:32.272 INFO:tasks.workunit.client.0.smithi090.stdout:2/778: symlink d0/de/lee 0 2024-03-11T19:46:32.491 INFO:tasks.workunit.client.0.smithi090.stdout:1/781: dwrite d5/d31/d36/f70 [0,4194304] 0 2024-03-11T19:46:33.341 INFO:tasks.workunit.client.0.smithi090.stdout:3/738: fsync d2/db/f62 0 2024-03-11T19:46:33.452 INFO:tasks.workunit.client.0.smithi090.stdout:2/779: readlink d0/l91 0 2024-03-11T19:46:33.876 INFO:tasks.workunit.client.0.smithi090.stdout:0/795: dwrite d2/d11/d3a/f80 [0,4194304] 0 2024-03-11T19:46:34.041 INFO:tasks.workunit.client.0.smithi090.stdout:7/766: dwrite d3/f4b [0,4194304] 0 2024-03-11T19:46:34.350 INFO:tasks.workunit.client.0.smithi090.stdout:3/739: mknod d2/de/ce7 0 2024-03-11T19:46:34.876 INFO:tasks.workunit.client.0.smithi090.stdout:9/781: read d1/dd/df/d1c/d21/dab/fca [5494751,108607] 0 2024-03-11T19:46:34.915 INFO:tasks.workunit.client.0.smithi090.stdout:6/835: creat d4/d13/d22/d42/d48/f119 x:0 0 0 2024-03-11T19:46:35.007 INFO:tasks.workunit.client.0.smithi090.stdout:1/782: mkdir d5/d31/d4c/daf/dfc/d103 0 2024-03-11T19:46:35.172 INFO:tasks.workunit.client.0.smithi090.stdout:4/837: dwrite d4/d18/fe8 [0,4194304] 0 2024-03-11T19:46:35.401 INFO:tasks.workunit.client.0.smithi090.stdout:6/836: dread d4/d13/d22/d34/d40/da4/f9a [0,4194304] 0 2024-03-11T19:46:35.636 INFO:tasks.workunit.client.0.smithi090.stdout:0/796: dread d2/d11/d3a/d64/fc6 [0,4194304] 0 2024-03-11T19:46:35.722 INFO:tasks.workunit.client.0.smithi090.stdout:0/797: dread d2/d11/f17 [0,4194304] 0 2024-03-11T19:46:36.224 INFO:tasks.workunit.client.0.smithi090.stdout:4/838: mkdir d4/d27/d28/d38/d64/d11a 0 2024-03-11T19:46:36.347 INFO:tasks.workunit.client.0.smithi090.stdout:0/798: fsync f0 0 2024-03-11T19:46:36.811 INFO:tasks.workunit.client.0.smithi090.stdout:8/849: rename d0/f58 to d0/d60/f117 0 2024-03-11T19:46:37.046 INFO:tasks.workunit.client.0.smithi090.stdout:5/775: dwrite d1/f49 [0,4194304] 0 2024-03-11T19:46:37.229 INFO:tasks.workunit.client.0.smithi090.stdout:1/783: creat d5/da/f104 x:0 0 0 2024-03-11T19:46:37.452 INFO:tasks.workunit.client.0.smithi090.stdout:4/839: unlink d4/d27/d28/d38/c8a 0 2024-03-11T19:46:37.843 INFO:tasks.workunit.client.0.smithi090.stdout:0/799: getdents d2/d11/d15/d29/d43/d58 0 2024-03-11T19:46:37.994 INFO:tasks.workunit.client.0.smithi090.stdout:9/782: write d1/dd/d18/d32/f45 [3075564,119254] 0 2024-03-11T19:46:38.043 INFO:tasks.workunit.client.0.smithi090.stdout:8/850: dread - d0/d3/d86/d93/dba/fd9 zero size 2024-03-11T19:46:38.518 INFO:tasks.workunit.client.0.smithi090.stdout:7/767: dwrite d3/dc/d1e/d7c/d96/d32/d71/d80/fce [0,4194304] 0 2024-03-11T19:46:38.670 INFO:tasks.workunit.client.0.smithi090.stdout:9/783: readlink d1/dd/df/lb7 0 2024-03-11T19:46:39.355 INFO:tasks.workunit.client.0.smithi090.stdout:7/768: fdatasync d3/dc/da6/deb/ff0 0 2024-03-11T19:46:39.424 INFO:tasks.workunit.client.0.smithi090.stdout:9/784: fsync d1/dd/d18/d3a/f86 0 2024-03-11T19:46:40.060 INFO:tasks.workunit.client.0.smithi090.stdout:7/769: creat d3/dc/d1e/d7c/d96/d32/da0/f109 x:0 0 0 2024-03-11T19:46:40.460 INFO:tasks.workunit.client.0.smithi090.stdout:3/740: rename d2/db/d15/d1a/dde to d2/db/d15/d1a/d75/de8 0 2024-03-11T19:46:40.480 INFO:tasks.workunit.client.0.smithi090.stdout:7/770: dread - d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/fc1 zero size 2024-03-11T19:46:41.382 INFO:tasks.workunit.client.0.smithi090.stdout:7/771: rmdir d3/dc/d1e/d7c/d96/d32/d45/d84/d8d 39 2024-03-11T19:46:41.410 INFO:tasks.workunit.client.0.smithi090.stdout:2/780: rename d0/d21/f42 to d0/d21/dce/da2/fef 0 2024-03-11T19:46:41.659 INFO:tasks.workunit.client.0.smithi090.stdout:4/840: dwrite d4/ff [0,4194304] 0 2024-03-11T19:46:41.849 INFO:tasks.workunit.client.0.smithi090.stdout:3/741: mkdir d2/db/d15/d69/dad/de9 0 2024-03-11T19:46:41.864 INFO:tasks.workunit.client.0.smithi090.stdout:1/784: write d5/f15 [3120494,87672] 0 2024-03-11T19:46:42.436 INFO:tasks.workunit.client.0.smithi090.stdout:3/742: stat d2/db/d15/d1a/d75/d8c/f8d 0 2024-03-11T19:46:42.477 INFO:tasks.workunit.client.0.smithi090.stdout:4/841: mkdir d4/dd8/db5/d11b 0 2024-03-11T19:46:42.542 INFO:tasks.workunit.client.0.smithi090.stdout:8/851: dwrite d0/d43/f64 [0,4194304] 0 2024-03-11T19:46:42.717 INFO:tasks.workunit.client.0.smithi090.stdout:9/785: dwrite d1/dd/df/d1c/d21/f9d [0,4194304] 0 2024-03-11T19:46:42.947 INFO:tasks.workunit.client.0.smithi090.stdout:3/743: chown d2/db/d15/d1a/d75/de8/d6a/dcb/f3b 45420193 1 2024-03-11T19:46:43.171 INFO:tasks.workunit.client.0.smithi090.stdout:1/785: creat d5/da/d1c/d3b/d8c/f105 x:0 0 0 2024-03-11T19:46:43.213 INFO:tasks.workunit.client.0.smithi090.stdout:1/786: chown d5/d65/d6b/dcf/feb 237 1 2024-03-11T19:46:43.933 INFO:tasks.workunit.client.0.smithi090.stdout:9/786: mkdir d1/dd/df/d1c/d21/d2a/dc2/d100 0 2024-03-11T19:46:44.056 INFO:tasks.workunit.client.0.smithi090.stdout:7/772: truncate d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/db5/de1/dc2/f65 3702998 0 2024-03-11T19:46:44.501 INFO:tasks.workunit.client.0.smithi090.stdout:2/781: dread d0/de/d12/f1a [0,4194304] 0 2024-03-11T19:46:44.515 INFO:tasks.workunit.client.0.smithi090.stdout:6/837: rename d4/d13/d22/d3a/c4f to d4/d13/d22/c11a 0 2024-03-11T19:46:44.720 INFO:tasks.workunit.client.0.smithi090.stdout:7/773: dread - d3/dc/d1e/d7c/d96/d32/d45/ff5 zero size 2024-03-11T19:46:45.060 INFO:tasks.workunit.client.0.smithi090.stdout:5/776: rename d1/d17/fb4 to d1/d17/d39/de8/f10b 0 2024-03-11T19:46:45.081 INFO:tasks.workunit.client.0.smithi090.stdout:2/782: mkdir d0/de/d12/d81/df0 0 2024-03-11T19:46:45.352 INFO:tasks.workunit.client.0.smithi090.stdout:6/838: dread d4/d13/d22/d34/d40/d7d/fa6 [0,4194304] 0 2024-03-11T19:46:45.365 INFO:tasks.workunit.client.0.smithi090.stdout:7/774: dread - d3/dc/d1e/d7c/d96/d32/d71/d80/ff9 zero size 2024-03-11T19:46:45.448 INFO:tasks.workunit.client.0.smithi090.stdout:7/775: stat d3/dc/d87 0 2024-03-11T19:46:45.632 INFO:tasks.workunit.client.0.smithi090.stdout:0/800: rename d2/d11/d15/d29/d3d/d7e/ca3 to d2/d11/d15/d29/d2e/db4/daf/cff 0 2024-03-11T19:46:45.867 INFO:tasks.workunit.client.0.smithi090.stdout:8/852: mknod d0/c118 0 2024-03-11T19:46:45.929 INFO:tasks.workunit.client.0.smithi090.stdout:4/842: dwrite d4/d18/fe8 [0,4194304] 0 2024-03-11T19:46:46.148 INFO:tasks.workunit.client.0.smithi090.stdout:4/843: write d4/ff [5687287,24292] 0 2024-03-11T19:46:46.425 INFO:tasks.workunit.client.0.smithi090.stdout:9/787: write d1/dd/df/d1c/fc9 [931385,74961] 0 2024-03-11T19:46:46.549 INFO:tasks.workunit.client.0.smithi090.stdout:5/777: dread d1/d17/d39/d51/d53/d7b/d3b/d76/d92/dc1/f107 [0,4194304] 0 2024-03-11T19:46:46.733 INFO:tasks.workunit.client.0.smithi090.stdout:7/776: mknod d3/dc/d87/c10a 0 2024-03-11T19:46:46.926 INFO:tasks.workunit.client.0.smithi090.stdout:2/783: chown d0/de/d12/d46/d4c/fe9 766703 1 2024-03-11T19:46:46.982 INFO:tasks.workunit.client.0.smithi090.stdout:8/853: read d0/d3/d86/d93/dba/fcf [2448176,22970] 0 2024-03-11T19:46:47.006 INFO:tasks.workunit.client.0.smithi090.stdout:8/854: chown d0/d3/db0 1193235 1 2024-03-11T19:46:47.165 INFO:tasks.workunit.client.0.smithi090.stdout:0/801: truncate d2/d11/d15/d29/d3d/d7b/fd7 4676346 0 2024-03-11T19:46:47.174 INFO:tasks.workunit.client.0.smithi090.stdout:4/844: mknod d4/dd8/d66/dcf/dd7/dd3/c11c 0 2024-03-11T19:46:47.388 INFO:tasks.workunit.client.0.smithi090.stdout:9/788: truncate d1/d49/f76 2077509 0 2024-03-11T19:46:47.413 INFO:tasks.workunit.client.0.smithi090.stdout:7/777: mknod d3/dc/d1e/d7c/d96/d32/d7b/d99/c10b 0 2024-03-11T19:46:47.670 INFO:tasks.workunit.client.0.smithi090.stdout:2/784: unlink d0/d21/dce/lb3 0 2024-03-11T19:46:47.682 INFO:tasks.workunit.client.0.smithi090.stdout:8/855: fsync d0/d3/d86/d93/dba/ffe 0 2024-03-11T19:46:47.708 INFO:tasks.workunit.client.0.smithi090.stdout:6/839: creat d4/d13/d22/d34/d37/f11b x:0 0 0 2024-03-11T19:46:48.158 INFO:tasks.workunit.client.0.smithi090.stdout:5/778: mknod d1/d17/c10c 0 2024-03-11T19:46:48.171 INFO:tasks.workunit.client.0.smithi090.stdout:4/845: truncate d4/d51/d8c/f9c 1885162 0 2024-03-11T19:46:48.189 INFO:tasks.workunit.client.0.smithi090.stdout:1/787: rename d5/d65/d6b/d7f/df7 to d5/da/d106 0 2024-03-11T19:46:48.311 INFO:tasks.workunit.client.0.smithi090.stdout:1/788: stat d5/da/dd/f1f 0 2024-03-11T19:46:48.351 INFO:tasks.workunit.client.0.smithi090.stdout:1/789: chown d5/d65/d6b/d7f/c9d 95621084 1 2024-03-11T19:46:48.539 INFO:tasks.workunit.client.0.smithi090.stdout:6/840: creat d4/d13/d22/d42/d48/d4e/f11c x:0 0 0 2024-03-11T19:46:48.559 INFO:tasks.workunit.client.0.smithi090.stdout:6/841: chown d4/d9b 110 1 2024-03-11T19:46:48.637 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:46:48.640 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:46:48.797 INFO:tasks.workunit.client.0.smithi090.stdout:4/846: readlink d4/d10e/l10f 0 2024-03-11T19:46:48.990 INFO:tasks.workunit.client.0.smithi090.stdout:3/744: rename d2/d2f/c85 to d2/db/d15/d1a/d3c/cea 0 2024-03-11T19:46:49.659 INFO:tasks.workunit.client.0.smithi090.stdout:1/790: truncate d5/da/d1c/d3b/d8c/fc4 643808 0 2024-03-11T19:46:50.189 INFO:tasks.workunit.client.0.smithi090.stdout:0/802: creat d2/d11/d15/d29/d2e/f100 x:0 0 0 2024-03-11T19:46:50.993 INFO:tasks.workunit.client.0.smithi090.stdout:9/789: rename d1/dd/df to d1/dd/d18/d25/d101 0 2024-03-11T19:46:51.006 INFO:tasks.workunit.client.0.smithi090.stdout:3/745: truncate d2/db/d15/d1a/d75/dc6/fc5 273177 0 2024-03-11T19:46:51.484 INFO:tasks.workunit.client.0.smithi090.stdout:4/847: mkdir d4/dd8/dfa/db7/d11d 0 2024-03-11T19:46:51.593 INFO:tasks.workunit.client.0.smithi090.stdout:5/779: rename d1/d26/dc8/fc9 to d1/d75/f10d 0 2024-03-11T19:46:51.833 INFO:tasks.workunit.client.0.smithi090.stdout:9/790: creat d1/dd/d18/d3a/d48/f102 x:0 0 0 2024-03-11T19:46:51.922 INFO:tasks.workunit.client.0.smithi090.stdout:3/746: creat d2/db/d15/d69/da2/feb x:0 0 0 2024-03-11T19:46:51.939 INFO:tasks.workunit.client.0.smithi090.stdout:7/778: dwrite d3/dc/d1e/d7c/d96/d32/d7b/f86 [0,4194304] 0 2024-03-11T19:46:52.096 INFO:tasks.workunit.client.0.smithi090.stdout:8/856: getdents d0/d3/d86/d93 0 2024-03-11T19:46:52.146 INFO:tasks.workunit.client.0.smithi090.stdout:1/791: symlink d5/da/l107 0 2024-03-11T19:46:52.382 INFO:tasks.workunit.client.0.smithi090.stdout:2/785: dwrite d0/de/f17 [0,4194304] 0 2024-03-11T19:46:53.327 INFO:tasks.workunit.client.0.smithi090.stdout:8/857: mkdir d0/d53/d59/d82/d119 0 2024-03-11T19:46:53.353 INFO:tasks.workunit.client.0.smithi090.stdout:8/858: chown d0/d3/d4c/d8d 249 1 2024-03-11T19:46:53.880 INFO:tasks.workunit.client.0.smithi090.stdout:9/791: rename d1/dd/c68 to d1/dd/d18/d25/d101/d1c/d77/deb/c103 0 2024-03-11T19:46:54.102 INFO:tasks.workunit.client.0.smithi090.stdout:7/779: symlink d3/dc/d1e/d7c/d96/d32/d67/l10c 0 2024-03-11T19:46:54.180 INFO:tasks.workunit.client.0.smithi090.stdout:6/842: dwrite d4/d13/d22/d34/d40/d7d/dc2/fdf [0,4194304] 0 2024-03-11T19:46:54.436 INFO:tasks.workunit.client.0.smithi090.stdout:2/786: truncate d0/de/d12/d46/d5d/d8d/fa6 1310400 0 2024-03-11T19:46:55.017 INFO:tasks.workunit.client.0.smithi090.stdout:3/747: creat d2/db/d15/d69/da2/dc7/ddd/fec x:0 0 0 2024-03-11T19:46:55.289 INFO:tasks.workunit.client.0.smithi090.stdout:9/792: read d1/dd/d18/d25/d101/d1c/d21/d2a/f29 [368455,20666] 0 2024-03-11T19:46:55.344 INFO:tasks.workunit.client.0.smithi090.stdout:9/793: stat d1/dd/d18/d25/d101/d1c/d21/d2a/f50 0 2024-03-11T19:46:55.451 INFO:tasks.workunit.client.0.smithi090.stdout:0/803: getdents d2/d11/d15/d29/d3d 0 2024-03-11T19:46:55.603 INFO:tasks.workunit.client.0.smithi090.stdout:6/843: fdatasync d4/d14/d25/fe6 0 2024-03-11T19:46:55.715 INFO:tasks.workunit.client.0.smithi090.stdout:7/780: creat d3/dc/d1e/d7c/d96/d32/da0/f10d x:0 0 0 2024-03-11T19:46:56.195 INFO:tasks.workunit.client.0.smithi090.stdout:6/844: dwrite d4/d13/d22/d34/d40/d7d/dc2/fdf [0,4194304] 0 2024-03-11T19:46:56.441 INFO:tasks.workunit.client.0.smithi090.stdout:2/787: dread d0/de/d12/d46/d4c/f61 [0,4194304] 0 2024-03-11T19:46:56.528 INFO:tasks.workunit.client.0.smithi090.stdout:0/804: read d2/d11/d3f/fa9 [2565723,84900] 0 2024-03-11T19:46:56.902 INFO:tasks.workunit.client.0.smithi090.stdout:5/780: creat d1/d17/d39/f10e x:0 0 0 2024-03-11T19:46:57.250 INFO:tasks.workunit.client.0.smithi090.stdout:8/859: rmdir d0/d60/db6/d110 0 2024-03-11T19:46:57.425 INFO:tasks.workunit.client.0.smithi090.stdout:7/781: dread d3/dc/d1e/d7c/d96/f9e [0,4194304] 0 2024-03-11T19:46:57.576 INFO:tasks.workunit.client.0.smithi090.stdout:5/781: fsync d1/f38 0 2024-03-11T19:46:57.593 INFO:tasks.workunit.client.0.smithi090.stdout:1/792: write d5/da/d1c/d3b/fcd [3207724,51237] 0 2024-03-11T19:46:58.047 INFO:tasks.workunit.client.0.smithi090.stdout:6/845: mknod d4/d13/d22/c11d 0 2024-03-11T19:46:58.199 INFO:tasks.workunit.client.0.smithi090.stdout:6/846: readlink d4/d13/d22/d3a/l4b 0 2024-03-11T19:46:58.222 INFO:tasks.workunit.client.0.smithi090.stdout:4/848: dwrite d4/fee [0,4194304] 0 2024-03-11T19:46:58.623 INFO:tasks.workunit.client.0.smithi090.stdout:9/794: creat d1/dd/d18/d3a/f104 x:0 0 0 2024-03-11T19:46:58.683 INFO:tasks.workunit.client.0.smithi090.stdout:6/847: dread d4/d14/d25/fe6 [0,4194304] 0 2024-03-11T19:46:58.758 INFO:tasks.workunit.client.0.smithi090.stdout:2/788: creat d0/de/d1f/ff1 x:0 0 0 2024-03-11T19:46:58.837 INFO:tasks.workunit.client.0.smithi090.stdout:2/789: rename d0/de to d0/de/d1f/df2 22 2024-03-11T19:46:58.865 INFO:tasks.workunit.client.0.smithi090.stdout:2/790: chown d0/de/d1f/ff1 383 1 2024-03-11T19:46:59.396 INFO:tasks.workunit.client.0.smithi090.stdout:3/748: dwrite d2/f9a [0,4194304] 0 2024-03-11T19:46:59.527 INFO:tasks.workunit.client.0.smithi090.stdout:5/782: truncate d1/d17/d39/d51/d53/d7b/d3b/d76/d92/fa2 633045 0 2024-03-11T19:46:59.683 INFO:tasks.workunit.client.0.smithi090.stdout:4/849: creat d4/d51/d8c/d6a/d100/f11e x:0 0 0 2024-03-11T19:46:59.693 INFO:tasks.workunit.client.0.smithi090.stdout:2/791: dread d0/de/d12/d46/d75/dbb/fd7 [0,4194304] 0 2024-03-11T19:46:59.744 INFO:tasks.workunit.client.0.smithi090.stdout:3/749: sync 2024-03-11T19:47:00.024 INFO:tasks.workunit.client.0.smithi090.stdout:9/795: symlink d1/dd/d18/d25/d101/d1c/d88/d96/l105 0 2024-03-11T19:47:00.028 INFO:tasks.workunit.client.0.smithi090.stdout:0/805: dwrite d2/fab [4194304,4194304] 0 2024-03-11T19:47:00.117 INFO:tasks.workunit.client.0.smithi090.stdout:1/793: mknod d5/d31/d76/c108 0 2024-03-11T19:47:00.165 INFO:tasks.workunit.client.0.smithi090.stdout:9/796: chown d1/dd/f69 49999231 1 2024-03-11T19:47:00.216 INFO:tasks.workunit.client.0.smithi090.stdout:9/797: chown d1/dd/d18/d3a/d70/ca1 14831274 1 2024-03-11T19:47:00.266 INFO:tasks.workunit.client.0.smithi090.stdout:9/798: write d1/dd/d18/d3a/f104 [116247,129847] 0 2024-03-11T19:47:00.290 INFO:tasks.workunit.client.0.smithi090.stdout:8/860: creat d0/d60/f11a x:0 0 0 2024-03-11T19:47:00.409 INFO:tasks.workunit.client.0.smithi090.stdout:5/783: dread d1/d17/f66 [0,4194304] 0 2024-03-11T19:47:00.502 INFO:tasks.workunit.client.0.smithi090.stdout:4/850: dread d4/d51/dbf/fe9 [0,4194304] 0 2024-03-11T19:47:00.575 INFO:tasks.workunit.client.0.smithi090.stdout:7/782: link d3/l1f d3/dc/d29/l10e 0 2024-03-11T19:47:00.849 INFO:tasks.workunit.client.0.smithi090.stdout:1/794: mkdir d5/d31/d4c/d109 0 2024-03-11T19:47:01.036 INFO:tasks.workunit.client.0.smithi090.stdout:0/806: truncate d2/d11/d15/d29/d2e/d6d/da4/fee 804746 0 2024-03-11T19:47:01.336 INFO:tasks.workunit.client.0.smithi090.stdout:8/861: rename d0/d3/d4c/l63 to d0/d3/d4c/d8d/df8/d113/l11b 0 2024-03-11T19:47:01.361 INFO:tasks.workunit.client.0.smithi090.stdout:8/862: readlink d0/d43/l35 0 2024-03-11T19:47:01.557 INFO:tasks.workunit.client.0.smithi090.stdout:5/784: mknod d1/d26/dc8/c10f 0 2024-03-11T19:47:01.581 INFO:tasks.workunit.client.0.smithi090.stdout:5/785: stat d1/d26/fd6 0 2024-03-11T19:47:01.619 INFO:tasks.workunit.client.0.smithi090.stdout:5/786: chown d1/d17/d39/d51/d73/f95 61 1 2024-03-11T19:47:01.692 INFO:tasks.workunit.client.0.smithi090.stdout:4/851: read - d4/dd8/d66/dcf/dd7/dd3/ffe zero size 2024-03-11T19:47:02.047 INFO:tasks.workunit.client.0.smithi090.stdout:3/750: mkdir d2/db/d15/d1a/d75/ded 0 2024-03-11T19:47:02.877 INFO:tasks.workunit.client.0.smithi090.stdout:1/795: dread d5/d65/fa6 [0,4194304] 0 2024-03-11T19:47:02.898 INFO:tasks.workunit.client.0.smithi090.stdout:0/807: dread d2/d11/d15/d29/d43/fbe [0,4194304] 0 2024-03-11T19:47:03.113 INFO:tasks.workunit.client.0.smithi090.stdout:4/852: dread d4/d18/fe8 [0,4194304] 0 2024-03-11T19:47:03.742 INFO:tasks.workunit.client.0.smithi090.stdout:9/799: creat d1/d47/f106 x:0 0 0 2024-03-11T19:47:04.236 INFO:tasks.workunit.client.0.smithi090.stdout:2/792: dwrite d0/de/d12/d46/d5d/f62 [0,4194304] 0 2024-03-11T19:47:04.371 INFO:tasks.workunit.client.0.smithi090.stdout:4/853: mkdir d4/d51/d8c/d6a/d8d/d11f 0 2024-03-11T19:47:04.397 INFO:tasks.workunit.client.0.smithi090.stdout:6/848: dwrite d4/d13/d22/d34/d40/fcb [0,4194304] 0 2024-03-11T19:47:04.722 INFO:tasks.workunit.client.0.smithi090.stdout:9/800: symlink d1/dd/d18/d25/d101/d1c/d21/d2a/da4/l107 0 2024-03-11T19:47:05.048 INFO:tasks.workunit.client.0.smithi090.stdout:7/783: rename d3/dc/d1e/d7c/d96/d32/d45/de0/l14 to d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/db5/de1/l10f 0 2024-03-11T19:47:05.054 INFO:tasks.workunit.client.0.smithi090.stdout:8/863: write d0/d43/dc/d90/fac [1614586,82150] 0 2024-03-11T19:47:05.309 INFO:tasks.workunit.client.0.smithi090.stdout:1/796: unlink d5/c16 0 2024-03-11T19:47:05.421 INFO:tasks.workunit.client.0.smithi090.stdout:5/787: dwrite d1/d75/f8c [0,4194304] 0 2024-03-11T19:47:05.556 INFO:tasks.workunit.client.0.smithi090.stdout:9/801: creat d1/d1d/d6a/f108 x:0 0 0 2024-03-11T19:47:05.726 INFO:tasks.workunit.client.0.smithi090.stdout:3/751: write d2/f12 [2665539,41140] 0 2024-03-11T19:47:05.760 INFO:tasks.workunit.client.0.smithi090.stdout:9/802: sync 2024-03-11T19:47:05.842 INFO:tasks.workunit.client.0.smithi090.stdout:8/864: rmdir d0/d3/d3e/d85 39 2024-03-11T19:47:05.954 INFO:tasks.workunit.client.0.smithi090.stdout:5/788: dread d1/f1e [0,4194304] 0 2024-03-11T19:47:06.146 INFO:tasks.workunit.client.0.smithi090.stdout:7/784: unlink d3/dc/d1e/d7c/d96/l6c 0 2024-03-11T19:47:06.403 INFO:tasks.workunit.client.0.smithi090.stdout:9/803: dread d1/d1d/d6a/fe0 [0,4194304] 0 2024-03-11T19:47:06.449 INFO:tasks.workunit.client.0.smithi090.stdout:3/752: dread d2/d2f/f30 [0,4194304] 0 2024-03-11T19:47:06.477 INFO:tasks.workunit.client.0.smithi090.stdout:2/793: rename d0/de/d12/d46/d4c/cb4 to d0/de/d12/dc8/cf3 0 2024-03-11T19:47:06.538 INFO:tasks.workunit.client.0.smithi090.stdout:7/785: chown d3/dc/d1e/d7c/d96/d32/d71/f77 12 1 2024-03-11T19:47:07.013 INFO:tasks.workunit.client.0.smithi090.stdout:5/789: unlink d1/d17/d39/d51/d53/d7b/d6f/ff0 0 2024-03-11T19:47:07.125 INFO:tasks.workunit.client.0.smithi090.stdout:0/808: dwrite d2/d11/d15/d29/d2e/db4/daf/fdb [0,4194304] 0 2024-03-11T19:47:07.342 INFO:tasks.workunit.client.0.smithi090.stdout:9/804: mknod d1/dd/d18/d32/c109 0 2024-03-11T19:47:07.367 INFO:tasks.workunit.client.0.smithi090.stdout:5/790: dread - d1/d17/d39/d51/d53/d7b/d3b/fdb zero size 2024-03-11T19:47:07.453 INFO:tasks.workunit.client.0.smithi090.stdout:2/794: mkdir d0/de/d1f/df4 0 2024-03-11T19:47:07.831 INFO:tasks.workunit.client.0.smithi090.stdout:3/753: symlink d2/db/d15/d1a/d75/de8/d6a/dcb/lee 0 2024-03-11T19:47:07.847 INFO:tasks.workunit.client.0.smithi090.stdout:3/754: chown d2/de 0 1 2024-03-11T19:47:07.889 INFO:tasks.workunit.client.0.smithi090.stdout:7/786: creat d3/dc/da6/f110 x:0 0 0 2024-03-11T19:47:08.724 INFO:tasks.workunit.client.0.smithi090.stdout:6/849: write d4/d13/d22/d34/d40/dd5/f5f [2318877,86176] 0 2024-03-11T19:47:09.252 INFO:tasks.workunit.client.0.smithi090.stdout:7/787: mkdir d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/db5/de1/dc6/d111 0 2024-03-11T19:47:09.346 INFO:tasks.workunit.client.0.smithi090.stdout:5/791: dread - d1/d17/fe4 zero size 2024-03-11T19:47:09.387 INFO:tasks.workunit.client.0.smithi090.stdout:5/792: stat d1/d17/d39/d51/d53/c58 0 2024-03-11T19:47:09.483 INFO:tasks.workunit.client.0.smithi090.stdout:9/805: creat d1/dd/d18/d3a/db6/de1/f10a x:0 0 0 2024-03-11T19:47:09.523 INFO:tasks.workunit.client.0.smithi090.stdout:4/854: write d4/d27/d28/d69/f99 [1574412,95779] 0 2024-03-11T19:47:09.608 INFO:tasks.workunit.client.0.smithi090.stdout:6/850: truncate d4/d14/d8a/f63 1934270 0 2024-03-11T19:47:10.002 INFO:tasks.workunit.client.0.smithi090.stdout:2/795: mknod d0/de/d12/d46/de6/cf5 0 2024-03-11T19:47:10.088 INFO:tasks.workunit.client.0.smithi090.stdout:7/788: rmdir d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/db5/de1/dc6/d103 39 2024-03-11T19:47:10.141 INFO:tasks.workunit.client.0.smithi090.stdout:7/789: chown d3/dc/d1e/d7c/d96/d32/d71/d80/c83 213 1 2024-03-11T19:47:10.317 INFO:tasks.workunit.client.0.smithi090.stdout:5/793: unlink d1/d17/d39/c5d 0 2024-03-11T19:47:10.382 INFO:tasks.workunit.client.0.smithi090.stdout:5/794: dread d1/d26/db5/ffa [0,4194304] 0 2024-03-11T19:47:11.152 INFO:tasks.workunit.client.0.smithi090.stdout:6/851: dread d4/d13/d22/d34/d83/d9c/fb0 [0,4194304] 0 2024-03-11T19:47:11.485 INFO:tasks.workunit.client.0.smithi090.stdout:8/865: rename d0/f65 to d0/d3/d98/da2/f11c 0 2024-03-11T19:47:12.046 INFO:tasks.workunit.client.0.smithi090.stdout:0/809: write d2/f1b [3081981,97893] 0 2024-03-11T19:47:12.258 INFO:tasks.workunit.client.0.smithi090.stdout:6/852: symlink d4/d13/d22/d3a/d67/l11e 0 2024-03-11T19:47:12.304 INFO:tasks.workunit.client.0.smithi090.stdout:6/853: chown d4/d13/d22/d34/d37/d7f/d90/fad 8 1 2024-03-11T19:47:12.495 INFO:tasks.workunit.client.0.smithi090.stdout:5/795: mkdir d1/d17/d39/d51/d53/d7b/d3b/d76/d92/dc1/dff/d110 0 2024-03-11T19:47:12.765 INFO:tasks.workunit.client.0.smithi090.stdout:1/797: rename d5/da/d1c/d6d/d80/lc0 to d5/d31/d76/d77/l10a 0 2024-03-11T19:47:12.792 INFO:tasks.workunit.client.0.smithi090.stdout:1/798: chown d5/da/d97/lf9 1 1 2024-03-11T19:47:13.426 INFO:tasks.workunit.client.0.smithi090.stdout:6/854: write d4/d13/d22/d34/d40/dd5/f3f [5140352,62719] 0 2024-03-11T19:47:13.503 INFO:tasks.workunit.client.0.smithi090.stdout:5/796: symlink d1/d17/d39/de8/l111 0 2024-03-11T19:47:13.693 INFO:tasks.workunit.client.0.smithi090.stdout:1/799: mkdir d5/d31/d76/d77/d10b 0 2024-03-11T19:47:13.858 INFO:tasks.workunit.client.0.smithi090.stdout:8/866: rename d0/d60/db6/db9 to d0/d3/d3e/d85/d116/d11d 0 2024-03-11T19:47:14.079 INFO:tasks.workunit.client.0.smithi090.stdout:3/755: write d2/db/d15/d49/f7f [573397,1316] 0 2024-03-11T19:47:14.182 INFO:tasks.workunit.client.0.smithi090.stdout:0/810: creat d2/d11/d15/d29/d43/df8/f101 x:0 0 0 2024-03-11T19:47:14.488 INFO:tasks.workunit.client.0.smithi090.stdout:4/855: write d4/dd8/f65 [1217387,81394] 0 2024-03-11T19:47:14.632 INFO:tasks.workunit.client.0.smithi090.stdout:6/855: creat d4/d13/d58/f11f x:0 0 0 2024-03-11T19:47:14.730 INFO:tasks.workunit.client.0.smithi090.stdout:5/797: creat d1/d17/d39/d51/d73/da6/f112 x:0 0 0 2024-03-11T19:47:14.806 INFO:tasks.workunit.client.0.smithi090.stdout:9/806: dwrite d1/dd/d18/d25/d101/d1c/d21/f9d [0,4194304] 0 2024-03-11T19:47:14.999 INFO:tasks.workunit.client.0.smithi090.stdout:3/756: symlink d2/d2f/dc4/lef 0 2024-03-11T19:47:15.215 INFO:tasks.workunit.client.0.smithi090.stdout:7/790: write d3/dc/d1e/d2a/f5e [3261161,23685] 0 2024-03-11T19:47:15.382 INFO:tasks.workunit.client.0.smithi090.stdout:0/811: unlink d2/d11/d15/d29/d2e/f32 0 2024-03-11T19:47:15.416 INFO:tasks.workunit.client.0.smithi090.stdout:0/812: chown d2/d11/d15/d29/fa5 406686050 1 2024-03-11T19:47:15.774 INFO:tasks.workunit.client.0.smithi090.stdout:4/856: dread d4/dd8/fa7 [0,4194304] 0 2024-03-11T19:47:16.055 INFO:tasks.workunit.client.0.smithi090.stdout:3/757: creat d2/db/d15/d1a/d75/de8/ff0 x:0 0 0 2024-03-11T19:47:16.300 INFO:tasks.workunit.client.0.smithi090.stdout:7/791: stat d3/dc/d1e/fa3 0 2024-03-11T19:47:16.616 INFO:tasks.workunit.client.0.smithi090.stdout:6/856: symlink d4/d13/d22/d3a/l120 0 2024-03-11T19:47:17.364 INFO:tasks.workunit.client.0.smithi090.stdout:7/792: creat d3/dc/d1e/d7c/d96/d32/d71/d80/f112 x:0 0 0 2024-03-11T19:47:17.368 INFO:tasks.workunit.client.0.smithi090.stdout:6/857: fdatasync d4/d13/d22/d42/d48/d4e/f11c 0 2024-03-11T19:47:17.443 INFO:tasks.workunit.client.0.smithi090.stdout:6/858: readlink d4/d13/d22/d34/d37/d7f/lc4 0 2024-03-11T19:47:17.482 INFO:tasks.workunit.client.0.smithi090.stdout:2/796: write d0/de/db6/fc1 [142162,11469] 0 2024-03-11T19:47:17.747 INFO:tasks.workunit.client.0.smithi090.stdout:9/807: creat d1/dd/d18/d25/d101/d1c/d21/dab/f10b x:0 0 0 2024-03-11T19:47:17.828 INFO:tasks.workunit.client.0.smithi090.stdout:3/758: mkdir d2/d45/df1 0 2024-03-11T19:47:18.586 INFO:tasks.workunit.client.0.smithi090.stdout:2/797: chown d0/f7 37932 1 2024-03-11T19:47:18.708 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:47:18.710 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:47:18.759 INFO:tasks.workunit.client.0.smithi090.stdout:4/857: link d4/d27/d28/d38/d64/lff d4/d51/d8c/d6a/d100/l120 0 2024-03-11T19:47:19.867 INFO:tasks.workunit.client.0.smithi090.stdout:1/800: write d5/da/dd/f1f [2113884,35845] 0 2024-03-11T19:47:20.030 INFO:tasks.workunit.client.0.smithi090.stdout:4/858: symlink d4/d18/l121 0 2024-03-11T19:47:20.806 INFO:tasks.workunit.client.0.smithi090.stdout:4/859: fsync d4/d27/d28/f30 0 2024-03-11T19:47:20.818 INFO:tasks.workunit.client.0.smithi090.stdout:7/793: unlink d3/f70 0 2024-03-11T19:47:20.826 INFO:tasks.workunit.client.0.smithi090.stdout:5/798: write d1/d17/d39/d51/d53/fc2 [800122,78807] 0 2024-03-11T19:47:21.154 INFO:tasks.workunit.client.0.smithi090.stdout:1/801: mkdir d5/d65/d6b/d7f/d10c 0 2024-03-11T19:47:21.171 INFO:tasks.workunit.client.0.smithi090.stdout:0/813: write d2/d11/d15/d29/d43/d58/f9e [963458,39899] 0 2024-03-11T19:47:21.424 INFO:tasks.workunit.client.0.smithi090.stdout:7/794: unlink d3/d55/f85 0 2024-03-11T19:47:21.451 INFO:tasks.workunit.client.0.smithi090.stdout:7/795: chown d3/d55/fb4 0 1 2024-03-11T19:47:21.550 INFO:tasks.workunit.client.0.smithi090.stdout:4/860: creat d4/d27/d28/ddc/f122 x:0 0 0 2024-03-11T19:47:22.224 INFO:tasks.workunit.client.0.smithi090.stdout:7/796: fsync d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/d9c/fdb 0 2024-03-11T19:47:22.854 INFO:tasks.workunit.client.0.smithi090.stdout:3/759: write d2/db/d15/d69/da2/dc7/fe3 [208085,57045] 0 2024-03-11T19:47:23.040 INFO:tasks.workunit.client.0.smithi090.stdout:7/797: mknod d3/dc/d1e/d7c/d96/d7f/c113 0 2024-03-11T19:47:23.057 INFO:tasks.workunit.client.0.smithi090.stdout:4/861: mknod d4/d27/d28/d38/c123 0 2024-03-11T19:47:23.076 INFO:tasks.workunit.client.0.smithi090.stdout:6/859: write d4/ff [2568987,12718] 0 2024-03-11T19:47:23.138 INFO:tasks.workunit.client.0.smithi090.stdout:4/862: readlink d4/d51/l6d 0 2024-03-11T19:47:23.196 INFO:tasks.workunit.client.0.smithi090.stdout:0/814: creat d2/d11/d15/d29/f102 x:0 0 0 2024-03-11T19:47:23.247 INFO:tasks.workunit.client.0.smithi090.stdout:4/863: readlink d4/d51/d8c/d6a/la2 0 2024-03-11T19:47:23.319 INFO:tasks.workunit.client.0.smithi090.stdout:1/802: link d5/da/d1c/c67 d5/da/dd/c10d 0 2024-03-11T19:47:23.453 INFO:tasks.workunit.client.0.smithi090.stdout:2/798: write d0/de/db6/fdc [4550329,67937] 0 2024-03-11T19:47:23.493 INFO:tasks.workunit.client.0.smithi090.stdout:6/860: dread d4/d13/d22/d34/d83/fbc [0,4194304] 0 2024-03-11T19:47:23.682 INFO:tasks.workunit.client.0.smithi090.stdout:4/864: sync 2024-03-11T19:47:23.741 INFO:tasks.workunit.client.0.smithi090.stdout:9/808: dwrite d1/dd/d18/d25/d101/d1c/d9f/fe4 [0,4194304] 0 2024-03-11T19:47:23.841 INFO:tasks.workunit.client.0.smithi090.stdout:8/867: rmdir d0/d60 39 2024-03-11T19:47:24.613 INFO:tasks.workunit.client.0.smithi090.stdout:7/798: unlink d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/db5/de1/dc2/d8e/f107 0 2024-03-11T19:47:24.961 INFO:tasks.workunit.client.0.smithi090.stdout:3/760: symlink d2/db/d15/d1a/d75/lf2 0 2024-03-11T19:47:25.307 INFO:tasks.workunit.client.0.smithi090.stdout:6/861: creat d4/d9b/def/f121 x:0 0 0 2024-03-11T19:47:25.313 INFO:tasks.workunit.client.0.smithi090.stdout:2/799: dread - d0/de/db6/d25/d8c/fb8 zero size 2024-03-11T19:47:25.471 INFO:tasks.workunit.client.0.smithi090.stdout:1/803: rename d5/da/d1c/d6d to d5/da/d23/dfe/d89/d10e 0 2024-03-11T19:47:25.820 INFO:tasks.workunit.client.0.smithi090.stdout:9/809: symlink d1/dd/d18/d25/l10c 0 2024-03-11T19:47:26.044 INFO:tasks.workunit.client.0.smithi090.stdout:0/815: getdents d2/d11/d3a/d64/dd5/de3 0 2024-03-11T19:47:26.072 INFO:tasks.workunit.client.0.smithi090.stdout:5/799: dwrite d1/d17/d39/d51/d53/d7b/f5b [0,4194304] 0 2024-03-11T19:47:26.525 INFO:tasks.workunit.client.0.smithi090.stdout:8/868: symlink d0/d60/db6/l11e 0 2024-03-11T19:47:26.559 INFO:tasks.workunit.client.0.smithi090.stdout:1/804: dread d5/da/f92 [0,4194304] 0 2024-03-11T19:47:26.916 INFO:tasks.workunit.client.0.smithi090.stdout:9/810: dread d1/d49/f76 [0,4194304] 0 2024-03-11T19:47:27.127 INFO:tasks.workunit.client.0.smithi090.stdout:0/816: fsync d2/f3 0 2024-03-11T19:47:28.022 INFO:tasks.workunit.client.0.smithi090.stdout:8/869: mkdir d0/d3/d4c/d8d/df8/d113/d11f 0 2024-03-11T19:47:28.229 INFO:tasks.workunit.client.0.smithi090.stdout:9/811: creat d1/dd/d18/d3a/db6/de1/f10d x:0 0 0 2024-03-11T19:47:28.848 INFO:tasks.workunit.client.0.smithi090.stdout:8/870: fdatasync d0/d43/f2f 0 2024-03-11T19:47:28.899 INFO:tasks.workunit.client.0.smithi090.stdout:4/865: dwrite d4/d27/d28/d69/f99 [0,4194304] 0 2024-03-11T19:47:29.016 INFO:tasks.workunit.client.0.smithi090.stdout:2/800: getdents d0/de 0 2024-03-11T19:47:29.248 INFO:tasks.workunit.client.0.smithi090.stdout:6/862: dwrite d4/d13/d22/d34/d40/fcb [0,4194304] 0 2024-03-11T19:47:29.375 INFO:tasks.workunit.client.0.smithi090.stdout:9/812: chown d1/dd/d18/d3a/da2/lf9 1 1 2024-03-11T19:47:29.979 INFO:tasks.workunit.client.0.smithi090.stdout:1/805: link d5/da/dd/l18 d5/d31/d4c/d109/l10f 0 2024-03-11T19:47:30.249 INFO:tasks.workunit.client.0.smithi090.stdout:7/799: write d3/dc/d1e/d7c/d96/d32/f4a [2453336,16949] 0 2024-03-11T19:47:30.251 INFO:tasks.workunit.client.0.smithi090.stdout:8/871: fdatasync d0/d43/f79 0 2024-03-11T19:47:30.888 INFO:tasks.workunit.client.0.smithi090.stdout:5/800: dwrite d1/d26/d19/f42 [0,4194304] 0 2024-03-11T19:47:31.090 INFO:tasks.workunit.client.0.smithi090.stdout:3/761: creat d2/db/d15/ff3 x:0 0 0 2024-03-11T19:47:31.126 INFO:tasks.workunit.client.0.smithi090.stdout:3/762: chown d2/d2f/dc4/lef 15215 1 2024-03-11T19:47:31.464 INFO:tasks.workunit.client.0.smithi090.stdout:8/872: unlink d0/d43/lae 0 2024-03-11T19:47:31.487 INFO:tasks.workunit.client.0.smithi090.stdout:8/873: stat d0/d3/d3e/lcb 0 2024-03-11T19:47:31.606 INFO:tasks.workunit.client.0.smithi090.stdout:0/817: write d2/fcb [1533359,6569] 0 2024-03-11T19:47:31.608 INFO:tasks.workunit.client.0.smithi090.stdout:7/800: creat d3/dc/d1e/d2a/f114 x:0 0 0 2024-03-11T19:47:32.886 INFO:tasks.workunit.client.0.smithi090.stdout:0/818: readlink d2/d11/d3a/la1 0 2024-03-11T19:47:33.079 INFO:tasks.workunit.client.0.smithi090.stdout:9/813: rename d1/d1d/f2b to d1/f10e 0 2024-03-11T19:47:33.219 INFO:tasks.workunit.client.0.smithi090.stdout:0/819: dread d2/d11/f1a [0,4194304] 0 2024-03-11T19:47:33.362 INFO:tasks.workunit.client.0.smithi090.stdout:1/806: link d5/d31/d76/l58 d5/d31/d4c/l110 0 2024-03-11T19:47:33.630 INFO:tasks.workunit.client.0.smithi090.stdout:3/763: creat d2/db/d15/d1a/d75/d8c/ff4 x:0 0 0 2024-03-11T19:47:34.048 INFO:tasks.workunit.client.0.smithi090.stdout:5/801: rmdir d1/d17/d39/d51/d53/d7b/d3b/d76/d92/d105 0 2024-03-11T19:47:34.133 INFO:tasks.workunit.client.0.smithi090.stdout:4/866: write d4/dd8/f86 [203698,93869] 0 2024-03-11T19:47:34.483 INFO:tasks.workunit.client.0.smithi090.stdout:2/801: dwrite d0/de/d1f/f1c [4194304,4194304] 0 2024-03-11T19:47:34.585 INFO:tasks.workunit.client.0.smithi090.stdout:3/764: symlink d2/d4f/d59/da1/lf5 0 2024-03-11T19:47:34.750 INFO:tasks.workunit.client.0.smithi090.stdout:4/867: creat d4/dd8/f124 x:0 0 0 2024-03-11T19:47:34.805 INFO:tasks.workunit.client.0.smithi090.stdout:4/868: stat d4/d27/d28/d69/d111 0 2024-03-11T19:47:34.850 INFO:tasks.workunit.client.0.smithi090.stdout:4/869: read d4/d27/d28/f30 [1209995,61601] 0 2024-03-11T19:47:35.150 INFO:tasks.workunit.client.0.smithi090.stdout:8/874: getdents d0/d43 0 2024-03-11T19:47:35.407 INFO:tasks.workunit.client.0.smithi090.stdout:6/863: write d4/d13/d22/d34/d40/d7d/f109 [45634,40042] 0 2024-03-11T19:47:35.896 INFO:tasks.workunit.client.0.smithi090.stdout:3/765: stat d2/db/d15/d1a/l26 0 2024-03-11T19:47:36.426 INFO:tasks.workunit.client.0.smithi090.stdout:7/801: write d3/fc8 [705702,49425] 0 2024-03-11T19:47:36.469 INFO:tasks.workunit.client.0.smithi090.stdout:6/864: rmdir d4/d13/d22/d34/d37/d7f 39 2024-03-11T19:47:36.505 INFO:tasks.workunit.client.0.smithi090.stdout:6/865: chown d4/d13/d22/d34/d83/d9c/fb0 1098 1 2024-03-11T19:47:36.714 INFO:tasks.workunit.client.0.smithi090.stdout:2/802: truncate d0/f4e 830934 0 2024-03-11T19:47:37.001 INFO:tasks.workunit.client.0.smithi090.stdout:0/820: dwrite d2/d11/d3a/d64/fc6 [0,4194304] 0 2024-03-11T19:47:37.125 INFO:tasks.workunit.client.0.smithi090.stdout:9/814: link d1/dd/d18/d25/d101/d1c/c42 d1/dd/c10f 0 2024-03-11T19:47:37.149 INFO:tasks.workunit.client.0.smithi090.stdout:4/870: getdents d4/dd8/dfa/db7/d11d 0 2024-03-11T19:47:37.350 INFO:tasks.workunit.client.0.smithi090.stdout:6/866: mkdir d4/d13/d22/d42/d48/d4e/d122 0 2024-03-11T19:47:37.546 INFO:tasks.workunit.client.0.smithi090.stdout:5/802: dwrite d1/d17/d39/d51/d53/d64/f6d [0,4194304] 0 2024-03-11T19:47:38.075 INFO:tasks.workunit.client.0.smithi090.stdout:2/803: mknod d0/d21/dce/da2/cf6 0 2024-03-11T19:47:38.396 INFO:tasks.workunit.client.0.smithi090.stdout:4/871: symlink d4/d10e/l125 0 2024-03-11T19:47:39.029 INFO:tasks.workunit.client.0.smithi090.stdout:1/807: write d5/d31/d36/f70 [880577,1724] 0 2024-03-11T19:47:39.174 INFO:tasks.workunit.client.0.smithi090.stdout:5/803: creat d1/d17/d39/d51/d53/d7b/d3b/d76/db7/ddf/f113 x:0 0 0 2024-03-11T19:47:39.369 INFO:tasks.workunit.client.0.smithi090.stdout:8/875: write d0/d43/f84 [4742109,72010] 0 2024-03-11T19:47:39.711 INFO:tasks.workunit.client.0.smithi090.stdout:4/872: unlink d4/d18/f41 0 2024-03-11T19:47:40.208 INFO:tasks.workunit.client.0.smithi090.stdout:1/808: creat d5/da/dd/f111 x:0 0 0 2024-03-11T19:47:40.470 INFO:tasks.workunit.client.0.smithi090.stdout:2/804: creat d0/de/d1f/df4/ff7 x:0 0 0 2024-03-11T19:47:40.512 INFO:tasks.workunit.client.0.smithi090.stdout:2/805: chown d0/de/db6/d25/c79 2410 1 2024-03-11T19:47:40.598 INFO:tasks.workunit.client.0.smithi090.stdout:8/876: truncate d0/d53/f6e 955187 0 2024-03-11T19:47:40.940 INFO:tasks.workunit.client.0.smithi090.stdout:4/873: creat d4/dd8/d66/f126 x:0 0 0 2024-03-11T19:47:41.385 INFO:tasks.workunit.client.0.smithi090.stdout:2/806: rename d0/de/d12/d46/d49 to d0/de/df8 0 2024-03-11T19:47:41.632 INFO:tasks.workunit.client.0.smithi090.stdout:0/821: dwrite d2/d11/d15/d29/d2e/f52 [0,4194304] 0 2024-03-11T19:47:41.804 INFO:tasks.workunit.client.0.smithi090.stdout:7/802: write d3/dc/d1e/f57 [718318,6774] 0 2024-03-11T19:47:41.809 INFO:tasks.workunit.client.0.smithi090.stdout:9/815: write d1/dd/d18/d3a/db6/de1/f10a [49242,105245] 0 2024-03-11T19:47:41.986 INFO:tasks.workunit.client.0.smithi090.stdout:5/804: creat d1/d17/d39/d51/d53/d7b/f114 x:0 0 0 2024-03-11T19:47:42.342 INFO:tasks.workunit.client.0.smithi090.stdout:6/867: write d4/d13/d22/d34/d83/fbc [1025198,99821] 0 2024-03-11T19:47:42.471 INFO:tasks.workunit.client.0.smithi090.stdout:5/805: sync 2024-03-11T19:47:42.488 INFO:tasks.workunit.client.0.smithi090.stdout:5/806: stat d1/d17/f79 0 2024-03-11T19:47:42.834 INFO:tasks.workunit.client.0.smithi090.stdout:9/816: rename d1/dd/d18/d25/d101/d1c/d21/d2a/fe8 to d1/dd/d18/d3a/dd8/f110 0 2024-03-11T19:47:42.956 INFO:tasks.workunit.client.0.smithi090.stdout:7/803: truncate d3/dc/d1e/d7c/d96/d32/d7b/f86 2239379 0 2024-03-11T19:47:43.364 INFO:tasks.workunit.client.0.smithi090.stdout:5/807: truncate d1/d17/d39/d51/d53/d7b/f44 4918989 0 2024-03-11T19:47:43.483 INFO:tasks.workunit.client.0.smithi090.stdout:0/822: mknod d2/d11/d15/d29/df5/c103 0 2024-03-11T19:47:43.598 INFO:tasks.workunit.client.0.smithi090.stdout:1/809: dwrite d5/d31/d76/f4d [0,4194304] 0 2024-03-11T19:47:43.929 INFO:tasks.workunit.client.0.smithi090.stdout:9/817: symlink d1/dd/l111 0 2024-03-11T19:47:44.107 INFO:tasks.workunit.client.0.smithi090.stdout:7/804: creat d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/db5/de1/dd2/f115 x:0 0 0 2024-03-11T19:47:44.169 INFO:tasks.workunit.client.0.smithi090.stdout:7/805: dread - d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/d9c/fdb zero size 2024-03-11T19:47:44.299 INFO:tasks.workunit.client.0.smithi090.stdout:8/877: link d0/d53/f7a d0/d3/d3e/f120 0 2024-03-11T19:47:44.317 INFO:tasks.workunit.client.0.smithi090.stdout:8/878: readlink d0/d53/lee 0 2024-03-11T19:47:44.819 INFO:tasks.workunit.client.0.smithi090.stdout:3/766: creat d2/db/d15/ff6 x:0 0 0 2024-03-11T19:47:45.173 INFO:tasks.workunit.client.0.smithi090.stdout:0/823: dread - d2/d11/d15/d29/d2e/f100 zero size 2024-03-11T19:47:45.259 INFO:tasks.workunit.client.0.smithi090.stdout:1/810: mkdir d5/da/dd/d112 0 2024-03-11T19:47:45.276 INFO:tasks.workunit.client.0.smithi090.stdout:1/811: rename d5 to d5/d65/d113 22 2024-03-11T19:47:45.586 INFO:tasks.workunit.client.0.smithi090.stdout:7/806: stat d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/db5/dde/c43 0 2024-03-11T19:47:45.856 INFO:tasks.workunit.client.0.smithi090.stdout:9/818: dread d1/d1d/f67 [4194304,4194304] 0 2024-03-11T19:47:45.940 INFO:tasks.workunit.client.0.smithi090.stdout:8/879: read d0/d43/dc/f75 [95143,1974] 0 2024-03-11T19:47:46.050 INFO:tasks.workunit.client.0.smithi090.stdout:5/808: symlink d1/d26/l115 0 2024-03-11T19:47:46.071 INFO:tasks.workunit.client.0.smithi090.stdout:6/868: dwrite d4/d13/d22/d34/d40/dd5/f3f [0,4194304] 0 2024-03-11T19:47:46.463 INFO:tasks.workunit.client.0.smithi090.stdout:4/874: link d4/c19 d4/d27/d28/d38/c127 0 2024-03-11T19:47:47.555 INFO:tasks.workunit.client.0.smithi090.stdout:8/880: mkdir d0/d3/db0/d121 0 2024-03-11T19:47:47.719 INFO:tasks.workunit.client.0.smithi090.stdout:6/869: unlink d4/d13/d22/d34/d40/dd5/d80/lc8 0 2024-03-11T19:47:47.815 INFO:tasks.workunit.client.0.smithi090.stdout:5/809: truncate d1/d17/d39/d51/d53/d7b/d3b/d76/db7/ddf/f101 434907 0 2024-03-11T19:47:48.126 INFO:tasks.workunit.client.0.smithi090.stdout:3/767: truncate d2/db/d15/d49/f7f 3992403 0 2024-03-11T19:47:48.827 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:47:48.830 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:47:49.096 INFO:tasks.workunit.client.0.smithi090.stdout:6/870: rename d4/d13/d22/d34/d40/da4/c82 to d4/d13/d22/d34/d40/dd5/d80/dd4/c123 0 2024-03-11T19:47:49.546 INFO:tasks.workunit.client.0.smithi090.stdout:7/807: dwrite d3/fc8 [0,4194304] 0 2024-03-11T19:47:49.633 INFO:tasks.workunit.client.0.smithi090.stdout:9/819: write d1/dd/d18/d25/d101/fd9 [846341,2476] 0 2024-03-11T19:47:49.828 INFO:tasks.workunit.client.0.smithi090.stdout:1/812: dwrite d5/f37 [0,4194304] 0 2024-03-11T19:47:49.943 INFO:tasks.workunit.client.0.smithi090.stdout:5/810: fdatasync d1/d26/db5/ff7 0 2024-03-11T19:47:50.149 INFO:tasks.workunit.client.0.smithi090.stdout:4/875: truncate d4/dd8/db5/fdb 750636 0 2024-03-11T19:47:50.161 INFO:tasks.workunit.client.0.smithi090.stdout:3/768: symlink d2/db/d15/d49/lf7 0 2024-03-11T19:47:50.206 INFO:tasks.workunit.client.0.smithi090.stdout:0/824: creat d2/d11/d15/d29/d2e/f104 x:0 0 0 2024-03-11T19:47:50.612 INFO:tasks.workunit.client.0.smithi090.stdout:7/808: creat d3/dc/d1e/d7c/d96/d32/d45/de0/f116 x:0 0 0 2024-03-11T19:47:51.447 INFO:tasks.workunit.client.0.smithi090.stdout:2/807: getdents d0/de/d12/d46/de8 0 2024-03-11T19:47:52.114 INFO:tasks.workunit.client.0.smithi090.stdout:5/811: truncate d1/d17/d39/d51/d53/d7b/f114 65433 0 2024-03-11T19:47:52.127 INFO:tasks.workunit.client.0.smithi090.stdout:5/812: chown d1/d17/la4 260111813 1 2024-03-11T19:47:52.563 INFO:tasks.workunit.client.0.smithi090.stdout:0/825: symlink d2/d11/d3a/d64/dac/dba/dc9/l105 0 2024-03-11T19:47:52.679 INFO:tasks.workunit.client.0.smithi090.stdout:6/871: rename d4/d13/d22/d3a/d6f/fb5 to d4/d13/d22/d3a/dd7/f124 0 2024-03-11T19:47:52.822 INFO:tasks.workunit.client.0.smithi090.stdout:7/809: symlink d3/dff/l117 0 2024-03-11T19:47:53.189 INFO:tasks.workunit.client.0.smithi090.stdout:7/810: dread d3/dc/d1e/d7c/d96/d32/d45/de0/f42 [0,4194304] 0 2024-03-11T19:47:53.241 INFO:tasks.workunit.client.0.smithi090.stdout:7/811: stat d3/dc/d87/cf2 0 2024-03-11T19:47:53.654 INFO:tasks.workunit.client.0.smithi090.stdout:2/808: rename d0/de/db6/d29/f7a to d0/de/db6/d25/d8c/ff9 0 2024-03-11T19:47:53.845 INFO:tasks.workunit.client.0.smithi090.stdout:9/820: link d1/d1d/d6a/f108 d1/dd/d18/d3a/f112 0 2024-03-11T19:47:53.989 INFO:tasks.workunit.client.0.smithi090.stdout:1/813: getdents d5/d31/d36 0 2024-03-11T19:47:54.321 INFO:tasks.workunit.client.0.smithi090.stdout:3/769: getdents d2/de 0 2024-03-11T19:47:54.579 INFO:tasks.workunit.client.0.smithi090.stdout:8/881: mknod d0/c122 0 2024-03-11T19:47:54.597 INFO:tasks.workunit.client.0.smithi090.stdout:4/876: write d4/d18/f25 [3056767,6524] 0 2024-03-11T19:47:54.694 INFO:tasks.workunit.client.0.smithi090.stdout:2/809: read d0/de/d1f/f55 [2499094,83071] 0 2024-03-11T19:47:54.713 INFO:tasks.workunit.client.0.smithi090.stdout:1/814: dread - d5/d65/d6b/dcf/feb zero size 2024-03-11T19:47:55.271 INFO:tasks.workunit.client.0.smithi090.stdout:3/770: creat d2/db/d15/d1a/ff8 x:0 0 0 2024-03-11T19:47:55.581 INFO:tasks.workunit.client.0.smithi090.stdout:0/826: link d2/d11/d15/d29/fbc d2/d11/d15/d29/d3d/f106 0 2024-03-11T19:47:55.938 INFO:tasks.workunit.client.0.smithi090.stdout:2/810: fdatasync d0/de/db6/d29/f41 0 2024-03-11T19:47:56.161 INFO:tasks.workunit.client.0.smithi090.stdout:5/813: link d1/d26/d19/l1d d1/d17/d39/d51/d53/d7b/d3b/d76/db7/ddf/l116 0 2024-03-11T19:47:56.316 INFO:tasks.workunit.client.0.smithi090.stdout:5/814: dread d1/d17/f30 [0,4194304] 0 2024-03-11T19:47:56.611 INFO:tasks.workunit.client.0.smithi090.stdout:3/771: truncate d2/db/d15/d1a/d75/d8c/fdc 209851 0 2024-03-11T19:47:56.674 INFO:tasks.workunit.client.0.smithi090.stdout:9/821: creat d1/d47/f113 x:0 0 0 2024-03-11T19:47:56.699 INFO:tasks.workunit.client.0.smithi090.stdout:3/772: read d2/db/d15/d1a/d75/de8/d6a/f97 [348539,2092] 0 2024-03-11T19:47:56.737 INFO:tasks.workunit.client.0.smithi090.stdout:9/822: readlink d1/d49/d9c/daf/ld2 0 2024-03-11T19:47:56.896 INFO:tasks.workunit.client.0.smithi090.stdout:4/877: mknod d4/d51/d8c/c128 0 2024-03-11T19:47:57.112 INFO:tasks.workunit.client.0.smithi090.stdout:0/827: creat d2/d11/d15/d29/d43/d58/f107 x:0 0 0 2024-03-11T19:47:57.255 INFO:tasks.workunit.client.0.smithi090.stdout:1/815: mkdir d5/d31/d76/d114 0 2024-03-11T19:47:57.908 INFO:tasks.workunit.client.0.smithi090.stdout:7/812: link d3/l41 d3/dc/d1e/d7c/d96/d32/d71/l118 0 2024-03-11T19:47:58.199 INFO:tasks.workunit.client.0.smithi090.stdout:7/813: dread d3/d55/fb4 [0,4194304] 0 2024-03-11T19:47:58.231 INFO:tasks.workunit.client.0.smithi090.stdout:3/773: creat d2/db/d15/d1a/d75/de8/d6a/ff9 x:0 0 0 2024-03-11T19:47:58.262 INFO:tasks.workunit.client.0.smithi090.stdout:8/882: write d0/d43/dc/f28 [1367131,43668] 0 2024-03-11T19:47:58.560 INFO:tasks.workunit.client.0.smithi090.stdout:1/816: truncate d5/d65/d6b/fa4 603318 0 2024-03-11T19:47:58.561 INFO:tasks.workunit.client.0.smithi090.stdout:0/828: creat d2/d11/d3a/d64/dac/f108 x:0 0 0 2024-03-11T19:47:58.797 INFO:tasks.workunit.client.0.smithi090.stdout:7/814: chown d3/ca 20 1 2024-03-11T19:47:59.199 INFO:tasks.workunit.client.0.smithi090.stdout:0/829: fdatasync d2/d11/f24 0 2024-03-11T19:47:59.267 INFO:tasks.workunit.client.0.smithi090.stdout:1/817: truncate d5/da/d23/dfe/f50 1487356 0 2024-03-11T19:47:59.567 INFO:tasks.workunit.client.0.smithi090.stdout:6/872: dwrite d4/d13/d22/d3a/dd7/f124 [0,4194304] 0 2024-03-11T19:47:59.671 INFO:tasks.workunit.client.0.smithi090.stdout:2/811: fsync d0/de/d12/d46/d4c/d64/fa3 0 2024-03-11T19:47:59.774 INFO:tasks.workunit.client.0.smithi090.stdout:9/823: rename d1/d49 to d1/dd/d18/d25/d101/d1c/d21/dab/d114 0 2024-03-11T19:48:00.090 INFO:tasks.workunit.client.0.smithi090.stdout:7/815: dwrite d3/dc/d1e/d7c/d96/d32/d45/de0/f116 [0,4194304] 0 2024-03-11T19:48:00.392 INFO:tasks.workunit.client.0.smithi090.stdout:4/878: symlink d4/l129 0 2024-03-11T19:48:00.479 INFO:tasks.workunit.client.0.smithi090.stdout:0/830: creat d2/d11/d15/d29/d2e/d6d/da4/f109 x:0 0 0 2024-03-11T19:48:00.961 INFO:tasks.workunit.client.0.smithi090.stdout:5/815: write d1/f1e [1229821,73391] 0 2024-03-11T19:48:01.008 INFO:tasks.workunit.client.0.smithi090.stdout:6/873: rmdir d4/d13/d22/d34/d40/da4 39 2024-03-11T19:48:01.353 INFO:tasks.workunit.client.0.smithi090.stdout:8/883: rename d0/d53/d59/d96/fde to d0/d43/dc/f123 0 2024-03-11T19:48:01.372 INFO:tasks.workunit.client.0.smithi090.stdout:8/884: readlink d0/d60/db6/l11e 0 2024-03-11T19:48:01.944 INFO:tasks.workunit.client.0.smithi090.stdout:8/885: sync 2024-03-11T19:48:02.113 INFO:tasks.workunit.client.0.smithi090.stdout:0/831: creat d2/d11/d15/d29/d2e/d6d/da4/d78/f10a x:0 0 0 2024-03-11T19:48:02.644 INFO:tasks.workunit.client.0.smithi090.stdout:5/816: fdatasync d1/d26/f20 0 2024-03-11T19:48:03.182 INFO:tasks.workunit.client.0.smithi090.stdout:8/886: creat d0/d3/d86/d93/dba/f124 x:0 0 0 2024-03-11T19:48:03.370 INFO:tasks.workunit.client.0.smithi090.stdout:0/832: creat d2/d11/d3a/d64/f10b x:0 0 0 2024-03-11T19:48:03.845 INFO:tasks.workunit.client.0.smithi090.stdout:5/817: mknod d1/d17/d39/d51/d53/d7b/da0/c117 0 2024-03-11T19:48:03.860 INFO:tasks.workunit.client.0.smithi090.stdout:1/818: write d5/d31/d76/d77/dbd/fd6 [919893,24367] 0 2024-03-11T19:48:04.095 INFO:tasks.workunit.client.0.smithi090.stdout:8/887: dread - d0/d60/fb5 zero size 2024-03-11T19:48:04.271 INFO:tasks.workunit.client.0.smithi090.stdout:0/833: truncate d2/d11/d3a/f42 5053920 0 2024-03-11T19:48:04.339 INFO:tasks.workunit.client.0.smithi090.stdout:0/834: readlink d2/d11/d15/d29/d2e/d6d/da4/l81 0 2024-03-11T19:48:04.376 INFO:tasks.workunit.client.0.smithi090.stdout:6/874: creat d4/d13/d22/d42/d48/f125 x:0 0 0 2024-03-11T19:48:04.534 INFO:tasks.workunit.client.0.smithi090.stdout:1/819: rmdir d5/da/d23/dfe 39 2024-03-11T19:48:04.624 INFO:tasks.workunit.client.0.smithi090.stdout:7/816: write d3/dc/da6/f102 [621718,49541] 0 2024-03-11T19:48:04.722 INFO:tasks.workunit.client.0.smithi090.stdout:9/824: dwrite d1/dd/d18/f65 [0,4194304] 0 2024-03-11T19:48:05.082 INFO:tasks.workunit.client.0.smithi090.stdout:6/875: mkdir d4/d13/d22/d34/d83/d9c/d126 0 2024-03-11T19:48:05.245 INFO:tasks.workunit.client.0.smithi090.stdout:4/879: dwrite d4/dd8/f124 [0,4194304] 0 2024-03-11T19:48:05.964 INFO:tasks.workunit.client.0.smithi090.stdout:3/774: creat d2/db/d15/ffa x:0 0 0 2024-03-11T19:48:06.172 INFO:tasks.workunit.client.0.smithi090.stdout:2/812: dwrite d0/fd [0,4194304] 0 2024-03-11T19:48:06.322 INFO:tasks.workunit.client.0.smithi090.stdout:8/888: truncate d0/d3/d86/fd6 1261888 0 2024-03-11T19:48:06.664 INFO:tasks.workunit.client.0.smithi090.stdout:8/889: chown d0/d3/d4c/d8d/f94 1 1 2024-03-11T19:48:06.745 INFO:tasks.workunit.client.0.smithi090.stdout:3/775: symlink d2/d2f/dc4/lfb 0 2024-03-11T19:48:06.754 INFO:tasks.workunit.client.0.smithi090.stdout:5/818: dwrite d1/d17/d39/f10e [0,4194304] 0 2024-03-11T19:48:07.509 INFO:tasks.workunit.client.0.smithi090.stdout:9/825: rename d1/c74 to d1/dbb/c115 0 2024-03-11T19:48:07.658 INFO:tasks.workunit.client.0.smithi090.stdout:5/819: unlink d1/d26/dc8/c10f 0 2024-03-11T19:48:07.924 INFO:tasks.workunit.client.0.smithi090.stdout:8/890: creat d0/d3/d4c/d8d/df8/d113/f125 x:0 0 0 2024-03-11T19:48:08.166 INFO:tasks.workunit.client.0.smithi090.stdout:2/813: truncate d0/de/d12/d46/d4c/fe9 202988 0 2024-03-11T19:48:08.653 INFO:tasks.workunit.client.0.smithi090.stdout:1/820: rename d5/d4e/d54/d94/c100 to d5/d31/d4c/daf/c115 0 2024-03-11T19:48:08.883 INFO:tasks.workunit.client.0.smithi090.stdout:9/826: creat d1/dd/d18/d25/d101/d1c/d9f/f116 x:0 0 0 2024-03-11T19:48:09.254 INFO:tasks.workunit.client.0.smithi090.stdout:3/776: mkdir d2/db/d15/d1a/d75/ded/dfc 0 2024-03-11T19:48:09.265 INFO:tasks.workunit.client.0.smithi090.stdout:8/891: truncate d0/d3/d86/f9c 3229592 0 2024-03-11T19:48:09.457 INFO:tasks.workunit.client.0.smithi090.stdout:2/814: mknod d0/de/d12/d46/d5d/d8d/cfa 0 2024-03-11T19:48:09.637 INFO:tasks.workunit.client.0.smithi090.stdout:6/876: link d4/d13/d22/d34/d40/leb d4/d13/d58/l127 0 2024-03-11T19:48:09.753 INFO:tasks.workunit.client.0.smithi090.stdout:7/817: dwrite d3/dc/d1e/d7c/d96/d32/d7b/d99/fc5 [0,4194304] 0 2024-03-11T19:48:10.158 INFO:tasks.workunit.client.0.smithi090.stdout:9/827: mknod d1/dd/d18/d25/d101/d1c/d9f/c117 0 2024-03-11T19:48:10.292 INFO:tasks.workunit.client.0.smithi090.stdout:0/835: dwrite d2/d11/d15/d29/d3d/d7e/fc0 [0,4194304] 0 2024-03-11T19:48:10.459 INFO:tasks.workunit.client.0.smithi090.stdout:4/880: truncate d4/dd8/f124 49969 0 2024-03-11T19:48:11.161 INFO:tasks.workunit.client.0.smithi090.stdout:2/815: rename d0/de/d1f/f55 to d0/de/d12/d81/df0/ffb 0 2024-03-11T19:48:11.302 INFO:tasks.workunit.client.0.smithi090.stdout:3/777: truncate d2/db/d15/d1a/d75/de8/d6a/dcb/f3b 914930 0 2024-03-11T19:48:12.619 INFO:tasks.workunit.client.0.smithi090.stdout:6/877: rename d4/d13/d22/d34/d40/d7d/dc2 to d4/d13/d22/d34/d40/dd5/d80/dd4/d128 0 2024-03-11T19:48:12.831 INFO:tasks.workunit.client.0.smithi090.stdout:2/816: chown d0/de/f16 41 1 2024-03-11T19:48:12.983 INFO:tasks.workunit.client.0.smithi090.stdout:0/836: chown d2/d11/d3a/f42 24407 1 2024-03-11T19:48:13.123 INFO:tasks.workunit.client.0.smithi090.stdout:1/821: write d5/d65/d6b/d7f/fd3 [999283,90963] 0 2024-03-11T19:48:13.912 INFO:tasks.workunit.client.0.smithi090.stdout:5/820: dwrite d1/d17/d39/d51/d53/d7b/f114 [0,4194304] 0 2024-03-11T19:48:14.071 INFO:tasks.workunit.client.0.smithi090.stdout:6/878: fsync d4/d13/d22/d42/d48/dd2/ff0 0 2024-03-11T19:48:14.160 INFO:tasks.workunit.client.0.smithi090.stdout:8/892: dwrite d0/d43/dc/f3a [0,4194304] 0 2024-03-11T19:48:14.279 INFO:tasks.workunit.client.0.smithi090.stdout:5/821: chown d1/d26/c7a 148 1 2024-03-11T19:48:14.281 INFO:tasks.workunit.client.0.smithi090.stdout:3/778: read - d2/db/d15/d69/dad/fe4 zero size 2024-03-11T19:48:14.295 INFO:tasks.workunit.client.0.smithi090.stdout:0/837: creat d2/d11/d3f/f10c x:0 0 0 2024-03-11T19:48:14.342 INFO:tasks.workunit.client.0.smithi090.stdout:8/893: chown d0/d3/fdd 1 1 2024-03-11T19:48:14.416 INFO:tasks.workunit.client.0.smithi090.stdout:8/894: chown d0/d3/d86/d93/dda/ff6 12647 1 2024-03-11T19:48:14.622 INFO:tasks.workunit.client.0.smithi090.stdout:7/818: link d3/d55/f64 d3/dc/d1e/d7c/d96/d32/d71/d80/f119 0 2024-03-11T19:48:14.649 INFO:tasks.workunit.client.0.smithi090.stdout:9/828: creat d1/dd/d18/d25/d101/d1c/f118 x:0 0 0 2024-03-11T19:48:14.738 INFO:tasks.workunit.client.0.smithi090.stdout:9/829: dread - d1/dd/d18/d3a/d48/fc1 zero size 2024-03-11T19:48:15.261 INFO:tasks.workunit.client.0.smithi090.stdout:6/879: stat d4/d13/d22/d34/d40/dd5/fff 0 2024-03-11T19:48:15.507 INFO:tasks.workunit.client.0.smithi090.stdout:5/822: symlink d1/d17/d39/d51/d53/d7b/d3b/d76/db7/de0/l118 0 2024-03-11T19:48:15.801 INFO:tasks.workunit.client.0.smithi090.stdout:4/881: write d4/d27/d28/ddc/f122 [818984,115937] 0 2024-03-11T19:48:15.871 INFO:tasks.workunit.client.0.smithi090.stdout:8/895: mknod d0/d3/d86/d93/dda/c126 0 2024-03-11T19:48:15.877 INFO:tasks.workunit.client.0.smithi090.stdout:0/838: dread d2/d11/f41 [0,4194304] 0 2024-03-11T19:48:15.905 INFO:tasks.workunit.client.0.smithi090.stdout:4/882: dread - d4/d51/f92 zero size 2024-03-11T19:48:16.477 INFO:tasks.workunit.client.0.smithi090.stdout:3/779: dread - d2/d45/fd0 zero size 2024-03-11T19:48:17.101 INFO:tasks.workunit.client.0.smithi090.stdout:4/883: symlink d4/d10e/l12a 0 2024-03-11T19:48:17.405 INFO:tasks.workunit.client.0.smithi090.stdout:0/839: rename d2/d11/d3f/fe0 to d2/d11/d3a/d64/dac/f10d 0 2024-03-11T19:48:17.467 INFO:tasks.workunit.client.0.smithi090.stdout:7/819: mknod d3/dc/d1e/d7c/d96/d32/d7b/c11a 0 2024-03-11T19:48:17.712 INFO:tasks.workunit.client.0.smithi090.stdout:3/780: readlink d2/l3e 0 2024-03-11T19:48:17.945 INFO:tasks.workunit.client.0.smithi090.stdout:4/884: fsync d4/dd8/dfa/db7/fd5 0 2024-03-11T19:48:18.040 INFO:tasks.workunit.client.0.smithi090.stdout:2/817: dwrite d0/de/d1f/f1c [4194304,4194304] 0 2024-03-11T19:48:18.190 INFO:tasks.workunit.client.0.smithi090.stdout:7/820: fsync d3/dc/d1e/f46 0 2024-03-11T19:48:18.334 INFO:tasks.workunit.client.0.smithi090.stdout:9/830: creat d1/dd/d18/d3a/f119 x:0 0 0 2024-03-11T19:48:18.652 INFO:tasks.workunit.client.0.smithi090.stdout:1/822: dwrite d5/d31/d36/f49 [4194304,4194304] 0 2024-03-11T19:48:18.663 INFO:tasks.workunit.client.0.smithi090.stdout:0/840: symlink d2/d11/d15/d29/d2e/l10e 0 2024-03-11T19:48:18.846 INFO:tasks.workunit.client.0.smithi090.stdout:0/841: readlink d2/d11/d15/d29/l2a 0 2024-03-11T19:48:18.905 INFO:tasks.workunit.client.0.smithi090.stdout:2/818: mknod d0/de/db6/d29/cfc 0 2024-03-11T19:48:18.939 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:48:18.941 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:48:19.099 INFO:tasks.workunit.client.0.smithi090.stdout:4/885: rename d4/dd8/dfa/db7/fe7 to d4/d27/d28/d38/d64/d119/f12b 0 2024-03-11T19:48:19.609 INFO:tasks.workunit.client.0.smithi090.stdout:9/831: creat d1/dd/d18/d25/d101/d1c/d88/f11a x:0 0 0 2024-03-11T19:48:19.631 INFO:tasks.workunit.client.0.smithi090.stdout:0/842: fsync d2/d11/d3a/f42 0 2024-03-11T19:48:20.023 INFO:tasks.workunit.client.0.smithi090.stdout:7/821: rename d3/dc/c6d to d3/dc/d1e/d7c/d96/d32/da0/c11b 0 2024-03-11T19:48:20.068 INFO:tasks.workunit.client.0.smithi090.stdout:1/823: dread d5/d4e/d54/d94/fb0 [0,4194304] 0 2024-03-11T19:48:20.472 INFO:tasks.workunit.client.0.smithi090.stdout:5/823: write d1/d26/f83 [432760,37548] 0 2024-03-11T19:48:20.684 INFO:tasks.workunit.client.0.smithi090.stdout:6/880: dwrite d4/d13/d22/d34/d83/fd6 [0,4194304] 0 2024-03-11T19:48:20.873 INFO:tasks.workunit.client.0.smithi090.stdout:0/843: mkdir d2/d11/d3a/da6/d10f 0 2024-03-11T19:48:20.963 INFO:tasks.workunit.client.0.smithi090.stdout:8/896: write d0/d43/dc/f8e [4349,56807] 0 2024-03-11T19:48:21.168 INFO:tasks.workunit.client.0.smithi090.stdout:1/824: mkdir d5/d31/d4c/d116 0 2024-03-11T19:48:21.639 INFO:tasks.workunit.client.0.smithi090.stdout:0/844: mknod d2/d11/d3a/d64/dac/c110 0 2024-03-11T19:48:22.143 INFO:tasks.workunit.client.0.smithi090.stdout:7/822: mkdir d3/dc/d1e/d7c/d96/d32/d67/dee/d11c 0 2024-03-11T19:48:22.512 INFO:tasks.workunit.client.0.smithi090.stdout:1/825: symlink d5/d65/d6b/d7f/d10c/l117 0 2024-03-11T19:48:23.132 INFO:tasks.workunit.client.0.smithi090.stdout:7/823: creat d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/db5/de1/dc6/d111/f11d x:0 0 0 2024-03-11T19:48:23.359 INFO:tasks.workunit.client.0.smithi090.stdout:3/781: creat d2/ffd x:0 0 0 2024-03-11T19:48:23.674 INFO:tasks.workunit.client.0.smithi090.stdout:7/824: sync 2024-03-11T19:48:23.870 INFO:tasks.workunit.client.0.smithi090.stdout:4/886: write d4/d27/f57 [5026233,70552] 0 2024-03-11T19:48:23.930 INFO:tasks.workunit.client.0.smithi090.stdout:2/819: write d0/fae [1947389,109749] 0 2024-03-11T19:48:23.962 INFO:tasks.workunit.client.0.smithi090.stdout:9/832: write d1/dd/d18/d25/d101/d1c/d21/f9d [4283190,95646] 0 2024-03-11T19:48:24.336 INFO:tasks.workunit.client.0.smithi090.stdout:5/824: dwrite d1/d17/d39/d51/d53/d7b/d6f/f8e [0,4194304] 0 2024-03-11T19:48:24.470 INFO:tasks.workunit.client.0.smithi090.stdout:9/833: chown d1/dd/d18/d25/d101/d1c/d21/d2a/l95 2791 1 2024-03-11T19:48:24.540 INFO:tasks.workunit.client.0.smithi090.stdout:6/881: write d4/d14/f6a [1202672,43786] 0 2024-03-11T19:48:24.585 INFO:tasks.workunit.client.0.smithi090.stdout:6/882: stat d4/d13/d58/c113 0 2024-03-11T19:48:24.601 INFO:tasks.workunit.client.0.smithi090.stdout:0/845: write d2/f1b [3101098,42546] 0 2024-03-11T19:48:24.620 INFO:tasks.workunit.client.0.smithi090.stdout:6/883: fsync d4/d13/d22/d34/d83/fd6 0 2024-03-11T19:48:24.664 INFO:tasks.workunit.client.0.smithi090.stdout:0/846: chown d2/d11/d15/d29/l2a 14257 1 2024-03-11T19:48:24.693 INFO:tasks.workunit.client.0.smithi090.stdout:3/782: creat d2/db/ffe x:0 0 0 2024-03-11T19:48:24.782 INFO:tasks.workunit.client.0.smithi090.stdout:7/825: link d3/dc/d1e/d7c/d96/d32/d7b/d99/c10b d3/dc/da6/c11e 0 2024-03-11T19:48:25.097 INFO:tasks.workunit.client.0.smithi090.stdout:4/887: creat d4/d27/d28/d69/d111/f12c x:0 0 0 2024-03-11T19:48:25.229 INFO:tasks.workunit.client.0.smithi090.stdout:8/897: rename d0/d3/d4c/d8d/df8/d113 to d0/d3/d86/d127 0 2024-03-11T19:48:25.481 INFO:tasks.workunit.client.0.smithi090.stdout:1/826: write d5/da/d1c/d3b/d8c/f105 [478682,82202] 0 2024-03-11T19:48:25.483 INFO:tasks.workunit.client.0.smithi090.stdout:6/884: creat d4/d13/d22/d34/d40/dd5/d80/ddb/f129 x:0 0 0 2024-03-11T19:48:25.585 INFO:tasks.workunit.client.0.smithi090.stdout:1/827: chown d5/da/d1c/c25 206 1 2024-03-11T19:48:26.065 INFO:tasks.workunit.client.0.smithi090.stdout:1/828: dread d5/d31/d36/fca [0,4194304] 0 2024-03-11T19:48:26.422 INFO:tasks.workunit.client.0.smithi090.stdout:9/834: rename d1/f58 to d1/dd/d18/d25/d101/d1c/d21/dab/d114/d9c/f11b 0 2024-03-11T19:48:26.446 INFO:tasks.workunit.client.0.smithi090.stdout:8/898: fsync d0/d43/dc/fd 0 2024-03-11T19:48:26.618 INFO:tasks.workunit.client.0.smithi090.stdout:8/899: sync 2024-03-11T19:48:26.794 INFO:tasks.workunit.client.0.smithi090.stdout:3/783: creat d2/d45/d55/fff x:0 0 0 2024-03-11T19:48:27.380 INFO:tasks.workunit.client.0.smithi090.stdout:7/826: rename d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/db5/dde/cd5 to d3/dc/d1e/d7c/d96/d32/d7b/d99/c11f 0 2024-03-11T19:48:27.561 INFO:tasks.workunit.client.0.smithi090.stdout:2/820: write d0/de/db6/fc1 [627041,64087] 0 2024-03-11T19:48:27.565 INFO:tasks.workunit.client.0.smithi090.stdout:9/835: creat d1/dd/d18/d25/d101/d1c/d21/d2a/da4/f11c x:0 0 0 2024-03-11T19:48:27.676 INFO:tasks.workunit.client.0.smithi090.stdout:5/825: write d1/d17/d39/d51/d53/d7b/da0/db9/dc6/fd7 [578863,48128] 0 2024-03-11T19:48:27.678 INFO:tasks.workunit.client.0.smithi090.stdout:8/900: dread - d0/d3/d4c/d8d/fd1 zero size 2024-03-11T19:48:27.829 INFO:tasks.workunit.client.0.smithi090.stdout:3/784: mkdir d2/db/d15/d69/d100 0 2024-03-11T19:48:28.022 INFO:tasks.workunit.client.0.smithi090.stdout:1/829: creat d5/da/dd/d112/f118 x:0 0 0 2024-03-11T19:48:28.398 INFO:tasks.workunit.client.0.smithi090.stdout:7/827: dread d3/f59 [0,4194304] 0 2024-03-11T19:48:28.748 INFO:tasks.workunit.client.0.smithi090.stdout:5/826: rename d1/d17/d39/d51/d53/d7b/d3b/d76/fa1 to d1/d17/d39/d51/d73/df6/f119 0 2024-03-11T19:48:28.886 INFO:tasks.workunit.client.0.smithi090.stdout:0/847: dwrite d2/d11/d15/f19 [0,4194304] 0 2024-03-11T19:48:29.027 INFO:tasks.workunit.client.0.smithi090.stdout:3/785: mknod d2/db/d15/d1a/d75/de8/d6a/c101 0 2024-03-11T19:48:29.150 INFO:tasks.workunit.client.0.smithi090.stdout:7/828: readlink d3/dc/d1e/lbe 0 2024-03-11T19:48:29.447 INFO:tasks.workunit.client.0.smithi090.stdout:4/888: dwrite d4/f46 [4194304,4194304] 0 2024-03-11T19:48:29.607 INFO:tasks.workunit.client.0.smithi090.stdout:6/885: write d4/d13/d22/d34/d83/d9c/fa5 [5129563,6893] 0 2024-03-11T19:48:29.999 INFO:tasks.workunit.client.0.smithi090.stdout:1/830: symlink d5/l119 0 2024-03-11T19:48:30.276 INFO:tasks.workunit.client.0.smithi090.stdout:0/848: mkdir d2/d94/d111 0 2024-03-11T19:48:30.495 INFO:tasks.workunit.client.0.smithi090.stdout:7/829: mkdir d3/dc/da6/d120 0 2024-03-11T19:48:31.177 INFO:tasks.workunit.client.0.smithi090.stdout:6/886: dread d4/d14/d8a/f63 [0,4194304] 0 2024-03-11T19:48:31.703 INFO:tasks.workunit.client.0.smithi090.stdout:2/821: link d0/de/db6/f47 d0/de/db6/d25/ffd 0 2024-03-11T19:48:31.832 INFO:tasks.workunit.client.0.smithi090.stdout:1/831: chown d5/da/d1c/d3b/fd0 127 1 2024-03-11T19:48:32.018 INFO:tasks.workunit.client.0.smithi090.stdout:5/827: creat d1/d17/d39/d51/d73/da6/d10a/f11a x:0 0 0 2024-03-11T19:48:32.232 INFO:tasks.workunit.client.0.smithi090.stdout:3/786: truncate d2/db/d15/d1a/d75/dc6/fc5 289977 0 2024-03-11T19:48:32.368 INFO:tasks.workunit.client.0.smithi090.stdout:9/836: dwrite d1/dd/d18/d3a/d57/f5b [0,4194304] 0 2024-03-11T19:48:32.500 INFO:tasks.workunit.client.0.smithi090.stdout:8/901: write d0/d43/dc/f114 [447353,82978] 0 2024-03-11T19:48:33.069 INFO:tasks.workunit.client.0.smithi090.stdout:2/822: creat d0/de/d1f/df4/ffe x:0 0 0 2024-03-11T19:48:33.408 INFO:tasks.workunit.client.0.smithi090.stdout:5/828: read d1/f4a [9117,113732] 0 2024-03-11T19:48:33.459 INFO:tasks.workunit.client.0.smithi090.stdout:1/832: dread d5/da/f4a [0,4194304] 0 2024-03-11T19:48:33.529 INFO:tasks.workunit.client.0.smithi090.stdout:3/787: stat d2/db/d15/d1a/d3c/c79 0 2024-03-11T19:48:34.535 INFO:tasks.workunit.client.0.smithi090.stdout:4/889: dwrite d4/f12 [0,4194304] 0 2024-03-11T19:48:35.026 INFO:tasks.workunit.client.0.smithi090.stdout:8/902: mkdir d0/d3/d128 0 2024-03-11T19:48:35.059 INFO:tasks.workunit.client.0.smithi090.stdout:6/887: creat d4/d13/d22/d34/d40/f12a x:0 0 0 2024-03-11T19:48:35.367 INFO:tasks.workunit.client.0.smithi090.stdout:4/890: dread d4/d51/d8c/d89/fbd [0,4194304] 0 2024-03-11T19:48:35.514 INFO:tasks.workunit.client.0.smithi090.stdout:8/903: creat d0/d43/dc/d90/f129 x:0 0 0 2024-03-11T19:48:35.530 INFO:tasks.workunit.client.0.smithi090.stdout:0/849: write d2/d11/d3a/d64/dac/fca [596330,59860] 0 2024-03-11T19:48:35.776 INFO:tasks.workunit.client.0.smithi090.stdout:0/850: sync 2024-03-11T19:48:35.972 INFO:tasks.workunit.client.0.smithi090.stdout:5/829: creat d1/d17/d39/f11b x:0 0 0 2024-03-11T19:48:35.997 INFO:tasks.workunit.client.0.smithi090.stdout:7/830: write d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/fcb [273133,95725] 0 2024-03-11T19:48:36.096 INFO:tasks.workunit.client.0.smithi090.stdout:8/904: creat d0/d43/dc/d90/f12a x:0 0 0 2024-03-11T19:48:36.106 INFO:tasks.workunit.client.0.smithi090.stdout:4/891: chown d4/d27/d28/l45 197 1 2024-03-11T19:48:36.468 INFO:tasks.workunit.client.0.smithi090.stdout:0/851: dread d2/d11/d15/d29/d2e/d6d/da4/fee [0,4194304] 0 2024-03-11T19:48:36.570 INFO:tasks.workunit.client.0.smithi090.stdout:5/830: creat d1/d17/d39/d51/d53/d7b/da0/dc0/f11c x:0 0 0 2024-03-11T19:48:36.595 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:13:56.353 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:48:36.612 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:13:56.371 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:48:36.660 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:13:56.358 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:48:36.670 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:13:56.360 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:48:36.672 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:13:56.370 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:48:36.691 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:13:56.381 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:48:36.746 INFO:tasks.workunit.client.0.smithi090.stdout:4/892: fsync d4/d51/d8c/d89/fc7 0 2024-03-11T19:48:36.781 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:13:56.539 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:48:36.867 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:13:56.557 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:48:36.871 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:13:56.569 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:48:37.083 INFO:tasks.workunit.client.0.smithi090.stdout:0/852: unlink d2/d11/d15/d29/c4b 0 2024-03-11T19:48:37.452 INFO:tasks.workunit.client.0.smithi090.stdout:5/831: mkdir d1/d26/d19/d11d 0 2024-03-11T19:48:37.505 INFO:tasks.workunit.client.0.smithi090.stdout:4/893: creat d4/d51/f12d x:0 0 0 2024-03-11T19:48:37.576 INFO:tasks.workunit.client.0.smithi090.stdout:6/888: getdents d4/d13/d22/d3a/d6f/dd3 0 2024-03-11T19:48:37.612 INFO:tasks.workunit.client.0.smithi090.stdout:2/823: write d0/d21/f6e [2575226,61440] 0 2024-03-11T19:48:38.314 INFO:tasks.workunit.client.0.smithi090.stdout:4/894: truncate d4/d18/fb0 4372797 0 2024-03-11T19:48:38.388 INFO:tasks.workunit.client.0.smithi090.stdout:2/824: creat d0/d21/dce/fff x:0 0 0 2024-03-11T19:48:38.396 INFO:tasks.workunit.client.0.smithi090.stdout:0/853: creat d2/d11/d15/d29/d3d/d8c/f112 x:0 0 0 2024-03-11T19:48:38.615 INFO:tasks.workunit.client.0.smithi090.stdout:4/895: write d4/d51/f12d [530297,3983] 0 2024-03-11T19:48:38.643 INFO:tasks.workunit.client.0.smithi090.stdout:1/833: dwrite d5/da/f92 [0,4194304] 0 2024-03-11T19:48:38.832 INFO:tasks.workunit.client.0.smithi090.stdout:7/831: getdents d3/dc/d1e/d7c/d96/d32/d7b 0 2024-03-11T19:48:39.186 INFO:tasks.workunit.client.0.smithi090.stdout:1/834: chown d5/d4e/d79/led 505 1 2024-03-11T19:48:39.258 INFO:tasks.workunit.client.0.smithi090.stdout:6/889: dread d4/d13/d22/d34/d83/d9c/fa5 [4194304,4194304] 0 2024-03-11T19:48:39.260 INFO:tasks.workunit.client.0.smithi090.stdout:2/825: read d0/f3 [1218380,65873] 0 2024-03-11T19:48:39.361 INFO:tasks.workunit.client.0.smithi090.stdout:3/788: dwrite d2/db/d15/ff3 [0,4194304] 0 2024-03-11T19:48:39.453 INFO:tasks.workunit.client.0.smithi090.stdout:1/835: chown d5/da/d23 224437 1 2024-03-11T19:48:39.530 INFO:tasks.workunit.client.0.smithi090.stdout:4/896: creat d4/dd8/d66/f12e x:0 0 0 2024-03-11T19:48:39.713 INFO:tasks.workunit.client.0.smithi090.stdout:8/905: write d0/d43/dc/f50 [423523,67592] 0 2024-03-11T19:48:40.137 INFO:tasks.workunit.client.0.smithi090.stdout:9/837: rmdir d1/d47 39 2024-03-11T19:48:40.404 INFO:tasks.workunit.client.0.smithi090.stdout:0/854: mknod d2/d11/d15/d29/d43/dcd/c113 0 2024-03-11T19:48:40.487 INFO:tasks.workunit.client.0.smithi090.stdout:5/832: dwrite d1/d26/f9f [0,4194304] 0 2024-03-11T19:48:40.684 INFO:tasks.workunit.client.0.smithi090.stdout:5/833: chown d1/d17/d39/d51/d53/d7b/d3b/cae 1 1 2024-03-11T19:48:40.750 INFO:tasks.workunit.client.0.smithi090.stdout:2/826: fsync d0/de/d1f/ff1 0 2024-03-11T19:48:40.758 INFO:tasks.workunit.client.0.smithi090.stdout:6/890: mknod d4/d13/d22/d3a/dd7/c12b 0 2024-03-11T19:48:40.773 INFO:tasks.workunit.client.0.smithi090.stdout:1/836: mkdir d5/da/d1c/d11a 0 2024-03-11T19:48:41.376 INFO:tasks.workunit.client.0.smithi090.stdout:7/832: mknod d3/dc/d1e/d7c/d96/d32/c121 0 2024-03-11T19:48:42.128 INFO:tasks.workunit.client.0.smithi090.stdout:5/834: readlink d1/l88 0 2024-03-11T19:48:42.629 INFO:tasks.workunit.client.0.smithi090.stdout:6/891: read - d4/d13/d22/d34/d40/f12a zero size 2024-03-11T19:48:43.089 INFO:tasks.workunit.client.0.smithi090.stdout:7/833: creat d3/d55/f122 x:0 0 0 2024-03-11T19:48:43.320 INFO:tasks.workunit.client.0.smithi090.stdout:4/897: dwrite d4/d27/d28/f63 [4194304,4194304] 0 2024-03-11T19:48:43.446 INFO:tasks.workunit.client.0.smithi090.stdout:0/855: mkdir d2/d11/d15/d29/d114 0 2024-03-11T19:48:43.802 INFO:tasks.workunit.client.0.smithi090.stdout:1/837: mknod d5/da/d23/c11b 0 2024-03-11T19:48:43.829 INFO:tasks.workunit.client.0.smithi090.stdout:1/838: chown d5/da/lf 23481 1 2024-03-11T19:48:44.287 INFO:tasks.workunit.client.0.smithi090.stdout:9/838: write d1/dd/d18/d25/d101/d1c/d77/ff6 [971702,2708] 0 2024-03-11T19:48:44.293 INFO:tasks.workunit.client.0.smithi090.stdout:6/892: mknod d4/d13/d22/d34/d83/d9c/db7/c12c 0 2024-03-11T19:48:44.310 INFO:tasks.workunit.client.0.smithi090.stdout:3/789: write d2/db/f92 [4031248,63386] 0 2024-03-11T19:48:44.965 INFO:tasks.workunit.client.0.smithi090.stdout:4/898: rmdir d4/dd8/d66 39 2024-03-11T19:48:45.038 INFO:tasks.workunit.client.0.smithi090.stdout:8/906: dwrite d0/d3/d98/da2/f11c [0,4194304] 0 2024-03-11T19:48:45.162 INFO:tasks.workunit.client.0.smithi090.stdout:0/856: read - d2/d11/d15/d36/fe4 zero size 2024-03-11T19:48:45.260 INFO:tasks.workunit.client.0.smithi090.stdout:8/907: read - d0/d3/d3e/fc2 zero size 2024-03-11T19:48:45.341 INFO:tasks.workunit.client.0.smithi090.stdout:1/839: mkdir d5/d4e/d54/d94/d11c 0 2024-03-11T19:48:45.871 INFO:tasks.workunit.client.0.smithi090.stdout:3/790: creat d2/db/d15/d69/dad/f102 x:0 0 0 2024-03-11T19:48:46.146 INFO:tasks.workunit.client.0.smithi090.stdout:3/791: sync 2024-03-11T19:48:46.215 INFO:tasks.workunit.client.0.smithi090.stdout:3/792: chown d2/db/d15/d1a/d75/ded 111614 1 2024-03-11T19:48:46.425 INFO:tasks.workunit.client.0.smithi090.stdout:4/899: truncate d4/d51/d8c/d89/f98 5226051 0 2024-03-11T19:48:46.468 INFO:tasks.workunit.client.0.smithi090.stdout:8/908: truncate d0/d3/f80 499972 0 2024-03-11T19:48:46.577 INFO:tasks.workunit.client.0.smithi090.stdout:8/909: sync 2024-03-11T19:48:46.695 INFO:tasks.workunit.client.0.smithi090.stdout:1/840: creat d5/d65/d6b/de7/f11d x:0 0 0 2024-03-11T19:48:47.097 INFO:tasks.workunit.client.0.smithi090.stdout:2/827: dwrite d0/de/d12/d46/d4c/fe0 [0,4194304] 0 2024-03-11T19:48:47.119 INFO:tasks.workunit.client.0.smithi090.stdout:9/839: rename d1/dd/d18/d25/d101/c27 to d1/d1d/d6a/c11d 0 2024-03-11T19:48:47.212 INFO:tasks.workunit.client.0.smithi090.stdout:9/840: sync 2024-03-11T19:48:47.331 INFO:tasks.workunit.client.0.smithi090.stdout:0/857: fdatasync d2/d11/d15/d29/d3d/d7b/fad 0 2024-03-11T19:48:47.605 INFO:tasks.workunit.client.0.smithi090.stdout:7/834: dwrite d3/dc/d1e/d7c/d96/fb8 [0,4194304] 0 2024-03-11T19:48:47.819 INFO:tasks.workunit.client.0.smithi090.stdout:1/841: truncate d5/d65/d6b/dcf/feb 376758 0 2024-03-11T19:48:47.871 INFO:tasks.workunit.client.0.smithi090.stdout:1/842: read d5/d4e/d54/f6e [8524024,101604] 0 2024-03-11T19:48:48.324 INFO:tasks.workunit.client.0.smithi090.stdout:2/828: symlink d0/de/d12/d46/d75/dbb/l100 0 2024-03-11T19:48:48.349 INFO:tasks.workunit.client.0.smithi090.stdout:5/835: write d1/d17/d39/d51/d53/d7b/d6f/f72 [633360,78003] 0 2024-03-11T19:48:49.059 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:48:49.062 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:48:49.257 INFO:tasks.workunit.client.0.smithi090.stdout:2/829: fdatasync d0/de/d12/d46/d5d/fcd 0 2024-03-11T19:48:49.400 INFO:tasks.workunit.client.0.smithi090.stdout:6/893: dwrite d4/d13/d22/d34/d40/d7d/fa6 [0,4194304] 0 2024-03-11T19:48:49.932 INFO:tasks.workunit.client.0.smithi090.stdout:0/858: link d2/d11/d15/d36/f7c d2/d11/d3a/d64/f115 0 2024-03-11T19:48:49.955 INFO:tasks.workunit.client.0.smithi090.stdout:0/859: stat d2/d11/d15/f74 0 2024-03-11T19:48:49.981 INFO:tasks.workunit.client.0.smithi090.stdout:0/860: chown d2/d11/d15/f74 29405899 1 2024-03-11T19:48:50.059 INFO:tasks.workunit.client.0.smithi090.stdout:4/900: write d4/d27/d28/fe6 [534136,53467] 0 2024-03-11T19:48:50.797 INFO:tasks.workunit.client.0.smithi090.stdout:8/910: dwrite d0/d43/f84 [4194304,4194304] 0 2024-03-11T19:48:50.847 INFO:tasks.workunit.client.0.smithi090.stdout:1/843: mknod d5/df5/c11e 0 2024-03-11T19:48:51.049 INFO:tasks.workunit.client.0.smithi090.stdout:6/894: dread d4/d13/d22/d34/d40/da4/f9a [0,4194304] 0 2024-03-11T19:48:51.175 INFO:tasks.workunit.client.0.smithi090.stdout:1/844: readlink d5/d31/d4c/l7e 0 2024-03-11T19:48:51.205 INFO:tasks.workunit.client.0.smithi090.stdout:8/911: readlink d0/d3/d98/da2/dcc/ldc 0 2024-03-11T19:48:51.347 INFO:tasks.workunit.client.0.smithi090.stdout:3/793: dwrite d2/db/d15/d49/f7f [0,4194304] 0 2024-03-11T19:48:51.478 INFO:tasks.workunit.client.0.smithi090.stdout:4/901: truncate d4/f4a 2321579 0 2024-03-11T19:48:51.882 INFO:tasks.workunit.client.0.smithi090.stdout:7/835: dwrite d3/dc/d1e/d7c/d96/d32/da0/f10d [0,4194304] 0 2024-03-11T19:48:51.990 INFO:tasks.workunit.client.0.smithi090.stdout:9/841: link d1/dd/d18/d3a/db6/de6/lff d1/dd/d18/d3a/l11e 0 2024-03-11T19:48:52.582 INFO:tasks.workunit.client.0.smithi090.stdout:1/845: truncate d5/da/d1c/d3b/fd0 1385566 0 2024-03-11T19:48:52.601 INFO:tasks.workunit.client.0.smithi090.stdout:5/836: write d1/d17/d39/d51/d53/d64/f108 [1046780,69264] 0 2024-03-11T19:48:53.009 INFO:tasks.workunit.client.0.smithi090.stdout:7/836: dread d3/dc/da6/f102 [0,4194304] 0 2024-03-11T19:48:53.285 INFO:tasks.workunit.client.0.smithi090.stdout:9/842: creat d1/dd/d18/d3a/dd8/f11f x:0 0 0 2024-03-11T19:48:53.393 INFO:tasks.workunit.client.0.smithi090.stdout:2/830: write d0/de/db6/d29/f2e [181331,63211] 0 2024-03-11T19:48:53.439 INFO:tasks.workunit.client.0.smithi090.stdout:4/902: mknod d4/dd8/c12f 0 2024-03-11T19:48:53.705 INFO:tasks.workunit.client.0.smithi090.stdout:0/861: dwrite d2/f37 [0,4194304] 0 2024-03-11T19:48:54.058 INFO:tasks.workunit.client.0.smithi090.stdout:5/837: creat d1/d17/d39/d51/d53/d7b/d3b/d76/db7/de0/f11e x:0 0 0 2024-03-11T19:48:54.469 INFO:tasks.workunit.client.0.smithi090.stdout:1/846: dread d5/da/d23/f2a [0,4194304] 0 2024-03-11T19:48:54.480 INFO:tasks.workunit.client.0.smithi090.stdout:4/903: chown d4/dd8/dfa/ff4 45246122 1 2024-03-11T19:48:54.671 INFO:tasks.workunit.client.0.smithi090.stdout:6/895: getdents d4/d13/d22/d34/d40/dd5 0 2024-03-11T19:48:54.721 INFO:tasks.workunit.client.0.smithi090.stdout:0/862: stat d2/d11/d15/d29/d3d/f106 0 2024-03-11T19:48:55.016 INFO:tasks.workunit.client.0.smithi090.stdout:3/794: dwrite d2/d2f/f9c [0,4194304] 0 2024-03-11T19:48:55.174 INFO:tasks.workunit.client.0.smithi090.stdout:2/831: creat d0/dad/f101 x:0 0 0 2024-03-11T19:48:55.622 INFO:tasks.workunit.client.0.smithi090.stdout:7/837: creat d3/dc/d1e/d7c/d96/d32/d45/f123 x:0 0 0 2024-03-11T19:48:55.846 INFO:tasks.workunit.client.0.smithi090.stdout:6/896: write d4/d13/d22/d34/d40/f12a [696277,95120] 0 2024-03-11T19:48:56.080 INFO:tasks.workunit.client.0.smithi090.stdout:3/795: truncate d2/fb1 668965 0 2024-03-11T19:48:56.181 INFO:tasks.workunit.client.0.smithi090.stdout:4/904: creat d4/dd8/d66/f130 x:0 0 0 2024-03-11T19:48:56.724 INFO:tasks.workunit.client.0.smithi090.stdout:6/897: symlink d4/d13/d22/d42/d48/d4e/l12d 0 2024-03-11T19:48:56.730 INFO:tasks.workunit.client.0.smithi090.stdout:7/838: dread d3/f7 [4194304,4194304] 0 2024-03-11T19:48:57.338 INFO:tasks.workunit.client.0.smithi090.stdout:5/838: getdents d1/d17/d39/d51/d73/da6 0 2024-03-11T19:48:57.663 INFO:tasks.workunit.client.0.smithi090.stdout:3/796: fsync d2/d45/f4a 0 2024-03-11T19:48:57.859 INFO:tasks.workunit.client.0.smithi090.stdout:2/832: creat d0/de/dd3/f102 x:0 0 0 2024-03-11T19:48:58.845 INFO:tasks.workunit.client.0.smithi090.stdout:0/863: dwrite d2/d11/d15/d29/d2e/db4/daf/ffa [0,4194304] 0 2024-03-11T19:48:59.214 INFO:tasks.workunit.client.0.smithi090.stdout:8/912: rename d0/f17 to d0/d3/d3e/d85/d116/f12b 0 2024-03-11T19:48:59.311 INFO:tasks.workunit.client.0.smithi090.stdout:5/839: write d1/d17/d39/d51/d53/d7b/f5b [1192673,114893] 0 2024-03-11T19:48:59.564 INFO:tasks.workunit.client.0.smithi090.stdout:4/905: dwrite d4/f2e [0,4194304] 0 2024-03-11T19:48:59.696 INFO:tasks.workunit.client.0.smithi090.stdout:0/864: creat d2/d11/d3a/f116 x:0 0 0 2024-03-11T19:48:59.957 INFO:tasks.workunit.client.0.smithi090.stdout:2/833: dwrite d0/de/d1f/f40 [0,4194304] 0 2024-03-11T19:49:00.543 INFO:tasks.workunit.client.0.smithi090.stdout:3/797: getdents d2/d4f/d59/da1 0 2024-03-11T19:49:00.805 INFO:tasks.workunit.client.0.smithi090.stdout:9/843: rename d1/dd/d18/d32 to d1/dd/d18/d25/d101/d1c/d21/d2a/dc2/d100/d120 0 2024-03-11T19:49:00.927 INFO:tasks.workunit.client.0.smithi090.stdout:4/906: dread d4/d27/d28/d69/f99 [0,4194304] 0 2024-03-11T19:49:00.955 INFO:tasks.workunit.client.0.smithi090.stdout:9/844: chown d1/dd/d18/d3a/d57/ff1 52519 1 2024-03-11T19:49:00.990 INFO:tasks.workunit.client.0.smithi090.stdout:4/907: chown d4/d51/d8c/d6a/d100/c115 55134 1 2024-03-11T19:49:01.145 INFO:tasks.workunit.client.0.smithi090.stdout:2/834: unlink d0/de/d12/d46/d75/dbb/fdd 0 2024-03-11T19:49:01.585 INFO:tasks.workunit.client.0.smithi090.stdout:1/847: rename d5/da/d23/dfe/d89/l99 to d5/da/d106/l11f 0 2024-03-11T19:49:01.935 INFO:tasks.workunit.client.0.smithi090.stdout:8/913: dwrite d0/f5a [0,4194304] 0 2024-03-11T19:49:02.045 INFO:tasks.workunit.client.0.smithi090.stdout:5/840: link d1/d17/d39/d51/d53/d7b/d3b/d76/db7/l109 d1/d17/d39/d51/d53/d7b/d3b/d76/db7/de0/l11f 0 2024-03-11T19:49:02.120 INFO:tasks.workunit.client.0.smithi090.stdout:0/865: creat d2/f117 x:0 0 0 2024-03-11T19:49:02.415 INFO:tasks.workunit.client.0.smithi090.stdout:3/798: symlink d2/db/d15/d1a/d75/dc6/l103 0 2024-03-11T19:49:02.434 INFO:tasks.workunit.client.0.smithi090.stdout:2/835: mknod d0/de/d12/d46/d4c/d64/c103 0 2024-03-11T19:49:02.633 INFO:tasks.workunit.client.0.smithi090.stdout:1/848: read d5/da/f68 [3144258,44167] 0 2024-03-11T19:49:03.656 INFO:tasks.workunit.client.0.smithi090.stdout:2/836: mknod d0/de/d12/d46/de6/c104 0 2024-03-11T19:49:03.789 INFO:tasks.workunit.client.0.smithi090.stdout:1/849: mkdir d5/d4e/d120 0 2024-03-11T19:49:03.797 INFO:tasks.workunit.client.0.smithi090.stdout:9/845: write d1/dd/d18/d25/d101/f4d [4656696,24195] 0 2024-03-11T19:49:04.118 INFO:tasks.workunit.client.0.smithi090.stdout:5/841: creat d1/d26/d19/d11d/f120 x:0 0 0 2024-03-11T19:49:04.143 INFO:tasks.workunit.client.0.smithi090.stdout:5/842: readlink d1/d26/d4d/l69 0 2024-03-11T19:49:04.385 INFO:tasks.workunit.client.0.smithi090.stdout:2/837: creat d0/de/db6/d25/d8c/f105 x:0 0 0 2024-03-11T19:49:04.504 INFO:tasks.workunit.client.0.smithi090.stdout:4/908: getdents d4/d27/d28/d38/d75 0 2024-03-11T19:49:04.683 INFO:tasks.workunit.client.0.smithi090.stdout:1/850: creat d5/da/dd/f121 x:0 0 0 2024-03-11T19:49:05.025 INFO:tasks.workunit.client.0.smithi090.stdout:8/914: write d0/d3/d86/d93/dba/fdf [4864343,99859] 0 2024-03-11T19:49:05.054 INFO:tasks.workunit.client.0.smithi090.stdout:5/843: read d1/d17/d39/d51/d53/d64/f6d [263616,123792] 0 2024-03-11T19:49:05.234 INFO:tasks.workunit.client.0.smithi090.stdout:4/909: mknod d4/d27/d28/d69/c131 0 2024-03-11T19:49:05.259 INFO:tasks.workunit.client.0.smithi090.stdout:3/799: write d2/db/ffe [33775,43974] 0 2024-03-11T19:49:05.302 INFO:tasks.workunit.client.0.smithi090.stdout:2/838: read d0/de/d12/d46/d5d/f62 [3598108,95225] 0 2024-03-11T19:49:05.515 INFO:tasks.workunit.client.0.smithi090.stdout:1/851: mkdir d5/d4e/d54/d94/d122 0 2024-03-11T19:49:06.276 INFO:tasks.workunit.client.0.smithi090.stdout:8/915: mknod d0/d43/dc/c12c 0 2024-03-11T19:49:06.292 INFO:tasks.workunit.client.0.smithi090.stdout:6/898: rename d4/d13/d22/d34/d40/dd5/d80/dd4/lfb to d4/d13/l12e 0 2024-03-11T19:49:06.318 INFO:tasks.workunit.client.0.smithi090.stdout:5/844: truncate d1/d26/f3f 4920965 0 2024-03-11T19:49:06.385 INFO:tasks.workunit.client.0.smithi090.stdout:2/839: rmdir d0/de/d12/d46/de8 39 2024-03-11T19:49:06.472 INFO:tasks.workunit.client.0.smithi090.stdout:8/916: chown d0/d3/d3e/d85/d116/d11d/cf2 8 1 2024-03-11T19:49:06.506 INFO:tasks.workunit.client.0.smithi090.stdout:9/846: truncate d1/d47/f90 1332662 0 2024-03-11T19:49:06.659 INFO:tasks.workunit.client.0.smithi090.stdout:3/800: truncate d2/db/d15/d69/dad/fa4 667064 0 2024-03-11T19:49:06.764 INFO:tasks.workunit.client.0.smithi090.stdout:1/852: unlink d5/d31/d76/d77/dbd/fd6 0 2024-03-11T19:49:07.533 INFO:tasks.workunit.client.0.smithi090.stdout:7/839: rename d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/ffc to d3/dc/da6/f124 0 2024-03-11T19:49:07.871 INFO:tasks.workunit.client.0.smithi090.stdout:8/917: truncate d0/d43/dc/d90/fac 1838214 0 2024-03-11T19:49:07.999 INFO:tasks.workunit.client.0.smithi090.stdout:1/853: read - d5/da/dd/d112/f118 zero size 2024-03-11T19:49:08.214 INFO:tasks.workunit.client.0.smithi090.stdout:0/866: rename d2/d11/d15/fe1 to d2/d11/d15/d29/d2e/db4/daf/f118 0 2024-03-11T19:49:08.407 INFO:tasks.workunit.client.0.smithi090.stdout:5/845: mkdir d1/d17/d121 0 2024-03-11T19:49:08.690 INFO:tasks.workunit.client.0.smithi090.stdout:7/840: truncate d3/dc/d1e/d7c/d96/d32/da0/fed 1035353 0 2024-03-11T19:49:08.871 INFO:tasks.workunit.client.0.smithi090.stdout:8/918: creat d0/d60/db6/f12d x:0 0 0 2024-03-11T19:49:09.350 INFO:tasks.workunit.client.0.smithi090.stdout:1/854: creat d5/d31/d76/d77/f123 x:0 0 0 2024-03-11T19:49:09.355 INFO:tasks.workunit.client.0.smithi090.stdout:0/867: truncate d2/d11/d15/d29/d43/d58/fc7 1961382 0 2024-03-11T19:49:09.554 INFO:tasks.workunit.client.0.smithi090.stdout:2/840: link d0/de/f16 d0/de/db6/d25/d8c/f106 0 2024-03-11T19:49:09.758 INFO:tasks.workunit.client.0.smithi090.stdout:6/899: mknod d4/d13/d22/d34/d37/d7f/c12f 0 2024-03-11T19:49:09.779 INFO:tasks.workunit.client.0.smithi090.stdout:4/910: dwrite d4/d27/d28/f30 [4194304,4194304] 0 2024-03-11T19:49:09.919 INFO:tasks.workunit.client.0.smithi090.stdout:6/900: stat d4/d13/d22/d42/d48/d4e/ca7 0 2024-03-11T19:49:10.338 INFO:tasks.workunit.client.0.smithi090.stdout:7/841: dread d3/dc/d1e/d7c/d96/d32/f3f [0,4194304] 0 2024-03-11T19:49:10.997 INFO:tasks.workunit.client.0.smithi090.stdout:9/847: write d1/dd/d18/d3a/fbe [476096,66839] 0 2024-03-11T19:49:11.286 INFO:tasks.workunit.client.0.smithi090.stdout:6/901: mknod d4/d13/d22/d3a/dd7/c130 0 2024-03-11T19:49:11.365 INFO:tasks.workunit.client.0.smithi090.stdout:3/801: write d2/d4f/fe2 [347902,53339] 0 2024-03-11T19:49:12.131 INFO:tasks.workunit.client.0.smithi090.stdout:7/842: creat d3/dff/f125 x:0 0 0 2024-03-11T19:49:12.475 INFO:tasks.workunit.client.0.smithi090.stdout:0/868: truncate d2/d11/d15/d29/d3d/d7e/f9d 1201481 0 2024-03-11T19:49:12.477 INFO:tasks.workunit.client.0.smithi090.stdout:1/855: creat d5/d31/d4c/d116/f124 x:0 0 0 2024-03-11T19:49:13.521 INFO:tasks.workunit.client.0.smithi090.stdout:1/856: creat d5/da/d106/f125 x:0 0 0 2024-03-11T19:49:13.522 INFO:tasks.workunit.client.0.smithi090.stdout:3/802: read - d2/db/d15/d69/dad/fc0 zero size 2024-03-11T19:49:13.867 INFO:tasks.workunit.client.0.smithi090.stdout:8/919: dwrite d0/d3/d4c/f88 [0,4194304] 0 2024-03-11T19:49:14.230 INFO:tasks.workunit.client.0.smithi090.stdout:1/857: chown d5/d31/d86/ce6 59 1 2024-03-11T19:49:14.409 INFO:tasks.workunit.client.0.smithi090.stdout:5/846: dwrite d1/d17/d39/d51/d53/d7b/d3b/d76/f77 [0,4194304] 0 2024-03-11T19:49:14.751 INFO:tasks.workunit.client.0.smithi090.stdout:7/843: creat d3/f126 x:0 0 0 2024-03-11T19:49:14.922 INFO:tasks.workunit.client.0.smithi090.stdout:2/841: dwrite d0/de/d12/d46/d4c/d64/fa3 [0,4194304] 0 2024-03-11T19:49:15.116 INFO:tasks.workunit.client.0.smithi090.stdout:3/803: symlink d2/db/d15/d69/dad/de9/l104 0 2024-03-11T19:49:15.129 INFO:tasks.workunit.client.0.smithi090.stdout:4/911: dwrite d4/d51/d8c/d6a/d8d/ff2 [0,4194304] 0 2024-03-11T19:49:15.298 INFO:tasks.workunit.client.0.smithi090.stdout:1/858: symlink d5/da/d23/dfe/d89/l126 0 2024-03-11T19:49:15.705 INFO:tasks.workunit.client.0.smithi090.stdout:9/848: link d1/dd/d18/d3a/f112 d1/dd/d18/d3a/d48/dc6/f121 0 2024-03-11T19:49:15.788 INFO:tasks.workunit.client.0.smithi090.stdout:9/849: dread d1/dd/d18/d25/d101/f85 [0,4194304] 0 2024-03-11T19:49:16.314 INFO:tasks.workunit.client.0.smithi090.stdout:4/912: dread - d4/dd8/db5/f118 zero size 2024-03-11T19:49:16.375 INFO:tasks.workunit.client.0.smithi090.stdout:3/804: truncate d2/db/f92 3285503 0 2024-03-11T19:49:16.464 INFO:tasks.workunit.client.0.smithi090.stdout:1/859: creat d5/da/d1c/d3b/d8c/f127 x:0 0 0 2024-03-11T19:49:16.678 INFO:tasks.workunit.client.0.smithi090.stdout:9/850: mknod d1/dd/d18/d3a/d57/c122 0 2024-03-11T19:49:17.230 INFO:tasks.workunit.client.0.smithi090.stdout:6/902: write d4/d13/d22/d34/d83/d9c/db7/f115 [371168,16087] 0 2024-03-11T19:49:17.535 INFO:tasks.workunit.client.0.smithi090.stdout:1/860: getdents d5/da/d1c/d11a 0 2024-03-11T19:49:17.744 INFO:tasks.workunit.client.0.smithi090.stdout:0/869: dwrite d2/d11/d15/d29/f5b [0,4194304] 0 2024-03-11T19:49:17.867 INFO:tasks.workunit.client.0.smithi090.stdout:8/920: write d0/d3/d86/d93/dba/f124 [370699,45983] 0 2024-03-11T19:49:18.010 INFO:tasks.workunit.client.0.smithi090.stdout:6/903: dread - d4/d13/d22/d3a/dd7/fdd zero size 2024-03-11T19:49:18.238 INFO:tasks.workunit.client.0.smithi090.stdout:1/861: symlink d5/d4e/d54/d94/l128 0 2024-03-11T19:49:18.361 INFO:tasks.workunit.client.0.smithi090.stdout:0/870: creat d2/d11/d3a/f119 x:0 0 0 2024-03-11T19:49:18.603 INFO:tasks.workunit.client.0.smithi090.stdout:8/921: rename d0/d3/d86/d93 to d0/d3/d86/d127/d12e 0 2024-03-11T19:49:18.666 INFO:tasks.workunit.client.0.smithi090.stdout:6/904: rmdir d4/d13/d22/d42/d48/dd2 39 2024-03-11T19:49:18.940 INFO:tasks.workunit.client.0.smithi090.stdout:1/862: mkdir d5/df5/d129 0 2024-03-11T19:49:19.070 INFO:tasks.workunit.client.0.smithi090.stdout:0/871: rename d2/f27 to d2/d94/d111/f11a 0 2024-03-11T19:49:19.128 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:49:19.131 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:49:19.270 INFO:tasks.workunit.client.0.smithi090.stdout:5/847: dwrite d1/d26/fc [0,4194304] 0 2024-03-11T19:49:19.760 INFO:tasks.workunit.client.0.smithi090.stdout:1/863: mkdir d5/da/d97/dd1/d12a 0 2024-03-11T19:49:19.768 INFO:tasks.workunit.client.0.smithi090.stdout:7/844: write d3/dc/da6/f102 [1557370,19602] 0 2024-03-11T19:49:20.423 INFO:tasks.workunit.client.0.smithi090.stdout:8/922: rename d0/d3/d86/fd6 to d0/d53/d59/f12f 0 2024-03-11T19:49:20.458 INFO:tasks.workunit.client.0.smithi090.stdout:7/845: truncate d3/d55/f122 387958 0 2024-03-11T19:49:20.909 INFO:tasks.workunit.client.0.smithi090.stdout:2/842: write d0/de/d12/d81/f87 [670115,12280] 0 2024-03-11T19:49:20.913 INFO:tasks.workunit.client.0.smithi090.stdout:4/913: write d4/d18/fe8 [2616347,121147] 0 2024-03-11T19:49:21.683 INFO:tasks.workunit.client.0.smithi090.stdout:0/872: rename d2/d11/d15/d29/d2e to d2/d11/d15/d29/de5/d11b 0 2024-03-11T19:49:21.807 INFO:tasks.workunit.client.0.smithi090.stdout:9/851: dwrite d1/dd/d18/d25/d101/d1c/d21/dab/d114/d9c/f11b [0,4194304] 0 2024-03-11T19:49:21.915 INFO:tasks.workunit.client.0.smithi090.stdout:3/805: dwrite d2/db/d15/f17 [0,4194304] 0 2024-03-11T19:49:21.980 INFO:tasks.workunit.client.0.smithi090.stdout:5/848: write d1/f38 [2989868,64920] 0 2024-03-11T19:49:22.058 INFO:tasks.workunit.client.0.smithi090.stdout:6/905: dwrite d4/d14/d8a/d74/f111 [0,4194304] 0 2024-03-11T19:49:22.270 INFO:tasks.workunit.client.0.smithi090.stdout:5/849: stat d1/d17/d39/df2/df9/fe6 0 2024-03-11T19:49:22.752 INFO:tasks.workunit.client.0.smithi090.stdout:6/906: read d4/d13/d22/d34/d40/da4/fd8 [1911326,33861] 0 2024-03-11T19:49:22.763 INFO:tasks.workunit.client.0.smithi090.stdout:1/864: dwrite d5/da/dd/f14 [0,4194304] 0 2024-03-11T19:49:22.783 INFO:tasks.workunit.client.0.smithi090.stdout:8/923: dread d0/d43/dc/f12 [0,4194304] 0 2024-03-11T19:49:22.982 INFO:tasks.workunit.client.0.smithi090.stdout:5/850: chown d1/d17/d39/d51/d53/d7b/da0/dc0/fcc 148524 1 2024-03-11T19:49:23.386 INFO:tasks.workunit.client.0.smithi090.stdout:9/852: truncate d1/dd/d18/ddd/fdf 917299 0 2024-03-11T19:49:23.446 INFO:tasks.workunit.client.0.smithi090.stdout:7/846: write d3/dc/d1e/d7c/d96/d32/da0/f109 [761626,40023] 0 2024-03-11T19:49:23.738 INFO:tasks.workunit.client.0.smithi090.stdout:2/843: write d0/de/d1f/df4/ff7 [702257,130232] 0 2024-03-11T19:49:24.013 INFO:tasks.workunit.client.0.smithi090.stdout:8/924: mknod d0/d3/d98/c130 0 2024-03-11T19:49:24.041 INFO:tasks.workunit.client.0.smithi090.stdout:8/925: chown d0/d3/d86/cfb 23 1 2024-03-11T19:49:24.088 INFO:tasks.workunit.client.0.smithi090.stdout:0/873: write f0 [5457020,45658] 0 2024-03-11T19:49:25.569 INFO:tasks.workunit.client.0.smithi090.stdout:4/914: getdents d4/d27 0 2024-03-11T19:49:25.749 INFO:tasks.workunit.client.0.smithi090.stdout:0/874: mkdir d2/d11/d3a/da6/d11c 0 2024-03-11T19:49:25.779 INFO:tasks.workunit.client.0.smithi090.stdout:6/907: write d4/d13/fa2 [2934496,29748] 0 2024-03-11T19:49:26.250 INFO:tasks.workunit.client.0.smithi090.stdout:3/806: dwrite d2/db/d15/d1a/d75/de8/d6a/fb5 [0,4194304] 0 2024-03-11T19:49:26.357 INFO:tasks.workunit.client.0.smithi090.stdout:1/865: dwrite d5/da/dd/f1f [0,4194304] 0 2024-03-11T19:49:26.480 INFO:tasks.workunit.client.0.smithi090.stdout:5/851: link d1/d17/la4 d1/d17/d39/d51/d53/d7b/d3b/d76/d92/dc1/dff/d110/l122 0 2024-03-11T19:49:26.499 INFO:tasks.workunit.client.0.smithi090.stdout:9/853: write d1/dd/d18/f65 [2899937,45326] 0 2024-03-11T19:49:26.501 INFO:tasks.workunit.client.0.smithi090.stdout:3/807: readlink d2/d4f/d59/la5 0 2024-03-11T19:49:26.513 INFO:tasks.workunit.client.0.smithi090.stdout:4/915: chown d4/d27/d28/d38/c4f 30363354 1 2024-03-11T19:49:26.558 INFO:tasks.workunit.client.0.smithi090.stdout:5/852: chown d1/d26/f54 29344850 1 2024-03-11T19:49:26.589 INFO:tasks.workunit.client.0.smithi090.stdout:3/808: dread d2/d45/f4a [0,4194304] 0 2024-03-11T19:49:26.734 INFO:tasks.workunit.client.0.smithi090.stdout:0/875: fdatasync d2/f1b 0 2024-03-11T19:49:26.808 INFO:tasks.workunit.client.0.smithi090.stdout:1/866: readlink d5/l10 0 2024-03-11T19:49:26.908 INFO:tasks.workunit.client.0.smithi090.stdout:9/854: sync 2024-03-11T19:49:26.908 INFO:tasks.workunit.client.0.smithi090.stdout:0/876: sync 2024-03-11T19:49:27.073 INFO:tasks.workunit.client.0.smithi090.stdout:0/877: sync 2024-03-11T19:49:27.156 INFO:tasks.workunit.client.0.smithi090.stdout:4/916: readlink d4/d27/d28/d38/d64/lff 0 2024-03-11T19:49:27.403 INFO:tasks.workunit.client.0.smithi090.stdout:7/847: dwrite d3/dc/d1e/d7c/d96/d32/d45/f9d [0,4194304] 0 2024-03-11T19:49:27.624 INFO:tasks.workunit.client.0.smithi090.stdout:8/926: write d0/d43/dc/f75 [426544,63966] 0 2024-03-11T19:49:27.638 INFO:tasks.workunit.client.0.smithi090.stdout:1/867: rmdir d5/da/d97 39 2024-03-11T19:49:27.674 INFO:tasks.workunit.client.0.smithi090.stdout:9/855: fdatasync d1/dd/f61 0 2024-03-11T19:49:28.311 INFO:tasks.workunit.client.0.smithi090.stdout:2/844: dwrite d0/de/d12/f24 [0,4194304] 0 2024-03-11T19:49:28.683 INFO:tasks.workunit.client.0.smithi090.stdout:6/908: write d4/ff [3258609,3096] 0 2024-03-11T19:49:28.844 INFO:tasks.workunit.client.0.smithi090.stdout:3/809: mknod d2/db/c105 0 2024-03-11T19:49:29.024 INFO:tasks.workunit.client.0.smithi090.stdout:5/853: write d1/f49 [3495266,85536] 0 2024-03-11T19:49:29.354 INFO:tasks.workunit.client.0.smithi090.stdout:1/868: truncate d5/d31/d36/f70 371864 0 2024-03-11T19:49:29.372 INFO:tasks.workunit.client.0.smithi090.stdout:7/848: read - d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/db5/de1/dd2/f115 zero size 2024-03-11T19:49:29.413 INFO:tasks.workunit.client.0.smithi090.stdout:1/869: chown d5/d65/d6b/lad 4898130 1 2024-03-11T19:49:30.577 INFO:tasks.workunit.client.0.smithi090.stdout:0/878: write d2/d11/d15/f74 [2387448,83155] 0 2024-03-11T19:49:30.628 INFO:tasks.workunit.client.0.smithi090.stdout:8/927: rmdir d0/d3/d98 39 2024-03-11T19:49:30.754 INFO:tasks.workunit.client.0.smithi090.stdout:5/854: read d1/d26/fd6 [1117634,92418] 0 2024-03-11T19:49:30.776 INFO:tasks.workunit.client.0.smithi090.stdout:8/928: chown d0/d60/db6/f12d 95353 1 2024-03-11T19:49:30.802 INFO:tasks.workunit.client.0.smithi090.stdout:9/856: creat d1/dd/d18/d3a/d48/dc6/f123 x:0 0 0 2024-03-11T19:49:30.905 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:14:50.569 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:49:30.905 INFO:tasks.workunit.client.0.smithi090.stdout:7/849: symlink d3/dc/d1e/d7c/d96/d32/d7b/d99/l127 0 2024-03-11T19:49:30.906 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:14:50.631 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:49:30.906 INFO:tasks.workunit.client.0.smithi090.stdout:7/850: chown d3/dc/d1e/l2e 27 1 2024-03-11T19:49:30.906 INFO:tasks.workunit.client.0.smithi090.stdout:1/870: truncate d5/da/f93 4880303 0 2024-03-11T19:49:30.916 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:14:50.675 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:49:30.976 INFO:tasks.workunit.client.0.smithi090.stdout:1/871: dread d5/da/d23/f2a [0,4194304] 0 2024-03-11T19:49:31.001 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:14:50.691 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:49:31.010 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:14:50.707 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:49:31.015 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:14:50.705 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:49:31.018 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:14:50.715 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:49:31.083 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:14:50.772 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:49:31.088 INFO:tasks.workunit.client.0.smithi090.stdout:4/917: link d4/d51/d8c/d6a/d100/l120 d4/d27/d28/d38/d64/d119/l132 0 2024-03-11T19:49:31.100 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:14:50.797 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:49:31.511 INFO:tasks.workunit.client.0.smithi090.stdout:5/855: fdatasync d1/d17/d39/d51/d53/fcf 0 2024-03-11T19:49:31.709 INFO:tasks.workunit.client.0.smithi090.stdout:5/856: sync 2024-03-11T19:49:31.748 INFO:tasks.workunit.client.0.smithi090.stdout:8/929: truncate d0/d43/dc/f50 6222703 0 2024-03-11T19:49:31.922 INFO:tasks.workunit.client.0.smithi090.stdout:3/810: dwrite d2/db/d15/d69/da2/dc7/ddd/fec [0,4194304] 0 2024-03-11T19:49:31.955 INFO:tasks.workunit.client.0.smithi090.stdout:1/872: fdatasync d5/f37 0 2024-03-11T19:49:32.066 INFO:tasks.workunit.client.0.smithi090.stdout:9/857: fsync d1/dd/d18/d3a/d48/dc6/f123 0 2024-03-11T19:49:32.196 INFO:tasks.workunit.client.0.smithi090.stdout:4/918: truncate d4/dd8/f86 601945 0 2024-03-11T19:49:32.383 INFO:tasks.workunit.client.0.smithi090.stdout:6/909: write d4/d13/d22/d42/d48/dd2/ff0 [916716,62606] 0 2024-03-11T19:49:32.708 INFO:tasks.workunit.client.0.smithi090.stdout:0/879: dwrite d2/f1c [0,4194304] 0 2024-03-11T19:49:33.336 INFO:tasks.workunit.client.0.smithi090.stdout:8/930: rmdir d0/d53/d59 39 2024-03-11T19:49:33.531 INFO:tasks.workunit.client.0.smithi090.stdout:5/857: fdatasync d1/d26/d19/d11d/f120 0 2024-03-11T19:49:33.912 INFO:tasks.workunit.client.0.smithi090.stdout:2/845: getdents d0/d21 0 2024-03-11T19:49:33.976 INFO:tasks.workunit.client.0.smithi090.stdout:1/873: mknod d5/d65/d6b/d7f/d10c/c12b 0 2024-03-11T19:49:34.478 INFO:tasks.workunit.client.0.smithi090.stdout:0/880: rename d2/d11/d15/d29/de5/d11b/d6d/da4/fee to d2/d11/d15/d29/d3d/d7e/dfc/f11d 0 2024-03-11T19:49:34.493 INFO:tasks.workunit.client.0.smithi090.stdout:3/811: dread d2/db/d15/d1a/d75/de8/d6a/dcb/faf [0,4194304] 0 2024-03-11T19:49:35.283 INFO:tasks.workunit.client.0.smithi090.stdout:5/858: dread - d1/d17/d39/d51/d53/d7b/d3b/d76/db7/ddf/dee/ff8 zero size 2024-03-11T19:49:35.588 INFO:tasks.workunit.client.0.smithi090.stdout:4/919: symlink d4/dd8/l133 0 2024-03-11T19:49:35.732 INFO:tasks.workunit.client.0.smithi090.stdout:0/881: chown d2/d11/d3a/d64/dac/dba/fbf 485386486 1 2024-03-11T19:49:35.954 INFO:tasks.workunit.client.0.smithi090.stdout:3/812: rmdir d2/db/d15/d1a/d75/de8/d6a 39 2024-03-11T19:49:36.281 INFO:tasks.workunit.client.0.smithi090.stdout:2/846: unlink d0/de/d1f/le2 0 2024-03-11T19:49:36.572 INFO:tasks.workunit.client.0.smithi090.stdout:9/858: dwrite d1/dd/d18/d25/d101/f4a [4194304,4194304] 0 2024-03-11T19:49:36.802 INFO:tasks.workunit.client.0.smithi090.stdout:7/851: link d3/dc/d1e/d7c/d96/d32/d45/de0/d48/fd7 d3/dc/d1e/d7c/d96/d32/d7b/f128 0 2024-03-11T19:49:36.826 INFO:tasks.workunit.client.0.smithi090.stdout:1/874: creat d5/d4e/d120/f12c x:0 0 0 2024-03-11T19:49:36.908 INFO:tasks.workunit.client.0.smithi090.stdout:6/910: link d4/d14/f110 d4/d13/d22/d34/d83/f131 0 2024-03-11T19:49:36.912 INFO:tasks.workunit.client.0.smithi090.stdout:4/920: rename d4/dd8/d66/fce to d4/dd8/d66/f134 0 2024-03-11T19:49:37.137 INFO:tasks.workunit.client.0.smithi090.stdout:8/931: write d0/d43/dc/da1/f112 [443302,10874] 0 2024-03-11T19:49:37.632 INFO:tasks.workunit.client.0.smithi090.stdout:9/859: fsync d1/dd/d18/d25/d101/d1c/d21/d2a/fa3 0 2024-03-11T19:49:37.771 INFO:tasks.workunit.client.0.smithi090.stdout:1/875: symlink d5/da/d1c/d3b/d8c/l12d 0 2024-03-11T19:49:37.817 INFO:tasks.workunit.client.0.smithi090.stdout:1/876: chown d5/d65/d6b/ce0 2 1 2024-03-11T19:49:38.163 INFO:tasks.workunit.client.0.smithi090.stdout:4/921: mknod d4/dd8/d66/dcf/dd7/dd3/c135 0 2024-03-11T19:49:38.324 INFO:tasks.workunit.client.0.smithi090.stdout:6/911: rename d4/d13/d22/d34/d40/dd5/f3f to d4/d13/d22/d34/d83/d9c/f132 0 2024-03-11T19:49:39.071 INFO:tasks.workunit.client.0.smithi090.stdout:9/860: dread d1/dd/d18/d3a/f104 [0,4194304] 0 2024-03-11T19:49:39.562 INFO:tasks.workunit.client.0.smithi090.stdout:5/859: write d1/d17/d39/d51/d53/fa5 [1376440,52906] 0 2024-03-11T19:49:39.617 INFO:tasks.workunit.client.0.smithi090.stdout:5/860: stat d1/d17/d39/d51/d73/da6/fbf 0 2024-03-11T19:49:39.863 INFO:tasks.workunit.client.0.smithi090.stdout:0/882: write d2/d11/d15/d29/d3d/d7e/fc0 [2621618,16470] 0 2024-03-11T19:49:40.055 INFO:tasks.workunit.client.0.smithi090.stdout:6/912: symlink d4/d13/d22/d34/d83/l133 0 2024-03-11T19:49:40.126 INFO:tasks.workunit.client.0.smithi090.stdout:8/932: mkdir d0/d53/d59/d131 0 2024-03-11T19:49:40.144 INFO:tasks.workunit.client.0.smithi090.stdout:3/813: write d2/f9 [4659251,37606] 0 2024-03-11T19:49:40.231 INFO:tasks.workunit.client.0.smithi090.stdout:2/847: creat d0/de/d12/f107 x:0 0 0 2024-03-11T19:49:40.430 INFO:tasks.workunit.client.0.smithi090.stdout:2/848: chown d0/d21/c43 331598 1 2024-03-11T19:49:40.507 INFO:tasks.workunit.client.0.smithi090.stdout:4/922: dread d4/d51/d8c/d89/f98 [0,4194304] 0 2024-03-11T19:49:40.828 INFO:tasks.workunit.client.0.smithi090.stdout:1/877: rename d5/d3c/fa5 to d5/d31/d4c/daf/dfc/f12e 0 2024-03-11T19:49:40.987 INFO:tasks.workunit.client.0.smithi090.stdout:4/923: dread d4/ff [0,4194304] 0 2024-03-11T19:49:41.289 INFO:tasks.workunit.client.0.smithi090.stdout:5/861: mknod d1/d17/d39/d51/d73/da6/d10a/c123 0 2024-03-11T19:49:41.388 INFO:tasks.workunit.client.0.smithi090.stdout:6/913: dread d4/d13/d22/d34/d40/d7d/f109 [0,4194304] 0 2024-03-11T19:49:41.539 INFO:tasks.workunit.client.0.smithi090.stdout:7/852: getdents d3/dc/d1e/d7c/d96/d32/d45/de0/d48 0 2024-03-11T19:49:41.563 INFO:tasks.workunit.client.0.smithi090.stdout:7/853: dread - d3/dc/fc4 zero size 2024-03-11T19:49:41.653 INFO:tasks.workunit.client.0.smithi090.stdout:0/883: rename d2/d11/d15/d29/d114 to d2/d11/df4/d11e 0 2024-03-11T19:49:41.686 INFO:tasks.workunit.client.0.smithi090.stdout:0/884: chown d2/d11/d15/d29/d43/d58 2769 1 2024-03-11T19:49:42.003 INFO:tasks.workunit.client.0.smithi090.stdout:4/924: unlink d4/dd8/d66/f76 0 2024-03-11T19:49:42.365 INFO:tasks.workunit.client.0.smithi090.stdout:6/914: read d4/d14/d8a/d74/f111 [3707754,45971] 0 2024-03-11T19:49:42.561 INFO:tasks.workunit.client.0.smithi090.stdout:2/849: creat d0/de/d12/d46/d75/dc5/de5/f108 x:0 0 0 2024-03-11T19:49:42.902 INFO:tasks.workunit.client.0.smithi090.stdout:0/885: dread d2/f14 [0,4194304] 0 2024-03-11T19:49:42.936 INFO:tasks.workunit.client.0.smithi090.stdout:9/861: write d1/dd/d18/d25/d101/f10 [214129,26162] 0 2024-03-11T19:49:43.693 INFO:tasks.workunit.client.0.smithi090.stdout:6/915: dread d4/d14/d8a/d74/f7b [0,4194304] 0 2024-03-11T19:49:43.851 INFO:tasks.workunit.client.0.smithi090.stdout:1/878: creat d5/d31/d76/f12f x:0 0 0 2024-03-11T19:49:43.868 INFO:tasks.workunit.client.0.smithi090.stdout:2/850: dread d0/de/d12/d46/d4c/d64/fa3 [0,4194304] 0 2024-03-11T19:49:44.395 INFO:tasks.workunit.client.0.smithi090.stdout:0/886: fsync d2/d11/d3a/d64/dac/f108 0 2024-03-11T19:49:44.399 INFO:tasks.workunit.client.0.smithi090.stdout:7/854: rmdir d3/dff 39 2024-03-11T19:49:44.417 INFO:tasks.workunit.client.0.smithi090.stdout:9/862: truncate d1/dd/d18/d3a/d48/dc6/f123 347701 0 2024-03-11T19:49:44.822 INFO:tasks.workunit.client.0.smithi090.stdout:3/814: dwrite d2/db/d15/d1a/d3c/f52 [4194304,4194304] 0 2024-03-11T19:49:45.023 INFO:tasks.workunit.client.0.smithi090.stdout:8/933: write d0/d43/dc/f50 [1074051,104453] 0 2024-03-11T19:49:45.340 INFO:tasks.workunit.client.0.smithi090.stdout:1/879: dread d5/d65/d6b/fa4 [0,4194304] 0 2024-03-11T19:49:45.365 INFO:tasks.workunit.client.0.smithi090.stdout:9/863: creat d1/dd/d18/d25/d101/d1c/d21/d2a/f124 x:0 0 0 2024-03-11T19:49:45.478 INFO:tasks.workunit.client.0.smithi090.stdout:5/862: dwrite d1/d17/d39/d51/d73/f94 [0,4194304] 0 2024-03-11T19:49:45.913 INFO:tasks.workunit.client.0.smithi090.stdout:2/851: symlink d0/de/df8/l109 0 2024-03-11T19:49:46.000 INFO:tasks.workunit.client.0.smithi090.stdout:4/925: getdents d4/d27/d28/d38/d64 0 2024-03-11T19:49:46.038 INFO:tasks.workunit.client.0.smithi090.stdout:9/864: dread d1/d1d/f67 [4194304,4194304] 0 2024-03-11T19:49:46.275 INFO:tasks.workunit.client.0.smithi090.stdout:8/934: mkdir d0/d3/db0/d132 0 2024-03-11T19:49:46.284 INFO:tasks.workunit.client.0.smithi090.stdout:3/815: dread d2/db/d15/d1a/d75/de8/d6a/dcb/f78 [0,4194304] 0 2024-03-11T19:49:46.447 INFO:tasks.workunit.client.0.smithi090.stdout:0/887: rename d2/d11/d15/d36/f7c to d2/d11/d15/d29/d3d/d7b/de6/f11f 0 2024-03-11T19:49:46.545 INFO:tasks.workunit.client.0.smithi090.stdout:5/863: fdatasync d1/d17/f30 0 2024-03-11T19:49:46.713 INFO:tasks.workunit.client.0.smithi090.stdout:4/926: readlink d4/dd8/dfa/lea 0 2024-03-11T19:49:46.936 INFO:tasks.workunit.client.0.smithi090.stdout:9/865: symlink d1/dd/d18/d3a/dd8/l125 0 2024-03-11T19:49:47.220 INFO:tasks.workunit.client.0.smithi090.stdout:6/916: write d4/d14/f6a [4553893,18606] 0 2024-03-11T19:49:47.440 INFO:tasks.workunit.client.0.smithi090.stdout:3/816: fsync d2/db/d15/d1a/d75/dc6/f77 0 2024-03-11T19:49:47.609 INFO:tasks.workunit.client.0.smithi090.stdout:8/935: truncate d0/d43/dc/fa3 43601 0 2024-03-11T19:49:48.053 INFO:tasks.workunit.client.0.smithi090.stdout:7/855: dwrite d3/dc/fe [4194304,4194304] 0 2024-03-11T19:49:48.446 INFO:tasks.workunit.client.0.smithi090.stdout:5/864: rename d1/d26/db5/ff7 to d1/d17/d39/d51/d53/d7b/da0/dc0/f124 0 2024-03-11T19:49:48.492 INFO:tasks.workunit.client.0.smithi090.stdout:5/865: dread - d1/d17/d39/d51/d53/d64/f93 zero size 2024-03-11T19:49:48.530 INFO:tasks.workunit.client.0.smithi090.stdout:4/927: fdatasync d4/dd8/d66/f12e 0 2024-03-11T19:49:48.986 INFO:tasks.workunit.client.0.smithi090.stdout:2/852: creat d0/de/d12/d46/d4c/d64/dd4/f10a x:0 0 0 2024-03-11T19:49:48.993 INFO:tasks.workunit.client.0.smithi090.stdout:6/917: mkdir d4/d9b/def/d134 0 2024-03-11T19:49:49.055 INFO:tasks.workunit.client.0.smithi090.stdout:6/918: chown d4/d14/d8a/f63 14 1 2024-03-11T19:49:49.265 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:49:49.268 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:49:49.488 INFO:tasks.workunit.client.0.smithi090.stdout:1/880: link d5/d31/d4c/le5 d5/d4e/d54/d94/d11c/l130 0 2024-03-11T19:49:50.294 INFO:tasks.workunit.client.0.smithi090.stdout:2/853: dread - d0/de/db6/d25/d8c/f105 zero size 2024-03-11T19:49:50.332 INFO:tasks.workunit.client.0.smithi090.stdout:5/866: dread d1/f4c [0,4194304] 0 2024-03-11T19:49:50.953 INFO:tasks.workunit.client.0.smithi090.stdout:4/928: mkdir d4/dd8/dfa/db7/d11d/d136 0 2024-03-11T19:49:50.972 INFO:tasks.workunit.client.0.smithi090.stdout:9/866: creat d1/dd/d18/d3a/f126 x:0 0 0 2024-03-11T19:49:51.024 INFO:tasks.workunit.client.0.smithi090.stdout:4/929: chown d4/cc5 5984 1 2024-03-11T19:49:51.772 INFO:tasks.workunit.client.0.smithi090.stdout:1/881: getdents d5/df5/d129 0 2024-03-11T19:49:51.973 INFO:tasks.workunit.client.0.smithi090.stdout:9/867: symlink d1/dd/d18/d25/d101/d1c/d9f/l127 0 2024-03-11T19:49:52.389 INFO:tasks.workunit.client.0.smithi090.stdout:2/854: fsync d0/de/f16 0 2024-03-11T19:49:52.431 INFO:tasks.workunit.client.0.smithi090.stdout:2/855: chown d0/de/db6/d25/c4a 15062 1 2024-03-11T19:49:52.480 INFO:tasks.workunit.client.0.smithi090.stdout:5/867: creat d1/d17/d39/d51/d53/d7b/d3b/d76/d92/dc1/dff/d110/f125 x:0 0 0 2024-03-11T19:49:52.708 INFO:tasks.workunit.client.0.smithi090.stdout:8/936: write d0/d60/db6/f12d [669791,35269] 0 2024-03-11T19:49:53.122 INFO:tasks.workunit.client.0.smithi090.stdout:0/888: write d2/d11/d15/d29/d43/d58/fc7 [2177556,108911] 0 2024-03-11T19:49:53.486 INFO:tasks.workunit.client.0.smithi090.stdout:5/868: dread - d1/d17/d39/d51/ff1 zero size 2024-03-11T19:49:53.762 INFO:tasks.workunit.client.0.smithi090.stdout:3/817: dwrite d2/db/d15/d69/dad/fa4 [0,4194304] 0 2024-03-11T19:49:53.789 INFO:tasks.workunit.client.0.smithi090.stdout:8/937: rename d0/d60/db6/f12d to d0/d3/d3e/d85/d116/f133 0 2024-03-11T19:49:53.808 INFO:tasks.workunit.client.0.smithi090.stdout:7/856: dwrite d3/f4 [4194304,4194304] 0 2024-03-11T19:49:54.026 INFO:tasks.workunit.client.0.smithi090.stdout:0/889: creat d2/d11/d15/d29/d3d/d8c/f120 x:0 0 0 2024-03-11T19:49:54.085 INFO:tasks.workunit.client.0.smithi090.stdout:6/919: write d4/f46 [2640087,30188] 0 2024-03-11T19:49:54.163 INFO:tasks.workunit.client.0.smithi090.stdout:2/856: truncate d0/de/d12/d46/fa8 793843 0 2024-03-11T19:49:54.174 INFO:tasks.workunit.client.0.smithi090.stdout:4/930: write d4/dd8/fa7 [2627337,5873] 0 2024-03-11T19:49:54.414 INFO:tasks.workunit.client.0.smithi090.stdout:3/818: stat d2/d4f/d59/cba 0 2024-03-11T19:49:54.891 INFO:tasks.workunit.client.0.smithi090.stdout:0/890: creat d2/d11/d15/d29/d3d/d8c/f121 x:0 0 0 2024-03-11T19:49:54.897 INFO:tasks.workunit.client.0.smithi090.stdout:6/920: unlink d4/d14/d8a/d74/l89 0 2024-03-11T19:49:55.177 INFO:tasks.workunit.client.0.smithi090.stdout:7/857: symlink d3/dc/d1e/d7c/d96/d32/d67/dee/l129 0 2024-03-11T19:49:55.206 INFO:tasks.workunit.client.0.smithi090.stdout:2/857: creat d0/de/db6/d25/f10b x:0 0 0 2024-03-11T19:49:55.210 INFO:tasks.workunit.client.0.smithi090.stdout:1/882: write d5/da/d1c/f39 [2161556,112936] 0 2024-03-11T19:49:55.247 INFO:tasks.workunit.client.0.smithi090.stdout:9/868: creat d1/dd/f128 x:0 0 0 2024-03-11T19:49:55.888 INFO:tasks.workunit.client.0.smithi090.stdout:4/931: symlink d4/dd8/l137 0 2024-03-11T19:49:55.911 INFO:tasks.workunit.client.0.smithi090.stdout:6/921: chown d4/d13/d22/d34/d40/fbb 9171 1 2024-03-11T19:49:55.949 INFO:tasks.workunit.client.0.smithi090.stdout:0/891: dread d2/d11/d15/d29/d3d/d7e/dfc/f11d [0,4194304] 0 2024-03-11T19:49:56.328 INFO:tasks.workunit.client.0.smithi090.stdout:4/932: dread d4/ff3 [0,4194304] 0 2024-03-11T19:49:56.425 INFO:tasks.workunit.client.0.smithi090.stdout:9/869: dread d1/d47/f90 [0,4194304] 0 2024-03-11T19:49:56.481 INFO:tasks.workunit.client.0.smithi090.stdout:5/869: write d1/d17/f66 [958986,84477] 0 2024-03-11T19:49:56.639 INFO:tasks.workunit.client.0.smithi090.stdout:6/922: rename d4/d13/d22/d34/f6d to d4/d13/d22/d42/d48/d4e/d122/f135 0 2024-03-11T19:49:56.894 INFO:tasks.workunit.client.0.smithi090.stdout:8/938: write d0/d3/d4c/d8d/f94 [3011276,81773] 0 2024-03-11T19:49:57.065 INFO:tasks.workunit.client.0.smithi090.stdout:4/933: truncate d4/d18/fe8 3458783 0 2024-03-11T19:49:57.224 INFO:tasks.workunit.client.0.smithi090.stdout:1/883: creat d5/da/d97/dd1/d12a/f131 x:0 0 0 2024-03-11T19:49:57.917 INFO:tasks.workunit.client.0.smithi090.stdout:9/870: rename d1/dd/d18/d25/d101/d1c/d21/d5c/c99 to d1/d1d/d6a/c129 0 2024-03-11T19:49:57.919 INFO:tasks.workunit.client.0.smithi090.stdout:7/858: creat d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/db5/de1/dc6/f12a x:0 0 0 2024-03-11T19:49:58.393 INFO:tasks.workunit.client.0.smithi090.stdout:1/884: dread - d5/da/dd/f111 zero size 2024-03-11T19:49:58.584 INFO:tasks.workunit.client.0.smithi090.stdout:4/934: dread d4/dd8/ff8 [0,4194304] 0 2024-03-11T19:49:58.657 INFO:tasks.workunit.client.0.smithi090.stdout:3/819: dwrite d2/ffd [0,4194304] 0 2024-03-11T19:49:58.666 INFO:tasks.workunit.client.0.smithi090.stdout:9/871: fsync d1/dd/d18/d3a/d48/f92 0 2024-03-11T19:49:58.857 INFO:tasks.workunit.client.0.smithi090.stdout:9/872: chown d1/dd/d18/d25/d101 5592280 1 2024-03-11T19:49:58.870 INFO:tasks.workunit.client.0.smithi090.stdout:3/820: chown d2/db/d15/d1a/d75/f80 289780 1 2024-03-11T19:49:58.905 INFO:tasks.workunit.client.0.smithi090.stdout:9/873: readlink d1/dd/d18/d25/l98 0 2024-03-11T19:49:59.368 INFO:tasks.workunit.client.0.smithi090.stdout:0/892: creat d2/d11/d15/d29/d3d/f122 x:0 0 0 2024-03-11T19:49:59.368 INFO:tasks.workunit.client.0.smithi090.stdout:8/939: creat d0/d53/d59/d96/de5/de7/f134 x:0 0 0 2024-03-11T19:49:59.523 INFO:tasks.workunit.client.0.smithi090.stdout:9/874: sync 2024-03-11T19:49:59.661 INFO:tasks.workunit.client.0.smithi090.stdout:2/858: dwrite d0/f7 [8388608,4194304] 0 2024-03-11T19:49:59.917 INFO:tasks.workunit.client.0.smithi090.stdout:5/870: write d1/d17/d39/d51/d53/d7b/fdd [2138385,67383] 0 2024-03-11T19:50:00.573 INFO:tasks.workunit.client.0.smithi090.stdout:1/885: creat d5/d65/d6b/de7/f132 x:0 0 0 2024-03-11T19:50:00.698 INFO:tasks.workunit.client.0.smithi090.stdout:1/886: stat d5/d31/d4c/l7e 0 2024-03-11T19:50:00.793 INFO:tasks.workunit.client.0.smithi090.stdout:3/821: mknod d2/d4f/d59/c106 0 2024-03-11T19:50:00.838 INFO:tasks.workunit.client.0.smithi090.stdout:1/887: stat d5/da/dd/f121 0 2024-03-11T19:50:00.880 INFO:tasks.workunit.client.0.smithi090.stdout:6/923: dwrite d4/d14/d8a/d74/f7b [0,4194304] 0 2024-03-11T19:50:01.024 INFO:tasks.workunit.client.0.smithi090.stdout:3/822: chown d2/d4f/d59/da1/lf5 1558469 1 2024-03-11T19:50:01.116 INFO:tasks.workunit.client.0.smithi090.stdout:0/893: fdatasync d2/d11/f17 0 2024-03-11T19:50:01.137 INFO:tasks.workunit.client.0.smithi090.stdout:9/875: truncate d1/dd/d18/d3a/f86 454165 0 2024-03-11T19:50:01.582 INFO:tasks.workunit.client.0.smithi090.stdout:2/859: dread d0/de/d12/d46/d4c/fe0 [0,4194304] 0 2024-03-11T19:50:01.931 INFO:tasks.workunit.client.0.smithi090.stdout:4/935: fdatasync d4/d51/d8c/fac 0 2024-03-11T19:50:02.161 INFO:tasks.workunit.client.0.smithi090.stdout:1/888: mknod d5/da/d97/dd1/d12a/c133 0 2024-03-11T19:50:02.404 INFO:tasks.workunit.client.0.smithi090.stdout:3/823: dread d2/db/f1c [0,4194304] 0 2024-03-11T19:50:02.443 INFO:tasks.workunit.client.0.smithi090.stdout:6/924: rename d4/d13/d22/d34/d83/d9c to d4/d13/d22/d42/d48/d4e/d136 0 2024-03-11T19:50:02.455 INFO:tasks.workunit.client.0.smithi090.stdout:3/824: chown d2/db/d15/d1a/d75 349 1 2024-03-11T19:50:03.000 INFO:tasks.workunit.client.0.smithi090.stdout:8/940: unlink d0/d3/d4c/d8d/ff4 0 2024-03-11T19:50:03.027 INFO:tasks.workunit.client.0.smithi090.stdout:7/859: write d3/dc/d1e/d7c/d96/d32/da0/fed [1534116,49629] 0 2024-03-11T19:50:03.093 INFO:tasks.workunit.client.0.smithi090.stdout:5/871: mkdir d1/d17/d39/d126 0 2024-03-11T19:50:03.099 INFO:tasks.workunit.client.0.smithi090.stdout:9/876: fdatasync d1/dd/d18/d25/d101/d1c/d77/fc4 0 2024-03-11T19:50:03.163 INFO:tasks.workunit.client.0.smithi090.stdout:5/872: read d1/d17/f30 [1946202,96747] 0 2024-03-11T19:50:03.592 INFO:tasks.workunit.client.0.smithi090.stdout:4/936: chown d4/d51/d8c/d6a/d100/l120 8433 1 2024-03-11T19:50:03.848 INFO:tasks.workunit.client.0.smithi090.stdout:1/889: mkdir d5/d31/d76/d77/d134 0 2024-03-11T19:50:04.752 INFO:tasks.workunit.client.0.smithi090.stdout:7/860: mknod d3/dc/d29/c12b 0 2024-03-11T19:50:05.194 INFO:tasks.workunit.client.0.smithi090.stdout:5/873: dread d1/d17/d39/d51/d53/d7b/d6f/f72 [0,4194304] 0 2024-03-11T19:50:05.782 INFO:tasks.workunit.client.0.smithi090.stdout:1/890: dread d5/d4e/d54/d94/f96 [4194304,4194304] 0 2024-03-11T19:50:06.590 INFO:tasks.workunit.client.0.smithi090.stdout:5/874: symlink d1/d17/d39/d51/d53/d7b/d3b/l127 0 2024-03-11T19:50:06.607 INFO:tasks.workunit.client.0.smithi090.stdout:0/894: write d2/d11/d15/d29/d3d/f106 [3777497,7765] 0 2024-03-11T19:50:07.031 INFO:tasks.workunit.client.0.smithi090.stdout:7/861: rename d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/db5/dde/c6e to d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/c12c 0 2024-03-11T19:50:07.392 INFO:tasks.workunit.client.0.smithi090.stdout:0/895: symlink d2/d11/df4/l123 0 2024-03-11T19:50:07.589 INFO:tasks.workunit.client.0.smithi090.stdout:2/860: dwrite d0/de/d12/d46/d5d/f62 [4194304,4194304] 0 2024-03-11T19:50:07.829 INFO:tasks.workunit.client.0.smithi090.stdout:6/925: write d4/d14/d8a/f63 [1993842,16223] 0 2024-03-11T19:50:07.840 INFO:tasks.workunit.client.0.smithi090.stdout:3/825: write d2/d2f/f9c [5190935,16942] 0 2024-03-11T19:50:07.931 INFO:tasks.workunit.client.0.smithi090.stdout:7/862: creat d3/dc/d1e/d7c/d96/d32/d67/f12d x:0 0 0 2024-03-11T19:50:08.046 INFO:tasks.workunit.client.0.smithi090.stdout:9/877: getdents d1/dd/d18/d25/d101/d1c/d21/d2a/dc2/d100/d120 0 2024-03-11T19:50:08.148 INFO:tasks.workunit.client.0.smithi090.stdout:0/896: dread d2/d11/d15/d29/f4a [0,4194304] 0 2024-03-11T19:50:08.497 INFO:tasks.workunit.client.0.smithi090.stdout:4/937: write d4/d27/d28/d69/f99 [5196681,92251] 0 2024-03-11T19:50:08.615 INFO:tasks.workunit.client.0.smithi090.stdout:4/938: dread d4/d51/dbf/fe9 [0,4194304] 0 2024-03-11T19:50:08.926 INFO:tasks.workunit.client.0.smithi090.stdout:8/941: write d0/d60/f117 [2110884,5294] 0 2024-03-11T19:50:08.948 INFO:tasks.workunit.client.0.smithi090.stdout:5/875: write d1/d17/d39/d51/d53/d7b/fa3 [2133369,100593] 0 2024-03-11T19:50:09.007 INFO:tasks.workunit.client.0.smithi090.stdout:5/876: readlink d1/d17/d39/d51/lef 0 2024-03-11T19:50:09.447 INFO:tasks.workunit.client.0.smithi090.stdout:6/926: chown d4/d13/d22/d34/cf1 1 1 2024-03-11T19:50:09.493 INFO:tasks.workunit.client.0.smithi090.stdout:0/897: read d2/f10 [7186755,55568] 0 2024-03-11T19:50:09.864 INFO:tasks.workunit.client.0.smithi090.stdout:2/861: dwrite d0/de/d12/d46/d5d/d63/fcc [0,4194304] 0 2024-03-11T19:50:10.076 INFO:tasks.workunit.client.0.smithi090.stdout:1/891: rename d5/d65/d6b/lc2 to d5/d31/d76/l135 0 2024-03-11T19:50:11.143 INFO:tasks.workunit.client.0.smithi090.stdout:3/826: mknod d2/d45/df1/c107 0 2024-03-11T19:50:11.318 INFO:tasks.workunit.client.0.smithi090.stdout:1/892: read d5/da/dd/f1f [1091571,15447] 0 2024-03-11T19:50:11.379 INFO:tasks.workunit.client.0.smithi090.stdout:4/939: mknod d4/d27/d28/d38/c138 0 2024-03-11T19:50:11.420 INFO:tasks.workunit.client.0.smithi090.stdout:2/862: dread d0/de/d12/d46/d75/f7c [0,4194304] 0 2024-03-11T19:50:11.774 INFO:tasks.workunit.client.0.smithi090.stdout:5/877: mknod d1/d26/c128 0 2024-03-11T19:50:11.897 INFO:tasks.workunit.client.0.smithi090.stdout:9/878: dwrite d1/dd/d18/d25/d101/f85 [0,4194304] 0 2024-03-11T19:50:12.179 INFO:tasks.workunit.client.0.smithi090.stdout:7/863: dwrite d3/dc/d1e/d7c/d96/d32/d45/de0/f35 [0,4194304] 0 2024-03-11T19:50:12.410 INFO:tasks.workunit.client.0.smithi090.stdout:8/942: rmdir d0/d3/db0/d121 0 2024-03-11T19:50:12.435 INFO:tasks.workunit.client.0.smithi090.stdout:6/927: creat d4/d14/f137 x:0 0 0 2024-03-11T19:50:12.591 INFO:tasks.workunit.client.0.smithi090.stdout:9/879: chown d1/dd/d18/d25/d101/d1c/d21/d2a/fa3 1 1 2024-03-11T19:50:12.649 INFO:tasks.workunit.client.0.smithi090.stdout:0/898: dwrite d2/d11/d3f/f76 [4194304,4194304] 0 2024-03-11T19:50:13.005 INFO:tasks.workunit.client.0.smithi090.stdout:8/943: stat d0/d3/d98 0 2024-03-11T19:50:13.006 INFO:tasks.workunit.client.0.smithi090.stdout:5/878: dread - d1/d17/d39/d51/d53/d7b/da0/dc0/f124 zero size 2024-03-11T19:50:13.402 INFO:tasks.workunit.client.0.smithi090.stdout:1/893: creat d5/da/d1c/d3b/f136 x:0 0 0 2024-03-11T19:50:13.439 INFO:tasks.workunit.client.0.smithi090.stdout:4/940: dwrite d4/d27/f114 [0,4194304] 0 2024-03-11T19:50:13.792 INFO:tasks.workunit.client.0.smithi090.stdout:0/899: fsync d2/d11/ff7 0 2024-03-11T19:50:13.883 INFO:tasks.workunit.client.0.smithi090.stdout:6/928: creat d4/d9b/def/d134/f138 x:0 0 0 2024-03-11T19:50:13.899 INFO:tasks.workunit.client.0.smithi090.stdout:8/944: unlink d0/d53/d59/c48 0 2024-03-11T19:50:13.915 INFO:tasks.workunit.client.0.smithi090.stdout:3/827: truncate d2/d2f/f9c 3427096 0 2024-03-11T19:50:13.934 INFO:tasks.workunit.client.0.smithi090.stdout:5/879: rmdir d1/d17/d39/d51/d53/d7b/d3b/d76 39 2024-03-11T19:50:13.949 INFO:tasks.workunit.client.0.smithi090.stdout:7/864: write d3/dc/d87/f90 [1561178,88992] 0 2024-03-11T19:50:14.018 INFO:tasks.workunit.client.0.smithi090.stdout:3/828: chown d2/d4f/d59 30244 1 2024-03-11T19:50:14.032 INFO:tasks.workunit.client.0.smithi090.stdout:8/945: chown d0/d3/d4c 154784 1 2024-03-11T19:50:14.033 INFO:tasks.workunit.client.0.smithi090.stdout:1/894: rmdir d5/d4e/d54/d94/d11c 39 2024-03-11T19:50:14.152 INFO:tasks.workunit.client.0.smithi090.stdout:3/829: chown d2/db/d15/d1a/d75/de8/d6a/cb2 1551803 1 2024-03-11T19:50:14.281 INFO:tasks.workunit.client.0.smithi090.stdout:4/941: creat d4/d51/d8c/d6a/d8d/f139 x:0 0 0 2024-03-11T19:50:14.814 INFO:tasks.workunit.client.0.smithi090.stdout:2/863: link d0/d21/dce/da2/cf6 d0/de/d12/d46/d75/dbb/c10c 0 2024-03-11T19:50:14.889 INFO:tasks.workunit.client.0.smithi090.stdout:9/880: write d1/dd/d18/d3a/fbe [1791412,14786] 0 2024-03-11T19:50:15.598 INFO:tasks.workunit.client.0.smithi090.stdout:7/865: rmdir d3/dc/d1e/d7c/d96 39 2024-03-11T19:50:15.692 INFO:tasks.workunit.client.0.smithi090.stdout:0/900: read d2/d11/d15/d29/de5/d11b/db4/daf/fdb [1057548,31129] 0 2024-03-11T19:50:15.842 INFO:tasks.workunit.client.0.smithi090.stdout:6/929: dread d4/d13/d22/d34/d40/fa1 [0,4194304] 0 2024-03-11T19:50:16.509 INFO:tasks.workunit.client.0.smithi090.stdout:9/881: mknod d1/dd/d18/d25/d101/d1c/d9f/c12a 0 2024-03-11T19:50:17.252 INFO:tasks.workunit.client.0.smithi090.stdout:6/930: creat d4/d13/d22/d34/d40/dd5/d80/dd4/d128/f139 x:0 0 0 2024-03-11T19:50:17.698 INFO:tasks.workunit.client.0.smithi090.stdout:3/830: creat d2/db/d15/d69/d100/f108 x:0 0 0 2024-03-11T19:50:17.882 INFO:tasks.workunit.client.0.smithi090.stdout:2/864: unlink d0/de/lee 0 2024-03-11T19:50:17.963 INFO:tasks.workunit.client.0.smithi090.stdout:9/882: fdatasync d1/dd/d18/d3a/da2/fc0 0 2024-03-11T19:50:18.419 INFO:tasks.workunit.client.0.smithi090.stdout:9/883: sync 2024-03-11T19:50:18.505 INFO:tasks.workunit.client.0.smithi090.stdout:5/880: link d1/d26/d4d/l61 d1/d17/d39/d51/d9a/l129 0 2024-03-11T19:50:18.614 INFO:tasks.workunit.client.0.smithi090.stdout:4/942: creat d4/dd8/f13a x:0 0 0 2024-03-11T19:50:18.774 INFO:tasks.workunit.client.0.smithi090.stdout:2/865: readlink d0/de/db6/d25/l57 0 2024-03-11T19:50:19.074 INFO:tasks.workunit.client.0.smithi090.stdout:2/866: dread d0/d21/fa9 [0,4194304] 0 2024-03-11T19:50:19.095 INFO:tasks.workunit.client.0.smithi090.stdout:1/895: write d5/f9a [1967897,89312] 0 2024-03-11T19:50:19.336 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:50:19.339 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:50:19.406 INFO:tasks.workunit.client.0.smithi090.stdout:9/884: symlink d1/dd/d18/d3a/d48/dc6/l12b 0 2024-03-11T19:50:19.415 INFO:tasks.workunit.client.0.smithi090.stdout:8/946: dwrite d0/d43/dc/d90/fac [0,4194304] 0 2024-03-11T19:50:19.560 INFO:tasks.workunit.client.0.smithi090.stdout:0/901: write d2/d11/d15/d29/de5/d11b/db4/fa8 [774905,55683] 0 2024-03-11T19:50:19.609 INFO:tasks.workunit.client.0.smithi090.stdout:8/947: stat d0/d60/db6/fca 0 2024-03-11T19:50:19.734 INFO:tasks.workunit.client.0.smithi090.stdout:6/931: fsync d4/d13/d22/d42/d48/d4e/d136/db7/fc7 0 2024-03-11T19:50:20.070 INFO:tasks.workunit.client.0.smithi090.stdout:6/932: dread d4/d13/d22/d34/d40/dd5/fff [0,4194304] 0 2024-03-11T19:50:20.466 INFO:tasks.workunit.client.0.smithi090.stdout:7/866: rmdir d3/dc/d1e/d7c/d96/d32/d67/dee/d11c 0 2024-03-11T19:50:20.724 INFO:tasks.workunit.client.0.smithi090.stdout:3/831: creat d2/db/f109 x:0 0 0 2024-03-11T19:50:20.844 INFO:tasks.workunit.client.0.smithi090.stdout:9/885: mknod d1/dd/d18/d3a/d48/dc6/c12c 0 2024-03-11T19:50:20.885 INFO:tasks.workunit.client.0.smithi090.stdout:9/886: stat d1/dd/d18/d25/d101/d1c/d77/fc4 0 2024-03-11T19:50:20.974 INFO:tasks.workunit.client.0.smithi090.stdout:0/902: unlink d2/d11/d15/d29/d3d/d8c/lb1 0 2024-03-11T19:50:21.025 INFO:tasks.workunit.client.0.smithi090.stdout:3/832: sync 2024-03-11T19:50:21.803 INFO:tasks.workunit.client.0.smithi090.stdout:2/867: truncate d0/de/f16 4007370 0 2024-03-11T19:50:21.914 INFO:tasks.workunit.client.0.smithi090.stdout:7/867: creat d3/dc/da6/f12e x:0 0 0 2024-03-11T19:50:22.182 INFO:tasks.workunit.client.0.smithi090.stdout:9/887: mkdir d1/dd/d18/d25/d101/d1c/d21/d2a/d12d 0 2024-03-11T19:50:22.193 INFO:tasks.workunit.client.0.smithi090.stdout:8/948: creat d0/d3/db0/d132/f135 x:0 0 0 2024-03-11T19:50:22.341 INFO:tasks.workunit.client.0.smithi090.stdout:0/903: symlink d2/d11/d3a/da6/l124 0 2024-03-11T19:50:22.580 INFO:tasks.workunit.client.0.smithi090.stdout:5/881: write d1/d17/d39/d51/d53/d7b/fdd [1816394,114128] 0 2024-03-11T19:50:22.580 INFO:tasks.workunit.client.0.smithi090.stdout:4/943: rename d4/dd8/d66/f126 to d4/dd8/f13b 0 2024-03-11T19:50:22.942 INFO:tasks.workunit.client.0.smithi090.stdout:7/868: truncate d3/dc/d1e/d7c/d96/d32/d45/de0/f4c 4229081 0 2024-03-11T19:50:23.198 INFO:tasks.workunit.client.0.smithi090.stdout:8/949: mknod d0/d3/d86/d127/d12e/c136 0 2024-03-11T19:50:23.246 INFO:tasks.workunit.client.0.smithi090.stdout:1/896: dwrite d5/da/f92 [0,4194304] 0 2024-03-11T19:50:23.580 INFO:tasks.workunit.client.0.smithi090.stdout:5/882: fdatasync d1/d17/fa9 0 2024-03-11T19:50:24.948 INFO:tasks.workunit.client.0.smithi090.stdout:8/950: fsync d0/d53/d59/d96/de5/de7/f134 0 2024-03-11T19:50:25.097 INFO:tasks.workunit.client.0.smithi090.stdout:6/933: link d4/d13/d22/d34/d40/dd5/f5f d4/d13/d22/f13a 0 2024-03-11T19:50:25.488 INFO:tasks.workunit.client.0.smithi090.stdout:3/833: dwrite d2/db/d15/d69/dad/fbf [4194304,4194304] 0 2024-03-11T19:50:25.989 INFO:tasks.workunit.client.0.smithi090.stdout:9/888: dwrite d1/dd/f3f [0,4194304] 0 2024-03-11T19:50:25.991 INFO:tasks.workunit.client.0.smithi090.stdout:4/944: write d4/d51/f72 [500820,85499] 0 2024-03-11T19:50:26.582 INFO:tasks.workunit.client.0.smithi090.stdout:1/897: creat d5/da/d1c/d3b/f137 x:0 0 0 2024-03-11T19:50:26.700 INFO:tasks.workunit.client.0.smithi090.stdout:2/868: dwrite d0/de/d12/d46/d75/f7c [0,4194304] 0 2024-03-11T19:50:26.887 INFO:tasks.workunit.client.0.smithi090.stdout:3/834: symlink d2/d2f/dc4/l10a 0 2024-03-11T19:50:26.910 INFO:tasks.workunit.client.0.smithi090.stdout:0/904: rename d2/d11/l22 to d2/d11/d15/d29/d3d/l125 0 2024-03-11T19:50:27.217 INFO:tasks.workunit.client.0.smithi090.stdout:8/951: creat d0/d3/d128/f137 x:0 0 0 2024-03-11T19:50:27.801 INFO:tasks.workunit.client.0.smithi090.stdout:2/869: creat d0/de/d12/d46/d5d/d63/f10d x:0 0 0 2024-03-11T19:50:27.822 INFO:tasks.workunit.client.0.smithi090.stdout:2/870: stat d0/d21 0 2024-03-11T19:50:27.861 INFO:tasks.workunit.client.0.smithi090.stdout:2/871: dread - d0/de/d12/d46/fb1 zero size 2024-03-11T19:50:27.897 INFO:tasks.workunit.client.0.smithi090.stdout:3/835: readlink d2/l2a 0 2024-03-11T19:50:28.046 INFO:tasks.workunit.client.0.smithi090.stdout:5/883: rename d1/d26/f9f to d1/d26/d4d/f12a 0 2024-03-11T19:50:28.762 INFO:tasks.workunit.client.0.smithi090.stdout:2/872: truncate d0/d21/f6e 8548308 0 2024-03-11T19:50:28.772 INFO:tasks.workunit.client.0.smithi090.stdout:3/836: mknod d2/db/d15/d69/dad/de9/c10b 0 2024-03-11T19:50:28.823 INFO:tasks.workunit.client.0.smithi090.stdout:2/873: chown d0/d21/fba 2390029 1 2024-03-11T19:50:29.043 INFO:tasks.workunit.client.0.smithi090.stdout:6/934: write d4/d13/d22/d42/d48/d4e/f11c [576920,89117] 0 2024-03-11T19:50:29.044 INFO:tasks.workunit.client.0.smithi090.stdout:0/905: creat d2/d11/d15/d29/df3/f126 x:0 0 0 2024-03-11T19:50:29.105 INFO:tasks.workunit.client.0.smithi090.stdout:6/935: stat d4/d14/d8a/f96 0 2024-03-11T19:50:29.145 INFO:tasks.workunit.client.0.smithi090.stdout:7/869: rename d3/dc/d1e/d7c/d96/d32/d45/ff5 to d3/dff/f12f 0 2024-03-11T19:50:29.223 INFO:tasks.workunit.client.0.smithi090.stdout:5/884: mkdir d1/d17/d39/df2/d12b 0 2024-03-11T19:50:29.914 INFO:tasks.workunit.client.0.smithi090.stdout:8/952: fdatasync d0/d3/d3e/f120 0 2024-03-11T19:50:30.297 INFO:tasks.workunit.client.0.smithi090.stdout:2/874: read - d0/de/d12/d46/d5d/d63/fd9 zero size 2024-03-11T19:50:30.543 INFO:tasks.workunit.client.0.smithi090.stdout:6/936: mknod d4/d13/d22/d34/d40/dd5/d80/ddb/c13b 0 2024-03-11T19:50:30.774 INFO:tasks.workunit.client.0.smithi090.stdout:1/898: write d5/da/d23/f2a [5422081,71378] 0 2024-03-11T19:50:30.843 INFO:tasks.workunit.client.0.smithi090.stdout:4/945: dwrite d4/dd8/d66/f130 [0,4194304] 0 2024-03-11T19:50:31.019 INFO:tasks.workunit.client.0.smithi090.stdout:3/837: fsync d2/db/d15/d1a/d75/de8/d6a/dcb/f3b 0 2024-03-11T19:50:31.195 INFO:tasks.workunit.client.0.smithi090.stdout:9/889: dwrite d1/dd/d18/d25/d101/f4d [0,4194304] 0 2024-03-11T19:50:31.769 INFO:tasks.workunit.client.0.smithi090.stdout:5/885: symlink d1/d17/d39/d51/d53/d7b/d3b/d76/d92/dc1/dff/l12c 0 2024-03-11T19:50:32.615 INFO:tasks.workunit.client.0.smithi090.stdout:2/875: unlink d0/de/db6/d25/c44 0 2024-03-11T19:50:32.761 INFO:tasks.workunit.client.0.smithi090.stdout:0/906: link d2/d11/d3f/l45 d2/d11/d15/d29/de5/d11b/d6d/da4/d78/l127 0 2024-03-11T19:50:33.046 INFO:tasks.workunit.client.0.smithi090.stdout:7/870: truncate d3/dc/d1e/d7c/d96/d32/d7b/f86 1697834 0 2024-03-11T19:50:33.079 INFO:tasks.workunit.client.0.smithi090.stdout:7/871: dread - d3/dc/d1e/ff8 zero size 2024-03-11T19:50:33.428 INFO:tasks.workunit.client.0.smithi090.stdout:1/899: mknod d5/df5/d129/c138 0 2024-03-11T19:50:34.181 INFO:tasks.workunit.client.0.smithi090.stdout:9/890: symlink d1/d47/d7c/dee/l12e 0 2024-03-11T19:50:34.201 INFO:tasks.workunit.client.0.smithi090.stdout:3/838: rename d2/db/d15/d1a/d75/dc6/fc5 to d2/db/d15/d69/da2/dc7/f10c 0 2024-03-11T19:50:34.458 INFO:tasks.workunit.client.0.smithi090.stdout:2/876: dread d0/de/f16 [0,4194304] 0 2024-03-11T19:50:34.533 INFO:tasks.workunit.client.0.smithi090.stdout:1/900: creat d5/d31/d4c/d116/f139 x:0 0 0 2024-03-11T19:50:34.598 INFO:tasks.workunit.client.0.smithi090.stdout:1/901: chown d5/da/d23/fbf 132319986 1 2024-03-11T19:50:34.984 INFO:tasks.workunit.client.0.smithi090.stdout:8/953: dwrite d0/d3/d3e/d85/fcd [0,4194304] 0 2024-03-11T19:50:35.082 INFO:tasks.workunit.client.0.smithi090.stdout:5/886: link d1/d17/d39/f10e d1/d17/d39/d51/d53/d7b/d3b/d76/db7/ddf/dee/f12d 0 2024-03-11T19:50:35.191 INFO:tasks.workunit.client.0.smithi090.stdout:6/937: write d4/d14/d8a/d74/f7b [3813492,49486] 0 2024-03-11T19:50:35.239 INFO:tasks.workunit.client.0.smithi090.stdout:3/839: stat d2/d45/l4d 0 2024-03-11T19:50:35.342 INFO:tasks.workunit.client.0.smithi090.stdout:7/872: rename d3/d55/l72 to d3/dc/d1e/d7c/d96/d32/d67/l130 0 2024-03-11T19:50:35.756 INFO:tasks.workunit.client.0.smithi090.stdout:2/877: truncate d0/de/d12/d46/d5d/d63/fd9 966253 0 2024-03-11T19:50:35.978 INFO:tasks.workunit.client.0.smithi090.stdout:5/887: chown d1/d17/la4 507265 1 2024-03-11T19:50:36.083 INFO:tasks.workunit.client.0.smithi090.stdout:6/938: dread - d4/d13/d22/d34/d37/f11b zero size 2024-03-11T19:50:36.259 INFO:tasks.workunit.client.0.smithi090.stdout:3/840: creat d2/db/d15/d1a/d75/de8/d6a/dcb/f10d x:0 0 0 2024-03-11T19:50:36.324 INFO:tasks.workunit.client.0.smithi090.stdout:1/902: symlink d5/d31/d4c/daf/dfc/d103/l13a 0 2024-03-11T19:50:36.771 INFO:tasks.workunit.client.0.smithi090.stdout:5/888: stat d1/d17/d39/d51/d53/d7b/d3b/d76/db7/ddf/f101 0 2024-03-11T19:50:37.005 INFO:tasks.workunit.client.0.smithi090.stdout:8/954: rename d0/f2d to d0/d53/d59/d96/de5/de7/f138 0 2024-03-11T19:50:37.050 INFO:tasks.workunit.client.0.smithi090.stdout:8/955: dread - d0/d3/d3e/d85/fbd zero size 2024-03-11T19:50:37.172 INFO:tasks.workunit.client.0.smithi090.stdout:3/841: read d2/db/f24 [4369663,6495] 0 2024-03-11T19:50:37.316 INFO:tasks.workunit.client.0.smithi090.stdout:4/946: dwrite d4/dd8/d66/f134 [0,4194304] 0 2024-03-11T19:50:37.680 INFO:tasks.workunit.client.0.smithi090.stdout:0/907: dwrite d2/d11/d15/d29/de5/d11b/db4/daf/ffa [0,4194304] 0 2024-03-11T19:50:37.846 INFO:tasks.workunit.client.0.smithi090.stdout:5/889: creat d1/d17/d39/de8/f12e x:0 0 0 2024-03-11T19:50:37.860 INFO:tasks.workunit.client.0.smithi090.stdout:1/903: read - d5/da/d1c/d3b/f136 zero size 2024-03-11T19:50:37.935 INFO:tasks.workunit.client.0.smithi090.stdout:7/873: creat d3/dc/d1e/d7c/d96/d32/d45/f131 x:0 0 0 2024-03-11T19:50:38.298 INFO:tasks.workunit.client.0.smithi090.stdout:8/956: dread d0/d3/f2c [0,4194304] 0 2024-03-11T19:50:38.470 INFO:tasks.workunit.client.0.smithi090.stdout:3/842: mknod d2/db/d15/d69/da2/dc7/c10e 0 2024-03-11T19:50:38.937 INFO:tasks.workunit.client.0.smithi090.stdout:5/890: symlink d1/d17/d39/d51/d53/d7b/d3b/d76/d92/dc1/l12f 0 2024-03-11T19:50:39.114 INFO:tasks.workunit.client.0.smithi090.stdout:7/874: rmdir d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab 39 2024-03-11T19:50:39.145 INFO:tasks.workunit.client.0.smithi090.stdout:4/947: dread d4/f4a [0,4194304] 0 2024-03-11T19:50:39.190 INFO:tasks.workunit.client.0.smithi090.stdout:0/908: dread d2/d11/d15/d29/d43/f98 [0,4194304] 0 2024-03-11T19:50:39.258 INFO:tasks.workunit.client.0.smithi090.stdout:2/878: rename d0/de/d12/cdb to d0/de/d12/d46/de8/deb/c10e 0 2024-03-11T19:50:39.308 INFO:tasks.workunit.client.0.smithi090.stdout:9/891: write d1/dd/d18/d25/d101/d1c/d21/d2a/f29 [3495973,76240] 0 2024-03-11T19:50:39.342 INFO:tasks.workunit.client.0.smithi090.stdout:8/957: truncate d0/d3/d4c/d8d/fd1 838730 0 2024-03-11T19:50:39.380 INFO:tasks.workunit.client.0.smithi090.stdout:3/843: rmdir d2/db/d15/d1a/d75 39 2024-03-11T19:50:39.617 INFO:tasks.workunit.client.0.smithi090.stdout:9/892: dread d1/dd/d18/d25/d101/d1c/d21/d2a/dc2/d100/d120/f45 [0,4194304] 0 2024-03-11T19:50:39.898 INFO:tasks.workunit.client.0.smithi090.stdout:6/939: dwrite d4/d14/f137 [0,4194304] 0 2024-03-11T19:50:40.766 INFO:tasks.workunit.client.0.smithi090.stdout:7/875: stat d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/db5/de1/dd2/cd6 0 2024-03-11T19:50:41.135 INFO:tasks.workunit.client.0.smithi090.stdout:1/904: write d5/d65/d6b/d7f/fab [878968,94211] 0 2024-03-11T19:50:41.894 INFO:tasks.workunit.client.0.smithi090.stdout:4/948: fdatasync d4/dd8/f86 0 2024-03-11T19:50:41.965 INFO:tasks.workunit.client.0.smithi090.stdout:6/940: read d4/d13/d22/d34/d83/d8d/f118 [2821093,21177] 0 2024-03-11T19:50:42.791 INFO:tasks.workunit.client.0.smithi090.stdout:1/905: fdatasync d5/da/d1c/d3b/fb3 0 2024-03-11T19:50:43.511 INFO:tasks.workunit.client.0.smithi090.stdout:9/893: write d1/dd/d18/d25/d101/d1c/d21/d2a/dc2/d100/d120/f34 [248434,28482] 0 2024-03-11T19:50:43.655 INFO:tasks.workunit.client.0.smithi090.stdout:4/949: mknod d4/d27/d28/d38/d75/c13c 0 2024-03-11T19:50:43.656 INFO:tasks.workunit.client.0.smithi090.stdout:8/958: link d0/d3/d4c/f88 d0/d60/db6/f139 0 2024-03-11T19:50:43.714 INFO:tasks.workunit.client.0.smithi090.stdout:8/959: stat d0/d43/dc/d90/f10e 0 2024-03-11T19:50:44.255 INFO:tasks.workunit.client.0.smithi090.stdout:5/891: dwrite d1/d17/d39/d51/d73/df6/f119 [0,4194304] 0 2024-03-11T19:50:44.713 INFO:tasks.workunit.client.0.smithi090.stdout:0/909: link d2/d11/df4/l123 d2/l128 0 2024-03-11T19:50:44.731 INFO:tasks.workunit.client.0.smithi090.stdout:6/941: symlink d4/l13c 0 2024-03-11T19:50:44.742 INFO:tasks.workunit.client.0.smithi090.stdout:3/844: write d2/db/d15/d1a/d75/de8/d6a/f97 [323129,111039] 0 2024-03-11T19:50:44.780 INFO:tasks.workunit.client.0.smithi090.stdout:1/906: stat d5/d4e/d54/d94/cbb 0 2024-03-11T19:50:44.836 INFO:tasks.workunit.client.0.smithi090.stdout:6/942: chown d4/d9b/def/d134 2740 1 2024-03-11T19:50:44.871 INFO:tasks.workunit.client.0.smithi090.stdout:1/907: readlink d5/d31/d76/d77/dbd/ld4 0 2024-03-11T19:50:45.285 INFO:tasks.workunit.client.0.smithi090.stdout:7/876: dwrite d3/dc/d1e/d7c/d96/d32/d45/de0/f4c [0,4194304] 0 2024-03-11T19:50:45.423 INFO:tasks.workunit.client.0.smithi090.stdout:4/950: truncate d4/d51/f12d 609264 0 2024-03-11T19:50:46.007 INFO:tasks.workunit.client.0.smithi090.stdout:2/879: getdents d0/de/df8 0 2024-03-11T19:50:46.014 INFO:tasks.workunit.client.0.smithi090.stdout:9/894: write d1/dd/d18/d3a/d48/fc1 [445678,110097] 0 2024-03-11T19:50:46.186 INFO:tasks.workunit.client.0.smithi090.stdout:3/845: dread d2/db/ffe [0,4194304] 0 2024-03-11T19:50:46.365 INFO:tasks.workunit.client.0.smithi090.stdout:0/910: fdatasync d2/d11/d15/d29/de5/d11b/fa0 0 2024-03-11T19:50:46.461 INFO:tasks.workunit.client.0.smithi090.stdout:1/908: chown d5/d31/d4c/daf/dfc/f12e 98161 1 2024-03-11T19:50:46.719 INFO:tasks.workunit.client.0.smithi090.stdout:8/960: fsync d0/f47 0 2024-03-11T19:50:46.764 INFO:tasks.workunit.client.0.smithi090.stdout:7/877: chown d3/dc/fa8 1454 1 2024-03-11T19:50:47.255 INFO:tasks.workunit.client.0.smithi090.stdout:5/892: write d1/f4a [1496168,63507] 0 2024-03-11T19:50:47.264 INFO:tasks.workunit.client.0.smithi090.stdout:9/895: creat d1/dd/d18/d3a/d48/dc6/f12f x:0 0 0 2024-03-11T19:50:47.331 INFO:tasks.workunit.client.0.smithi090.stdout:2/880: mknod d0/de/d12/d46/d5d/d63/c10f 0 2024-03-11T19:50:47.756 INFO:tasks.workunit.client.0.smithi090.stdout:0/911: unlink d2/d11/d3a/c8b 0 2024-03-11T19:50:47.885 INFO:tasks.workunit.client.0.smithi090.stdout:1/909: mknod d5/d65/d6b/d7f/d10c/c13b 0 2024-03-11T19:50:48.188 INFO:tasks.workunit.client.0.smithi090.stdout:8/961: fdatasync d0/d3/d86/d127/d12e/dba/fcf 0 2024-03-11T19:50:48.381 INFO:tasks.workunit.client.0.smithi090.stdout:4/951: mknod d4/d51/d8c/c13d 0 2024-03-11T19:50:48.576 INFO:tasks.workunit.client.0.smithi090.stdout:2/881: rmdir d0/de/db6 39 2024-03-11T19:50:49.021 INFO:tasks.workunit.client.0.smithi090.stdout:6/943: dwrite d4/d14/f110 [0,4194304] 0 2024-03-11T19:50:49.240 INFO:tasks.workunit.client.0.smithi090.stdout:9/896: truncate d1/dd/d18/d25/d101/d1c/d9f/f8d 1464825 0 2024-03-11T19:50:49.260 INFO:tasks.workunit.client.0.smithi090.stdout:9/897: chown d1/lb9 3210 1 2024-03-11T19:50:49.325 INFO:tasks.workunit.client.0.smithi090.stdout:0/912: rename d2/d11/f41 to d2/d11/d15/d29/d43/df8/f129 0 2024-03-11T19:50:49.405 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:50:49.408 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:50:49.420 INFO:tasks.workunit.client.0.smithi090.stdout:1/910: chown d5/d65/d6b/de7/l101 10519227 1 2024-03-11T19:50:49.656 INFO:tasks.workunit.client.0.smithi090.stdout:8/962: creat d0/d53/d59/d82/f13a x:0 0 0 2024-03-11T19:50:49.665 INFO:tasks.workunit.client.0.smithi090.stdout:1/911: dread d5/d65/f69 [0,4194304] 0 2024-03-11T19:50:49.738 INFO:tasks.workunit.client.0.smithi090.stdout:8/963: chown d0/d3/d3e/f4e 171 1 2024-03-11T19:50:49.771 INFO:tasks.workunit.client.0.smithi090.stdout:8/964: chown d0/d3/d3e/d85/ced 8 1 2024-03-11T19:50:49.806 INFO:tasks.workunit.client.0.smithi090.stdout:7/878: write d3/dc/d1e/d2a/fcc [1877574,68929] 0 2024-03-11T19:50:49.807 INFO:tasks.workunit.client.0.smithi090.stdout:4/952: fsync d4/d18/f6e 0 2024-03-11T19:50:51.034 INFO:tasks.workunit.client.0.smithi090.stdout:3/846: link d2/db/d15/d49/lf7 d2/db/d15/d1a/d75/de8/d6a/dcb/l10f 0 2024-03-11T19:50:51.067 INFO:tasks.workunit.client.0.smithi090.stdout:3/847: readlink d2/db/d15/d1a/d3c/la3 0 2024-03-11T19:50:51.491 INFO:tasks.workunit.client.0.smithi090.stdout:6/944: truncate d4/d13/fa2 219744 0 2024-03-11T19:50:51.509 INFO:tasks.workunit.client.0.smithi090.stdout:0/913: fsync d2/d11/d15/d29/de5/d11b/fa0 0 2024-03-11T19:50:51.567 INFO:tasks.workunit.client.0.smithi090.stdout:6/945: chown d4/d13/d22/d3a/dd7 381282 1 2024-03-11T19:50:51.856 INFO:tasks.workunit.client.0.smithi090.stdout:8/965: read d0/d3/d3e/f104 [188354,34610] 0 2024-03-11T19:50:51.941 INFO:tasks.workunit.client.0.smithi090.stdout:4/953: mknod d4/d51/d8c/d6a/c13e 0 2024-03-11T19:50:52.115 INFO:tasks.workunit.client.0.smithi090.stdout:7/879: fsync d3/dc/d1e/d7c/d96/fdc 0 2024-03-11T19:50:52.141 INFO:tasks.workunit.client.0.smithi090.stdout:7/880: readlink d3/d55/lc9 0 2024-03-11T19:50:52.161 INFO:tasks.workunit.client.0.smithi090.stdout:7/881: chown d3/d55/lc9 4875636 1 2024-03-11T19:50:52.302 INFO:tasks.workunit.client.0.smithi090.stdout:2/882: mknod d0/de/d12/c110 0 2024-03-11T19:50:52.331 INFO:tasks.workunit.client.0.smithi090.stdout:5/893: link d1/d17/d39/d51/d73/l9c d1/d17/d39/d51/d73/da6/l130 0 2024-03-11T19:50:52.756 INFO:tasks.workunit.client.0.smithi090.stdout:3/848: creat d2/db/d15/d1a/d75/de8/d6a/f110 x:0 0 0 2024-03-11T19:50:53.716 INFO:tasks.workunit.client.0.smithi090.stdout:8/966: creat d0/d43/dc/da1/f13b x:0 0 0 2024-03-11T19:50:53.863 INFO:tasks.workunit.client.0.smithi090.stdout:4/954: read d4/d27/d28/d69/f99 [4843373,48250] 0 2024-03-11T19:50:54.063 INFO:tasks.workunit.client.0.smithi090.stdout:1/912: dwrite d5/d4e/d54/d94/f96 [4194304,4194304] 0 2024-03-11T19:50:54.545 INFO:tasks.workunit.client.0.smithi090.stdout:3/849: truncate d2/d2f/f32 550328 0 2024-03-11T19:50:54.570 INFO:tasks.workunit.client.0.smithi090.stdout:3/850: readlink d2/d2f/dc4/lef 0 2024-03-11T19:50:54.830 INFO:tasks.workunit.client.0.smithi090.stdout:8/967: truncate d0/d43/dc/f8e 1219886 0 2024-03-11T19:50:54.882 INFO:tasks.workunit.client.0.smithi090.stdout:8/968: chown d0/d3/d98 1 1 2024-03-11T19:50:54.899 INFO:tasks.workunit.client.0.smithi090.stdout:8/969: chown d0/f47 321227464 1 2024-03-11T19:50:55.052 INFO:tasks.workunit.client.0.smithi090.stdout:7/882: symlink d3/dc/d1e/d7c/d96/d32/d45/l132 0 2024-03-11T19:50:55.075 INFO:tasks.workunit.client.0.smithi090.stdout:7/883: readlink d3/dc/d1e/lbe 0 2024-03-11T19:50:55.104 INFO:tasks.workunit.client.0.smithi090.stdout:4/955: truncate d4/f2e 1446831 0 2024-03-11T19:50:55.618 INFO:tasks.workunit.client.0.smithi090.stdout:5/894: mkdir d1/d17/d131 0 2024-03-11T19:50:55.633 INFO:tasks.workunit.client.0.smithi090.stdout:6/946: write d4/d13/d22/d42/d48/fc5 [2349293,120649] 0 2024-03-11T19:50:55.863 INFO:tasks.workunit.client.0.smithi090.stdout:6/947: dread d4/d13/d22/d34/d40/fa1 [0,4194304] 0 2024-03-11T19:50:55.910 INFO:tasks.workunit.client.0.smithi090.stdout:3/851: fdatasync d2/ff 0 2024-03-11T19:50:56.692 INFO:tasks.workunit.client.0.smithi090.stdout:7/884: readlink d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/db5/de1/dc2/ld3 0 2024-03-11T19:50:56.823 INFO:tasks.workunit.client.0.smithi090.stdout:5/895: creat d1/d26/db5/f132 x:0 0 0 2024-03-11T19:50:57.158 INFO:tasks.workunit.client.0.smithi090.stdout:9/898: dread d1/dd/d18/d25/d101/d1c/d9f/f8d [0,4194304] 0 2024-03-11T19:50:57.652 INFO:tasks.workunit.client.0.smithi090.stdout:0/914: getdents d2/d11/d15/d29/de5/d11b/d6d/da4/d78 0 2024-03-11T19:50:57.862 INFO:tasks.workunit.client.0.smithi090.stdout:2/883: dwrite d0/f4e [0,4194304] 0 2024-03-11T19:50:58.215 INFO:tasks.workunit.client.0.smithi090.stdout:9/899: dread d1/dd/d18/d3a/d48/dc6/f123 [0,4194304] 0 2024-03-11T19:50:58.341 INFO:tasks.workunit.client.0.smithi090.stdout:1/913: dwrite d5/da/f47 [0,4194304] 0 2024-03-11T19:50:58.595 INFO:tasks.workunit.client.0.smithi090.stdout:7/885: mknod d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/c133 0 2024-03-11T19:50:58.606 INFO:tasks.workunit.client.0.smithi090.stdout:0/915: symlink d2/d11/d3a/l12a 0 2024-03-11T19:50:58.858 INFO:tasks.workunit.client.0.smithi090.stdout:3/852: creat d2/db/d15/d1a/f111 x:0 0 0 2024-03-11T19:50:58.864 INFO:tasks.workunit.client.0.smithi090.stdout:2/884: stat d0/de/d12/d81/d93/cb9 0 2024-03-11T19:50:58.995 INFO:tasks.workunit.client.0.smithi090.stdout:4/956: creat d4/d27/d28/d38/d75/f13f x:0 0 0 2024-03-11T19:50:59.299 INFO:tasks.workunit.client.0.smithi090.stdout:1/914: fsync d5/d31/d4c/daf/dfc/f12e 0 2024-03-11T19:50:59.365 INFO:tasks.workunit.client.0.smithi090.stdout:7/886: mknod d3/dc/d1e/d7c/d96/d7f/c134 0 2024-03-11T19:50:59.785 INFO:tasks.workunit.client.0.smithi090.stdout:3/853: symlink d2/db/d15/d1a/d75/de8/d6a/dcb/l112 0 2024-03-11T19:50:59.863 INFO:tasks.workunit.client.0.smithi090.stdout:6/948: dwrite d4/d13/d22/d34/d40/fa1 [4194304,4194304] 0 2024-03-11T19:51:00.009 INFO:tasks.workunit.client.0.smithi090.stdout:5/896: dwrite d1/f38 [0,4194304] 0 2024-03-11T19:51:00.475 INFO:tasks.workunit.client.0.smithi090.stdout:8/970: rmdir d0 39 2024-03-11T19:51:00.513 INFO:tasks.workunit.client.0.smithi090.stdout:0/916: getdents d2/d11/d15/d29/d3d/d7e/dfc/def 0 2024-03-11T19:51:00.575 INFO:tasks.workunit.client.0.smithi090.stdout:1/915: dread - d5/da/f104 zero size 2024-03-11T19:51:00.742 INFO:tasks.workunit.client.0.smithi090.stdout:9/900: dwrite d1/dd/d18/d25/d101/d1c/d21/d2a/f50 [0,4194304] 0 2024-03-11T19:51:01.491 INFO:tasks.workunit.client.0.smithi090.stdout:7/887: dwrite d3/dc/f30 [4194304,4194304] 0 2024-03-11T19:51:01.644 INFO:tasks.workunit.client.0.smithi090.stdout:6/949: dread d4/d13/d22/d42/d48/d4e/fae [0,4194304] 0 2024-03-11T19:51:01.675 INFO:tasks.workunit.client.0.smithi090.stdout:8/971: chown d0/d53/l74 1598978483 1 2024-03-11T19:51:01.676 INFO:tasks.workunit.client.0.smithi090.stdout:7/888: chown d3/dc/f3a 2603 1 2024-03-11T19:51:01.875 INFO:tasks.workunit.client.0.smithi090.stdout:0/917: mkdir d2/d11/d3a/d12b 0 2024-03-11T19:51:01.877 INFO:tasks.workunit.client.0.smithi090.stdout:2/885: creat d0/de/d12/d46/d4c/f111 x:0 0 0 2024-03-11T19:51:02.083 INFO:tasks.workunit.client.0.smithi090.stdout:1/916: rmdir d5/d65/d6b/de7 39 2024-03-11T19:51:02.116 INFO:tasks.workunit.client.0.smithi090.stdout:1/917: chown d5/da/d23/dfe/fda 138 1 2024-03-11T19:51:02.133 INFO:tasks.workunit.client.0.smithi090.stdout:1/918: stat d5/d31/d36 0 2024-03-11T19:51:02.314 INFO:tasks.workunit.client.0.smithi090.stdout:9/901: symlink d1/dd/d18/d3a/d48/l130 0 2024-03-11T19:51:02.466 INFO:tasks.workunit.client.0.smithi090.stdout:3/854: truncate d2/d45/f7a 23054 0 2024-03-11T19:51:02.710 INFO:tasks.workunit.client.0.smithi090.stdout:5/897: fdatasync d1/d17/d39/d51/d53/d7b/f44 0 2024-03-11T19:51:02.891 INFO:tasks.workunit.client.0.smithi090.stdout:7/889: chown d3/dc/d1e/d7c/d96/d32/lf4 109 1 2024-03-11T19:51:03.158 INFO:tasks.workunit.client.0.smithi090.stdout:0/918: rmdir d2/d11/d3a/d64/dac/dba/dd3 39 2024-03-11T19:51:03.230 INFO:tasks.workunit.client.0.smithi090.stdout:2/886: read d0/f2 [7692334,28902] 0 2024-03-11T19:51:03.444 INFO:tasks.workunit.client.0.smithi090.stdout:1/919: read d5/d31/d86/fd9 [78538,36115] 0 2024-03-11T19:51:03.595 INFO:tasks.workunit.client.0.smithi090.stdout:9/902: symlink d1/d47/d7c/dee/l131 0 2024-03-11T19:51:03.664 INFO:tasks.workunit.client.0.smithi090.stdout:3/855: creat d2/d2f/f113 x:0 0 0 2024-03-11T19:51:03.756 INFO:tasks.workunit.client.0.smithi090.stdout:4/957: link d4/d27/d28/d69/f99 d4/dd8/f140 0 2024-03-11T19:51:04.163 INFO:tasks.workunit.client.0.smithi090.stdout:5/898: mkdir d1/d17/d39/d51/d53/d7b/d3b/d76/d92/dc1/d133 0 2024-03-11T19:51:04.705 INFO:tasks.workunit.client.0.smithi090.stdout:2/887: dread - d0/dad/f101 zero size 2024-03-11T19:51:04.745 INFO:tasks.workunit.client.0.smithi090.stdout:2/888: chown d0/f4e 1536019 1 2024-03-11T19:51:04.851 INFO:tasks.workunit.client.0.smithi090.stdout:6/950: mkdir d4/d13d 0 2024-03-11T19:51:04.851 INFO:tasks.workunit.client.0.smithi090.stdout:9/903: unlink d1/dd/d18/d25/d101/d1c/d9f/lb0 0 2024-03-11T19:51:05.132 INFO:tasks.workunit.client.0.smithi090.stdout:4/958: unlink d4/d51/d8c/d89/lae 0 2024-03-11T19:51:05.310 INFO:tasks.workunit.client.0.smithi090.stdout:7/890: mkdir d3/dc/d1e/d7c/d96/d32/d135 0 2024-03-11T19:51:05.866 INFO:tasks.workunit.client.0.smithi090.stdout:5/899: truncate d1/d17/d39/d51/d73/da6/fbf 844714 0 2024-03-11T19:51:06.104 INFO:tasks.workunit.client.0.smithi090.stdout:2/889: mkdir d0/de/d12/d46/d75/d112 0 2024-03-11T19:51:06.298 INFO:tasks.workunit.client.0.smithi090.stdout:9/904: truncate d1/dd/d18/d3a/d70/fe5 5023406 0 2024-03-11T19:51:06.567 INFO:tasks.workunit.client.0.smithi090.stdout:8/972: link d0/d60/l83 d0/d3/d3e/d85/d116/d11d/l13c 0 2024-03-11T19:51:06.571 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:16:26.329 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:51:06.573 INFO:tasks.workunit.client.0.smithi090.stdout:4/959: dread - d4/d51/ff7 zero size 2024-03-11T19:51:06.580 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:16:26.339 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:51:06.687 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:16:26.445 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:51:06.702 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:16:26.398 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:51:06.707 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:16:26.404 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:51:06.715 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:16:26.404 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:51:06.718 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:16:26.407 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:51:06.771 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:16:26.467 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:51:06.809 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:16:26.497 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:51:06.922 INFO:tasks.workunit.client.0.smithi090.stdout:1/920: dwrite d5/d31/d76/d77/fff [0,4194304] 0 2024-03-11T19:51:07.010 INFO:tasks.workunit.client.0.smithi090.stdout:7/891: mkdir d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/db5/de1/dc2/dd0/d136 0 2024-03-11T19:51:07.034 INFO:tasks.workunit.client.0.smithi090.stdout:1/921: readlink d5/d31/d36/lae 0 2024-03-11T19:51:07.086 INFO:tasks.workunit.client.0.smithi090.stdout:1/922: dread d5/d31/d86/fd9 [0,4194304] 0 2024-03-11T19:51:07.341 INFO:tasks.workunit.client.0.smithi090.stdout:0/919: creat d2/d11/d15/f12c x:0 0 0 2024-03-11T19:51:07.503 INFO:tasks.workunit.client.0.smithi090.stdout:6/951: creat d4/d13/d22/d42/d48/d4e/d136/d126/f13e x:0 0 0 2024-03-11T19:51:07.611 INFO:tasks.workunit.client.0.smithi090.stdout:8/973: rename d0/d3/d86/d127/d12e/dda/f102 to d0/d53/d59/d131/f13d 0 2024-03-11T19:51:08.844 INFO:tasks.workunit.client.0.smithi090.stdout:0/920: creat d2/d11/d15/d29/de5/f12d x:0 0 0 2024-03-11T19:51:09.588 INFO:tasks.workunit.client.0.smithi090.stdout:3/856: dwrite d2/db/d15/d1a/d75/dc6/f27 [0,4194304] 0 2024-03-11T19:51:09.721 INFO:tasks.workunit.client.0.smithi090.stdout:5/900: write d1/d17/d39/d51/d53/d7b/f114 [589309,53110] 0 2024-03-11T19:51:09.775 INFO:tasks.workunit.client.0.smithi090.stdout:5/901: readlink d1/d26/l115 0 2024-03-11T19:51:09.797 INFO:tasks.workunit.client.0.smithi090.stdout:5/902: chown d1/d26/l47 268 1 2024-03-11T19:51:09.929 INFO:tasks.workunit.client.0.smithi090.stdout:9/905: creat d1/dd/d18/d25/d101/d1c/d21/dab/f132 x:0 0 0 2024-03-11T19:51:09.990 INFO:tasks.workunit.client.0.smithi090.stdout:9/906: chown d1/dd/d18/d25/d101/d1c/d21/dab/d114/c84 4 1 2024-03-11T19:51:10.322 INFO:tasks.workunit.client.0.smithi090.stdout:4/960: dwrite d4/d27/f3a [0,4194304] 0 2024-03-11T19:51:10.783 INFO:tasks.workunit.client.0.smithi090.stdout:7/892: dwrite d3/dc/fc4 [0,4194304] 0 2024-03-11T19:51:10.906 INFO:tasks.workunit.client.0.smithi090.stdout:5/903: symlink d1/d17/d39/d51/d53/d7b/da0/l134 0 2024-03-11T19:51:11.017 INFO:tasks.workunit.client.0.smithi090.stdout:0/921: mkdir d2/d11/d3a/d64/dac/dba/dec/dfd/d12e 0 2024-03-11T19:51:11.057 INFO:tasks.workunit.client.0.smithi090.stdout:1/923: write d5/d31/d76/f4d [3343650,86368] 0 2024-03-11T19:51:11.349 INFO:tasks.workunit.client.0.smithi090.stdout:9/907: truncate d1/dd/d18/d3a/fbe 90712 0 2024-03-11T19:51:11.366 INFO:tasks.workunit.client.0.smithi090.stdout:2/890: write d0/de/d1f/f1c [6260685,59762] 0 2024-03-11T19:51:11.655 INFO:tasks.workunit.client.0.smithi090.stdout:4/961: symlink d4/dd8/d66/dcf/dd7/l141 0 2024-03-11T19:51:11.725 INFO:tasks.workunit.client.0.smithi090.stdout:8/974: write d0/d43/dc/f28 [6010600,120660] 0 2024-03-11T19:51:11.801 INFO:tasks.workunit.client.0.smithi090.stdout:3/857: mkdir d2/db/d15/d1a/d75/dc6/d114 0 2024-03-11T19:51:12.524 INFO:tasks.workunit.client.0.smithi090.stdout:6/952: write d4/d13/d22/f13a [2631439,80197] 0 2024-03-11T19:51:12.549 INFO:tasks.workunit.client.0.smithi090.stdout:7/893: mknod d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/db5/de1/dc6/d111/c137 0 2024-03-11T19:51:12.577 INFO:tasks.workunit.client.0.smithi090.stdout:0/922: truncate d2/d11/d3a/d64/dac/fca 1975430 0 2024-03-11T19:51:12.605 INFO:tasks.workunit.client.0.smithi090.stdout:5/904: chown d1/d17/d39/d51/d53/d7b/d3b/d76/d92/fa2 6112024 1 2024-03-11T19:51:12.722 INFO:tasks.workunit.client.0.smithi090.stdout:5/905: chown d1/d17/d39/d51/d53/d7b/fdd 15415884 1 2024-03-11T19:51:13.127 INFO:tasks.workunit.client.0.smithi090.stdout:3/858: symlink d2/db/d15/d69/da2/dc7/l115 0 2024-03-11T19:51:13.178 INFO:tasks.workunit.client.0.smithi090.stdout:3/859: readlink d2/db/d15/d1a/d75/de8/d6a/dcb/l112 0 2024-03-11T19:51:13.656 INFO:tasks.workunit.client.0.smithi090.stdout:7/894: mknod d3/dc/d1e/d7c/d96/d32/d67/c138 0 2024-03-11T19:51:13.692 INFO:tasks.workunit.client.0.smithi090.stdout:0/923: symlink d2/d11/d3a/dc8/dd0/l12f 0 2024-03-11T19:51:13.693 INFO:tasks.workunit.client.0.smithi090.stdout:5/906: creat d1/d26/d4d/f135 x:0 0 0 2024-03-11T19:51:13.816 INFO:tasks.workunit.client.0.smithi090.stdout:0/924: readlink d2/d11/d15/d29/de5/d11b/l10e 0 2024-03-11T19:51:13.866 INFO:tasks.workunit.client.0.smithi090.stdout:7/895: stat d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/db5/cc3 0 2024-03-11T19:51:14.128 INFO:tasks.workunit.client.0.smithi090.stdout:9/908: rename d1/dd/d18/d3a/l11e to d1/dd/d18/d25/d101/d1c/d21/d2a/d12d/l133 0 2024-03-11T19:51:14.243 INFO:tasks.workunit.client.0.smithi090.stdout:3/860: chown d2/fb1 11528 1 2024-03-11T19:51:14.402 INFO:tasks.workunit.client.0.smithi090.stdout:5/907: read - d1/d26/d19/fb6 zero size 2024-03-11T19:51:14.438 INFO:tasks.workunit.client.0.smithi090.stdout:5/908: dread - d1/d26/d4d/f135 zero size 2024-03-11T19:51:14.574 INFO:tasks.workunit.client.0.smithi090.stdout:0/925: dread - d2/d11/d15/d29/d43/d58/f107 zero size 2024-03-11T19:51:14.595 INFO:tasks.workunit.client.0.smithi090.stdout:0/926: readlink d2/d11/l59 0 2024-03-11T19:51:14.695 INFO:tasks.workunit.client.0.smithi090.stdout:7/896: read d3/dc/d1e/d7c/d96/d32/faf [3717926,130810] 0 2024-03-11T19:51:14.983 INFO:tasks.workunit.client.0.smithi090.stdout:6/953: rename d4/d13/d22/d3a to d4/d14/d107/d13f 0 2024-03-11T19:51:15.065 INFO:tasks.workunit.client.0.smithi090.stdout:3/861: mknod d2/db/d15/d69/da2/dc7/ddd/c116 0 2024-03-11T19:51:15.223 INFO:tasks.workunit.client.0.smithi090.stdout:5/909: creat d1/d17/d39/d51/d9a/d80/f136 x:0 0 0 2024-03-11T19:51:15.274 INFO:tasks.workunit.client.0.smithi090.stdout:7/897: sync 2024-03-11T19:51:15.613 INFO:tasks.workunit.client.0.smithi090.stdout:0/927: mkdir d2/d11/d15/d29/df3/d130 0 2024-03-11T19:51:15.808 INFO:tasks.workunit.client.0.smithi090.stdout:3/862: creat d2/db/d15/d1a/d75/d8c/f117 x:0 0 0 2024-03-11T19:51:15.812 INFO:tasks.workunit.client.0.smithi090.stdout:1/924: write d5/da/d97/fa8 [1001464,41927] 0 2024-03-11T19:51:16.222 INFO:tasks.workunit.client.0.smithi090.stdout:5/910: fdatasync d1/d17/d39/d51/d73/df6/f119 0 2024-03-11T19:51:16.231 INFO:tasks.workunit.client.0.smithi090.stdout:2/891: write d0/de/f16 [2049177,78119] 0 2024-03-11T19:51:16.471 INFO:tasks.workunit.client.0.smithi090.stdout:4/962: dwrite d4/ff [4194304,4194304] 0 2024-03-11T19:51:16.909 INFO:tasks.workunit.client.0.smithi090.stdout:9/909: rename d1/d1d/d6a/ld0 to d1/dd/d18/d25/d101/d1c/d88/d96/l134 0 2024-03-11T19:51:16.972 INFO:tasks.workunit.client.0.smithi090.stdout:0/928: dread d2/d11/d3a/f70 [0,4194304] 0 2024-03-11T19:51:17.056 INFO:tasks.workunit.client.0.smithi090.stdout:0/929: readlink d2/d11/d3a/d64/dac/dba/lce 0 2024-03-11T19:51:17.284 INFO:tasks.workunit.client.0.smithi090.stdout:6/954: fsync d4/d13/d22/d34/d40/fcb 0 2024-03-11T19:51:17.355 INFO:tasks.workunit.client.0.smithi090.stdout:8/975: write d0/d3/d86/f9c [3311361,6464] 0 2024-03-11T19:51:17.538 INFO:tasks.workunit.client.0.smithi090.stdout:2/892: read d0/de/f17 [2948656,10279] 0 2024-03-11T19:51:18.213 INFO:tasks.workunit.client.0.smithi090.stdout:9/910: creat d1/dd/d18/d25/d101/d1c/d21/d2a/d12d/f135 x:0 0 0 2024-03-11T19:51:18.747 INFO:tasks.workunit.client.0.smithi090.stdout:6/955: truncate d4/ff 964057 0 2024-03-11T19:51:18.759 INFO:tasks.workunit.client.0.smithi090.stdout:3/863: truncate d2/db/d15/f1e 4421710 0 2024-03-11T19:51:18.784 INFO:tasks.workunit.client.0.smithi090.stdout:8/976: symlink d0/d3/d86/d127/d12e/dba/l13e 0 2024-03-11T19:51:18.859 INFO:tasks.workunit.client.0.smithi090.stdout:8/977: chown d0/d3/d86/l8b 15 1 2024-03-11T19:51:19.058 INFO:tasks.workunit.client.0.smithi090.stdout:4/963: mkdir d4/d18/d105/d142 0 2024-03-11T19:51:19.093 INFO:tasks.workunit.client.0.smithi090.stdout:4/964: chown d4/d51/d8c/d89/fe4 6727590 1 2024-03-11T19:51:19.414 INFO:tasks.workunit.client.0.smithi090.stdout:4/965: dread d4/d27/d28/fbe [0,4194304] 0 2024-03-11T19:51:19.443 INFO:tasks.workunit.client.0.smithi090.stdout:4/966: fsync d4/f8 0 2024-03-11T19:51:19.453 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:51:19.456 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:51:19.493 INFO:tasks.workunit.client.0.smithi090.stdout:1/925: rename d5/d31/d76/l53 to d5/d4e/l13c 0 2024-03-11T19:51:19.551 INFO:tasks.workunit.client.0.smithi090.stdout:0/930: symlink d2/d11/l131 0 2024-03-11T19:51:19.705 INFO:tasks.workunit.client.0.smithi090.stdout:9/911: symlink d1/dd/d18/d3a/db6/de6/l136 0 2024-03-11T19:51:19.753 INFO:tasks.workunit.client.0.smithi090.stdout:7/898: link d3/dc/d1e/d7c/d96/fea d3/dc/d1e/d7c/d96/d32/d45/f139 0 2024-03-11T19:51:19.976 INFO:tasks.workunit.client.0.smithi090.stdout:6/956: chown d4/d13/d22/d34/d40/l84 270477741 1 2024-03-11T19:51:20.046 INFO:tasks.workunit.client.0.smithi090.stdout:8/978: truncate d0/d3/d3e/d85/d116/f12b 3391643 0 2024-03-11T19:51:20.152 INFO:tasks.workunit.client.0.smithi090.stdout:2/893: truncate d0/de/d12/d46/d5d/d8d/fa6 385011 0 2024-03-11T19:51:20.378 INFO:tasks.workunit.client.0.smithi090.stdout:8/979: dread d0/d3/d86/d127/d12e/dba/fcf [0,4194304] 0 2024-03-11T19:51:20.474 INFO:tasks.workunit.client.0.smithi090.stdout:1/926: rename d5/d4e/d120/f12c to d5/df5/d129/f13d 0 2024-03-11T19:51:20.570 INFO:tasks.workunit.client.0.smithi090.stdout:0/931: readlink d2/d11/d15/d29/de5/d11b/d6d/da4/d78/l127 0 2024-03-11T19:51:21.438 INFO:tasks.workunit.client.0.smithi090.stdout:5/911: write d1/f49 [3435544,90129] 0 2024-03-11T19:51:21.511 INFO:tasks.workunit.client.0.smithi090.stdout:5/912: readlink d1/d17/d39/d51/d73/da6/l130 0 2024-03-11T19:51:21.703 INFO:tasks.workunit.client.0.smithi090.stdout:2/894: mknod d0/de/d12/dc8/c113 0 2024-03-11T19:51:21.942 INFO:tasks.workunit.client.0.smithi090.stdout:8/980: mknod d0/d53/d59/c13f 0 2024-03-11T19:51:22.346 INFO:tasks.workunit.client.0.smithi090.stdout:1/927: rename d5/d4e/d54/d94 to d5/da/d23/dfe/d89/d10e/d80/d13e 0 2024-03-11T19:51:22.827 INFO:tasks.workunit.client.0.smithi090.stdout:7/899: truncate d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/db5/de1/dc2/f65 335476 0 2024-03-11T19:51:23.595 INFO:tasks.workunit.client.0.smithi090.stdout:8/981: symlink d0/d3/d86/d127/d12e/dba/l140 0 2024-03-11T19:51:23.825 INFO:tasks.workunit.client.0.smithi090.stdout:3/864: write d2/ffd [671210,65155] 0 2024-03-11T19:51:24.169 INFO:tasks.workunit.client.0.smithi090.stdout:7/900: dread d3/dc/d87/f90 [0,4194304] 0 2024-03-11T19:51:24.271 INFO:tasks.workunit.client.0.smithi090.stdout:5/913: creat d1/d17/d39/d126/f137 x:0 0 0 2024-03-11T19:51:24.361 INFO:tasks.workunit.client.0.smithi090.stdout:2/895: creat d0/de/dd3/f114 x:0 0 0 2024-03-11T19:51:24.532 INFO:tasks.workunit.client.0.smithi090.stdout:4/967: write d4/d27/d28/d38/d64/fa3 [2017567,58333] 0 2024-03-11T19:51:24.950 INFO:tasks.workunit.client.0.smithi090.stdout:5/914: symlink d1/d26/db5/l138 0 2024-03-11T19:51:25.069 INFO:tasks.workunit.client.0.smithi090.stdout:5/915: dread d1/d17/d39/d51/d53/d7b/d3b/d76/d92/fa2 [0,4194304] 0 2024-03-11T19:51:25.071 INFO:tasks.workunit.client.0.smithi090.stdout:2/896: fsync d0/de/d1f/f1c 0 2024-03-11T19:51:25.313 INFO:tasks.workunit.client.0.smithi090.stdout:9/912: write d1/dd/d18/d3a/f104 [424623,3937] 0 2024-03-11T19:51:25.484 INFO:tasks.workunit.client.0.smithi090.stdout:4/968: dread d4/d51/d8c/d6a/d8d/ff2 [0,4194304] 0 2024-03-11T19:51:25.700 INFO:tasks.workunit.client.0.smithi090.stdout:7/901: creat d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/db5/de1/dc6/d103/f13a x:0 0 0 2024-03-11T19:51:26.043 INFO:tasks.workunit.client.0.smithi090.stdout:6/957: write d4/d13/d22/d34/d40/dd5/fff [2155778,66882] 0 2024-03-11T19:51:26.317 INFO:tasks.workunit.client.0.smithi090.stdout:2/897: symlink d0/de/d12/d81/d93/l115 0 2024-03-11T19:51:26.653 INFO:tasks.workunit.client.0.smithi090.stdout:4/969: symlink d4/d51/dbf/l143 0 2024-03-11T19:51:26.890 INFO:tasks.workunit.client.0.smithi090.stdout:8/982: getdents d0/d53 0 2024-03-11T19:51:27.031 INFO:tasks.workunit.client.0.smithi090.stdout:0/932: write d2/d11/d15/d29/d3d/d7b/f96 [740000,16723] 0 2024-03-11T19:51:27.365 INFO:tasks.workunit.client.0.smithi090.stdout:4/970: symlink d4/dd8/dfa/db7/d11d/l144 0 2024-03-11T19:51:27.799 INFO:tasks.workunit.client.0.smithi090.stdout:6/958: creat d4/d13/d22/d42/d48/dd2/dfa/f140 x:0 0 0 2024-03-11T19:51:28.208 INFO:tasks.workunit.client.0.smithi090.stdout:8/983: creat d0/d3/d4c/d8d/f141 x:0 0 0 2024-03-11T19:51:28.213 INFO:tasks.workunit.client.0.smithi090.stdout:2/898: rename d0/de/d1f/f76 to d0/de/db6/d29/f116 0 2024-03-11T19:51:28.454 INFO:tasks.workunit.client.0.smithi090.stdout:6/959: mkdir d4/d9b/def/d134/d141 0 2024-03-11T19:51:28.661 INFO:tasks.workunit.client.0.smithi090.stdout:0/933: rename d2/d11/d3a/f70 to d2/d94/f132 0 2024-03-11T19:51:28.740 INFO:tasks.workunit.client.0.smithi090.stdout:1/928: write d5/d4e/d79/fc8 [294849,26749] 0 2024-03-11T19:51:28.811 INFO:tasks.workunit.client.0.smithi090.stdout:8/984: unlink d0/d43/dc/l4f 0 2024-03-11T19:51:29.108 INFO:tasks.workunit.client.0.smithi090.stdout:0/934: fdatasync d2/d11/d3a/f75 0 2024-03-11T19:51:29.452 INFO:tasks.workunit.client.0.smithi090.stdout:1/929: rename d5/d31/d36/fca to d5/da/d23/dfe/d89/d10e/d80/d13e/d122/f13f 0 2024-03-11T19:51:30.274 INFO:tasks.workunit.client.0.smithi090.stdout:1/930: truncate d5/f37 4007738 0 2024-03-11T19:51:30.281 INFO:tasks.workunit.client.0.smithi090.stdout:3/865: write d2/d45/d55/fff [617945,47434] 0 2024-03-11T19:51:30.338 INFO:tasks.workunit.client.0.smithi090.stdout:3/866: readlink d2/d2f/l54 0 2024-03-11T19:51:31.010 INFO:tasks.workunit.client.0.smithi090.stdout:3/867: mknod d2/d4f/c118 0 2024-03-11T19:51:31.438 INFO:tasks.workunit.client.0.smithi090.stdout:3/868: mkdir d2/db/d15/d69/d119 0 2024-03-11T19:51:32.157 INFO:tasks.workunit.client.0.smithi090.stdout:5/916: dwrite d1/d26/d19/fb6 [0,4194304] 0 2024-03-11T19:51:32.532 INFO:tasks.workunit.client.0.smithi090.stdout:9/913: dwrite d1/dd/d18/d25/d101/d1c/d9f/f8d [0,4194304] 0 2024-03-11T19:51:32.886 INFO:tasks.workunit.client.0.smithi090.stdout:5/917: creat d1/d17/d39/de8/f139 x:0 0 0 2024-03-11T19:51:33.108 INFO:tasks.workunit.client.0.smithi090.stdout:6/960: write d4/d14/f6a [1838947,2833] 0 2024-03-11T19:51:33.270 INFO:tasks.workunit.client.0.smithi090.stdout:7/902: dwrite d3/dc/d1e/d7c/d96/d32/d7b/d99/f100 [0,4194304] 0 2024-03-11T19:51:33.394 INFO:tasks.workunit.client.0.smithi090.stdout:4/971: dwrite d4/d27/d28/d69/d111/f12c [0,4194304] 0 2024-03-11T19:51:33.822 INFO:tasks.workunit.client.0.smithi090.stdout:2/899: dwrite d0/d21/f6e [8388608,4194304] 0 2024-03-11T19:51:34.086 INFO:tasks.workunit.client.0.smithi090.stdout:5/918: read d1/d17/d39/f10e [3837727,102098] 0 2024-03-11T19:51:34.263 INFO:tasks.workunit.client.0.smithi090.stdout:8/985: dwrite d0/d43/dc/d90/f9e [0,4194304] 0 2024-03-11T19:51:34.590 INFO:tasks.workunit.client.0.smithi090.stdout:7/903: truncate d3/f7 8620962 0 2024-03-11T19:51:34.698 INFO:tasks.workunit.client.0.smithi090.stdout:6/961: unlink d4/d13/d22/d34/d40/dd5/dc1/c100 0 2024-03-11T19:51:34.878 INFO:tasks.workunit.client.0.smithi090.stdout:1/931: write d5/d4e/d79/f98 [2293278,111091] 0 2024-03-11T19:51:35.158 INFO:tasks.workunit.client.0.smithi090.stdout:9/914: creat d1/f137 x:0 0 0 2024-03-11T19:51:35.198 INFO:tasks.workunit.client.0.smithi090.stdout:9/915: stat d1/dd/d18/d3a/d70/c87 0 2024-03-11T19:51:35.430 INFO:tasks.workunit.client.0.smithi090.stdout:8/986: mkdir d0/d60/db6/d142 0 2024-03-11T19:51:35.493 INFO:tasks.workunit.client.0.smithi090.stdout:8/987: dread d0/d43/dc/fd [0,4194304] 0 2024-03-11T19:51:35.696 INFO:tasks.workunit.client.0.smithi090.stdout:7/904: creat d3/dc/d1e/d7c/d96/d32/d45/de0/d48/f13b x:0 0 0 2024-03-11T19:51:35.847 INFO:tasks.workunit.client.0.smithi090.stdout:3/869: write d2/db/d15/d69/dad/fe4 [771329,64319] 0 2024-03-11T19:51:35.919 INFO:tasks.workunit.client.0.smithi090.stdout:1/932: mknod d5/da/d97/df6/c140 0 2024-03-11T19:51:36.091 INFO:tasks.workunit.client.0.smithi090.stdout:5/919: symlink d1/d17/d39/d51/d53/d7b/d6f/l13a 0 2024-03-11T19:51:36.147 INFO:tasks.workunit.client.0.smithi090.stdout:5/920: dread - d1/d17/d39/d51/d53/d7b/d3b/fdb zero size 2024-03-11T19:51:36.731 INFO:tasks.workunit.client.0.smithi090.stdout:7/905: dread d3/dc/d1e/d2a/f5e [0,4194304] 0 2024-03-11T19:51:36.803 INFO:tasks.workunit.client.0.smithi090.stdout:7/906: chown d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/db5/de1/dc2/dd0 7037122 1 2024-03-11T19:51:36.943 INFO:tasks.workunit.client.0.smithi090.stdout:1/933: creat d5/d31/d4c/daf/dfc/d103/f141 x:0 0 0 2024-03-11T19:51:37.124 INFO:tasks.workunit.client.0.smithi090.stdout:9/916: mknod d1/dd/d18/d3a/c138 0 2024-03-11T19:51:37.385 INFO:tasks.workunit.client.0.smithi090.stdout:2/900: link d0/de/db6/d29/cfc d0/de/d12/d46/d75/c117 0 2024-03-11T19:51:37.621 INFO:tasks.workunit.client.0.smithi090.stdout:0/935: write d2/d94/f132 [595092,21579] 0 2024-03-11T19:51:38.389 INFO:tasks.workunit.client.0.smithi090.stdout:1/934: creat d5/d31/d76/f142 x:0 0 0 2024-03-11T19:51:39.055 INFO:tasks.workunit.client.0.smithi090.stdout:9/917: getdents d1/dd/d18/d25/d101/d1c/d21/dab/d114 0 2024-03-11T19:51:39.289 INFO:tasks.workunit.client.0.smithi090.stdout:4/972: dwrite d4/dd8/f7e [0,4194304] 0 2024-03-11T19:51:39.591 INFO:tasks.workunit.client.0.smithi090.stdout:2/901: getdents d0/dad 0 2024-03-11T19:51:39.806 INFO:tasks.workunit.client.0.smithi090.stdout:4/973: sync 2024-03-11T19:51:39.832 INFO:tasks.workunit.client.0.smithi090.stdout:4/974: read - d4/d51/f92 zero size 2024-03-11T19:51:40.134 INFO:tasks.workunit.client.0.smithi090.stdout:1/935: mknod d5/d31/d76/c143 0 2024-03-11T19:51:40.547 INFO:tasks.workunit.client.0.smithi090.stdout:2/902: rmdir d0/de/db6 39 2024-03-11T19:51:41.137 INFO:tasks.workunit.client.0.smithi090.stdout:6/962: write d4/d14/f21 [4570221,125914] 0 2024-03-11T19:51:41.441 INFO:tasks.workunit.client.0.smithi090.stdout:3/870: dwrite d2/db/f24 [0,4194304] 0 2024-03-11T19:51:41.558 INFO:tasks.workunit.client.0.smithi090.stdout:1/936: creat d5/d31/d76/d114/f144 x:0 0 0 2024-03-11T19:51:41.895 INFO:tasks.workunit.client.0.smithi090.stdout:5/921: dwrite d1/d17/d39/d51/d53/d64/f6d [0,4194304] 0 2024-03-11T19:51:42.372 INFO:tasks.workunit.client.0.smithi090.stdout:7/907: dwrite d3/dc/f30 [8388608,4194304] 0 2024-03-11T19:51:42.486 INFO:tasks.workunit.client.0.smithi090.stdout:0/936: write d2/d11/d3a/d64/dac/f108 [666042,116180] 0 2024-03-11T19:51:42.588 INFO:tasks.workunit.client.0.smithi090.stdout:7/908: sync 2024-03-11T19:51:42.723 INFO:tasks.workunit.client.0.smithi090.stdout:0/937: sync 2024-03-11T19:51:42.751 INFO:tasks.workunit.client.0.smithi090.stdout:7/909: sync 2024-03-11T19:51:43.028 INFO:tasks.workunit.client.0.smithi090.stdout:6/963: mknod d4/d13/d22/d34/d40/c142 0 2024-03-11T19:51:43.061 INFO:tasks.workunit.client.0.smithi090.stdout:8/988: write d0/d3/d4c/d8d/fef [4926861,22446] 0 2024-03-11T19:51:43.130 INFO:tasks.workunit.client.0.smithi090.stdout:7/910: dread d3/dc/f30 [0,4194304] 0 2024-03-11T19:51:43.203 INFO:tasks.workunit.client.0.smithi090.stdout:6/964: read d4/d14/d107/d13f/f56 [3732050,107161] 0 2024-03-11T19:51:43.308 INFO:tasks.workunit.client.0.smithi090.stdout:1/937: symlink d5/da/d1c/d11a/l145 0 2024-03-11T19:51:43.409 INFO:tasks.workunit.client.0.smithi090.stdout:0/938: fdatasync d2/fab 0 2024-03-11T19:51:43.667 INFO:tasks.workunit.client.0.smithi090.stdout:2/903: getdents d0/de/d12/d46/d75/dc5/de5 0 2024-03-11T19:51:43.733 INFO:tasks.workunit.client.0.smithi090.stdout:8/989: mknod d0/d53/d59/d82/c143 0 2024-03-11T19:51:43.853 INFO:tasks.workunit.client.0.smithi090.stdout:8/990: sync 2024-03-11T19:51:44.021 INFO:tasks.workunit.client.0.smithi090.stdout:7/911: rename d3/dc/d1e/c1d to d3/dc/da6/deb/c13c 0 2024-03-11T19:51:44.616 INFO:tasks.workunit.client.0.smithi090.stdout:0/939: mknod d2/d11/d15/d29/de5/d11b/d6d/c133 0 2024-03-11T19:51:44.631 INFO:tasks.workunit.client.0.smithi090.stdout:9/918: write d1/dd/d18/d25/d101/d1c/d21/d2a/f29 [787016,99612] 0 2024-03-11T19:51:44.710 INFO:tasks.workunit.client.0.smithi090.stdout:0/940: stat d2/l86 0 2024-03-11T19:51:44.974 INFO:tasks.workunit.client.0.smithi090.stdout:4/975: dwrite d4/f12 [4194304,4194304] 0 2024-03-11T19:51:45.473 INFO:tasks.workunit.client.0.smithi090.stdout:2/904: creat d0/de/d12/d46/d75/d112/f118 x:0 0 0 2024-03-11T19:51:45.539 INFO:tasks.workunit.client.0.smithi090.stdout:0/941: truncate d2/f3 2959231 0 2024-03-11T19:51:45.684 INFO:tasks.workunit.client.0.smithi090.stdout:9/919: dread d1/dd/d18/d25/d101/d1c/d21/f9d [0,4194304] 0 2024-03-11T19:51:45.693 INFO:tasks.workunit.client.0.smithi090.stdout:0/942: chown d2/d11/l131 0 1 2024-03-11T19:51:45.745 INFO:tasks.workunit.client.0.smithi090.stdout:0/943: chown d2/d11/d6e/fdf 11545 1 2024-03-11T19:51:46.486 INFO:tasks.workunit.client.0.smithi090.stdout:7/912: rmdir d3/d55 39 2024-03-11T19:51:46.502 INFO:tasks.workunit.client.0.smithi090.stdout:5/922: write d1/d17/d39/d51/d73/da6/f112 [863794,6151] 0 2024-03-11T19:51:46.616 INFO:tasks.workunit.client.0.smithi090.stdout:2/905: stat d0/de/db6/d29/c7e 0 2024-03-11T19:51:46.777 INFO:tasks.workunit.client.0.smithi090.stdout:1/938: write d5/da/f104 [933151,62783] 0 2024-03-11T19:51:46.800 INFO:tasks.workunit.client.0.smithi090.stdout:8/991: write d0/d53/d59/d82/f100 [46158,130279] 0 2024-03-11T19:51:47.003 INFO:tasks.workunit.client.0.smithi090.stdout:6/965: dwrite d4/d13/d22/d42/d48/d4e/d136/fa5 [0,4194304] 0 2024-03-11T19:51:47.154 INFO:tasks.workunit.client.0.smithi090.stdout:4/976: mknod d4/dd8/d66/da1/d112/c145 0 2024-03-11T19:51:47.161 INFO:tasks.workunit.client.0.smithi090.stdout:9/920: creat d1/dd/d18/d3a/db6/de6/f139 x:0 0 0 2024-03-11T19:51:47.183 INFO:tasks.workunit.client.0.smithi090.stdout:3/871: dwrite d2/f5f [0,4194304] 0 2024-03-11T19:51:47.477 INFO:tasks.workunit.client.0.smithi090.stdout:8/992: mknod d0/d53/c144 0 2024-03-11T19:51:47.538 INFO:tasks.workunit.client.0.smithi090.stdout:8/993: chown d0/d3/d3e/d85/fe9 559349656 1 2024-03-11T19:51:48.077 INFO:tasks.workunit.client.0.smithi090.stdout:1/939: fsync d5/f15 0 2024-03-11T19:51:48.194 INFO:tasks.workunit.client.0.smithi090.stdout:6/966: fsync d4/d9b/ff7 0 2024-03-11T19:51:48.615 INFO:tasks.workunit.client.0.smithi090.stdout:3/872: dread d2/db/d15/d1a/d75/de8/d6a/dcb/fac [0,4194304] 0 2024-03-11T19:51:48.701 INFO:tasks.workunit.client.0.smithi090.stdout:9/921: rename d1/dd/d18/d25/d101/d1c/d21/d2a/da4/fa8 to d1/d47/d7c/dee/f13a 0 2024-03-11T19:51:48.912 INFO:tasks.workunit.client.0.smithi090.stdout:9/922: sync 2024-03-11T19:51:48.947 INFO:tasks.workunit.client.0.smithi090.stdout:5/923: creat d1/d26/f13b x:0 0 0 2024-03-11T19:51:48.977 INFO:tasks.workunit.client.0.smithi090.stdout:0/944: write d2/d11/d15/f12c [657815,107889] 0 2024-03-11T19:51:49.255 INFO:tasks.workunit.client.0.smithi090.stdout:2/906: creat d0/de/d12/d46/d4c/f119 x:0 0 0 2024-03-11T19:51:49.503 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:51:49.506 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:51:49.528 INFO:tasks.workunit.client.0.smithi090.stdout:6/967: mknod d4/d13/d22/d34/d40/dd5/d80/c143 0 2024-03-11T19:51:49.913 INFO:tasks.workunit.client.0.smithi090.stdout:3/873: readlink d2/db/d15/d1a/d75/de8/d6a/dcb/ld4 0 2024-03-11T19:51:50.070 INFO:tasks.workunit.client.0.smithi090.stdout:7/913: getdents d3/dc/d1e 0 2024-03-11T19:51:50.107 INFO:tasks.workunit.client.0.smithi090.stdout:5/924: creat d1/d26/d4d/f13c x:0 0 0 2024-03-11T19:51:50.664 INFO:tasks.workunit.client.0.smithi090.stdout:2/907: mkdir d0/de/d12/d81/df0/d11a 0 2024-03-11T19:51:50.719 INFO:tasks.workunit.client.0.smithi090.stdout:2/908: stat d0/de/d12/d46/d75/f7c 0 2024-03-11T19:51:51.098 INFO:tasks.workunit.client.0.smithi090.stdout:1/940: creat d5/da/f146 x:0 0 0 2024-03-11T19:51:51.281 INFO:tasks.workunit.client.0.smithi090.stdout:5/925: creat d1/d17/d39/d51/d53/d64/f13d x:0 0 0 2024-03-11T19:51:51.481 INFO:tasks.workunit.client.0.smithi090.stdout:7/914: dread - d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/db5/de1/dc2/dd0/ff1 zero size 2024-03-11T19:51:51.708 INFO:tasks.workunit.client.0.smithi090.stdout:5/926: dwrite d1/d17/d39/d51/d73/da6/f112 [0,4194304] 0 2024-03-11T19:51:52.275 INFO:tasks.workunit.client.0.smithi090.stdout:2/909: read d0/d21/dce/da2/fef [3574698,32222] 0 2024-03-11T19:51:52.509 INFO:tasks.workunit.client.0.smithi090.stdout:4/977: dwrite d4/d27/d28/d69/f99 [0,4194304] 0 2024-03-11T19:51:52.991 INFO:tasks.workunit.client.0.smithi090.stdout:9/923: dwrite d1/dd/d18/d25/d101/d1c/d21/d2a/f75 [0,4194304] 0 2024-03-11T19:51:53.147 INFO:tasks.workunit.client.0.smithi090.stdout:0/945: link d2/f1b d2/d11/d15/d29/de5/d11b/d6d/da4/f134 0 2024-03-11T19:51:53.249 INFO:tasks.workunit.client.0.smithi090.stdout:6/968: dwrite d4/d13/d22/d42/dee/f104 [0,4194304] 0 2024-03-11T19:51:53.485 INFO:tasks.workunit.client.0.smithi090.stdout:7/915: mknod d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/db5/de1/dc6/d103/c13d 0 2024-03-11T19:51:53.634 INFO:tasks.workunit.client.0.smithi090.stdout:5/927: read d1/d17/d39/d51/d73/da6/fbf [507451,61995] 0 2024-03-11T19:51:53.808 INFO:tasks.workunit.client.0.smithi090.stdout:0/946: dread d2/d11/d3f/fa9 [0,4194304] 0 2024-03-11T19:51:54.474 INFO:tasks.workunit.client.0.smithi090.stdout:8/994: write d0/d53/faa [252117,126546] 0 2024-03-11T19:51:54.828 INFO:tasks.workunit.client.0.smithi090.stdout:3/874: dwrite d2/d45/f70 [0,4194304] 0 2024-03-11T19:51:55.055 INFO:tasks.workunit.client.0.smithi090.stdout:6/969: creat d4/d13/d22/d34/d40/dd5/d80/dd4/f144 x:0 0 0 2024-03-11T19:51:55.232 INFO:tasks.workunit.client.0.smithi090.stdout:9/924: dread d1/dd/d18/d3a/d57/fe9 [0,4194304] 0 2024-03-11T19:51:55.388 INFO:tasks.workunit.client.0.smithi090.stdout:0/947: truncate d2/d11/f24 265668 0 2024-03-11T19:51:55.395 INFO:tasks.workunit.client.0.smithi090.stdout:5/928: stat d1/d17/d39/d51/c87 0 2024-03-11T19:51:55.432 INFO:tasks.workunit.client.0.smithi090.stdout:7/916: chown d3/dc/d1e/d7c/d96/d32/d7b/f128 469 1 2024-03-11T19:51:55.502 INFO:tasks.workunit.client.0.smithi090.stdout:0/948: stat d2/d11/d15/d29/d3d/d7e/dfc 0 2024-03-11T19:51:55.526 INFO:tasks.workunit.client.0.smithi090.stdout:4/978: mkdir d4/dd8/db5/d11b/d146 0 2024-03-11T19:51:55.594 INFO:tasks.workunit.client.0.smithi090.stdout:5/929: stat d1/d17/d39/d51/d53/d7b/d3b/de5 0 2024-03-11T19:51:55.595 INFO:tasks.workunit.client.0.smithi090.stdout:7/917: stat d3/dc/d1e/d7c/d96/d32/d45/cbf 0 2024-03-11T19:51:55.654 INFO:tasks.workunit.client.0.smithi090.stdout:5/930: chown d1/d75/c9b 155 1 2024-03-11T19:51:55.852 INFO:tasks.workunit.client.0.smithi090.stdout:1/941: creat d5/da/d23/dfe/d89/d10e/f147 x:0 0 0 2024-03-11T19:51:56.159 INFO:tasks.workunit.client.0.smithi090.stdout:3/875: creat d2/db/d15/d1a/d3c/f11a x:0 0 0 2024-03-11T19:51:56.398 INFO:tasks.workunit.client.0.smithi090.stdout:9/925: truncate d1/dd/d18/d25/d101/d1c/d21/dab/d114/d9c/f11b 4394026 0 2024-03-11T19:51:56.553 INFO:tasks.workunit.client.0.smithi090.stdout:0/949: creat d2/d11/d3a/d64/dd5/f135 x:0 0 0 2024-03-11T19:51:56.754 INFO:tasks.workunit.client.0.smithi090.stdout:7/918: unlink d3/dff/f125 0 2024-03-11T19:51:56.811 INFO:tasks.workunit.client.0.smithi090.stdout:7/919: chown d3/dc/d1e/d7c/d96/d32/da0 24 1 2024-03-11T19:51:56.991 INFO:tasks.workunit.client.0.smithi090.stdout:8/995: getdents d0/d3/d86/d127/d11f 0 2024-03-11T19:51:57.410 INFO:tasks.workunit.client.0.smithi090.stdout:2/910: write d0/de/d12/f78 [860393,104180] 0 2024-03-11T19:51:57.422 INFO:tasks.workunit.client.0.smithi090.stdout:9/926: dread - d1/dd/d18/d25/d101/d1c/d21/dab/d114/d9c/daf/ff5 zero size 2024-03-11T19:51:57.822 INFO:tasks.workunit.client.0.smithi090.stdout:4/979: mkdir d4/d27/d28/d7a/d104/d147 0 2024-03-11T19:51:57.902 INFO:tasks.workunit.client.0.smithi090.stdout:5/931: getdents d1/d17/d39/d51/d53/d7b/d3b/d76/d92/dc1/d133 0 2024-03-11T19:51:58.031 INFO:tasks.workunit.client.0.smithi090.stdout:7/920: read d3/d55/fb4 [1865820,92454] 0 2024-03-11T19:51:58.056 INFO:tasks.workunit.client.0.smithi090.stdout:7/921: chown d3/dc/d1e/f63 61980331 1 2024-03-11T19:51:58.445 INFO:tasks.workunit.client.0.smithi090.stdout:2/911: fdatasync d0/de/db6/d25/d8c/f105 0 2024-03-11T19:51:59.008 INFO:tasks.workunit.client.0.smithi090.stdout:6/970: write d4/d13/d22/d34/d40/dd5/d80/ddb/ddc/f102 [160311,36407] 0 2024-03-11T19:51:59.291 INFO:tasks.workunit.client.0.smithi090.stdout:1/942: link d5/d31/d76/c143 d5/d4e/d54/c148 0 2024-03-11T19:52:00.092 INFO:tasks.workunit.client.0.smithi090.stdout:2/912: unlink d0/de/db6/d25/l7b 0 2024-03-11T19:52:00.410 INFO:tasks.workunit.client.0.smithi090.stdout:3/876: dwrite d2/db/d15/d1a/d75/d8c/f117 [0,4194304] 0 2024-03-11T19:52:00.727 INFO:tasks.workunit.client.0.smithi090.stdout:1/943: rename d5/da/d23/l7a to d5/d4e/d120/l149 0 2024-03-11T19:52:01.020 INFO:tasks.workunit.client.0.smithi090.stdout:8/996: write d0/d43/dc/da1/f112 [1055830,21869] 0 2024-03-11T19:52:01.051 INFO:tasks.workunit.client.0.smithi090.stdout:0/950: dwrite d2/fcb [0,4194304] 0 2024-03-11T19:52:01.212 INFO:tasks.workunit.client.0.smithi090.stdout:0/951: chown d2/d11/d15/d29/d43/d58/l67 1 1 2024-03-11T19:52:01.255 INFO:tasks.workunit.client.0.smithi090.stdout:2/913: mkdir d0/de/d12/d46/d75/dc5/d11b 0 2024-03-11T19:52:01.749 INFO:tasks.workunit.client.0.smithi090.stdout:5/932: link d1/d17/d39/d51/d53/d7b/da0/db9/dc6/fd7 d1/d26/d4d/f13e 0 2024-03-11T19:52:01.827 INFO:tasks.workunit.client.0.smithi090.stdout:1/944: dread - d5/da/d1c/d3b/f137 zero size 2024-03-11T19:52:01.995 INFO:tasks.workunit.client.0.smithi090.stdout:8/997: fsync d0/d43/dc/fd 0 2024-03-11T19:52:02.071 INFO:tasks.workunit.client.0.smithi090.stdout:7/922: write d3/dc/d1e/d7c/d96/d32/d45/de0/f116 [2959982,124555] 0 2024-03-11T19:52:02.130 INFO:tasks.workunit.client.0.smithi090.stdout:5/933: chown d1/d26/db5/l138 7023732 1 2024-03-11T19:52:02.294 INFO:tasks.workunit.client.0.smithi090.stdout:0/952: readlink d2/d11/l1d 0 2024-03-11T19:52:02.490 INFO:tasks.workunit.client.0.smithi090.stdout:4/980: dwrite d4/d27/f114 [0,4194304] 0 2024-03-11T19:52:02.496 INFO:tasks.workunit.client.0.smithi090.stdout:9/927: dwrite d1/dd/d18/d25/f97 [0,4194304] 0 2024-03-11T19:52:02.829 INFO:tasks.workunit.client.0.smithi090.stdout:2/914: dread d0/de/db6/d25/fc7 [0,4194304] 0 2024-03-11T19:52:03.466 INFO:tasks.workunit.client.0.smithi090.stdout:6/971: getdents d4/d13/d22 0 2024-03-11T19:52:03.966 INFO:tasks.workunit.client.0.smithi090.stdout:4/981: read d4/dd8/db5/fdb [722229,89008] 0 2024-03-11T19:52:04.083 INFO:tasks.workunit.client.0.smithi090.stdout:0/953: truncate d2/d11/d15/f74 1552403 0 2024-03-11T19:52:04.307 INFO:tasks.workunit.client.0.smithi090.stdout:7/923: read d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/fcb [273048,95807] 0 2024-03-11T19:52:04.503 INFO:tasks.workunit.client.0.smithi090.stdout:8/998: truncate d0/d43/dc/d90/f9e 915094 0 2024-03-11T19:52:04.657 INFO:tasks.workunit.client.0.smithi090.stdout:9/928: mknod d1/dd/d18/d25/d101/d1c/d77/deb/c13b 0 2024-03-11T19:52:04.660 INFO:tasks.workunit.client.0.smithi090.stdout:8/999: readlink d0/d3/d98/lea 0 2024-03-11T19:52:04.679 INFO:tasks.workunit.client.0.smithi090.stdout:2/915: dread d0/f18 [0,4194304] 0 2024-03-11T19:52:04.706 INFO:tasks.workunit.client.0.smithi090.stdout:3/877: rename d2/d2f/dc4/lfb to d2/db/d15/d1a/d75/de8/l11b 0 2024-03-11T19:52:04.726 INFO:tasks.workunit.client.0.smithi090.stdout:9/929: chown d1/dd/d18/d3a/d57 390 1 2024-03-11T19:52:04.816 INFO:tasks.workunit.client.0.smithi090.stdout:3/878: chown d2/db/d15/d1a/d75/dc6 51 1 2024-03-11T19:52:04.890 INFO:tasks.workunit.client.0.smithi090.stdout:3/879: read - d2/db/d15/d1a/d75/de8/d6a/fdf zero size 2024-03-11T19:52:05.034 INFO:tasks.workunit.client.0.smithi090.stdout:1/945: truncate d5/da/d23/dfe/d89/d10e/d80/d13e/d122/f13f 8455367 0 2024-03-11T19:52:05.142 INFO:tasks.workunit.client.0.smithi090.stdout:4/982: mkdir d4/d51/d8c/d89/d148 0 2024-03-11T19:52:05.213 INFO:tasks.workunit.client.0.smithi090.stdout:0/954: rmdir d2/d11/d3a/d64/dac/dba/dec/dfd 39 2024-03-11T19:52:05.882 INFO:tasks.workunit.client.0.smithi090.stdout:6/972: rename d4/l13c to d4/d13/d22/d34/d40/dd5/l145 0 2024-03-11T19:52:06.261 INFO:tasks.workunit.client.0.smithi090.stdout:1/946: truncate d5/d31/d4c/f73 1527028 0 2024-03-11T19:52:06.276 INFO:tasks.workunit.client.0.smithi090.stdout:4/983: symlink d4/dd8/dfa/db7/l149 0 2024-03-11T19:52:06.681 INFO:tasks.workunit.client.0.smithi090.stdout:0/955: mknod d2/d11/d3a/d64/c136 0 2024-03-11T19:52:06.698 INFO:tasks.workunit.client.0.smithi090.stdout:5/934: rmdir d1/d17/d39/d51/d53/d7b/d3b/d76/db7/ddf/ded 0 2024-03-11T19:52:06.779 INFO:tasks.workunit.client.0.smithi090.stdout:7/924: mknod d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/c13e 0 2024-03-11T19:52:08.146 INFO:tasks.workunit.client.0.smithi090.stdout:7/925: rmdir d3/dc/d1e 39 2024-03-11T19:52:08.180 INFO:tasks.workunit.client.0.smithi090.stdout:7/926: chown d3/dc/d29/c12b 1 1 2024-03-11T19:52:08.316 INFO:tasks.workunit.client.0.smithi090.stdout:3/880: rename d2/db/d15/d1a/d75/d8c/cd3 to d2/db/d15/d69/dad/c11c 0 2024-03-11T19:52:08.364 INFO:tasks.workunit.client.0.smithi090.stdout:3/881: readlink d2/db/d15/d1a/d75/de8/d6a/dcb/lee 0 2024-03-11T19:52:08.428 INFO:tasks.workunit.client.0.smithi090.stdout:2/916: link d0/d21/c43 d0/de/d1f/c11c 0 2024-03-11T19:52:08.787 INFO:tasks.workunit.client.0.smithi090.stdout:0/956: mknod d2/d11/d3a/d64/dac/dba/dec/dfd/c137 0 2024-03-11T19:52:08.813 INFO:tasks.workunit.client.0.smithi090.stdout:0/957: stat d2/d11/d15/d29/cb5 0 2024-03-11T19:52:08.853 INFO:tasks.workunit.client.0.smithi090.stdout:3/882: sync 2024-03-11T19:52:09.413 INFO:tasks.workunit.client.0.smithi090.stdout:5/935: getdents d1/d17/d39/d51/d53/d7b/d3b/de5 0 2024-03-11T19:52:09.528 INFO:tasks.workunit.client.0.smithi090.stdout:2/917: dread - d0/de/d12/d46/d4c/f111 zero size 2024-03-11T19:52:09.661 INFO:tasks.workunit.client.0.smithi090.stdout:9/930: getdents d1/d47/d7c 0 2024-03-11T19:52:09.786 INFO:tasks.workunit.client.0.smithi090.stdout:4/984: creat d4/d27/d28/f14a x:0 0 0 2024-03-11T19:52:10.012 INFO:tasks.workunit.client.0.smithi090.stdout:6/973: write d4/d9b/fea [578403,76670] 0 2024-03-11T19:52:10.130 INFO:tasks.workunit.client.0.smithi090.stdout:3/883: dread - d2/db/d15/d1a/d75/de8/ff0 zero size 2024-03-11T19:52:10.347 INFO:tasks.workunit.client.0.smithi090.stdout:7/927: stat d3/dc/d1e/d7c/d96/d32/c38 0 2024-03-11T19:52:10.510 INFO:tasks.workunit.client.0.smithi090.stdout:5/936: creat d1/d17/d39/d51/d53/d7b/d3b/d76/db7/ddf/dee/f13f x:0 0 0 2024-03-11T19:52:10.674 INFO:tasks.workunit.client.0.smithi090.stdout:9/931: dread - d1/dd/d18/d25/d101/d1c/d9f/f116 zero size 2024-03-11T19:52:10.839 INFO:tasks.workunit.client.0.smithi090.stdout:4/985: dread - d4/dd8/f13b zero size 2024-03-11T19:52:10.961 INFO:tasks.workunit.client.0.smithi090.stdout:1/947: dwrite d5/d31/d76/f142 [0,4194304] 0 2024-03-11T19:52:11.091 INFO:tasks.workunit.client.0.smithi090.stdout:6/974: rename d4/d13/d22/d42/d48/f125 to d4/d13d/f146 0 2024-03-11T19:52:11.169 INFO:tasks.workunit.client.0.smithi090.stdout:6/975: readlink d4/d13/d22/l4c 0 2024-03-11T19:52:11.420 INFO:tasks.workunit.client.0.smithi090.stdout:7/928: fdatasync d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/ffd 0 2024-03-11T19:52:11.565 INFO:tasks.workunit.client.0.smithi090.stdout:7/929: sync 2024-03-11T19:52:11.609 INFO:tasks.workunit.client.0.smithi090.stdout:2/918: mkdir d0/de/db6/d83/d11d 0 2024-03-11T19:52:12.395 INFO:tasks.workunit.client.0.smithi090.stdout:1/948: creat d5/da/d97/df6/f14a x:0 0 0 2024-03-11T19:52:12.541 INFO:tasks.workunit.client.0.smithi090.stdout:9/932: fsync d1/dd/d18/d25/d101/d1c/d21/d2a/dc2/d100/d120/fde 0 2024-03-11T19:52:12.574 INFO:tasks.workunit.client.0.smithi090.stdout:6/976: mkdir d4/d13/d22/d34/d37/d7f/d90/d147 0 2024-03-11T19:52:12.595 INFO:tasks.workunit.client.0.smithi090.stdout:0/958: dwrite d2/d11/d15/d29/fc1 [0,4194304] 0 2024-03-11T19:52:12.630 INFO:tasks.workunit.client.0.smithi090.stdout:2/919: mknod d0/d21/c11e 0 2024-03-11T19:52:12.782 INFO:tasks.workunit.client.0.smithi090.stdout:3/884: dwrite d2/db/d15/d1a/d75/de8/d6a/fa0 [4194304,4194304] 0 2024-03-11T19:52:13.011 INFO:tasks.workunit.client.0.smithi090.stdout:4/986: rename d4/d18/fe8 to d4/d27/d28/d69/d111/f14b 0 2024-03-11T19:52:13.073 INFO:tasks.workunit.client.0.smithi090.stdout:4/987: chown d4/d27/d28/d38/d64/d119/l132 49691 1 2024-03-11T19:52:13.080 INFO:tasks.workunit.client.0.smithi090.stdout:4/988: stat d4/f12 0 2024-03-11T19:52:13.314 INFO:tasks.workunit.client.0.smithi090.stdout:2/920: dread - d0/de/dd3/f114 zero size 2024-03-11T19:52:14.272 INFO:tasks.workunit.client.0.smithi090.stdout:4/989: truncate d4/d51/d8c/d89/f98 2007165 0 2024-03-11T19:52:14.498 INFO:tasks.workunit.client.0.smithi090.stdout:7/930: creat d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/f13f x:0 0 0 2024-03-11T19:52:14.600 INFO:tasks.workunit.client.0.smithi090.stdout:5/937: link d1/d17/d39/df2/df9/ccd d1/d26/c140 0 2024-03-11T19:52:14.640 INFO:tasks.workunit.client.0.smithi090.stdout:7/931: sync 2024-03-11T19:52:14.654 INFO:tasks.workunit.client.0.smithi090.stdout:7/932: stat d3/dc/l9f 0 2024-03-11T19:52:15.324 INFO:tasks.workunit.client.0.smithi090.stdout:7/933: mknod d3/dc/d1e/d7c/d96/d32/d67/dee/c140 0 2024-03-11T19:52:15.344 INFO:tasks.workunit.client.0.smithi090.stdout:5/938: chown d1/d26/f3f 0 1 2024-03-11T19:52:15.348 INFO:tasks.workunit.client.0.smithi090.stdout:7/934: readlink d3/d55/lc9 0 2024-03-11T19:52:15.603 INFO:tasks.workunit.client.0.smithi090.stdout:2/921: symlink d0/de/d12/d46/l11f 0 2024-03-11T19:52:15.614 INFO:tasks.workunit.client.0.smithi090.stdout:9/933: write d1/dd/f128 [374136,123988] 0 2024-03-11T19:52:15.702 INFO:tasks.workunit.client.0.smithi090.stdout:9/934: readlink d1/dd/d18/d3a/da2/lf9 0 2024-03-11T19:52:15.751 INFO:tasks.workunit.client.0.smithi090.stdout:9/935: chown d1/dd/d18/d25/d101/d1c/d21/d2a/ced 17235656 1 2024-03-11T19:52:15.953 INFO:tasks.workunit.client.0.smithi090.stdout:1/949: getdents d5/df5/d129 0 2024-03-11T19:52:15.979 INFO:tasks.workunit.client.0.smithi090.stdout:6/977: write d4/d14/d107/d13f/f65 [8549313,106764] 0 2024-03-11T19:52:15.987 INFO:tasks.workunit.client.0.smithi090.stdout:5/939: fsync d1/d17/d39/d51/d53/d7b/d3b/d76/d92/fa2 0 2024-03-11T19:52:16.101 INFO:tasks.workunit.client.0.smithi090.stdout:7/935: creat d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/f141 x:0 0 0 2024-03-11T19:52:16.107 INFO:tasks.workunit.client.0.smithi090.stdout:6/978: sync 2024-03-11T19:52:16.211 INFO:tasks.workunit.client.0.smithi090.stdout:2/922: creat d0/de/d12/d81/d93/f120 x:0 0 0 2024-03-11T19:52:16.215 INFO:tasks.workunit.client.0.smithi090.stdout:3/885: write d2/db/d15/d1a/d75/d8c/ff4 [850000,62162] 0 2024-03-11T19:52:16.246 INFO:tasks.workunit.client.0.smithi090.stdout:4/990: link d4/dd8/f7e d4/dd8/db5/d11b/f14c 0 2024-03-11T19:52:16.516 INFO:tasks.workunit.client.0.smithi090.stdout:9/936: dread d1/dd/d18/d25/d101/f10 [0,4194304] 0 2024-03-11T19:52:16.544 INFO:tasks.workunit.client.0.smithi090.stdout:0/959: write d2/d11/d15/d29/de5/d11b/db4/daf/fb3 [73196,112421] 0 2024-03-11T19:52:17.132 INFO:tasks.workunit.client.0.smithi090.stdout:5/940: symlink d1/d26/d4d/l141 0 2024-03-11T19:52:17.244 INFO:tasks.workunit.client.0.smithi090.stdout:6/979: mknod d4/d13/d22/d34/d40/dd5/dc1/c148 0 2024-03-11T19:52:17.278 INFO:tasks.workunit.client.0.smithi090.stdout:6/980: chown d4/d13/d22/d34/d37/d7f/c117 232560160 1 2024-03-11T19:52:17.317 INFO:tasks.workunit.client.0.smithi090.stdout:2/923: mkdir d0/de/d12/d81/d121 0 2024-03-11T19:52:17.387 INFO:tasks.workunit.client.0.smithi090.stdout:4/991: truncate d4/dd8/d66/dcf/dd7/f107 174717 0 2024-03-11T19:52:17.585 INFO:tasks.workunit.client.0.smithi090.stdout:9/937: creat d1/dd/d18/d25/d101/d1c/d21/d2a/dc2/d100/d120/f13c x:0 0 0 2024-03-11T19:52:17.612 INFO:tasks.workunit.client.0.smithi090.stdout:9/938: stat d1/d47/d7c/dee/l131 0 2024-03-11T19:52:18.184 INFO:tasks.workunit.client.0.smithi090.stdout:5/941: mknod d1/d17/d39/df2/df9/c142 0 2024-03-11T19:52:18.630 INFO:tasks.workunit.client.0.smithi090.stdout:2/924: creat d0/de/d12/d46/d75/dc5/de5/f122 x:0 0 0 2024-03-11T19:52:18.642 INFO:tasks.workunit.client.0.smithi090.stdout:6/981: dread - d4/d13/d22/d42/d48/f119 zero size 2024-03-11T19:52:18.776 INFO:tasks.workunit.client.0.smithi090.stdout:6/982: chown d4/d13/d22/d42/d48/d4e/d136/ce4 714073 1 2024-03-11T19:52:18.873 INFO:tasks.workunit.client.0.smithi090.stdout:0/960: mkdir d2/d11/d15/d29/d138 0 2024-03-11T19:52:19.097 INFO:tasks.workunit.client.0.smithi090.stdout:9/939: creat d1/dd/d18/d3a/d70/f13d x:0 0 0 2024-03-11T19:52:19.165 INFO:tasks.workunit.client.0.smithi090.stdout:2/925: dwrite d0/de/d12/d46/d75/dc5/de5/f122 [0,4194304] 0 2024-03-11T19:52:19.347 INFO:tasks.workunit.client.0.smithi090.stdout:2/926: chown d0/de/db6/d25/c4a 1 1 2024-03-11T19:52:19.474 INFO:tasks.workunit.client.0.smithi090.stdout:5/942: truncate d1/d26/f54 7658150 0 2024-03-11T19:52:19.578 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:52:19.582 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:52:19.605 INFO:tasks.workunit.client.0.smithi090.stdout:3/886: rename d2/db/d15/d1a/d75/f80 to d2/db/d15/d1a/d75/f11d 0 2024-03-11T19:52:19.632 INFO:tasks.workunit.client.0.smithi090.stdout:3/887: chown d2/db/d15/d1a/d75/d8c 228561 1 2024-03-11T19:52:20.012 INFO:tasks.workunit.client.0.smithi090.stdout:0/961: dread d2/d11/d15/d29/d3d/d7e/f9d [0,4194304] 0 2024-03-11T19:52:20.290 INFO:tasks.workunit.client.0.smithi090.stdout:2/927: mkdir d0/de/d12/d46/de6/d123 0 2024-03-11T19:52:20.536 INFO:tasks.workunit.client.0.smithi090.stdout:7/936: write d3/f7 [5517203,74939] 0 2024-03-11T19:52:20.617 INFO:tasks.workunit.client.0.smithi090.stdout:3/888: truncate d2/f12 1805576 0 2024-03-11T19:52:20.668 INFO:tasks.workunit.client.0.smithi090.stdout:1/950: dwrite d5/da/f93 [4194304,4194304] 0 2024-03-11T19:52:20.791 INFO:tasks.workunit.client.0.smithi090.stdout:6/983: mknod d4/d13/d22/d34/d37/c149 0 2024-03-11T19:52:21.214 INFO:tasks.workunit.client.0.smithi090.stdout:5/943: mkdir d1/d17/d39/df2/d12b/d143 0 2024-03-11T19:52:21.384 INFO:tasks.workunit.client.0.smithi090.stdout:4/992: write d4/dd8/db5/feb [4872559,72612] 0 2024-03-11T19:52:21.435 INFO:tasks.workunit.client.0.smithi090.stdout:3/889: rename d2/de/c4e to d2/d2f/c11e 0 2024-03-11T19:52:21.725 INFO:tasks.workunit.client.0.smithi090.stdout:1/951: mkdir d5/d31/d4c/daf/dfc/d14b 0 2024-03-11T19:52:22.354 INFO:tasks.workunit.client.0.smithi090.stdout:5/944: readlink d1/d17/d39/d51/d53/d7b/d3b/d76/db7/ddf/l116 0 2024-03-11T19:52:22.372 INFO:tasks.workunit.client.0.smithi090.stdout:5/945: chown d1/d17/d131 58 1 2024-03-11T19:52:22.905 INFO:tasks.workunit.client.0.smithi090.stdout:3/890: dread d2/d2f/f9c [0,4194304] 0 2024-03-11T19:52:23.033 INFO:tasks.workunit.client.0.smithi090.stdout:1/952: stat d5/d31/d76/d77/dbd/ld2 0 2024-03-11T19:52:23.331 INFO:tasks.workunit.client.0.smithi090.stdout:9/940: dwrite d1/dd/d18/d3a/d48/fbc [4194304,4194304] 0 2024-03-11T19:52:23.516 INFO:tasks.workunit.client.0.smithi090.stdout:7/937: mkdir d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/db5/de1/dc2/dd0/d136/d142 0 2024-03-11T19:52:24.045 INFO:tasks.workunit.client.0.smithi090.stdout:0/962: dwrite d2/d11/d15/d29/d43/d58/f91 [0,4194304] 0 2024-03-11T19:52:24.406 INFO:tasks.workunit.client.0.smithi090.stdout:3/891: mknod d2/db/d15/d69/c11f 0 2024-03-11T19:52:24.421 INFO:tasks.workunit.client.0.smithi090.stdout:1/953: mknod d5/da/d23/dfe/d89/c14c 0 2024-03-11T19:52:24.679 INFO:tasks.workunit.client.0.smithi090.stdout:2/928: dwrite d0/de/f16 [0,4194304] 0 2024-03-11T19:52:24.835 INFO:tasks.workunit.client.0.smithi090.stdout:7/938: symlink d3/de6/l143 0 2024-03-11T19:52:25.086 INFO:tasks.workunit.client.0.smithi090.stdout:5/946: mknod d1/d17/d39/d51/d53/d7b/d3b/de5/c144 0 2024-03-11T19:52:25.142 INFO:tasks.workunit.client.0.smithi090.stdout:0/963: symlink d2/d94/d111/l139 0 2024-03-11T19:52:25.168 INFO:tasks.workunit.client.0.smithi090.stdout:0/964: chown d2/d11/d15/d29/fa5 19 1 2024-03-11T19:52:25.492 INFO:tasks.workunit.client.0.smithi090.stdout:6/984: link d4/d13/d22/c11a d4/d13/d22/d42/d48/dd2/dfa/c14a 0 2024-03-11T19:52:25.830 INFO:tasks.workunit.client.0.smithi090.stdout:4/993: dwrite d4/dd8/dfa/db7/fd5 [0,4194304] 0 2024-03-11T19:52:25.962 INFO:tasks.workunit.client.0.smithi090.stdout:1/954: fsync d5/da/d23/f2d 0 2024-03-11T19:52:26.012 INFO:tasks.workunit.client.0.smithi090.stdout:4/994: chown d4/d51/l56 78299407 1 2024-03-11T19:52:26.758 INFO:tasks.workunit.client.0.smithi090.stdout:6/985: creat d4/d13/d22/d34/d40/dd5/d80/ddb/ddc/f14b x:0 0 0 2024-03-11T19:52:26.810 INFO:tasks.workunit.client.0.smithi090.stdout:6/986: chown d4/d13/d22/d42/d48/d4e/d136/d126/f13e 203 1 2024-03-11T19:52:26.958 INFO:tasks.workunit.client.0.smithi090.stdout:3/892: symlink d2/db/l120 0 2024-03-11T19:52:27.342 INFO:tasks.workunit.client.0.smithi090.stdout:1/955: mkdir d5/da/d106/d14d 0 2024-03-11T19:52:27.927 INFO:tasks.workunit.client.0.smithi090.stdout:3/893: fsync d2/db/f84 0 2024-03-11T19:52:27.969 INFO:tasks.workunit.client.0.smithi090.stdout:9/941: write d1/dd/d18/d3a/d57/ff1 [50526,128421] 0 2024-03-11T19:52:28.051 INFO:tasks.workunit.client.0.smithi090.stdout:3/894: chown d2/db/d15/d69/da2/dc7/fe3 3740583 1 2024-03-11T19:52:28.230 INFO:tasks.workunit.client.0.smithi090.stdout:4/995: creat d4/d27/d28/d7a/d104/d147/f14d x:0 0 0 2024-03-11T19:52:28.382 INFO:tasks.workunit.client.0.smithi090.stdout:1/956: dread - d5/da/d1c/d3b/d8c/f127 zero size 2024-03-11T19:52:28.668 INFO:tasks.workunit.client.0.smithi090.stdout:5/947: creat d1/d17/d39/d51/d53/d7b/f145 x:0 0 0 2024-03-11T19:52:28.834 INFO:tasks.workunit.client.0.smithi090.stdout:9/942: mkdir d1/dd/d18/d25/d101/d1c/d77/deb/d13e 0 2024-03-11T19:52:29.201 INFO:tasks.workunit.client.0.smithi090.stdout:3/895: symlink d2/db/d15/d69/dad/l121 0 2024-03-11T19:52:29.228 INFO:tasks.workunit.client.0.smithi090.stdout:3/896: chown d2/d2f/l54 96222128 1 2024-03-11T19:52:29.266 INFO:tasks.workunit.client.0.smithi090.stdout:4/996: mknod d4/dd8/db5/c14e 0 2024-03-11T19:52:29.318 INFO:tasks.workunit.client.0.smithi090.stdout:7/939: link d3/dc/d1e/d7c/d96/d32/d67/l130 d3/dc/da6/d120/l144 0 2024-03-11T19:52:29.723 INFO:tasks.workunit.client.0.smithi090.stdout:9/943: truncate d1/dd/d18/d25/d101/d1c/d21/dab/f10b 551349 0 2024-03-11T19:52:29.744 INFO:tasks.workunit.client.0.smithi090.stdout:3/897: dread - d2/db/f109 zero size 2024-03-11T19:52:30.137 INFO:tasks.workunit.client.0.smithi090.stdout:2/929: write d0/de/db6/d29/f2e [418117,21092] 0 2024-03-11T19:52:30.561 INFO:tasks.workunit.client.0.smithi090.stdout:5/948: mkdir d1/d17/d39/d51/d53/d7b/d6f/d146 0 2024-03-11T19:52:30.629 INFO:tasks.workunit.client.0.smithi090.stdout:4/997: fdatasync d4/dd8/db5/d11b/f14c 0 2024-03-11T19:52:30.746 INFO:tasks.workunit.client.0.smithi090.stdout:9/944: creat d1/dd/d18/d3a/dd8/f13f x:0 0 0 2024-03-11T19:52:31.225 INFO:tasks.workunit.client.0.smithi090.stdout:0/965: rename d2/cb to d2/d94/d111/c13a 0 2024-03-11T19:52:31.524 INFO:tasks.workunit.client.0.smithi090.stdout:2/930: unlink d0/de/d12/d46/d4c/d71/f8a 0 2024-03-11T19:52:31.578 INFO:tasks.workunit.client.0.smithi090.stdout:2/931: readlink d0/de/db6/d25/d8c/lbd 0 2024-03-11T19:52:31.707 INFO:tasks.workunit.client.0.smithi090.stdout:6/987: rename d4/d13/d22/d34/d40/dd5/d80/dd4/d128/f139 to d4/d13/d22/d34/d40/dd5/d80/ddb/ddc/f14c 0 2024-03-11T19:52:32.140 INFO:tasks.workunit.client.0.smithi090.stdout:2/932: chown d0/de/f68 10329 1 2024-03-11T19:52:32.345 INFO:tasks.workunit.client.0.smithi090.stdout:3/898: rename d2/db/d15/d69/da2/dc7/l115 to d2/db/d15/d69/dad/de9/l122 0 2024-03-11T19:52:32.377 INFO:tasks.workunit.client.0.smithi090.stdout:3/899: chown d2/db/ffe 36 1 2024-03-11T19:52:32.410 INFO:tasks.workunit.client.0.smithi090.stdout:3/900: chown d2/db/d15/d1a/d3c/c79 29536270 1 2024-03-11T19:52:32.859 INFO:tasks.workunit.client.0.smithi090.stdout:1/957: write d5/da/dd/f1f [5217070,41481] 0 2024-03-11T19:52:33.037 INFO:tasks.workunit.client.0.smithi090.stdout:3/901: truncate d2/f5a 3207937 0 2024-03-11T19:52:33.054 INFO:tasks.workunit.client.0.smithi090.stdout:4/998: rename d4/ff to d4/d27/d28/d7a/f14f 0 2024-03-11T19:52:33.242 INFO:tasks.workunit.client.0.smithi090.stdout:0/966: creat d2/d11/d3a/d64/dac/dba/dec/f13b x:0 0 0 2024-03-11T19:52:33.727 INFO:tasks.workunit.client.0.smithi090.stdout:9/945: rename d1/dd/d18/d25/d101/d1c/d21/d2a/fba to d1/dd/d18/d3a/dd8/f140 0 2024-03-11T19:52:33.772 INFO:tasks.workunit.client.0.smithi090.stdout:9/946: chown d1/dd/d18/d3a/d57 7092 1 2024-03-11T19:52:33.916 INFO:tasks.workunit.client.0.smithi090.stdout:6/988: creat d4/d14/d8a/f14d x:0 0 0 2024-03-11T19:52:34.076 INFO:tasks.workunit.client.0.smithi090.stdout:7/940: write d3/dc/d1e/d7c/d96/d32/d45/de0/f116 [4564656,37093] 0 2024-03-11T19:52:34.359 INFO:tasks.workunit.client.0.smithi090.stdout:0/967: truncate d2/d11/d15/d29/d3d/d8c/f120 54847 0 2024-03-11T19:52:34.395 INFO:tasks.workunit.client.0.smithi090.stdout:0/968: stat d2/d11/d15/d29/de5/d11b/d6d/da4 0 2024-03-11T19:52:34.973 INFO:tasks.workunit.client.0.smithi090.stdout:9/947: creat d1/dd/d18/d25/d101/d1c/d21/dab/d114/f141 x:0 0 0 2024-03-11T19:52:35.008 INFO:tasks.workunit.client.0.smithi090.stdout:9/948: stat d1/dd/d18/d3a/dd8/f11f 0 2024-03-11T19:52:35.034 INFO:tasks.workunit.client.0.smithi090.stdout:6/989: creat d4/d13/d22/d34/d40/dd5/dc1/f14e x:0 0 0 2024-03-11T19:52:35.644 INFO:tasks.workunit.client.0.smithi090.stdout:5/949: dwrite d1/d17/d39/d51/d53/d7b/d3b/d76/f77 [0,4194304] 0 2024-03-11T19:52:35.883 INFO:tasks.workunit.client.0.smithi090.stdout:4/999: mknod d4/d10e/d113/c150 0 2024-03-11T19:52:36.001 INFO:tasks.workunit.client.0.smithi090.stdout:2/933: write d0/de/db6/f47 [482832,82122] 0 2024-03-11T19:52:36.070 INFO:tasks.workunit.client.0.smithi090.stdout:6/990: dread d4/d13/d22/d34/d37/d7f/d90/fad [0,4194304] 0 2024-03-11T19:52:36.108 INFO:tasks.workunit.client.0.smithi090.stdout:7/941: dread d3/dc/d1e/d7c/d96/d32/d71/fdf [0,4194304] 0 2024-03-11T19:52:36.209 INFO:tasks.workunit.client.0.smithi090.stdout:6/991: readlink d4/d14/d107/d13f/dd7/la3 0 2024-03-11T19:52:36.492 INFO:tasks.workunit.client.0.smithi090.stdout:9/949: fdatasync d1/dd/d18/d3a/fbe 0 2024-03-11T19:52:36.948 INFO:tasks.workunit.client.0.smithi090.stdout:7/942: truncate d3/dc/d87/f90 145769 0 2024-03-11T19:52:36.980 INFO:tasks.workunit.client.0.smithi090.stdout:7/943: readlink d3/dc/d29/la2 0 2024-03-11T19:52:37.310 INFO:tasks.workunit.client.0.smithi090.stdout:3/902: rename d2/de to d2/db/d15/d69/d119/d123 0 2024-03-11T19:52:37.483 INFO:tasks.workunit.client.0.smithi090.stdout:1/958: dwrite d5/d31/d36/f70 [0,4194304] 0 2024-03-11T19:52:37.684 INFO:tasks.workunit.client.0.smithi090.stdout:2/934: mkdir d0/de/d12/d46/d124 0 2024-03-11T19:52:37.889 INFO:tasks.workunit.client.0.smithi090.stdout:9/950: symlink d1/dd/d18/l142 0 2024-03-11T19:52:38.130 INFO:tasks.workunit.client.0.smithi090.stdout:3/903: fsync d2/db/ffe 0 2024-03-11T19:52:38.245 INFO:tasks.workunit.client.0.smithi090.stdout:1/959: creat d5/da/d97/f14e x:0 0 0 2024-03-11T19:52:38.312 INFO:tasks.workunit.client.0.smithi090.stdout:0/969: write d2/d11/d15/d29/de5/d11b/d6d/da4/f134 [5374000,4969] 0 2024-03-11T19:52:38.684 INFO:tasks.workunit.client.0.smithi090.stdout:2/935: dread d0/de/d1f/f40 [0,4194304] 0 2024-03-11T19:52:38.718 INFO:tasks.workunit.client.0.smithi090.stdout:2/936: chown d0/de/d12/c110 20 1 2024-03-11T19:52:39.280 INFO:tasks.workunit.client.0.smithi090.stdout:5/950: write d1/d17/d39/d51/d53/d7b/d6f/f8e [4205684,42817] 0 2024-03-11T19:52:39.291 INFO:tasks.workunit.client.0.smithi090.stdout:6/992: write d4/f62 [6361069,106942] 0 2024-03-11T19:52:39.432 INFO:tasks.workunit.client.0.smithi090.stdout:3/904: truncate d2/db/d15/d1a/d75/de8/d6a/fb5 3130451 0 2024-03-11T19:52:39.492 INFO:tasks.workunit.client.0.smithi090.stdout:1/960: dread - d5/da/d23/dfe/d89/d10e/f147 zero size 2024-03-11T19:52:39.687 INFO:tasks.workunit.client.0.smithi090.stdout:0/970: dread - d2/d11/d15/d29/d3d/d7b/de6/f11f zero size 2024-03-11T19:52:40.005 INFO:tasks.workunit.client.0.smithi090.stdout:2/937: creat d0/de/dd3/f125 x:0 0 0 2024-03-11T19:52:40.134 INFO:tasks.workunit.client.0.smithi090.stdout:7/944: creat d3/dc/d1e/d7c/d96/d32/d7b/f145 x:0 0 0 2024-03-11T19:52:40.444 INFO:tasks.workunit.client.0.smithi090.stdout:5/951: creat d1/d17/d39/d51/d9a/f147 x:0 0 0 2024-03-11T19:52:40.477 INFO:tasks.workunit.client.0.smithi090.stdout:6/993: read d4/d14/d107/d13f/f65 [7511847,124431] 0 2024-03-11T19:52:40.596 INFO:tasks.workunit.client.0.smithi090.stdout:1/961: stat d5/d31/d4c/f91 0 2024-03-11T19:52:40.924 INFO:tasks.workunit.client.0.smithi090.stdout:0/971: creat d2/d11/d3a/da6/f13c x:0 0 0 2024-03-11T19:52:40.988 INFO:tasks.workunit.client.0.smithi090.stdout:1/962: dread d5/da/f4a [0,4194304] 0 2024-03-11T19:52:41.053 INFO:tasks.workunit.client.0.smithi090.stdout:7/945: rmdir d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/db5/de1/dc6 39 2024-03-11T19:52:41.120 INFO:tasks.workunit.client.0.smithi090.stdout:7/946: chown d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/db5/cc3 10328 1 2024-03-11T19:52:41.160 INFO:tasks.workunit.client.0.smithi090.stdout:7/947: read - d3/dc/d1e/d7c/d96/fb1 zero size 2024-03-11T19:52:41.317 INFO:tasks.workunit.client.0.smithi090.stdout:3/905: mknod d2/db/d15/d1a/d75/dc6/d114/c124 0 2024-03-11T19:52:41.548 INFO:tasks.workunit.client.0.smithi090.stdout:5/952: symlink d1/d17/d39/d51/d53/d7b/d3b/d76/d92/dc1/dff/d110/l148 0 2024-03-11T19:52:42.058 INFO:tasks.workunit.client.0.smithi090.stdout:2/938: symlink d0/de/d12/d46/d75/dc5/d11b/l126 0 2024-03-11T19:52:42.169 INFO:tasks.workunit.client.0.smithi090.stdout:0/972: creat d2/d11/d15/d29/de5/f13d x:0 0 0 2024-03-11T19:52:42.215 INFO:tasks.workunit.client.0.smithi090.stdout:0/973: chown d2/d11/d3a/d64/dac/dba/dec 14572021 1 2024-03-11T19:52:42.405 INFO:tasks.workunit.client.0.smithi090.stdout:1/963: chown d5/da/d23/f2b 7 1 2024-03-11T19:52:42.642 INFO:tasks.workunit.client.0.smithi090.stdout:3/906: rename d2/db/d15/d69/dad/fbf to d2/db/d15/d69/da2/dc7/f125 0 2024-03-11T19:52:42.696 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:18:02.455 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:52:42.710 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:18:02.469 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:52:42.721 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:18:02.417 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:52:42.735 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:18:02.431 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:52:42.736 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:18:02.424 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:52:42.752 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:18:02.440 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:52:42.842 INFO:tasks.workunit.client.0.smithi090.stdout:5/953: symlink d1/d17/d39/d51/d73/l149 0 2024-03-11T19:52:42.871 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:18:02.567 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:52:42.881 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:18:02.569 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:52:42.929 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:18:02.688 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:52:43.192 INFO:tasks.workunit.client.0.smithi090.stdout:0/974: creat d2/d11/df4/d11e/f13e x:0 0 0 2024-03-11T19:52:44.054 INFO:tasks.workunit.client.0.smithi090.stdout:0/975: unlink d2/d11/d15/d29/d3d/d7e/f9d 0 2024-03-11T19:52:44.111 INFO:tasks.workunit.client.0.smithi090.stdout:9/951: dwrite d1/dd/d18/d3a/f112 [0,4194304] 0 2024-03-11T19:52:44.186 INFO:tasks.workunit.client.0.smithi090.stdout:3/907: truncate d2/d2f/f32 1283766 0 2024-03-11T19:52:44.238 INFO:tasks.workunit.client.0.smithi090.stdout:9/952: readlink d1/dd/d18/d3a/d48/l130 0 2024-03-11T19:52:44.372 INFO:tasks.workunit.client.0.smithi090.stdout:9/953: dread d1/dd/d18/d25/d101/d1c/d21/f9d [0,4194304] 0 2024-03-11T19:52:44.890 INFO:tasks.workunit.client.0.smithi090.stdout:3/908: creat d2/db/d15/d69/da2/dc7/f126 x:0 0 0 2024-03-11T19:52:44.964 INFO:tasks.workunit.client.0.smithi090.stdout:9/954: dread d1/dd/f128 [0,4194304] 0 2024-03-11T19:52:45.075 INFO:tasks.workunit.client.0.smithi090.stdout:7/948: mknod d3/dc/d1e/d7c/d96/d32/c146 0 2024-03-11T19:52:45.391 INFO:tasks.workunit.client.0.smithi090.stdout:6/994: dwrite d4/f101 [0,4194304] 0 2024-03-11T19:52:45.557 INFO:tasks.workunit.client.0.smithi090.stdout:3/909: mkdir d2/db/d15/d49/d127 0 2024-03-11T19:52:45.567 INFO:tasks.workunit.client.0.smithi090.stdout:7/949: chown d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/db5/de1/dc2/d8e 25183 1 2024-03-11T19:52:45.694 INFO:tasks.workunit.client.0.smithi090.stdout:9/955: dread - d1/dd/d18/d3a/d70/f13d zero size 2024-03-11T19:52:46.140 INFO:tasks.workunit.client.0.smithi090.stdout:6/995: creat d4/d14/d107/f14f x:0 0 0 2024-03-11T19:52:46.193 INFO:tasks.workunit.client.0.smithi090.stdout:3/910: read d2/f12 [1225500,128136] 0 2024-03-11T19:52:46.377 INFO:tasks.workunit.client.0.smithi090.stdout:2/939: write d0/de/dd3/f102 [629649,88909] 0 2024-03-11T19:52:46.588 INFO:tasks.workunit.client.0.smithi090.stdout:2/940: chown d0/de/d12/d46 218939 1 2024-03-11T19:52:46.613 INFO:tasks.workunit.client.0.smithi090.stdout:7/950: dread d3/dc/d1e/d7c/d96/d32/d45/de0/f35 [0,4194304] 0 2024-03-11T19:52:46.648 INFO:tasks.workunit.client.0.smithi090.stdout:1/964: write d5/da/d23/f2d [4948168,95213] 0 2024-03-11T19:52:46.909 INFO:tasks.workunit.client.0.smithi090.stdout:6/996: stat d4/d13/d22/d42/f72 0 2024-03-11T19:52:47.092 INFO:tasks.workunit.client.0.smithi090.stdout:3/911: rename d2/d45/f70 to d2/db/d15/d69/d100/f128 0 2024-03-11T19:52:47.262 INFO:tasks.workunit.client.0.smithi090.stdout:2/941: read - d0/de/db6/d29/f116 zero size 2024-03-11T19:52:47.335 INFO:tasks.workunit.client.0.smithi090.stdout:1/965: write d5/da/d97/f14e [51592,12142] 0 2024-03-11T19:52:47.549 INFO:tasks.workunit.client.0.smithi090.stdout:7/951: fsync d3/dc/d1e/d7c/d96/d32/d45/de0/f4c 0 2024-03-11T19:52:47.904 INFO:tasks.workunit.client.0.smithi090.stdout:2/942: truncate d0/de/d12/d46/f98 116147 0 2024-03-11T19:52:48.039 INFO:tasks.workunit.client.0.smithi090.stdout:1/966: rename d5/da/dd/c5a to d5/df5/c14f 0 2024-03-11T19:52:48.145 INFO:tasks.workunit.client.0.smithi090.stdout:5/954: write d1/d26/d4d/f13e [84128,72625] 0 2024-03-11T19:52:48.238 INFO:tasks.workunit.client.0.smithi090.stdout:7/952: truncate d3/dc/d1e/d7c/d96/d32/d45/f9d 603730 0 2024-03-11T19:52:48.393 INFO:tasks.workunit.client.0.smithi090.stdout:7/953: sync 2024-03-11T19:52:48.753 INFO:tasks.workunit.client.0.smithi090.stdout:7/954: dread d3/dc/d1e/d7c/d96/d32/f3f [0,4194304] 0 2024-03-11T19:52:49.093 INFO:tasks.workunit.client.0.smithi090.stdout:7/955: dread d3/f3b [0,4194304] 0 2024-03-11T19:52:49.126 INFO:tasks.workunit.client.0.smithi090.stdout:1/967: rmdir d5/d31/d76/d114 39 2024-03-11T19:52:49.171 INFO:tasks.workunit.client.0.smithi090.stdout:1/968: chown d5/d65/d6b/d7f/fab 3742 1 2024-03-11T19:52:49.601 INFO:tasks.workunit.client.0.smithi090.stdout:0/976: dwrite d2/d11/d15/d29/f9a [0,4194304] 0 2024-03-11T19:52:49.678 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:52:49.681 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:52:49.752 INFO:tasks.workunit.client.0.smithi090.stdout:1/969: chown d5/da/d23/c24 132228 1 2024-03-11T19:52:49.785 INFO:tasks.workunit.client.0.smithi090.stdout:0/977: chown d2/d11/d15/d29/df5/c103 0 1 2024-03-11T19:52:49.810 INFO:tasks.workunit.client.0.smithi090.stdout:0/978: readlink d2/d11/l131 0 2024-03-11T19:52:50.108 INFO:tasks.workunit.client.0.smithi090.stdout:5/955: rename d1/d17/d39/d51/d53/d7b/da0/db9/dc6/lcb to d1/d17/d39/d51/d53/d7b/d3b/d76/l14a 0 2024-03-11T19:52:50.166 INFO:tasks.workunit.client.0.smithi090.stdout:1/970: mkdir d5/d65/d6b/d7f/d150 0 2024-03-11T19:52:50.183 INFO:tasks.workunit.client.0.smithi090.stdout:9/956: write d1/dd/f128 [603634,15306] 0 2024-03-11T19:52:50.257 INFO:tasks.workunit.client.0.smithi090.stdout:9/957: chown d1/d47/f106 114 1 2024-03-11T19:52:50.580 INFO:tasks.workunit.client.0.smithi090.stdout:7/956: creat d3/dc/d1e/d7c/d96/d32/d71/f147 x:0 0 0 2024-03-11T19:52:50.693 INFO:tasks.workunit.client.0.smithi090.stdout:5/956: readlink d1/l29 0 2024-03-11T19:52:50.882 INFO:tasks.workunit.client.0.smithi090.stdout:0/979: rename d2/d11/d15/d29/de5/d11b/d6d/da4/d78/l127 to d2/d94/l13f 0 2024-03-11T19:52:51.040 INFO:tasks.workunit.client.0.smithi090.stdout:9/958: write d1/dd/d18/d3a/f112 [3405972,52053] 0 2024-03-11T19:52:51.416 INFO:tasks.workunit.client.0.smithi090.stdout:1/971: rename d5/da/d23/f2d to d5/da/d97/df6/f151 0 2024-03-11T19:52:51.609 INFO:tasks.workunit.client.0.smithi090.stdout:7/957: dread d3/f7 [0,4194304] 0 2024-03-11T19:52:51.717 INFO:tasks.workunit.client.0.smithi090.stdout:1/972: dread d5/da/d1c/fdb [0,4194304] 0 2024-03-11T19:52:51.773 INFO:tasks.workunit.client.0.smithi090.stdout:9/959: rmdir d1/dd/d18/d3a/d57 39 2024-03-11T19:52:51.849 INFO:tasks.workunit.client.0.smithi090.stdout:1/973: dread d5/d31/d4c/daf/dfc/f12e [0,4194304] 0 2024-03-11T19:52:51.924 INFO:tasks.workunit.client.0.smithi090.stdout:7/958: sync 2024-03-11T19:52:52.564 INFO:tasks.workunit.client.0.smithi090.stdout:6/997: creat d4/d13/f150 x:0 0 0 2024-03-11T19:52:52.566 INFO:tasks.workunit.client.0.smithi090.stdout:5/957: rename d1/d17/d39/d51/d53/d7b/d3b/d76/db7/ddf/f101 to d1/d17/d39/de8/f14b 0 2024-03-11T19:52:52.598 INFO:tasks.workunit.client.0.smithi090.stdout:2/943: write d0/de/f17 [3795833,40770] 0 2024-03-11T19:52:52.844 INFO:tasks.workunit.client.0.smithi090.stdout:5/958: chown d1/d17/d39/d51/d73/f95 19 1 2024-03-11T19:52:52.895 INFO:tasks.workunit.client.0.smithi090.stdout:3/912: dwrite d2/db/d15/f1e [0,4194304] 0 2024-03-11T19:52:53.260 INFO:tasks.workunit.client.0.smithi090.stdout:9/960: chown d1/dd/d18/d25/d101/l9e 747377750 1 2024-03-11T19:52:53.464 INFO:tasks.workunit.client.0.smithi090.stdout:6/998: dread d4/d13/d22/d34/d40/da4/fd8 [0,4194304] 0 2024-03-11T19:52:53.563 INFO:tasks.workunit.client.0.smithi090.stdout:0/980: write d2/d11/d3a/d64/f10b [692723,66777] 0 2024-03-11T19:52:53.741 INFO:tasks.workunit.client.0.smithi090.stdout:6/999: dread - d4/d13/d22/d34/d40/dd5/fbe zero size 2024-03-11T19:52:54.124 INFO:tasks.workunit.client.0.smithi090.stdout:5/959: symlink d1/d17/d39/d51/d53/d7b/d3b/d76/db7/l14c 0 2024-03-11T19:52:54.459 INFO:tasks.workunit.client.0.smithi090.stdout:9/961: mknod d1/dd/d18/d25/d101/d1c/d77/deb/c143 0 2024-03-11T19:52:54.630 INFO:tasks.workunit.client.0.smithi090.stdout:0/981: rmdir d2/d11/d15/d29/df3 39 2024-03-11T19:52:54.979 INFO:tasks.workunit.client.0.smithi090.stdout:0/982: dwrite d2/d11/d15/d29/f9a [4194304,4194304] 0 2024-03-11T19:52:55.095 INFO:tasks.workunit.client.0.smithi090.stdout:1/974: creat d5/da/d23/dfe/f152 x:0 0 0 2024-03-11T19:52:55.138 INFO:tasks.workunit.client.0.smithi090.stdout:0/983: sync 2024-03-11T19:52:55.705 INFO:tasks.workunit.client.0.smithi090.stdout:9/962: dread d1/dd/d18/d25/d101/d1c/d77/fc4 [0,4194304] 0 2024-03-11T19:52:55.789 INFO:tasks.workunit.client.0.smithi090.stdout:1/975: creat d5/d65/f153 x:0 0 0 2024-03-11T19:52:55.802 INFO:tasks.workunit.client.0.smithi090.stdout:9/963: dread d1/dd/d18/d3a/d48/dc6/f123 [0,4194304] 0 2024-03-11T19:52:55.986 INFO:tasks.workunit.client.0.smithi090.stdout:0/984: chown d2/d11/d15/f74 0 1 2024-03-11T19:52:56.189 INFO:tasks.workunit.client.0.smithi090.stdout:2/944: link d0/de/db6/c80 d0/de/d12/d46/de6/c127 0 2024-03-11T19:52:56.399 INFO:tasks.workunit.client.0.smithi090.stdout:3/913: getdents d2/d2f 0 2024-03-11T19:52:56.627 INFO:tasks.workunit.client.0.smithi090.stdout:9/964: mkdir d1/dd/d18/d25/d101/d1c/d77/d144 0 2024-03-11T19:52:56.757 INFO:tasks.workunit.client.0.smithi090.stdout:7/959: dwrite d3/dc/d1e/d7c/d96/d32/d45/d84/d8d/dab/db5/de1/dc2/dd0/ff1 [0,4194304] 0 2024-03-11T19:52:56.896 INFO:tasks.workunit.client.0.smithi090.stdout:9/965: readlink d1/dd/d18/ddd/lf4 0 2024-03-11T19:52:57.341 INFO:tasks.workunit.client.0.smithi090.stdout:1/976: mknod d5/d65/d6b/c154 0 2024-03-11T19:52:57.754 INFO:tasks.workunit.client.0.smithi090.stdout:3/914: unlink d2/db/d15/d1a/d75/d8c/cc3 0 2024-03-11T19:52:57.855 INFO:tasks.workunit.client.0.smithi090.stdout:9/966: dread - d1/dd/df3/ff8 zero size 2024-03-11T19:52:57.927 INFO:tasks.workunit.client.0.smithi090.stdout:3/915: dread d2/fb9 [0,4194304] 0 2024-03-11T19:52:58.613 INFO:tasks.workunit.client.0.smithi090.stdout:5/960: dwrite d1/d17/d39/d51/d53/d7b/d3b/fe1 [4194304,4194304] 0 2024-03-11T19:52:58.847 INFO:tasks.workunit.client.0.smithi090.stdout:9/967: dread - d1/dd/d18/d25/d101/d1c/d21/d2a/dc2/d100/d120/f13c zero size 2024-03-11T19:52:59.101 INFO:tasks.workunit.client.0.smithi090.stdout:3/916: dread d2/db/d15/d49/f7f [0,4194304] 0 2024-03-11T19:52:59.284 INFO:tasks.workunit.client.0.smithi090.stdout:5/961: truncate d1/d17/d39/d51/d53/fac 191984 0 2024-03-11T19:52:59.558 INFO:tasks.workunit.client.0.smithi090.stdout:2/945: write d0/de/d12/d46/d75/f7c [4279795,10740] 0 2024-03-11T19:53:00.043 INFO:tasks.workunit.client.0.smithi090.stdout:0/985: write d2/d11/d15/d29/fbd [1213487,22932] 0 2024-03-11T19:53:00.116 INFO:tasks.workunit.client.0.smithi090.stdout:0/986: read d2/d11/d15/d29/fbd [2963877,79548] 0 2024-03-11T19:53:00.212 INFO:tasks.workunit.client.0.smithi090.stdout:5/962: mknod d1/d17/d39/df2/d12b/d143/c14d 0 2024-03-11T19:53:00.298 INFO:tasks.workunit.client.0.smithi090.stdout:1/977: write d5/da/d23/dfe/fd8 [97860,30220] 0 2024-03-11T19:53:00.872 INFO:tasks.workunit.client.0.smithi090.stdout:7/960: rename d3/dc/d1e/d7c/d96/d32/d45/d84 to d3/d55/d148 0 2024-03-11T19:53:00.893 INFO:tasks.workunit.client.0.smithi090.stdout:5/963: dread - d1/d26/f13b zero size 2024-03-11T19:53:01.122 INFO:tasks.workunit.client.0.smithi090.stdout:5/964: sync 2024-03-11T19:53:01.141 INFO:tasks.workunit.client.0.smithi090.stdout:5/965: chown d1/d26/lb 5746 1 2024-03-11T19:53:01.220 INFO:tasks.workunit.client.0.smithi090.stdout:3/917: getdents d2/db/d15/d1a/d75/de8/d6a/dcb 0 2024-03-11T19:53:01.351 INFO:tasks.workunit.client.0.smithi090.stdout:1/978: dread d5/d4e/d79/f82 [0,4194304] 0 2024-03-11T19:53:01.530 INFO:tasks.workunit.client.0.smithi090.stdout:9/968: creat d1/dd/d18/d25/d101/d1c/d21/dab/d114/f145 x:0 0 0 2024-03-11T19:53:01.575 INFO:tasks.workunit.client.0.smithi090.stdout:9/969: stat d1/dd/d18/d3a/d48/dc6/f12f 0 2024-03-11T19:53:02.030 INFO:tasks.workunit.client.0.smithi090.stdout:5/966: mkdir d1/d26/dc8/d14e 0 2024-03-11T19:53:02.326 INFO:tasks.workunit.client.0.smithi090.stdout:1/979: truncate d5/da/d97/df6/f14a 947684 0 2024-03-11T19:53:02.448 INFO:tasks.workunit.client.0.smithi090.stdout:9/970: truncate d1/dd/f80 1483774 0 2024-03-11T19:53:02.968 INFO:tasks.workunit.client.0.smithi090.stdout:1/980: creat d5/da/d23/dfe/d89/d10e/d80/f155 x:0 0 0 2024-03-11T19:53:02.992 INFO:tasks.workunit.client.0.smithi090.stdout:9/971: dread - d1/dd/d18/d25/d101/d1c/d21/d2a/da4/f11c zero size 2024-03-11T19:53:03.096 INFO:tasks.workunit.client.0.smithi090.stdout:9/972: sync 2024-03-11T19:53:03.128 INFO:tasks.workunit.client.0.smithi090.stdout:2/946: write d0/de/d12/d46/d75/dc5/de5/f108 [968180,5087] 0 2024-03-11T19:53:03.135 INFO:tasks.workunit.client.0.smithi090.stdout:0/987: write d2/d11/d15/d29/d3d/d7b/f96 [1079436,124387] 0 2024-03-11T19:53:03.236 INFO:tasks.workunit.client.0.smithi090.stdout:2/947: sync 2024-03-11T19:53:03.612 INFO:tasks.workunit.client.0.smithi090.stdout:1/981: creat d5/da/d97/dd1/f156 x:0 0 0 2024-03-11T19:53:03.632 INFO:tasks.workunit.client.0.smithi090.stdout:9/973: truncate d1/dd/f3f 628035 0 2024-03-11T19:53:03.850 INFO:tasks.workunit.client.0.smithi090.stdout:2/948: rmdir d0/de/d12/d46/d75/d112 39 2024-03-11T19:53:04.244 INFO:tasks.workunit.client.0.smithi090.stdout:7/961: write d3/d55/d148/d8d/dab/db5/de1/dc6/f12a [144352,8522] 0 2024-03-11T19:53:04.248 INFO:tasks.workunit.client.0.smithi090.stdout:3/918: getdents d2/db/d15/d69/d119/d123 0 2024-03-11T19:53:04.278 INFO:tasks.workunit.client.0.smithi090.stdout:5/967: link d1/d26/f52 d1/d17/d39/d51/d53/d7b/da0/f14f 0 2024-03-11T19:53:04.452 INFO:tasks.workunit.client.0.smithi090.stdout:7/962: stat d3/dc/d1e/d7c/d96/d32/d67/dee/c140 0 2024-03-11T19:53:04.809 INFO:tasks.workunit.client.0.smithi090.stdout:2/949: creat d0/de/d12/d81/d93/f128 x:0 0 0 2024-03-11T19:53:04.906 INFO:tasks.workunit.client.0.smithi090.stdout:5/968: dread d1/d17/d39/d51/d53/fac [0,4194304] 0 2024-03-11T19:53:05.466 INFO:tasks.workunit.client.0.smithi090.stdout:2/950: creat d0/de/db6/d29/f129 x:0 0 0 2024-03-11T19:53:05.813 INFO:tasks.workunit.client.0.smithi090.stdout:5/969: dread d1/f2f [0,4194304] 0 2024-03-11T19:53:05.920 INFO:tasks.workunit.client.0.smithi090.stdout:0/988: dwrite d2/d11/d15/d29/d43/d58/f9e [0,4194304] 0 2024-03-11T19:53:06.215 INFO:tasks.workunit.client.0.smithi090.stdout:1/982: write d5/da/d23/dfe/d89/d10e/fe8 [496237,94385] 0 2024-03-11T19:53:06.530 INFO:tasks.workunit.client.0.smithi090.stdout:0/989: symlink d2/d11/d15/dcc/l140 0 2024-03-11T19:53:07.214 INFO:tasks.workunit.client.0.smithi090.stdout:0/990: fdatasync d2/d11/d15/d29/de5/f12d 0 2024-03-11T19:53:07.330 INFO:tasks.workunit.client.0.smithi090.stdout:3/919: dwrite d2/db/d15/d1a/d75/dc6/f27 [0,4194304] 0 2024-03-11T19:53:07.513 INFO:tasks.workunit.client.0.smithi090.stdout:7/963: dwrite d3/d55/d148/d8d/dab/f141 [0,4194304] 0 2024-03-11T19:53:07.731 INFO:tasks.workunit.client.0.smithi090.stdout:7/964: sync 2024-03-11T19:53:07.780 INFO:tasks.workunit.client.0.smithi090.stdout:3/920: rmdir d2 39 2024-03-11T19:53:08.035 INFO:tasks.workunit.client.0.smithi090.stdout:7/965: chown d3/dc/d1e/d7c/d96/d32/d7b/d99/f100 1027973 1 2024-03-11T19:53:08.054 INFO:tasks.workunit.client.0.smithi090.stdout:2/951: dwrite d0/de/d12/d46/d4c/d64/dd4/f10a [0,4194304] 0 2024-03-11T19:53:08.163 INFO:tasks.workunit.client.0.smithi090.stdout:0/991: unlink d2/d11/d15/lcf 0 2024-03-11T19:53:08.187 INFO:tasks.workunit.client.0.smithi090.stdout:9/974: symlink d1/dd/l146 0 2024-03-11T19:53:08.202 INFO:tasks.workunit.client.0.smithi090.stdout:1/983: write d5/da/d23/dfe/d89/d10e/d80/fb9 [964421,75093] 0 2024-03-11T19:53:08.216 INFO:tasks.workunit.client.0.smithi090.stdout:5/970: write d1/d17/d39/d51/d73/da6/f112 [3813658,25057] 0 2024-03-11T19:53:08.265 INFO:tasks.workunit.client.0.smithi090.stdout:0/992: stat d2/l128 0 2024-03-11T19:53:08.322 INFO:tasks.workunit.client.0.smithi090.stdout:5/971: sync 2024-03-11T19:53:08.433 INFO:tasks.workunit.client.0.smithi090.stdout:2/952: read d0/de/dd3/f102 [571898,32364] 0 2024-03-11T19:53:08.525 INFO:tasks.workunit.client.0.smithi090.stdout:9/975: symlink d1/dbb/l147 0 2024-03-11T19:53:08.633 INFO:tasks.workunit.client.0.smithi090.stdout:0/993: mkdir d2/d11/d3a/dc8/dd0/d141 0 2024-03-11T19:53:08.686 INFO:tasks.workunit.client.0.smithi090.stdout:7/966: dread d3/dc/d1e/d7c/d96/d32/d71/d80/f119 [0,4194304] 0 2024-03-11T19:53:08.787 INFO:tasks.workunit.client.0.smithi090.stdout:7/967: read d3/dc/d1e/d7c/d96/d32/d45/de0/d48/fd7 [3233144,115007] 0 2024-03-11T19:53:09.022 INFO:tasks.workunit.client.0.smithi090.stdout:1/984: mkdir d5/da/d23/dfe/d157 0 2024-03-11T19:53:09.058 INFO:tasks.workunit.client.0.smithi090.stdout:1/985: dread d5/d4e/d79/f82 [0,4194304] 0 2024-03-11T19:53:09.156 INFO:tasks.workunit.client.0.smithi090.stdout:3/921: rename d2/db/d15/d1a/d75/de8/d6a/dcb/l112 to d2/db/d15/d1a/d75/dc6/d114/l129 0 2024-03-11T19:53:09.279 INFO:tasks.workunit.client.0.smithi090.stdout:7/968: fdatasync d3/f68 0 2024-03-11T19:53:09.282 INFO:tasks.workunit.client.0.smithi090.stdout:5/972: symlink d1/d17/d39/d51/d53/d7b/l150 0 2024-03-11T19:53:09.906 INFO:tasks.workunit.client.0.smithi090.stdout:2/953: symlink d0/de/d12/d46/d5d/d8d/ddf/l12a 0 2024-03-11T19:53:10.074 INFO:tasks.workunit.client.0.smithi090.stdout:5/973: dread - d1/d17/d39/d51/d53/d7b/da0/dc0/f11c zero size 2024-03-11T19:53:10.097 INFO:tasks.workunit.client.0.smithi090.stdout:5/974: chown d1/d17/d131 6 1 2024-03-11T19:53:10.471 INFO:tasks.workunit.client.0.smithi090.stdout:7/969: dread d3/dc/d1e/d7c/d96/d32/d45/de0/f116 [0,4194304] 0 2024-03-11T19:53:10.677 INFO:tasks.workunit.client.0.smithi090.stdout:3/922: mknod d2/d2f/dd5/c12a 0 2024-03-11T19:53:11.203 INFO:tasks.workunit.client.0.smithi090.stdout:9/976: rename d1/dd/c15 to d1/dd/d18/d3a/c148 0 2024-03-11T19:53:11.617 INFO:tasks.workunit.client.0.smithi090.stdout:2/954: link d0/de/db6/d29/f129 d0/de/db6/d83/dbe/f12b 0 2024-03-11T19:53:11.983 INFO:tasks.workunit.client.0.smithi090.stdout:7/970: rename d3/dc/d1e/d7c/d96/d32/d45/de0/c97 to d3/d55/d148/d8d/dab/db5/de1/dc6/d103/d105/c149 0 2024-03-11T19:53:12.200 INFO:tasks.workunit.client.0.smithi090.stdout:0/994: dread d2/d11/d3a/d64/dac/fca [0,4194304] 0 2024-03-11T19:53:12.613 INFO:tasks.workunit.client.0.smithi090.stdout:1/986: mkdir d5/da/d158 0 2024-03-11T19:53:12.647 INFO:tasks.workunit.client.0.smithi090.stdout:5/975: write d1/f2f [524167,41391] 0 2024-03-11T19:53:12.727 INFO:tasks.workunit.client.0.smithi090.stdout:9/977: write d1/d1d/d6a/fcd [632900,107790] 0 2024-03-11T19:53:12.774 INFO:tasks.workunit.client.0.smithi090.stdout:1/987: sync 2024-03-11T19:53:12.943 INFO:tasks.workunit.client.0.smithi090.stdout:0/995: stat d2/d11/d3f/c51 0 2024-03-11T19:53:13.017 INFO:tasks.workunit.client.0.smithi090.stdout:2/955: mkdir d0/de/d12/d46/d75/d112/d12c 0 2024-03-11T19:53:13.097 INFO:tasks.workunit.client.0.smithi090.stdout:3/923: dwrite d2/db/f109 [0,4194304] 0 2024-03-11T19:53:13.239 INFO:tasks.workunit.client.0.smithi090.stdout:7/971: symlink d3/dc/d1e/d7c/d96/d32/d45/l14a 0 2024-03-11T19:53:13.322 INFO:tasks.workunit.client.0.smithi090.stdout:3/924: stat d2/db/d15/d69/da2/dc7/c10e 0 2024-03-11T19:53:13.339 INFO:tasks.workunit.client.0.smithi090.stdout:7/972: sync 2024-03-11T19:53:13.346 INFO:tasks.workunit.client.0.smithi090.stdout:3/925: readlink d2/l6e 0 2024-03-11T19:53:13.670 INFO:tasks.workunit.client.0.smithi090.stdout:5/976: mkdir d1/d17/d39/d51/d53/d7b/d3b/d76/db7/d151 0 2024-03-11T19:53:13.699 INFO:tasks.workunit.client.0.smithi090.stdout:0/996: rename d2/d11/d15/d29/d3d/d8c/f121 to d2/d11/df4/d11e/f142 0 2024-03-11T19:53:13.772 INFO:tasks.workunit.client.0.smithi090.stdout:3/926: mkdir d2/db/d15/d1a/d75/dc6/d114/d12b 0 2024-03-11T19:53:14.286 INFO:tasks.workunit.client.0.smithi090.stdout:3/927: rmdir d2/db/d15/d1a/d75/dc6/d114 39 2024-03-11T19:53:14.296 INFO:tasks.workunit.client.0.smithi090.stdout:1/988: write d5/d65/d6b/de7/f132 [150837,107028] 0 2024-03-11T19:53:15.145 INFO:tasks.workunit.client.0.smithi090.stdout:9/978: dwrite d1/dd/d18/d25/d101/d1c/d21/d2a/dc2/d100/d120/f34 [0,4194304] 0 2024-03-11T19:53:15.305 INFO:tasks.workunit.client.0.smithi090.stdout:9/979: chown d1/dd/d18/d3a/da2 11932098 1 2024-03-11T19:53:15.752 INFO:tasks.workunit.client.0.smithi090.stdout:2/956: link d0/de/d12/d46/d4c/d64/c103 d0/c12d 0 2024-03-11T19:53:15.782 INFO:tasks.workunit.client.0.smithi090.stdout:7/973: rename d3/d55/d148/d8d/dab/ffe to d3/d55/d148/d8d/dab/db5/de1/dc6/f14b 0 2024-03-11T19:53:15.809 INFO:tasks.workunit.client.0.smithi090.stdout:5/977: write d1/d17/d39/d51/d53/d7b/f37 [3521442,16051] 0 2024-03-11T19:53:15.896 INFO:tasks.workunit.client.0.smithi090.stdout:2/957: chown d0/de/db6/d25/d8c/fd8 2046835 1 2024-03-11T19:53:15.957 INFO:tasks.workunit.client.0.smithi090.stdout:7/974: chown d3/d55/d148/d8d/dab/db5/de1/dc2/dd0/d136/d142 10 1 2024-03-11T19:53:16.027 INFO:tasks.workunit.client.0.smithi090.stdout:2/958: dread d0/de/db6/d25/fc7 [0,4194304] 0 2024-03-11T19:53:16.528 INFO:tasks.workunit.client.0.smithi090.stdout:2/959: symlink d0/de/d12/d46/de8/l12e 0 2024-03-11T19:53:16.827 INFO:tasks.workunit.client.0.smithi090.stdout:7/975: fsync d3/dc/d1e/f4f 0 2024-03-11T19:53:17.081 INFO:tasks.workunit.client.0.smithi090.stdout:5/978: dread d1/d17/d39/f68 [0,4194304] 0 2024-03-11T19:53:17.143 INFO:tasks.workunit.client.0.smithi090.stdout:0/997: dwrite d2/d94/f132 [0,4194304] 0 2024-03-11T19:53:17.151 INFO:tasks.workunit.client.0.smithi090.stdout:9/980: getdents d1/dd/d18/d25/d101/d1c/d21/d2a 0 2024-03-11T19:53:17.351 INFO:tasks.workunit.client.0.smithi090.stdout:2/960: fdatasync d0/de/d12/d46/d5d/fcd 0 2024-03-11T19:53:17.362 INFO:tasks.workunit.client.0.smithi090.stdout:7/976: chown d3/dc/fc4 10 1 2024-03-11T19:53:17.654 INFO:tasks.workunit.client.0.smithi090.stdout:0/998: symlink d2/d11/d6e/l143 0 2024-03-11T19:53:17.660 INFO:tasks.workunit.client.0.smithi090.stdout:3/928: dwrite d2/db/d15/ffa [0,4194304] 0 2024-03-11T19:53:17.874 INFO:tasks.workunit.client.0.smithi090.stdout:2/961: chown d0/de/db6/f47 12967633 1 2024-03-11T19:53:17.900 INFO:tasks.workunit.client.0.smithi090.stdout:1/989: dwrite d5/d4e/d54/f6e [0,4194304] 0 2024-03-11T19:53:18.149 INFO:tasks.workunit.client.0.smithi090.stdout:3/929: dread - d2/db/d15/d1a/d75/de8/d6a/dcb/f10d zero size 2024-03-11T19:53:18.190 INFO:tasks.workunit.client.0.smithi090.stdout:3/930: dread - d2/db/d15/d69/dad/fc0 zero size 2024-03-11T19:53:18.398 INFO:tasks.workunit.client.0.smithi090.stdout:9/981: creat d1/dd/d18/d25/d101/d1c/d21/d2a/f149 x:0 0 0 2024-03-11T19:53:18.442 INFO:tasks.workunit.client.0.smithi090.stdout:7/977: mknod d3/dc/d1e/d7c/d96/d32/d45/de0/d48/c14c 0 2024-03-11T19:53:18.488 INFO:tasks.workunit.client.0.smithi090.stdout:1/990: read d5/da/f104 [380240,40670] 0 2024-03-11T19:53:18.589 INFO:tasks.workunit.client.0.smithi090.stdout:2/962: truncate d0/f2 7448788 0 2024-03-11T19:53:18.817 INFO:tasks.workunit.client.0.smithi090.stdout:2/963: sync 2024-03-11T19:53:18.946 INFO:tasks.workunit.client.0.smithi090.stdout:2/964: readlink d0/l91 0 2024-03-11T19:53:18.957 INFO:tasks.workunit.client.0.smithi090.stdout:5/979: write d1/d17/d39/d51/d53/d7b/d3b/d76/db7/fea [978006,21486] 0 2024-03-11T19:53:18.961 INFO:tasks.workunit.client.0.smithi090.stdout:3/931: write d2/db/d15/d1a/d75/fc8 [2515704,22689] 0 2024-03-11T19:53:19.078 INFO:tasks.workunit.client.0.smithi090.stdout:2/965: dread - d0/de/d12/d46/d4c/f111 zero size 2024-03-11T19:53:19.098 INFO:tasks.workunit.client.0.smithi090.stdout:0/999: symlink d2/d11/d3a/d64/dac/dba/dec/l144 0 2024-03-11T19:53:19.184 INFO:tasks.workunit.client.0.smithi090.stdout:2/966: sync 2024-03-11T19:53:19.260 INFO:tasks.workunit.client.0.smithi090.stdout:1/991: unlink d5/d31/d76/c143 0 2024-03-11T19:53:19.444 INFO:tasks.workunit.client.0.smithi090.stdout:3/932: mkdir d2/db/d15/d69/d12c 0 2024-03-11T19:53:19.446 INFO:tasks.workunit.client.0.smithi090.stdout:5/980: fsync d1/d17/f79 0 2024-03-11T19:53:19.473 INFO:tasks.workunit.client.0.smithi090.stdout:7/978: mknod d3/dc/d1e/d7c/d96/d32/d7b/c14d 0 2024-03-11T19:53:19.479 INFO:tasks.workunit.client.0.smithi090.stdout:9/982: rmdir d1/dd/d18/d25/d101/d1c/d21/dab/d114/d9c 39 2024-03-11T19:53:19.790 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:53:19.794 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:53:19.941 INFO:tasks.workunit.client.0.smithi090.stdout:1/992: mknod d5/da/d23/dfe/d89/c159 0 2024-03-11T19:53:20.135 INFO:tasks.workunit.client.0.smithi090.stdout:9/983: mknod d1/d1d/c14a 0 2024-03-11T19:53:20.442 INFO:tasks.workunit.client.0.smithi090.stdout:7/979: mknod d3/d55/d148/d8d/dab/db5/c14e 0 2024-03-11T19:53:20.770 INFO:tasks.workunit.client.0.smithi090.stdout:5/981: rename d1/d17/d39/d51/d53/d7b/d3b/d76/db7/fe9 to d1/d17/d39/de8/f152 0 2024-03-11T19:53:20.835 INFO:tasks.workunit.client.0.smithi090.stdout:2/967: truncate d0/de/db6/f31 3427458 0 2024-03-11T19:53:21.221 INFO:tasks.workunit.client.0.smithi090.stdout:1/993: dread d5/da/d97/df6/f14a [0,4194304] 0 2024-03-11T19:53:21.701 INFO:tasks.workunit.client.0.smithi090.stdout:9/984: rename d1/dd/d18/l142 to d1/dd/d18/d3a/d70/l14b 0 2024-03-11T19:53:22.105 INFO:tasks.workunit.client.0.smithi090.stdout:2/968: dread d0/de/d12/d46/d5d/d8d/fcb [0,4194304] 0 2024-03-11T19:53:22.242 INFO:tasks.workunit.client.0.smithi090.stdout:1/994: rename d5/da/d1c/d3b/f137 to d5/da/d97/f15a 0 2024-03-11T19:53:22.540 INFO:tasks.workunit.client.0.smithi090.stdout:3/933: dwrite d2/db/d15/d1a/d75/de8/d6a/f97 [0,4194304] 0 2024-03-11T19:53:22.672 INFO:tasks.workunit.client.0.smithi090.stdout:2/969: chown d0/l7f 75233 1 2024-03-11T19:53:22.732 INFO:tasks.workunit.client.0.smithi090.stdout:3/934: write d2/db/d15/d1a/d75/de8/d6a/d9b/fae [5111988,96952] 0 2024-03-11T19:53:22.851 INFO:tasks.workunit.client.0.smithi090.stdout:9/985: getdents d1/dd/d18/d25/d101/d1c/d77/deb/d13e 0 2024-03-11T19:53:23.338 INFO:tasks.workunit.client.0.smithi090.stdout:7/980: write d3/dc/d1e/d7c/d96/d32/d45/de0/f53 [4945769,130224] 0 2024-03-11T19:53:23.428 INFO:tasks.workunit.client.0.smithi090.stdout:5/982: getdents d1/d17 0 2024-03-11T19:53:23.521 INFO:tasks.workunit.client.0.smithi090.stdout:5/983: chown d1/d17/d39/d51/d53/fb2 874048401 1 2024-03-11T19:53:24.047 INFO:tasks.workunit.client.0.smithi090.stdout:7/981: mkdir d3/dc/d1e/d7c/d96/d32/d67/d14f 0 2024-03-11T19:53:24.109 INFO:tasks.workunit.client.0.smithi090.stdout:1/995: write d5/da/d23/dfe/d89/d10e/d80/d13e/fb0 [936813,55060] 0 2024-03-11T19:53:24.117 INFO:tasks.workunit.client.0.smithi090.stdout:9/986: write d1/d47/f90 [1320935,19282] 0 2024-03-11T19:53:24.118 INFO:tasks.workunit.client.0.smithi090.stdout:3/935: write d2/fb1 [301893,125696] 0 2024-03-11T19:53:24.122 INFO:tasks.workunit.client.0.smithi090.stdout:5/984: readlink d1/d17/d39/d51/d53/d7b/d3b/d76/db7/l109 0 2024-03-11T19:53:24.246 INFO:tasks.workunit.client.0.smithi090.stdout:7/982: stat d3/d55/d148/d8d/dab 0 2024-03-11T19:53:24.340 INFO:tasks.workunit.client.0.smithi090.stdout:2/970: dwrite d0/f3 [0,4194304] 0 2024-03-11T19:53:24.419 INFO:tasks.workunit.client.0.smithi090.stdout:1/996: mknod d5/da/d97/c15b 0 2024-03-11T19:53:24.489 INFO:tasks.workunit.client.0.smithi090.stdout:3/936: truncate d2/db/d15/f1e 3234947 0 2024-03-11T19:53:24.500 INFO:tasks.workunit.client.0.smithi090.stdout:5/985: chown d1/d17/d39/d51/d53/d7b/d3b/d76/db7/de0/f11e 340 1 2024-03-11T19:53:25.181 INFO:tasks.workunit.client.0.smithi090.stdout:7/983: truncate d3/dc/d1e/ff 5571334 0 2024-03-11T19:53:25.273 INFO:tasks.workunit.client.0.smithi090.stdout:5/986: dread - d1/d26/db5/f132 zero size 2024-03-11T19:53:25.457 INFO:tasks.workunit.client.0.smithi090.stdout:9/987: truncate d1/dd/d18/d3a/dd8/f110 501914 0 2024-03-11T19:53:25.613 INFO:tasks.workunit.client.0.smithi090.stdout:3/937: creat d2/d45/df1/f12d x:0 0 0 2024-03-11T19:53:26.241 INFO:tasks.workunit.client.0.smithi090.stdout:9/988: stat d1/l7d 0 2024-03-11T19:53:26.273 INFO:tasks.workunit.client.0.smithi090.stdout:9/989: chown d1/dd/d18/d3a/dd8/f11f 810461540 1 2024-03-11T19:53:26.307 INFO:tasks.workunit.client.0.smithi090.stdout:1/997: write d5/d65/d6b/d7f/fab [6163,99185] 0 2024-03-11T19:53:26.383 INFO:tasks.workunit.client.0.smithi090.stdout:2/971: rename d0/de/d12/d46/d5d/d8d/fa6 to d0/de/d12/d81/d121/f12f 0 2024-03-11T19:53:26.652 INFO:tasks.workunit.client.0.smithi090.stdout:7/984: fsync d3/dc/d1e/d7c/d96/fea 0 2024-03-11T19:53:26.745 INFO:tasks.workunit.client.0.smithi090.stdout:9/990: chown d1/dd/d18/d25/d101/d1c/d21/dab/d114/d9c/daf/ld2 23 1 2024-03-11T19:53:26.789 INFO:tasks.workunit.client.0.smithi090.stdout:9/991: chown d1/dd/d18/d25/d101/d1c/d21/d2a/da4 3193 1 2024-03-11T19:53:26.814 INFO:tasks.workunit.client.0.smithi090.stdout:1/998: fdatasync d5/da/d23/dfe/d89/d10e/fec 0 2024-03-11T19:53:27.116 INFO:tasks.workunit.client.0.smithi090.stdout:5/987: unlink d1/l88 0 2024-03-11T19:53:27.876 INFO:tasks.workunit.client.0.smithi090.stdout:1/999: read d5/d4e/d79/fc8 [195896,28733] 0 2024-03-11T19:53:27.910 INFO:tasks.workunit.client.0.smithi090.stdout:7/985: dread d3/dc/d1e/d7c/d96/d32/da0/f10d [0,4194304] 0 2024-03-11T19:53:28.192 INFO:tasks.workunit.client.0.smithi090.stdout:3/938: write d2/d2f/f113 [245734,83793] 0 2024-03-11T19:53:28.654 INFO:tasks.workunit.client.0.smithi090.stdout:9/992: link d1/dd/d18/d3a/d70/l14b d1/dd/d18/d3a/da2/l14c 0 2024-03-11T19:53:28.809 INFO:tasks.workunit.client.0.smithi090.stdout:3/939: mknod d2/d4f/c12e 0 2024-03-11T19:53:28.899 INFO:tasks.workunit.client.0.smithi090.stdout:7/986: symlink d3/dc/d1e/d7c/d96/d32/d135/l150 0 2024-03-11T19:53:28.946 INFO:tasks.workunit.client.0.smithi090.stdout:2/972: write d0/de/d12/d46/d5d/d63/f10d [200594,62760] 0 2024-03-11T19:53:29.053 INFO:tasks.workunit.client.0.smithi090.stdout:2/973: dread d0/de/d12/d46/f48 [0,4194304] 0 2024-03-11T19:53:29.282 INFO:tasks.workunit.client.0.smithi090.stdout:5/988: getdents d1 0 2024-03-11T19:53:29.288 INFO:tasks.workunit.client.0.smithi090.stdout:9/993: unlink d1/dd/d18/d3a/da2/fac 0 2024-03-11T19:53:29.314 INFO:tasks.workunit.client.0.smithi090.stdout:3/940: creat d2/d2f/dd5/f12f x:0 0 0 2024-03-11T19:53:29.341 INFO:tasks.workunit.client.0.smithi090.stdout:7/987: truncate d3/dc/d1e/d2a/f114 248372 0 2024-03-11T19:53:29.415 INFO:tasks.workunit.client.0.smithi090.stdout:2/974: mknod d0/de/d12/d46/d75/dc5/c130 0 2024-03-11T19:53:29.988 INFO:tasks.workunit.client.0.smithi090.stdout:5/989: mkdir d1/d26/d4d/d153 0 2024-03-11T19:53:30.366 INFO:tasks.workunit.client.0.smithi090.stdout:7/988: creat d3/d55/d148/d8d/dab/db5/de1/dc6/d103/d105/f151 x:0 0 0 2024-03-11T19:53:30.455 INFO:tasks.workunit.client.0.smithi090.stdout:9/994: dread d1/dd/d18/d25/d101/f4a [0,4194304] 0 2024-03-11T19:53:30.526 INFO:tasks.workunit.client.0.smithi090.stdout:5/990: read d1/d26/d19/f42 [2666788,26865] 0 2024-03-11T19:53:30.622 INFO:tasks.workunit.client.0.smithi090.stdout:7/989: chown d3/dc/d1e/d7c/d96/d32/d7b/f128 56 1 2024-03-11T19:53:31.060 INFO:tasks.workunit.client.0.smithi090.stdout:2/975: creat d0/de/d12/f131 x:0 0 0 2024-03-11T19:53:31.879 INFO:tasks.workunit.client.0.smithi090.stdout:7/990: chown d3/dc/f33 0 1 2024-03-11T19:53:32.130 INFO:tasks.workunit.client.0.smithi090.stdout:3/941: dwrite d2/db/d15/d1a/d75/de8/d6a/dcb/f3b [0,4194304] 0 2024-03-11T19:53:32.293 INFO:tasks.workunit.client.0.smithi090.stdout:2/976: symlink d0/de/d1f/l132 0 2024-03-11T19:53:32.309 INFO:tasks.workunit.client.0.smithi090.stdout:9/995: mkdir d1/dd/d18/d25/d101/d1c/d21/dab/d114/d9c/df7/d14d 0 2024-03-11T19:53:32.416 INFO:tasks.workunit.client.0.smithi090.stdout:5/991: write d1/d17/d39/f11b [734149,13849] 0 2024-03-11T19:53:32.668 INFO:tasks.workunit.client.0.smithi090.stdout:5/992: dread d1/d17/d39/d51/d53/d7b/f44 [0,4194304] 0 2024-03-11T19:53:32.975 INFO:tasks.workunit.client.0.smithi090.stdout:5/993: symlink d1/d17/d39/df2/d12b/l154 0 2024-03-11T19:53:33.337 INFO:tasks.workunit.client.0.smithi090.stdout:2/977: write d0/dad/f101 [599216,83939] 0 2024-03-11T19:53:33.558 INFO:tasks.workunit.client.0.smithi090.stdout:3/942: link d2/l3e d2/db/d15/d69/d100/l130 0 2024-03-11T19:53:33.609 INFO:tasks.workunit.client.0.smithi090.stdout:7/991: dwrite d3/dc/fe [0,4194304] 0 2024-03-11T19:53:33.880 INFO:tasks.workunit.client.0.smithi090.stdout:5/994: write d1/d17/d39/d51/d53/d7b/d3b/d76/db7/ddf/dee/ff8 [562180,130634] 0 2024-03-11T19:53:33.893 INFO:tasks.workunit.client.0.smithi090.stdout:2/978: creat d0/de/d12/d46/d4c/d71/f133 x:0 0 0 2024-03-11T19:53:33.969 INFO:tasks.workunit.client.0.smithi090.stdout:3/943: sync 2024-03-11T19:53:34.055 INFO:tasks.workunit.client.0.smithi090.stdout:7/992: creat d3/dc/da6/d120/f152 x:0 0 0 2024-03-11T19:53:34.112 INFO:tasks.workunit.client.0.smithi090.stdout:2/979: mknod d0/de/dd3/c134 0 2024-03-11T19:53:34.174 INFO:tasks.workunit.client.0.smithi090.stdout:9/996: dwrite d1/d47/d7c/dee/f13a [0,4194304] 0 2024-03-11T19:53:34.249 INFO:tasks.workunit.client.0.smithi090.stdout:5/995: mkdir d1/d17/d39/d51/d53/d64/d155 0 2024-03-11T19:53:34.272 INFO:tasks.workunit.client.0.smithi090.stdout:3/944: dread - d2/db/d15/d1a/f111 zero size 2024-03-11T19:53:34.285 INFO:tasks.workunit.client.0.smithi090.stdout:9/997: sync 2024-03-11T19:53:34.370 INFO:tasks.workunit.client.0.smithi090.stdout:3/945: read - d2/db/d15/d1a/d75/de8/d6a/dcb/f10d zero size 2024-03-11T19:53:34.393 INFO:tasks.workunit.client.0.smithi090.stdout:2/980: creat d0/de/f135 x:0 0 0 2024-03-11T19:53:34.458 INFO:tasks.workunit.client.0.smithi090.stdout:7/993: read - d3/dc/d1e/d7c/d96/d32/d71/d80/f112 zero size 2024-03-11T19:53:34.467 INFO:tasks.workunit.client.0.smithi090.stdout:9/998: read d1/d1d/d6a/f108 [948163,23237] 0 2024-03-11T19:53:34.576 INFO:tasks.workunit.client.0.smithi090.stdout:3/946: symlink d2/d45/d55/l131 0 2024-03-11T19:53:34.603 INFO:tasks.workunit.client.0.smithi090.stdout:2/981: creat d0/de/db6/d29/f136 x:0 0 0 2024-03-11T19:53:34.608 INFO:tasks.workunit.client.0.smithi090.stdout:5/996: creat d1/d17/d39/d51/f156 x:0 0 0 2024-03-11T19:53:34.643 INFO:tasks.workunit.client.0.smithi090.stdout:9/999: symlink d1/dd/l14e 0 2024-03-11T19:53:34.794 INFO:tasks.workunit.client.0.smithi090.stdout:7/994: fdatasync d3/dc/d1e/d7c/d96/d32/d45/f131 0 2024-03-11T19:53:34.886 INFO:tasks.workunit.client.0.smithi090.stdout:3/947: creat d2/db/d15/d1a/d75/ded/f132 x:0 0 0 2024-03-11T19:53:34.917 INFO:tasks.workunit.client.0.smithi090.stdout:3/948: chown d2/d2f/dd5/f12f 118385 1 2024-03-11T19:53:35.130 INFO:tasks.workunit.client.0.smithi090.stdout:2/982: dread d0/de/d12/d46/d75/dc5/de5/f108 [0,4194304] 0 2024-03-11T19:53:35.349 INFO:tasks.workunit.client.0.smithi090.stdout:3/949: creat d2/d2f/dd5/f133 x:0 0 0 2024-03-11T19:53:35.376 INFO:tasks.workunit.client.0.smithi090.stdout:7/995: fdatasync d3/dc/d1e/d2a/f114 0 2024-03-11T19:53:35.504 INFO:tasks.workunit.client.0.smithi090.stdout:7/996: sync 2024-03-11T19:53:35.872 INFO:tasks.workunit.client.0.smithi090.stdout:2/983: dread d0/de/d12/f24 [0,4194304] 0 2024-03-11T19:53:35.965 INFO:tasks.workunit.client.0.smithi090.stdout:7/997: truncate d3/dc/da6/f102 189860 0 2024-03-11T19:53:36.496 INFO:tasks.workunit.client.0.smithi090.stdout:2/984: creat d0/de/d12/d46/d75/dc5/de5/f137 x:0 0 0 2024-03-11T19:53:36.628 INFO:tasks.workunit.client.0.smithi090.stdout:3/950: getdents d2/db/d15/d1a/d75/dc6/d114/d12b 0 2024-03-11T19:53:36.861 INFO:tasks.workunit.client.0.smithi090.stdout:5/997: write d1/d17/d39/d51/d53/d7b/d6f/f8e [2660430,91426] 0 2024-03-11T19:53:37.021 INFO:tasks.workunit.client.0.smithi090.stdout:7/998: fdatasync d3/d55/f122 0 2024-03-11T19:53:37.125 INFO:tasks.workunit.client.0.smithi090.stdout:3/951: rename d2/db/d15/d1a/d75/de8/d6a/cb2 to d2/db/d15/d69/da2/c134 0 2024-03-11T19:53:37.606 INFO:tasks.workunit.client.0.smithi090.stdout:5/998: creat d1/d17/d39/df2/d12b/f157 x:0 0 0 2024-03-11T19:53:38.034 INFO:tasks.workunit.client.0.smithi090.stdout:3/952: fsync d2/d45/df1/f12d 0 2024-03-11T19:53:38.064 INFO:tasks.workunit.client.0.smithi090.stdout:3/953: chown d2/fb9 26 1 2024-03-11T19:53:38.101 INFO:tasks.workunit.client.0.smithi090.stdout:3/954: stat d2/db/d15/d69/d119/d123/c20 0 2024-03-11T19:53:38.293 INFO:tasks.workunit.client.0.smithi090.stdout:5/999: truncate d1/d26/db5/ffa 55212 0 2024-03-11T19:53:38.680 INFO:tasks.workunit.client.0.smithi090.stdout:7/999: mkdir d3/dc/d1e/d7c/d96/d32/d153 0 2024-03-11T19:53:39.194 INFO:tasks.workunit.client.0.smithi090.stdout:2/985: write d0/de/f16 [1574215,23162] 0 2024-03-11T19:53:39.518 INFO:tasks.workunit.client.0.smithi090.stdout:3/955: dwrite d2/d4f/fe2 [0,4194304] 0 2024-03-11T19:53:39.643 INFO:tasks.workunit.client.0.smithi090.stdout:3/956: dread - d2/d2f/dd5/f133 zero size 2024-03-11T19:53:40.554 INFO:tasks.workunit.client.0.smithi090.stdout:2/986: dwrite d0/de/d12/d46/d4c/f119 [0,4194304] 0 2024-03-11T19:53:40.857 INFO:tasks.workunit.client.0.smithi090.stdout:3/957: link d2/d45/l4d d2/d4f/d59/l135 0 2024-03-11T19:53:40.862 INFO:tasks.workunit.client.0.smithi090.stdout:2/987: rmdir d0/de/d12/d46/d5d/d8d 39 2024-03-11T19:53:41.215 INFO:tasks.workunit.client.0.smithi090.stdout:2/988: chown d0/de/d12/d46/f98 28716719 1 2024-03-11T19:53:41.671 INFO:tasks.workunit.client.0.smithi090.stdout:2/989: creat d0/de/d12/d46/d75/f138 x:0 0 0 2024-03-11T19:53:41.675 INFO:tasks.workunit.client.0.smithi090.stdout:3/958: dwrite d2/d45/faa [0,4194304] 0 2024-03-11T19:53:42.044 INFO:tasks.workunit.client.0.smithi090.stdout:2/990: mknod d0/de/d12/d46/d5d/d8d/ddf/c139 0 2024-03-11T19:53:42.143 INFO:tasks.workunit.client.0.smithi090.stdout:3/959: getdents d2/db/d15/d69 0 2024-03-11T19:53:42.172 INFO:tasks.workunit.client.0.smithi090.stdout:3/960: chown d2/db/d15/d1a 9532 1 2024-03-11T19:53:42.359 INFO:tasks.workunit.client.0.smithi090.stdout:2/991: link d0/de/d12/f1a d0/de/d12/dc8/f13a 0 2024-03-11T19:53:42.364 INFO:tasks.workunit.client.0.smithi090.stdout:3/961: dread - d2/db/d15/d69/da2/dc7/f126 zero size 2024-03-11T19:53:42.504 INFO:tasks.workunit.client.0.smithi090.stdout:2/992: truncate d0/de/f17 566605 0 2024-03-11T19:53:42.706 INFO:tasks.workunit.client.0.smithi090.stdout:3/962: truncate d2/db/d15/d1a/f33 1353689 0 2024-03-11T19:53:42.849 INFO:tasks.workunit.client.0.smithi090.stdout:3/963: truncate d2/f67 1163840 0 2024-03-11T19:53:42.866 INFO:tasks.workunit.client.0.smithi090.stdout:2/993: creat d0/de/d12/d46/d75/f13b x:0 0 0 2024-03-11T19:53:42.961 INFO:tasks.workunit.client.0.smithi090.stdout:2/994: dread - d0/de/db6/d29/fca zero size 2024-03-11T19:53:43.024 INFO:tasks.workunit.client.0.smithi090.stdout:3/964: mknod d2/db/d15/d69/d119/d123/c136 0 2024-03-11T19:53:43.139 INFO:tasks.workunit.client.0.smithi090.stdout:2/995: symlink d0/de/d12/d46/d75/d112/d12c/l13c 0 2024-03-11T19:53:43.195 INFO:tasks.workunit.client.0.smithi090.stdout:3/965: fdatasync d2/db/d15/d1a/d75/ded/f132 0 2024-03-11T19:53:43.760 INFO:tasks.workunit.client.0.smithi090.stdout:3/966: creat d2/d2f/dd5/f137 x:0 0 0 2024-03-11T19:53:44.227 INFO:tasks.workunit.client.0.smithi090.stdout:2/996: link d0/fd d0/de/db6/d83/dbe/f13d 0 2024-03-11T19:53:44.450 INFO:tasks.workunit.client.0.smithi090.stdout:3/967: dread d2/f6 [0,4194304] 0 2024-03-11T19:53:44.656 INFO:tasks.workunit.client.0.smithi090.stdout:2/997: read d0/de/d12/dc8/f13a [174020,128722] 0 2024-03-11T19:53:45.134 INFO:tasks.workunit.client.0.smithi090.stdout:2/998: truncate d0/de/d12/d46/f48 1065929 0 2024-03-11T19:53:45.324 INFO:tasks.workunit.client.0.smithi090.stdout:2/999: mknod d0/de/d12/d46/d5d/d8d/c13e 0 2024-03-11T19:53:46.020 INFO:tasks.workunit.client.0.smithi090.stdout:3/968: dwrite d2/db/d15/d1a/d75/de8/d6a/f99 [0,4194304] 0 2024-03-11T19:53:46.351 INFO:tasks.workunit.client.0.smithi090.stdout:3/969: creat d2/d4f/f138 x:0 0 0 2024-03-11T19:53:46.395 INFO:tasks.workunit.client.0.smithi090.stdout:3/970: chown d2/db/d15/d49/cd1 8187304 1 2024-03-11T19:53:46.416 INFO:tasks.workunit.client.0.smithi090.stdout:3/971: readlink d2/d2f/dc4/lc9 0 2024-03-11T19:53:46.559 INFO:tasks.workunit.client.0.smithi090.stdout:3/972: creat d2/db/d15/d69/d119/f139 x:0 0 0 2024-03-11T19:53:46.664 INFO:tasks.workunit.client.0.smithi090.stdout:3/973: creat d2/d2f/f13a x:0 0 0 2024-03-11T19:53:46.913 INFO:tasks.workunit.client.0.smithi090.stdout:3/974: write d2/db/d15/d1a/d75/dc6/f27 [2004606,130518] 0 2024-03-11T19:53:46.998 INFO:tasks.workunit.client.0.smithi090.stdout:3/975: symlink d2/d2f/dd5/l13b 0 2024-03-11T19:53:47.202 INFO:tasks.workunit.client.0.smithi090.stdout:3/976: creat d2/db/d15/d1a/d75/de8/d6a/d9b/f13c x:0 0 0 2024-03-11T19:53:47.238 INFO:tasks.workunit.client.0.smithi090.stdout:3/977: chown d2/d4f/d59/da1/lcc 11 1 2024-03-11T19:53:47.354 INFO:tasks.workunit.client.0.smithi090.stdout:3/978: dread d2/d2f/f9c [0,4194304] 0 2024-03-11T19:53:47.487 INFO:tasks.workunit.client.0.smithi090.stdout:3/979: fsync d2/db/d15/d69/da2/dc7/ddd/fec 0 2024-03-11T19:53:47.640 INFO:tasks.workunit.client.0.smithi090.stdout:3/980: mknod d2/db/d15/d1a/d75/de8/d6a/d9b/c13d 0 2024-03-11T19:53:47.882 INFO:tasks.workunit.client.0.smithi090.stdout:3/981: creat d2/db/d15/d49/f13e x:0 0 0 2024-03-11T19:53:48.021 INFO:tasks.workunit.client.0.smithi090.stdout:3/982: unlink d2/d2f/l7b 0 2024-03-11T19:53:48.369 INFO:tasks.workunit.client.0.smithi090.stdout:3/983: dread d2/f9a [0,4194304] 0 2024-03-11T19:53:48.425 INFO:tasks.workunit.client.0.smithi090.stdout:3/984: chown d2/db/d15/d1a/d75/de8/d6a/dcb/lee 488173 1 2024-03-11T19:53:48.446 INFO:tasks.workunit.client.0.smithi090.stdout:3/985: chown d2/db/f1c 55384823 1 2024-03-11T19:53:48.750 INFO:tasks.workunit.client.0.smithi090.stdout:3/986: write d2/db/d15/f17 [2122666,98330] 0 2024-03-11T19:53:49.040 INFO:tasks.workunit.client.0.smithi090.stdout:3/987: write d2/db/d15/d69/da2/dc7/ddd/fec [845751,19004] 0 2024-03-11T19:53:49.087 INFO:tasks.workunit.client.0.smithi090.stdout:3/988: chown d2/db/d15/d49/fd7 1698951942 1 2024-03-11T19:53:49.329 INFO:tasks.workunit.client.0.smithi090.stdout:3/989: rename d2/db/d15/d1a to d2/d45/df1/d13f 0 2024-03-11T19:53:49.411 INFO:tasks.workunit.client.0.smithi090.stdout:3/990: truncate d2/db/d15/ff6 931429 0 2024-03-11T19:53:49.667 INFO:tasks.workunit.client.0.smithi090.stdout:3/991: rename d2/db/d15/f1e to d2/d45/df1/d13f/d75/ded/dfc/f140 0 2024-03-11T19:53:49.909 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:53:49.912 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:53:50.066 INFO:tasks.workunit.client.0.smithi090.stdout:3/992: write d2/ffd [2013711,109240] 0 2024-03-11T19:53:50.916 INFO:tasks.workunit.client.0.smithi090.stdout:3/993: write d2/d45/df1/d13f/d75/dc6/fb7 [843532,90161] 0 2024-03-11T19:53:51.023 INFO:tasks.workunit.client.0.smithi090.stdout:3/994: fsync d2/fb9 0 2024-03-11T19:53:51.281 INFO:tasks.workunit.client.0.smithi090.stdout:3/995: write d2/d45/fd0 [714559,45247] 0 2024-03-11T19:53:51.471 INFO:tasks.workunit.client.0.smithi090.stdout:3/996: dread d2/d45/df1/d13f/d75/dc6/f27 [0,4194304] 0 2024-03-11T19:53:51.560 INFO:tasks.workunit.client.0.smithi090.stdout:3/997: readlink d2/db/d15/le5 0 2024-03-11T19:53:51.662 INFO:tasks.workunit.client.0.smithi090.stdout:3/998: creat d2/d2f/dd5/f141 x:0 0 0 2024-03-11T19:53:52.004 INFO:tasks.workunit.client.0.smithi090.stdout:3/999: creat d2/d45/df1/d13f/d75/dc6/d114/d12b/f142 x:0 0 0 2024-03-11T19:53:52.029 INFO:tasks.workunit.client.0.smithi090.stderr:+ rm -rf -- ./tmp.t3HdW5tyEI 2024-03-11T19:54:20.046 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:54:20.049 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:54:50.099 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:54:50.103 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:55:20.219 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:55:20.224 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:55:50.330 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:55:50.334 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:56:20.562 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:56:20.566 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:56:50.618 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:56:50.621 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:56:52.589 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:22:12.351 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:56:52.602 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:22:12.363 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:56:52.632 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:22:12.329 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:56:52.639 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:22:12.329 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:56:52.648 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:22:12.346 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:56:52.660 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:22:12.350 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:56:52.839 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:22:12.529 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:56:52.843 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:22:12.541 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:56:52.869 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:22:12.632 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:57:20.697 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:57:20.701 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:57:50.761 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:57:50.765 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:57:53.949 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:23:13.713 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:57:53.958 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:23:13.722 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:57:54.061 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:23:13.826 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:57:54.063 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:23:13.762 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:57:54.073 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:23:13.771 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:57:54.077 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:23:13.768 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:57:54.083 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:23:13.774 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:57:54.147 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:23:13.846 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:57:54.174 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:23:13.865 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:58:20.885 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:58:20.888 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:58:51.004 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:58:51.007 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:59:21.115 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:59:21.118 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:59:30.016 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:24:49.782 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:59:30.024 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:24:49.790 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:59:30.129 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:24:49.828 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:59:30.132 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:24:49.832 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:59:30.142 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:24:49.833 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:59:30.144 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:24:49.836 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:59:30.186 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:24:49.878 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T19:59:30.206 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:24:49.906 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T19:59:30.212 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:24:49.979 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T19:59:51.210 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T19:59:51.213 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:00:21.346 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:00:21.349 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:00:51.398 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:00:51.401 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:00:59.482 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:26:19.250 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:00:59.491 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:26:19.259 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:00:59.522 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:26:19.223 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:00:59.524 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:26:19.217 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:00:59.536 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:26:19.237 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:00:59.536 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:26:19.229 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:00:59.617 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:26:19.385 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:00:59.650 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:26:19.350 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:00:59.691 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:26:19.384 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:01:21.509 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:01:21.512 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:01:51.595 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:01:51.598 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:02:21.664 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:02:21.668 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:02:51.718 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:02:51.722 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:03:21.770 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:03:21.773 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:03:51.907 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:03:51.910 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:04:22.001 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:04:22.004 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:04:52.139 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:04:52.142 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:05:06.819 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:30:26.590 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:05:06.841 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:30:26.613 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:05:06.869 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:30:26.572 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:05:06.872 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:30:26.567 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:05:06.882 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:30:26.585 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:05:06.890 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:30:26.586 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:05:07.011 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:30:26.783 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:05:07.061 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:30:26.756 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:05:07.099 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:30:26.802 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:05:22.182 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:05:22.185 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:05:48.706 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-11T20:05:48.706 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-11T20:05:52.237 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:05:52.241 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:06:17.330 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:31:37.102 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:06:17.334 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:31:37.107 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:06:17.399 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:31:37.172 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:06:17.449 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:31:37.153 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:06:17.450 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:31:37.146 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:06:17.451 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:31:37.155 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:06:17.456 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:31:37.152 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:06:17.497 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:31:37.201 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:06:17.525 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:31:37.221 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:06:22.306 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:06:22.309 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:06:52.401 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:06:52.404 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:07:22.479 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:07:22.482 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:07:52.548 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:07:52.551 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:07:53.270 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:33:13.043 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:07:53.273 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:33:13.047 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:07:53.340 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:33:13.113 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:07:53.385 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:33:13.089 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:07:53.386 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:33:13.091 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:07:53.402 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:33:13.099 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:07:53.406 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:33:13.103 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:07:53.478 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:33:13.182 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:07:53.492 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:33:13.188 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:08:22.610 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:08:22.613 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:08:52.667 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:08:52.671 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:09:14.707 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:34:34.481 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:09:14.720 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:34:34.494 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:09:14.755 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:34:34.452 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:09:14.760 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:34:34.465 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:09:14.762 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:34:34.459 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:09:14.773 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:34:34.478 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:09:14.857 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:34:34.631 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:09:14.871 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:34:34.576 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:09:14.874 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:34:34.571 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:09:22.724 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:09:22.727 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:09:52.845 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:09:52.849 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:10:23.081 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:10:23.084 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:10:53.130 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:10:53.133 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:11:23.262 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:11:23.265 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:11:53.349 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:11:53.352 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:12:23.427 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:12:23.430 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:12:53.480 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:12:53.483 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:13:23.608 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:13:23.612 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:13:23.978 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:38:43.754 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:13:23.993 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:38:43.769 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:13:24.026 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:38:43.732 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:13:24.028 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:38:43.726 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:13:24.036 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:38:43.742 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:13:24.049 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:38:43.747 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:13:24.133 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:38:43.909 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:13:24.212 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:38:43.918 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:13:24.247 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:38:43.945 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:13:53.762 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:13:53.765 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:14:23.823 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:14:23.826 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:14:40.568 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:40:00.345 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:14:40.670 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:40:00.447 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:14:40.691 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:40:00.397 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:14:40.697 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:40:00.403 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:14:40.762 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:40:00.539 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:14:40.777 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:40:00.483 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:14:40.783 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:40:00.481 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:14:40.786 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:40:00.485 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:14:40.856 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:40:00.554 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:14:53.869 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:14:53.871 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:15:23.929 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:15:23.932 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:15:54.067 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:15:54.070 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:16:16.239 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:41:36.016 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:16:16.248 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:41:36.025 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:16:16.376 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:41:36.083 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:16:16.381 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:41:36.088 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:16:16.440 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:41:36.147 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:16:16.451 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:41:36.229 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:16:16.504 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:41:36.203 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:16:16.505 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:41:36.204 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:16:16.559 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:41:36.257 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:16:24.187 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:16:24.191 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:16:54.315 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:16:54.318 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:17:24.392 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:17:24.395 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:17:30.465 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:42:50.242 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:17:30.475 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:42:50.253 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:17:30.495 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:42:50.202 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:17:30.507 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:42:50.207 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:17:30.514 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:42:50.222 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:17:30.523 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:42:50.222 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:17:30.553 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:42:50.331 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:17:30.628 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:42:50.335 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:17:30.694 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:42:50.393 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:17:54.439 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:17:54.442 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:18:24.600 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:18:24.603 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:18:54.703 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:18:54.706 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:19:24.789 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:19:24.796 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:19:54.944 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:19:54.947 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:20:24.995 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:20:24.998 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:20:55.058 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:20:55.061 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:21:25.111 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:21:25.114 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:21:39.668 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:46:59.446 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:21:39.674 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:46:59.453 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:21:39.706 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:46:59.415 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:21:39.717 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:46:59.418 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:21:39.717 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:46:59.426 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:21:39.728 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:46:59.429 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:21:39.876 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:46:59.655 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:21:39.893 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:46:59.602 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:21:39.926 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:46:59.627 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:21:55.232 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:21:55.236 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:22:25.310 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:22:25.313 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:22:55.599 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:22:55.602 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:22:58.771 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:48:18.550 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:22:58.891 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:48:18.670 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:22:58.900 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:48:18.609 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:22:58.910 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:48:18.620 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:22:58.991 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:48:18.692 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:22:59.003 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:48:18.704 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:22:59.013 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:48:18.722 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:22:59.018 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:48:18.796 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:22:59.106 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:48:18.807 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:23:25.651 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:23:25.656 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:23:55.779 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:23:55.782 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:24:25.819 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:24:25.823 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:24:39.593 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:49:59.372 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:24:39.610 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:49:59.389 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:24:39.707 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:49:59.486 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:24:39.710 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:49:59.420 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:24:39.717 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:49:59.427 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:24:39.719 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:49:59.421 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:24:39.728 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:49:59.429 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:24:39.801 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:49:59.503 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:24:39.809 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:49:59.518 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:24:55.876 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:24:55.879 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:25:25.997 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:25:26.000 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:25:45.540 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:51:05.320 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:25:45.562 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:51:05.341 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:25:45.582 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:51:05.292 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:25:45.611 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:51:05.321 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:25:45.619 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:51:05.321 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:25:45.629 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:51:05.332 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:25:45.635 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:51:05.415 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:25:45.720 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:51:05.430 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:25:45.756 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:51:05.458 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:25:56.049 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:25:56.051 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:26:26.102 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:26:26.106 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:26:56.161 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:26:56.164 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:27:26.235 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:27:26.238 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:27:56.380 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:27:56.383 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:28:26.424 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:28:26.428 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:28:56.520 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:28:56.523 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:29:26.607 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:29:26.611 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:29:54.711 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:55:14.489 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:29:54.721 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:55:14.499 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:29:54.742 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:55:14.443 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:29:54.747 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:55:14.456 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:29:54.749 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:55:14.451 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:29:54.776 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:55:14.486 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:29:54.894 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:55:14.673 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:29:54.953 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:55:14.654 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:29:55.027 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:55:14.736 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:29:56.720 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:29:56.722 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:30:26.841 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:30:26.845 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:30:56.969 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:30:56.972 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:31:22.296 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:56:42.072 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:31:22.299 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:56:42.076 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:31:22.368 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:56:42.145 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:31:22.409 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:56:42.108 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:31:22.410 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:56:42.110 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:31:22.415 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:56:42.123 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:31:22.425 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:56:42.132 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:31:22.470 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:56:42.170 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:31:22.528 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:56:42.235 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:31:27.098 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:31:27.102 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:31:57.143 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:31:57.146 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:32:27.224 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:32:27.227 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:32:57.277 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:32:57.280 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:32:58.218 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:58:17.992 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:32:58.224 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:58:17.999 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:32:58.305 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:58:18.080 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:32:58.338 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:58:18.044 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:32:58.340 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:58:18.038 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:32:58.341 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:58:18.047 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:32:58.353 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:58:18.051 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:32:58.416 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:58:18.114 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:32:58.433 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:58:18.140 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:33:27.411 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:33:27.415 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:33:57.542 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:33:57.546 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:34:01.382 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:59:21.155 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:34:01.393 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:59:21.167 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:34:01.429 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:59:21.134 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:34:01.431 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:59:21.128 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:34:01.442 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:59:21.136 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:34:01.442 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:59:21.144 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:34:01.530 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:01:59:21.227 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:34:01.539 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:01:59:21.313 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:34:01.543 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:01:59:21.248 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:34:27.751 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:34:27.755 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:34:57.869 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:34:57.872 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:35:28.004 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:35:28.008 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:35:58.142 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:35:58.145 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:36:28.288 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:36:28.291 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:36:58.408 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:36:58.411 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:37:28.595 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:37:28.599 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:37:58.649 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:37:58.652 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:38:09.856 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:02:03:29.626 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:38:09.892 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:02:03:29.663 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:38:09.919 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:02:03:29.613 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:38:09.921 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:02:03:29.624 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:38:09.926 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:02:03:29.620 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:38:09.929 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:02:03:29.632 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:38:10.090 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:02:03:29.792 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:38:10.127 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:02:03:29.822 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:38:10.249 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:02:03:30.020 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:38:29.003 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:38:29.006 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:38:59.072 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:38:59.075 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:39:29.118 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:39:29.123 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:39:41.240 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:02:05:01.010 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:39:41.244 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:02:05:01.014 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:39:41.349 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:02:05:01.042 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:39:41.352 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:02:05:01.046 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:39:41.353 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:02:05:01.054 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:39:41.358 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:02:05:01.127 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:39:41.362 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:02:05:01.064 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:39:41.430 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:02:05:01.123 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:39:41.438 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:02:05:01.140 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:39:59.191 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:39:59.195 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:40:29.317 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:40:29.320 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:40:59.456 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:40:59.460 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:41:22.040 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:02:06:41.809 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:41:22.044 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:02:06:41.813 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:41:22.138 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:02:06:41.907 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:41:22.154 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:02:06:41.855 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:41:22.160 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:02:06:41.861 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:41:22.173 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:02:06:41.865 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:41:22.178 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:02:06:41.871 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:41:22.229 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:02:06:41.922 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:41:22.244 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:02:06:41.945 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:41:29.609 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:41:29.612 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:41:59.712 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:41:59.715 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:42:17.567 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:02:07:37.335 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:42:17.573 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:02:07:37.342 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:42:17.609 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:02:07:37.310 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:42:17.614 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:02:07:37.307 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:42:17.621 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:02:07:37.314 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:42:17.621 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:02:07:37.322 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:42:17.650 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:02:07:37.419 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:42:17.694 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:02:07:37.384 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:42:17.722 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:02:07:37.423 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:42:29.782 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:42:29.786 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:42:59.835 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:42:59.839 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:43:29.947 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:43:29.950 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:44:00.076 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:44:00.079 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:44:30.129 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:44:30.133 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:45:00.196 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:45:00.198 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:45:30.245 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:45:30.248 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:46:00.610 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:46:00.613 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:46:26.336 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:02:11:46.103 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:46:26.362 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:02:11:46.130 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:46:26.381 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:02:11:46.073 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:46:26.386 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:02:11:46.086 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:46:26.389 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:02:11:46.081 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:46:26.398 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:02:11:46.098 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:46:26.557 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:02:11:46.324 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:46:26.568 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:02:11:46.268 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:46:26.568 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:02:11:46.260 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:46:30.651 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:46:30.654 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:47:00.764 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:47:00.767 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:47:30.834 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:47:30.837 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:48:00.887 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:48:00.890 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:48:04.499 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:02:13:24.266 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:48:04.501 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:02:13:24.268 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:48:04.563 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:02:13:24.329 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:48:04.620 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:02:13:24.319 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:48:04.622 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:02:13:24.322 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:48:04.623 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:02:13:24.314 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:48:04.625 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:02:13:24.317 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:48:04.679 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:02:13:24.378 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:48:04.700 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:02:13:24.391 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:48:31.007 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:48:31.010 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:49:01.079 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:49:01.082 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:49:31.200 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:49:31.202 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:49:40.290 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:02:15:00.057 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:49:40.297 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:02:15:00.064 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:49:40.348 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:02:15:00.115 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:49:40.412 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:02:15:00.111 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:49:40.414 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:02:15:00.105 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:49:40.415 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:02:15:00.114 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:49:40.424 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:02:15:00.115 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:49:40.461 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:02:15:00.160 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:49:40.482 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:02:15:00.173 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:50:01.319 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:50:01.323 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:50:31.372 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:50:31.375 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:50:32.901 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:02:15:52.668 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:50:32.921 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:02:15:52.687 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:50:32.948 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:02:15:52.647 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:50:32.952 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:02:15:52.643 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:50:32.955 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:02:15:52.655 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:50:32.966 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:02:15:52.658 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:50:33.023 INFO:tasks.ceph.mon.b.smithi092.stderr:==00:02:15:52.723 27638== Warning: unimplemented fcntl command: 1036 2024-03-11T20:50:33.025 INFO:tasks.ceph.mon.a.smithi090.stderr:==00:02:15:52.791 27658== Warning: unimplemented fcntl command: 1036 2024-03-11T20:50:33.075 INFO:tasks.ceph.mon.c.smithi092.stderr:==00:02:15:52.766 27637== Warning: unimplemented fcntl command: 1036 2024-03-11T20:51:01.444 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:51:01.447 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:51:19.220 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-03-11T20:51:19.221 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-11T20:51:19.542 DEBUG:teuthology.parallel:result is None 2024-03-11T20:51:19.542 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-11T20:51:20.128 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-11T20:51:20.128 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-03-11T20:51:20.138 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-03-11T20:51:20.139 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-11T20:51:20.139 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-11T20:51:20.145 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-11T20:51:20.145 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-11T20:51:20.199 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T20:51:20.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.372+0000 7f5807d1b640 1 Processor -- start 2024-03-11T20:51:20.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.372+0000 7f5807d1b640 1 -- start start 2024-03-11T20:51:20.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.376+0000 7f5807d1b640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f580015a950 0x7f5800165630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:20.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.376+0000 7f5807d1b640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5800159d80 con 0x7f5800153fc0 2024-03-11T20:51:20.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.376+0000 7f5807d1b640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58000c6e10 con 0x7f58000cc3a0 2024-03-11T20:51:20.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.376+0000 7f5807d1b640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58000c7990 con 0x7f580015a950 2024-03-11T20:51:20.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.376+0000 7f5805ab7640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f5800153fc0 0x7f580006bc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:47260/0 (socket says 172.21.15.90:47260) 2024-03-11T20:51:20.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.376+0000 7f5805ab7640 1 -- 172.21.15.90:0/4182994064 learned_addr learned my addr 172.21.15.90:0/4182994064 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T20:51:20.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.376+0000 7f58062b8640 1 --2- 172.21.15.90:0/4182994064 >> v2:172.21.15.90:3300/0 conn(0x7f580015a950 0x7f5800165630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:20.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.380+0000 7f58062b8640 1 -- 172.21.15.90:0/4182994064 >> v1:172.21.15.92:6789/0 conn(0x7f58000cc3a0 legacy=0x7f580015a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:20.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.380+0000 7f58062b8640 1 -- 172.21.15.90:0/4182994064 >> v1:172.21.15.90:6789/0 conn(0x7f5800153fc0 legacy=0x7f580006bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:20.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.380+0000 7f58062b8640 1 -- 172.21.15.90:0/4182994064 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58000c70f0 con 0x7f580015a950 2024-03-11T20:51:20.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.392+0000 7f58062b8640 1 --2- 172.21.15.90:0/4182994064 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f580015a950 0x7f5800165630 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f57f8009d90 tx=0x7f57f801f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a4889fdf7959c81 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:20.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.396+0000 7f5804ab5640 1 -- 172.21.15.90:0/4182994064 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f80af600 con 0x7f580015a950 2024-03-11T20:51:20.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.396+0000 7f5804ab5640 1 -- 172.21.15.90:0/4182994064 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f80ab050 con 0x7f580015a950 2024-03-11T20:51:20.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.396+0000 7f5804ab5640 1 -- 172.21.15.90:0/4182994064 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f80af980 con 0x7f580015a950 2024-03-11T20:51:20.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.400+0000 7f5807d1b640 1 -- 172.21.15.90:0/4182994064 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f580015a950 msgr2=0x7f5800165630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:20.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.400+0000 7f5807d1b640 1 --2- 172.21.15.90:0/4182994064 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f580015a950 0x7f5800165630 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f57f8009d90 tx=0x7f57f801f150 comp rx=0 tx=0).stop 2024-03-11T20:51:20.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.400+0000 7f5807d1b640 1 -- 172.21.15.90:0/4182994064 shutdown_connections 2024-03-11T20:51:20.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.400+0000 7f5807d1b640 1 --2- 172.21.15.90:0/4182994064 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f580015a950 0x7f5800165630 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:20.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.400+0000 7f5807d1b640 1 -- 172.21.15.90:0/4182994064 >> 172.21.15.90:0/4182994064 conn(0x7f58000cb880 msgr2=0x7f58000cbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:20.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.400+0000 7f5807d1b640 1 -- 172.21.15.90:0/4182994064 shutdown_connections 2024-03-11T20:51:20.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.400+0000 7f5807d1b640 1 -- 172.21.15.90:0/4182994064 wait complete. 2024-03-11T20:51:20.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.400+0000 7f5807d1b640 1 Processor -- start 2024-03-11T20:51:20.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.400+0000 7f5807d1b640 1 -- start start 2024-03-11T20:51:20.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.400+0000 7f5807d1b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58000cc3a0 0x7f58001492b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:20.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.400+0000 7f5807d1b640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f5800153fc0 0x7f5800071b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:20.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.400+0000 7f5807d1b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f580015a950 0x7f5800072090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:20.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.400+0000 7f5807d1b640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5800148bb0 con 0x7f580015a950 2024-03-11T20:51:20.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.400+0000 7f5807d1b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5800149e30 con 0x7f58000cc3a0 2024-03-11T20:51:20.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.400+0000 7f5807d1b640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f580014b330 con 0x7f5800153fc0 2024-03-11T20:51:20.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.400+0000 7f58062b8640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f580015a950 0x7f5800072090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:20.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.404+0000 7f58062b8640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f580015a950 0x7f5800072090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55950/0 (socket says 172.21.15.90:55950) 2024-03-11T20:51:20.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.404+0000 7f58062b8640 1 -- 172.21.15.90:0/2532729735 learned_addr learned my addr 172.21.15.90:0/2532729735 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:51:20.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.404+0000 7f58052b6640 1 --2- 172.21.15.90:0/2532729735 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f5800153fc0 0x7f5800071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:20.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.404+0000 7f5805ab7640 1 --2- 172.21.15.90:0/2532729735 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58000cc3a0 0x7f58001492b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:20.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.408+0000 7f58062b8640 1 -- 172.21.15.90:0/2532729735 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f5800153fc0 msgr2=0x7f5800071b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:20.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.408+0000 7f58062b8640 1 --2- 172.21.15.90:0/2532729735 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f5800153fc0 0x7f5800071b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:20.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.408+0000 7f58062b8640 1 -- 172.21.15.90:0/2532729735 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58000cc3a0 msgr2=0x7f58001492b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:20.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.408+0000 7f58062b8640 1 --2- 172.21.15.90:0/2532729735 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58000cc3a0 0x7f58001492b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:20.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.408+0000 7f58062b8640 1 -- 172.21.15.90:0/2532729735 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f8009a70 con 0x7f580015a950 2024-03-11T20:51:20.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.412+0000 7f58062b8640 1 --2- 172.21.15.90:0/2532729735 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f580015a950 0x7f5800072090 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f57f8009d90 tx=0x7f57f801bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:20.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.416+0000 7f57f67fc640 1 -- 172.21.15.90:0/2532729735 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f80b9410 con 0x7f580015a950 2024-03-11T20:51:20.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.416+0000 7f57f67fc640 1 -- 172.21.15.90:0/2532729735 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f80b9570 con 0x7f580015a950 2024-03-11T20:51:20.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.416+0000 7f5807d1b640 1 -- 172.21.15.90:0/2532729735 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58001488d0 con 0x7f580015a950 2024-03-11T20:51:20.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.416+0000 7f57f67fc640 1 -- 172.21.15.90:0/2532729735 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f80b9920 con 0x7f580015a950 2024-03-11T20:51:20.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.416+0000 7f5807d1b640 1 -- 172.21.15.90:0/2532729735 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58000c3cf0 con 0x7f580015a950 2024-03-11T20:51:20.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.416+0000 7f5807d1b640 1 -- 172.21.15.90:0/2532729735 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58000c3870 con 0x7f580015a950 2024-03-11T20:51:20.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.700+0000 7f57f67fc640 1 -- 172.21.15.90:0/2532729735 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f57f80b6050 con 0x7f580015a950 2024-03-11T20:51:20.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.700+0000 7f57f67fc640 1 --2- 172.21.15.90:0/2532729735 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f57e4066e90 0x7f57e4069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:20.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.700+0000 7f5805ab7640 1 --2- 172.21.15.90:0/2532729735 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f57e4066e90 0x7f57e4069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:20.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.704+0000 7f5805ab7640 1 --2- 172.21.15.90:0/2532729735 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f57e4066e90 0x7f57e4069350 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f57fc005890 tx=0x7f57fc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:20.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.740+0000 7f57f67fc640 1 -- 172.21.15.90:0/2532729735 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f57f80f4260 con 0x7f580015a950 2024-03-11T20:51:20.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.740+0000 7f57f67fc640 1 -- 172.21.15.90:0/2532729735 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f57f80f0540 con 0x7f580015a950 2024-03-11T20:51:20.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.912+0000 7f5807d1b640 1 -- 172.21.15.90:0/2532729735 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f58000c3870 con 0x7f580015a950 2024-03-11T20:51:20.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:20.996+0000 7f57f67fc640 1 -- 172.21.15.90:0/2532729735 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+18819 (secure 0 0 0) 0x7f58000c3870 con 0x7f580015a950 2024-03-11T20:51:20.996 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T20:51:20.996 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":30,"fsid":"76d56b10-6c29-476a-804b-8482ea90f9e2","created":"2024-03-11T18:34:54.521741+0000","modified":"2024-03-11T18:37:03.877940+0000","last_up_change":"2024-03-11T18:35:16.207500+0000","last_in_change":"2024-03-11T18:35:05.213634+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T18:35:16.193842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T18:35:54.001469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T18:35:55.917321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T18:36:02.210943+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"34decff9-7280-4d84-9665-d6af8ea6de84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6819","nonce":1755667921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6821","nonce":1755667921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6825","nonce":1755667921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6823","nonce":1755667921}]},"public_addr":"172.21.15.90:6819/1755667921","cluster_addr":"172.21.15.90:6821/1755667921","heartbeat_back_addr":"172.21.15.90:6825/1755667921","heartbeat_front_addr":"172.21.15.90:6823/1755667921","state":["exists","up"]},{"osd":1,"uuid":"8ecdce9e-5486-415a-a609-dd16e945440e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6807","nonce":438682081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6809","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6811","nonce":438682081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6817","nonce":438682081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6815","nonce":438682081}]},"public_addr":"172.21.15.90:6807/438682081","cluster_addr":"172.21.15.90:6811/438682081","heartbeat_back_addr":"172.21.15.90:6817/438682081","heartbeat_front_addr":"172.21.15.90:6815/438682081","state":["exists","up"]},{"osd":2,"uuid":"737486d4-0e90-4c82-9e79-bf37c812e5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6827","nonce":2837704738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6829","nonce":2837704738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6833","nonce":2837704738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6831","nonce":2837704738}]},"public_addr":"172.21.15.90:6827/2837704738","cluster_addr":"172.21.15.90:6829/2837704738","heartbeat_back_addr":"172.21.15.90:6833/2837704738","heartbeat_front_addr":"172.21.15.90:6831/2837704738","state":["exists","up"]},{"osd":3,"uuid":"698bd661-bbef-4699-a619-f7bfe23ce12c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6803","nonce":240987971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6805","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6806","nonce":240987971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6813","nonce":240987971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6810","nonce":240987971}]},"public_addr":"172.21.15.90:6803/240987971","cluster_addr":"172.21.15.90:6806/240987971","heartbeat_back_addr":"172.21.15.90:6813/240987971","heartbeat_front_addr":"172.21.15.90:6810/240987971","state":["exists","up"]},{"osd":4,"uuid":"aba9e31f-73d9-43fe-a10b-2dc5832968b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6801","nonce":502685275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6803","nonce":502685275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6807","nonce":502685275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6805","nonce":502685275}]},"public_addr":"172.21.15.92:6801/502685275","cluster_addr":"172.21.15.92:6803/502685275","heartbeat_back_addr":"172.21.15.92:6807/502685275","heartbeat_front_addr":"172.21.15.92:6805/502685275","state":["exists","up"]},{"osd":5,"uuid":"bdd56766-4b28-4007-8de3-b5b08cf335d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6809","nonce":200142070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6811","nonce":200142070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6815","nonce":200142070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6813","nonce":200142070}]},"public_addr":"172.21.15.92:6809/200142070","cluster_addr":"172.21.15.92:6811/200142070","heartbeat_back_addr":"172.21.15.92:6815/200142070","heartbeat_front_addr":"172.21.15.92:6813/200142070","state":["exists","up"]},{"osd":6,"uuid":"6c580a59-610f-455e-a96b-033299dcc118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6817","nonce":1919010204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6819","nonce":1919010204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6823","nonce":1919010204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6821","nonce":1919010204}]},"public_addr":"172.21.15.92:6817/1919010204","cluster_addr":"172.21.15.92:6819/1919010204","heartbeat_back_addr":"172.21.15.92:6823/1919010204","heartbeat_front_addr":"172.21.15.92:6821/1919010204","state":["exists","up"]},{"osd":7,"uuid":"958184cc-aebf-4c5b-91a0-2a3e38a6b42b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6825","nonce":3475925719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6827","nonce":3475925719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6831","nonce":3475925719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6829","nonce":3475925719}]},"public_addr":"172.21.15.92:6825/3475925719","cluster_addr":"172.21.15.92:6827/3475925719","heartbeat_back_addr":"172.21.15.92:6831/3475925719","heartbeat_front_addr":"172.21.15.92:6829/3475925719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.816294+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.048084+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.137975+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.968246+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.016829+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.931676+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.041673+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.079404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.33","mappings":[{"from":3,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T20:51:20.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.000+0000 7f5807d1b640 1 -- 172.21.15.90:0/2532729735 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f57e4066e90 msgr2=0x7f57e4069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:20.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.000+0000 7f5807d1b640 1 --2- 172.21.15.90:0/2532729735 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f57e4066e90 0x7f57e4069350 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f57fc005890 tx=0x7f57fc009040 comp rx=0 tx=0).stop 2024-03-11T20:51:20.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.000+0000 7f5807d1b640 1 -- 172.21.15.90:0/2532729735 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f580015a950 msgr2=0x7f5800072090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:21.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.000+0000 7f5807d1b640 1 --2- 172.21.15.90:0/2532729735 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f580015a950 0x7f5800072090 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f57f8009d90 tx=0x7f57f801bef0 comp rx=0 tx=0).stop 2024-03-11T20:51:21.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.000+0000 7f5807d1b640 1 -- 172.21.15.90:0/2532729735 shutdown_connections 2024-03-11T20:51:21.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.000+0000 7f5807d1b640 1 --2- 172.21.15.90:0/2532729735 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f57e4066e90 0x7f57e4069350 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:21.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.000+0000 7f5807d1b640 1 --2- 172.21.15.90:0/2532729735 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f580015a950 0x7f5800072090 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:21.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.000+0000 7f5807d1b640 1 --2- 172.21.15.90:0/2532729735 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f5800153fc0 0x7f5800071b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:21.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.000+0000 7f5807d1b640 1 --2- 172.21.15.90:0/2532729735 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f58000cc3a0 0x7f58001492b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:21.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.000+0000 7f5807d1b640 1 -- 172.21.15.90:0/2532729735 >> 172.21.15.90:0/2532729735 conn(0x7f58000cb880 msgr2=0x7f5800158d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:21.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.000+0000 7f5807d1b640 1 -- 172.21.15.90:0/2532729735 shutdown_connections 2024-03-11T20:51:21.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.000+0000 7f5807d1b640 1 -- 172.21.15.90:0/2532729735 wait complete. 2024-03-11T20:51:21.020 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-11T18:35:16.193842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-11T18:35:54.001469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-11T18:35:55.917321+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-11T18:36:02.210943+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-11T20:51:21.021 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-11T20:51:21.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.152+0000 7f41f57dd640 1 Processor -- start 2024-03-11T20:51:21.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.152+0000 7f41f57dd640 1 -- start start 2024-03-11T20:51:21.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.152+0000 7f41f57dd640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f41f0151b30 0x7f41f006e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:21.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.152+0000 7f41f57dd640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f41f0070b00 0x7f41f00c5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:21.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.152+0000 7f41f57dd640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41f006f430 con 0x7f41f00c4af0 2024-03-11T20:51:21.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.152+0000 7f41f57dd640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41f00c65b0 con 0x7f41f0151b30 2024-03-11T20:51:21.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.152+0000 7f41f57dd640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41f00c6320 con 0x7f41f0070b00 2024-03-11T20:51:21.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.168+0000 7f41ee7fc640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f41f0070b00 0x7f41f00c5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:21.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.172+0000 7f41ee7fc640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f41f0070b00 0x7f41f00c5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.90:46314/0 (socket says 172.21.15.90:46314) 2024-03-11T20:51:21.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.172+0000 7f41ee7fc640 1 -- 172.21.15.90:0/2021731027 learned_addr learned my addr 172.21.15.90:0/2021731027 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:51:21.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.208+0000 7f41eeffd640 1 --2- 172.21.15.90:0/2021731027 >> v2:172.21.15.90:3300/0 conn(0x7f41f0151b30 0x7f41f006e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:21.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.212+0000 7f41eeffd640 1 -- 172.21.15.90:0/2021731027 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f41f0070b00 msgr2=0x7f41f00c5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:21.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.212+0000 7f41eeffd640 1 --2- 172.21.15.90:0/2021731027 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f41f0070b00 0x7f41f00c5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:21.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.212+0000 7f41eeffd640 1 -- 172.21.15.90:0/2021731027 >> v1:172.21.15.90:6789/0 conn(0x7f41f00c4af0 legacy=0x7f41f0157c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:21.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.212+0000 7f41eeffd640 1 -- 172.21.15.90:0/2021731027 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f006e820 con 0x7f41f0151b30 2024-03-11T20:51:21.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.220+0000 7f41eeffd640 1 --2- 172.21.15.90:0/2021731027 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f41f0151b30 0x7f41f006e210 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f41e8009d90 tx=0x7f41e801c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e889e0cdc5e7e490 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:21.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.228+0000 7f41edffb640 1 -- 172.21.15.90:0/2021731027 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e80a5980 con 0x7f41f0151b30 2024-03-11T20:51:21.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.228+0000 7f41edffb640 1 -- 172.21.15.90:0/2021731027 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41e80a5ae0 con 0x7f41f0151b30 2024-03-11T20:51:21.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.228+0000 7f41edffb640 1 -- 172.21.15.90:0/2021731027 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e80a5e70 con 0x7f41f0151b30 2024-03-11T20:51:21.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.228+0000 7f41f57dd640 1 -- 172.21.15.90:0/2021731027 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f41f0151b30 msgr2=0x7f41f006e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:21.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.228+0000 7f41f57dd640 1 --2- 172.21.15.90:0/2021731027 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f41f0151b30 0x7f41f006e210 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f41e8009d90 tx=0x7f41e801c900 comp rx=0 tx=0).stop 2024-03-11T20:51:21.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.232+0000 7f41f57dd640 1 -- 172.21.15.90:0/2021731027 shutdown_connections 2024-03-11T20:51:21.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.232+0000 7f41f57dd640 1 --2- 172.21.15.90:0/2021731027 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f41f0070b00 0x7f41f00c5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:21.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.232+0000 7f41f57dd640 1 --2- 172.21.15.90:0/2021731027 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f41f0151b30 0x7f41f006e210 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:21.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.232+0000 7f41f57dd640 1 -- 172.21.15.90:0/2021731027 >> 172.21.15.90:0/2021731027 conn(0x7f41f006ac00 msgr2=0x7f41f006b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:21.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.232+0000 7f41f57dd640 1 -- 172.21.15.90:0/2021731027 shutdown_connections 2024-03-11T20:51:21.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.232+0000 7f41f57dd640 1 -- 172.21.15.90:0/2021731027 wait complete. 2024-03-11T20:51:21.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.232+0000 7f41f57dd640 1 Processor -- start 2024-03-11T20:51:21.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.232+0000 7f41f57dd640 1 -- start start 2024-03-11T20:51:21.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.236+0000 7f41f57dd640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f41f0070b00 0x7f41f00c9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:21.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.236+0000 7f41f57dd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41f00c4af0 0x7f41f014cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:21.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.236+0000 7f41f57dd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f41f0151b30 0x7f41f01f36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:21.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.236+0000 7f41f57dd640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f014d620 con 0x7f41f0151b30 2024-03-11T20:51:21.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.236+0000 7f41f57dd640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f014c0e0 con 0x7f41f00c4af0 2024-03-11T20:51:21.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.236+0000 7f41f57dd640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41f01f3bf0 con 0x7f41f0070b00 2024-03-11T20:51:21.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.236+0000 7f41ef7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f41f0151b30 0x7f41f01f36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:21.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.236+0000 7f41ef7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f41f0151b30 0x7f41f01f36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55974/0 (socket says 172.21.15.90:55974) 2024-03-11T20:51:21.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.236+0000 7f41ef7fe640 1 -- 172.21.15.90:0/3969954843 learned_addr learned my addr 172.21.15.90:0/3969954843 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:51:21.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.240+0000 7f41ef7fe640 1 -- 172.21.15.90:0/3969954843 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f41f0070b00 msgr2=0x7f41f00c9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:21.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.240+0000 7f41ef7fe640 1 --2- 172.21.15.90:0/3969954843 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f41f0070b00 0x7f41f00c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:21.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.240+0000 7f41ef7fe640 1 -- 172.21.15.90:0/3969954843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41f00c4af0 msgr2=0x7f41f014cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:21.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.240+0000 7f41ef7fe640 1 --2- 172.21.15.90:0/3969954843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41f00c4af0 0x7f41f014cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:21.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.240+0000 7f41ef7fe640 1 -- 172.21.15.90:0/3969954843 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e8009a70 con 0x7f41f0151b30 2024-03-11T20:51:21.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.244+0000 7f41ef7fe640 1 --2- 172.21.15.90:0/3969954843 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f41f0151b30 0x7f41f01f36b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f41e000c3c0 tx=0x7f41e000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:21.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.248+0000 7f41d77fe640 1 -- 172.21.15.90:0/3969954843 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e0011790 con 0x7f41f0151b30 2024-03-11T20:51:21.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.248+0000 7f41f57dd640 1 -- 172.21.15.90:0/3969954843 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41f0156e50 con 0x7f41f0151b30 2024-03-11T20:51:21.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.248+0000 7f41d77fe640 1 -- 172.21.15.90:0/3969954843 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41e00118f0 con 0x7f41f0151b30 2024-03-11T20:51:21.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.248+0000 7f41d77fe640 1 -- 172.21.15.90:0/3969954843 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e0011ca0 con 0x7f41f0151b30 2024-03-11T20:51:21.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.248+0000 7f41f57dd640 1 -- 172.21.15.90:0/3969954843 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41f014c2c0 con 0x7f41f0151b30 2024-03-11T20:51:21.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.248+0000 7f41f57dd640 1 -- 172.21.15.90:0/3969954843 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41f004eef0 con 0x7f41f0151b30 2024-03-11T20:51:21.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.272+0000 7f41d77fe640 1 -- 172.21.15.90:0/3969954843 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f41e001c460 con 0x7f41f0151b30 2024-03-11T20:51:21.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.272+0000 7f41d77fe640 1 --2- 172.21.15.90:0/3969954843 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f41dc066e90 0x7f41dc069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:21.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.272+0000 7f41eeffd640 1 --2- 172.21.15.90:0/3969954843 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f41dc066e90 0x7f41dc069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:21.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.272+0000 7f41eeffd640 1 --2- 172.21.15.90:0/3969954843 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f41dc066e90 0x7f41dc069350 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f41e80af000 tx=0x7f41e8004ac0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:21.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.276+0000 7f41d77fe640 1 -- 172.21.15.90:0/3969954843 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f41e0056670 con 0x7f41f0151b30 2024-03-11T20:51:21.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.564+0000 7f41d77fe640 1 -- 172.21.15.90:0/3969954843 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f41e0052210 con 0x7f41f0151b30 2024-03-11T20:51:21.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.740+0000 7f41f57dd640 1 -- 172.21.15.90:0/3969954843 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f41f00c8930 con 0x7f41f0151b30 2024-03-11T20:51:21.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.744+0000 7f41d77fe640 1 -- 172.21.15.90:0/3969954843 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f41e00558d0 con 0x7f41f0151b30 2024-03-11T20:51:21.745 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 1 2024-03-11T20:51:21.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.748+0000 7f41f57dd640 1 -- 172.21.15.90:0/3969954843 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f41dc066e90 msgr2=0x7f41dc069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:21.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.748+0000 7f41f57dd640 1 --2- 172.21.15.90:0/3969954843 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f41dc066e90 0x7f41dc069350 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f41e80af000 tx=0x7f41e8004ac0 comp rx=0 tx=0).stop 2024-03-11T20:51:21.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.748+0000 7f41f57dd640 1 -- 172.21.15.90:0/3969954843 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f41f0151b30 msgr2=0x7f41f01f36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:21.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.748+0000 7f41f57dd640 1 --2- 172.21.15.90:0/3969954843 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f41f0151b30 0x7f41f01f36b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f41e000c3c0 tx=0x7f41e000c880 comp rx=0 tx=0).stop 2024-03-11T20:51:21.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.748+0000 7f41f57dd640 1 -- 172.21.15.90:0/3969954843 shutdown_connections 2024-03-11T20:51:21.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.748+0000 7f41f57dd640 1 --2- 172.21.15.90:0/3969954843 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f41dc066e90 0x7f41dc069350 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:21.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.748+0000 7f41f57dd640 1 --2- 172.21.15.90:0/3969954843 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f41f0151b30 0x7f41f01f36b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:21.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.748+0000 7f41f57dd640 1 --2- 172.21.15.90:0/3969954843 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f41f00c4af0 0x7f41f014cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:21.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.748+0000 7f41f57dd640 1 --2- 172.21.15.90:0/3969954843 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f41f0070b00 0x7f41f00c9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:21.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.748+0000 7f41f57dd640 1 -- 172.21.15.90:0/3969954843 >> 172.21.15.90:0/3969954843 conn(0x7f41f006ac00 msgr2=0x7f41f01614c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:21.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.748+0000 7f41f57dd640 1 -- 172.21.15.90:0/3969954843 shutdown_connections 2024-03-11T20:51:21.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.748+0000 7f41f57dd640 1 -- 172.21.15.90:0/3969954843 wait complete. 2024-03-11T20:51:21.764 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-11T20:51:21.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.892+0000 7f4732f81640 1 Processor -- start 2024-03-11T20:51:21.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.892+0000 7f4732f81640 1 -- start start 2024-03-11T20:51:21.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.896+0000 7f4732f81640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f472c06b860 0x7f472c0c3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:21.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.896+0000 7f4732f81640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f472c0c7430 con 0x7f472c0c95f0 2024-03-11T20:51:21.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.896+0000 7f4732f81640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f472c152130 con 0x7f472c151d50 2024-03-11T20:51:21.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.896+0000 7f4732f81640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f472c0c7a40 con 0x7f472c06b860 2024-03-11T20:51:21.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.896+0000 7f473151e640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f472c0c95f0 0x7f472c0c5a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:47276/0 (socket says 172.21.15.90:47276) 2024-03-11T20:51:21.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.896+0000 7f473151e640 1 -- 172.21.15.90:0/2819612077 learned_addr learned my addr 172.21.15.90:0/2819612077 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T20:51:21.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.896+0000 7f472bfff640 1 --2- 172.21.15.90:0/2819612077 >> v2:172.21.15.92:3300/0 conn(0x7f472c06b860 0x7f472c0c3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:21.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.900+0000 7f472b7fe640 1 -- 172.21.15.90:0/2819612077 <== mon.0 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2144296742 0 0) 0x7f47240014a0 con 0x7f472c0c95f0 2024-03-11T20:51:21.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.900+0000 7f472b7fe640 1 -- 172.21.15.90:0/2819612077 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4718003530 con 0x7f472c0c95f0 2024-03-11T20:51:21.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.904+0000 7f472b7fe640 1 -- 172.21.15.90:0/2819612077 <== mon.0 v1:172.21.15.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4234773624 0 0) 0x7f4718003530 con 0x7f472c0c95f0 2024-03-11T20:51:21.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.904+0000 7f472b7fe640 1 -- 172.21.15.90:0/2819612077 >> v2:172.21.15.92:3300/0 conn(0x7f472c06b860 msgr2=0x7f472c0c3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:21.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.904+0000 7f472b7fe640 1 --2- 172.21.15.90:0/2819612077 >> v2:172.21.15.92:3300/0 conn(0x7f472c06b860 0x7f472c0c3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:21.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.904+0000 7f472b7fe640 1 -- 172.21.15.90:0/2819612077 >> v1:172.21.15.92:6789/0 conn(0x7f472c151d50 legacy=0x7f472c157970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:21.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.904+0000 7f472b7fe640 1 -- 172.21.15.90:0/2819612077 --> v1:172.21.15.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f472c0c6e70 con 0x7f472c0c95f0 2024-03-11T20:51:21.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.904+0000 7f472b7fe640 1 -- 172.21.15.90:0/2819612077 <== mon.0 v1:172.21.15.90:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 691735435 0 0) 0x7f4724001b90 con 0x7f472c0c95f0 2024-03-11T20:51:21.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.904+0000 7f472b7fe640 1 -- 172.21.15.90:0/2819612077 >> v1:172.21.15.90:6789/0 conn(0x7f472c0c95f0 legacy=0x7f472c0c5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:21.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.904+0000 7f472b7fe640 1 --2- 172.21.15.90:0/2819612077 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f47180039d0 0x7f4718003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:21.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.904+0000 7f472b7fe640 1 --2- 172.21.15.90:0/2819612077 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47180047d0 0x7f4718004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:21.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.904+0000 7f472b7fe640 1 --2- 172.21.15.90:0/2819612077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4718005190 0x7f471800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:21.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.904+0000 7f472b7fe640 1 -- 172.21.15.90:0/2819612077 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f472c0c7a40 con 0x7f47180047d0 2024-03-11T20:51:21.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.904+0000 7f472b7fe640 1 -- 172.21.15.90:0/2819612077 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471800c070 con 0x7f4718005190 2024-03-11T20:51:21.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.904+0000 7f472b7fe640 1 -- 172.21.15.90:0/2819612077 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f471800c230 con 0x7f47180039d0 2024-03-11T20:51:21.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.904+0000 7f4730d1d640 1 --2- 172.21.15.90:0/2819612077 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f47180039d0 0x7f4718003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:21.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.904+0000 7f472bfff640 1 --2- 172.21.15.90:0/2819612077 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47180047d0 0x7f4718004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:21.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.908+0000 7f472bfff640 1 -- 172.21.15.90:0/2819612077 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f47180039d0 msgr2=0x7f4718003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:21.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.908+0000 7f472bfff640 1 --2- 172.21.15.90:0/2819612077 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f47180039d0 0x7f4718003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:21.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.908+0000 7f472bfff640 1 -- 172.21.15.90:0/2819612077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4718005190 msgr2=0x7f471800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:21.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.908+0000 7f472bfff640 1 --2- 172.21.15.90:0/2819612077 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4718005190 0x7f471800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:21.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.908+0000 7f472bfff640 1 -- 172.21.15.90:0/2819612077 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4718003530 con 0x7f47180047d0 2024-03-11T20:51:21.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.908+0000 7f473151e640 1 -- 172.21.15.90:0/2819612077 reap_dead start 2024-03-11T20:51:21.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.912+0000 7f472bfff640 1 --2- 172.21.15.90:0/2819612077 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47180047d0 0x7f4718004c50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4714009ae0 tx=0x7f471401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78ddb03761431ac4 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:21.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.916+0000 7f472b7fe640 1 -- 172.21.15.90:0/2819612077 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47140a5660 con 0x7f47180047d0 2024-03-11T20:51:21.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.916+0000 7f472b7fe640 1 -- 172.21.15.90:0/2819612077 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47140a57c0 con 0x7f47180047d0 2024-03-11T20:51:21.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.916+0000 7f472b7fe640 1 -- 172.21.15.90:0/2819612077 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47140a5b70 con 0x7f47180047d0 2024-03-11T20:51:21.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.916+0000 7f4732f81640 1 -- 172.21.15.90:0/2819612077 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47180047d0 msgr2=0x7f4718004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:21.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.916+0000 7f4732f81640 1 --2- 172.21.15.90:0/2819612077 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47180047d0 0x7f4718004c50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4714009ae0 tx=0x7f471401ca70 comp rx=0 tx=0).stop 2024-03-11T20:51:21.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.916+0000 7f4732f81640 1 -- 172.21.15.90:0/2819612077 shutdown_connections 2024-03-11T20:51:21.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.916+0000 7f4732f81640 1 --2- 172.21.15.90:0/2819612077 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47180047d0 0x7f4718004c50 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:21.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.916+0000 7f4732f81640 1 -- 172.21.15.90:0/2819612077 >> 172.21.15.90:0/2819612077 conn(0x7f472c071aa0 msgr2=0x7f472c071eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:21.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.916+0000 7f4732f81640 1 -- 172.21.15.90:0/2819612077 shutdown_connections 2024-03-11T20:51:21.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.916+0000 7f4732f81640 1 -- 172.21.15.90:0/2819612077 wait complete. 2024-03-11T20:51:21.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.920+0000 7f4732f81640 1 Processor -- start 2024-03-11T20:51:21.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.920+0000 7f4732f81640 1 -- start start 2024-03-11T20:51:21.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.920+0000 7f4732f81640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47180047d0 0x7f472c149890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:21.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.920+0000 7f4732f81640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f472c1540e0 0x7f472c152c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:21.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.920+0000 7f4732f81640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f472c14b280 0x7f472c14b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:21.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.920+0000 7f4732f81640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f472c149dd0 con 0x7f47180047d0 2024-03-11T20:51:21.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.920+0000 7f4732f81640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f472c149360 con 0x7f472c14b280 2024-03-11T20:51:21.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.920+0000 7f4732f81640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f472c1f0910 con 0x7f472c1540e0 2024-03-11T20:51:21.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.920+0000 7f4730d1d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47180047d0 0x7f472c149890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:21.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.920+0000 7f4730d1d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47180047d0 0x7f472c149890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55984/0 (socket says 172.21.15.90:55984) 2024-03-11T20:51:21.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.920+0000 7f4730d1d640 1 -- 172.21.15.90:0/752775956 learned_addr learned my addr 172.21.15.90:0/752775956 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:51:21.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.924+0000 7f473151e640 1 --2- 172.21.15.90:0/752775956 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f472c14b280 0x7f472c14b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:21.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.924+0000 7f4730d1d640 1 -- 172.21.15.90:0/752775956 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f472c1540e0 msgr2=0x7f472c152c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:21.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.924+0000 7f4730d1d640 1 --2- 172.21.15.90:0/752775956 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f472c1540e0 0x7f472c152c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:21.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.924+0000 7f4730d1d640 1 -- 172.21.15.90:0/752775956 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f472c14b280 msgr2=0x7f472c14b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:21.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.924+0000 7f4730d1d640 1 --2- 172.21.15.90:0/752775956 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f472c14b280 0x7f472c14b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:21.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.924+0000 7f4730d1d640 1 -- 172.21.15.90:0/752775956 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f472c1495b0 con 0x7f47180047d0 2024-03-11T20:51:21.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.928+0000 7f4730d1d640 1 --2- 172.21.15.90:0/752775956 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47180047d0 0x7f472c149890 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f471c00f770 tx=0x7f471c00fc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:21.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.956+0000 7f47297fa640 1 -- 172.21.15.90:0/752775956 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471c001350 con 0x7f47180047d0 2024-03-11T20:51:21.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.956+0000 7f47297fa640 1 -- 172.21.15.90:0/752775956 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f471c0050d0 con 0x7f47180047d0 2024-03-11T20:51:21.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.956+0000 7f47297fa640 1 -- 172.21.15.90:0/752775956 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471c005480 con 0x7f47180047d0 2024-03-11T20:51:21.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.956+0000 7f4732f81640 1 -- 172.21.15.90:0/752775956 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f472c152410 con 0x7f47180047d0 2024-03-11T20:51:21.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.956+0000 7f4732f81640 1 -- 172.21.15.90:0/752775956 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f472c157740 con 0x7f47180047d0 2024-03-11T20:51:21.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.956+0000 7f4732f81640 1 -- 172.21.15.90:0/752775956 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f472c04eef0 con 0x7f47180047d0 2024-03-11T20:51:21.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.984+0000 7f47297fa640 1 -- 172.21.15.90:0/752775956 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f471c00d730 con 0x7f47180047d0 2024-03-11T20:51:21.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.984+0000 7f47297fa640 1 --2- 172.21.15.90:0/752775956 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f470c066e70 0x7f470c069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:21.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.984+0000 7f472bfff640 1 --2- 172.21.15.90:0/752775956 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f470c066e70 0x7f470c069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:21.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:21.984+0000 7f472bfff640 1 --2- 172.21.15.90:0/752775956 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f470c066e70 0x7f470c069330 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f47140af000 tx=0x7f4714008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:22.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.264+0000 7f47297fa640 1 -- 172.21.15.90:0/752775956 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f471c0565d0 con 0x7f47180047d0 2024-03-11T20:51:22.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.264+0000 7f47297fa640 1 -- 172.21.15.90:0/752775956 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f471c052170 con 0x7f47180047d0 2024-03-11T20:51:22.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.460+0000 7f4732f81640 1 -- 172.21.15.90:0/752775956 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f472c06fda0 con 0x7f47180047d0 2024-03-11T20:51:22.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.472+0000 7f47297fa640 1 -- 172.21.15.90:0/752775956 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f471c055830 con 0x7f47180047d0 2024-03-11T20:51:22.473 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 32 2024-03-11T20:51:22.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.476+0000 7f4732f81640 1 -- 172.21.15.90:0/752775956 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f470c066e70 msgr2=0x7f470c069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:22.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.476+0000 7f4732f81640 1 --2- 172.21.15.90:0/752775956 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f470c066e70 0x7f470c069330 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f47140af000 tx=0x7f4714008040 comp rx=0 tx=0).stop 2024-03-11T20:51:22.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.476+0000 7f4732f81640 1 -- 172.21.15.90:0/752775956 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47180047d0 msgr2=0x7f472c149890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:22.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.476+0000 7f4732f81640 1 --2- 172.21.15.90:0/752775956 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47180047d0 0x7f472c149890 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f471c00f770 tx=0x7f471c00fc30 comp rx=0 tx=0).stop 2024-03-11T20:51:22.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.476+0000 7f4732f81640 1 -- 172.21.15.90:0/752775956 shutdown_connections 2024-03-11T20:51:22.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.476+0000 7f4732f81640 1 --2- 172.21.15.90:0/752775956 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f470c066e70 0x7f470c069330 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:22.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.476+0000 7f4732f81640 1 --2- 172.21.15.90:0/752775956 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f472c14b280 0x7f472c14b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:22.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.476+0000 7f4732f81640 1 --2- 172.21.15.90:0/752775956 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f472c1540e0 0x7f472c152c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:22.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.476+0000 7f4732f81640 1 --2- 172.21.15.90:0/752775956 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f47180047d0 0x7f472c149890 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:22.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.476+0000 7f4732f81640 1 -- 172.21.15.90:0/752775956 >> 172.21.15.90:0/752775956 conn(0x7f472c071aa0 msgr2=0x7f472c14d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:22.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.476+0000 7f4732f81640 1 -- 172.21.15.90:0/752775956 shutdown_connections 2024-03-11T20:51:22.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.476+0000 7f4732f81640 1 -- 172.21.15.90:0/752775956 wait complete. 2024-03-11T20:51:22.493 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-11T20:51:22.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.620+0000 7f00f89ee640 1 Processor -- start 2024-03-11T20:51:22.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.620+0000 7f00f89ee640 1 -- start start 2024-03-11T20:51:22.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.624+0000 7f00f89ee640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f00f4153fc0 0x7f00f4159be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:22.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.624+0000 7f00f89ee640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00f41543a0 con 0x7f00f40c46f0 2024-03-11T20:51:22.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.624+0000 7f00f89ee640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00f415ab60 con 0x7f00f40cb840 2024-03-11T20:51:22.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.624+0000 7f00f89ee640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00f40b2dc0 con 0x7f00f4153fc0 2024-03-11T20:51:22.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.624+0000 7f00e9d9b640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f00f40cb840 0x7f00f406c3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.90:58226/0 (socket says 172.21.15.90:58226) 2024-03-11T20:51:22.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.624+0000 7f00e9d9b640 1 -- 172.21.15.90:0/2551252300 learned_addr learned my addr 172.21.15.90:0/2551252300 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T20:51:22.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.624+0000 7f00f259c640 1 --2- 172.21.15.90:0/2551252300 >> v2:172.21.15.92:3300/0 conn(0x7f00f4153fc0 0x7f00f4159be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:22.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.632+0000 7f00f1d9b640 1 -- 172.21.15.90:0/2551252300 <== mon.1 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 583458283 0 0) 0x7f00f415ab60 con 0x7f00f40cb840 2024-03-11T20:51:22.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.632+0000 7f00f1d9b640 1 -- 172.21.15.90:0/2551252300 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00e0003530 con 0x7f00f40cb840 2024-03-11T20:51:22.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.636+0000 7f00f1d9b640 1 -- 172.21.15.90:0/2551252300 <== mon.0 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1287742433 0 0) 0x7f00f41543a0 con 0x7f00f40c46f0 2024-03-11T20:51:22.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.636+0000 7f00f1d9b640 1 -- 172.21.15.90:0/2551252300 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00f415ab60 con 0x7f00f40c46f0 2024-03-11T20:51:22.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.636+0000 7f00f259c640 1 -- 172.21.15.90:0/2551252300 >> v1:172.21.15.92:6789/0 conn(0x7f00f40cb840 legacy=0x7f00f406c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:22.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.636+0000 7f00f259c640 1 -- 172.21.15.90:0/2551252300 >> v1:172.21.15.90:6789/0 conn(0x7f00f40c46f0 legacy=0x7f00f40cf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:22.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.636+0000 7f00f259c640 1 -- 172.21.15.90:0/2551252300 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f415a2e0 con 0x7f00f4153fc0 2024-03-11T20:51:22.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.644+0000 7f00f259c640 1 --2- 172.21.15.90:0/2551252300 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00f4153fc0 0x7f00f4159be0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f00e40097c0 tx=0x7f00e401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ac082b31e88b404 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:22.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.660+0000 7f00f1d9b640 1 -- 172.21.15.90:0/2551252300 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e40ac410 con 0x7f00f4153fc0 2024-03-11T20:51:22.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.660+0000 7f00f1d9b640 1 -- 172.21.15.90:0/2551252300 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00e40ac570 con 0x7f00f4153fc0 2024-03-11T20:51:22.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.660+0000 7f00f1d9b640 1 -- 172.21.15.90:0/2551252300 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e40ac900 con 0x7f00f4153fc0 2024-03-11T20:51:22.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.660+0000 7f00f89ee640 1 -- 172.21.15.90:0/2551252300 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00f4153fc0 msgr2=0x7f00f4159be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:22.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.660+0000 7f00f89ee640 1 --2- 172.21.15.90:0/2551252300 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00f4153fc0 0x7f00f4159be0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f00e40097c0 tx=0x7f00e401c4a0 comp rx=0 tx=0).stop 2024-03-11T20:51:22.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.660+0000 7f00f89ee640 1 -- 172.21.15.90:0/2551252300 shutdown_connections 2024-03-11T20:51:22.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.660+0000 7f00f89ee640 1 --2- 172.21.15.90:0/2551252300 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00f4153fc0 0x7f00f4159be0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:22.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.660+0000 7f00f89ee640 1 -- 172.21.15.90:0/2551252300 >> 172.21.15.90:0/2551252300 conn(0x7f00f4072730 msgr2=0x7f00f4072b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:22.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.660+0000 7f00f89ee640 1 -- 172.21.15.90:0/2551252300 shutdown_connections 2024-03-11T20:51:22.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.660+0000 7f00f89ee640 1 -- 172.21.15.90:0/2551252300 wait complete. 2024-03-11T20:51:22.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.664+0000 7f00f89ee640 1 Processor -- start 2024-03-11T20:51:22.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.664+0000 7f00f89ee640 1 -- start start 2024-03-11T20:51:22.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.664+0000 7f00f89ee640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f00f40c46f0 0x7f00f4152aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:22.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.664+0000 7f00f89ee640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00f40cb840 0x7f00f415b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:22.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.664+0000 7f00f89ee640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f00f4153fc0 0x7f00f415b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:22.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.664+0000 7f00f89ee640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f4141840 con 0x7f00f40c46f0 2024-03-11T20:51:22.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.664+0000 7f00f89ee640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f41439a0 con 0x7f00f40cb840 2024-03-11T20:51:22.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.664+0000 7f00f89ee640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00f4141a30 con 0x7f00f4153fc0 2024-03-11T20:51:22.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.664+0000 7f00f259c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f00f40c46f0 0x7f00f4152aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:22.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.664+0000 7f00f259c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f00f40c46f0 0x7f00f4152aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55990/0 (socket says 172.21.15.90:55990) 2024-03-11T20:51:22.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.664+0000 7f00f259c640 1 -- 172.21.15.90:0/1735681627 learned_addr learned my addr 172.21.15.90:0/1735681627 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:51:22.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.664+0000 7f00f2d9d640 1 --2- 172.21.15.90:0/1735681627 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f00f4153fc0 0x7f00f415b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:22.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.668+0000 7f00e9d9b640 1 --2- 172.21.15.90:0/1735681627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00f40cb840 0x7f00f415b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:22.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.668+0000 7f00f259c640 1 -- 172.21.15.90:0/1735681627 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f00f4153fc0 msgr2=0x7f00f415b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:22.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.668+0000 7f00f259c640 1 --2- 172.21.15.90:0/1735681627 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f00f4153fc0 0x7f00f415b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:22.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.668+0000 7f00f259c640 1 -- 172.21.15.90:0/1735681627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00f40cb840 msgr2=0x7f00f415b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:22.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.668+0000 7f00f259c640 1 --2- 172.21.15.90:0/1735681627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00f40cb840 0x7f00f415b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:22.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.668+0000 7f00f259c640 1 -- 172.21.15.90:0/1735681627 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00e4009420 con 0x7f00f40c46f0 2024-03-11T20:51:22.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.672+0000 7f00f259c640 1 --2- 172.21.15.90:0/1735681627 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f00f40c46f0 0x7f00f4152aa0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f00e401a0d0 tx=0x7f00e4019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:22.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.676+0000 7f00eb7fe640 1 -- 172.21.15.90:0/1735681627 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e40b5400 con 0x7f00f40c46f0 2024-03-11T20:51:22.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.676+0000 7f00eb7fe640 1 -- 172.21.15.90:0/1735681627 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00e40b55a0 con 0x7f00f40c46f0 2024-03-11T20:51:22.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.676+0000 7f00f89ee640 1 -- 172.21.15.90:0/1735681627 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00f41433e0 con 0x7f00f40c46f0 2024-03-11T20:51:22.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.676+0000 7f00eb7fe640 1 -- 172.21.15.90:0/1735681627 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e40b5950 con 0x7f00f40c46f0 2024-03-11T20:51:22.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.676+0000 7f00f89ee640 1 -- 172.21.15.90:0/1735681627 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00f415ade0 con 0x7f00f40c46f0 2024-03-11T20:51:22.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.676+0000 7f00f89ee640 1 -- 172.21.15.90:0/1735681627 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00f404eef0 con 0x7f00f40c46f0 2024-03-11T20:51:22.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.696+0000 7f00eb7fe640 1 -- 172.21.15.90:0/1735681627 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f00e40b3040 con 0x7f00f40c46f0 2024-03-11T20:51:22.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.696+0000 7f00eb7fe640 1 --2- 172.21.15.90:0/1735681627 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f00d4066ec0 0x7f00d4069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:22.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.696+0000 7f00e9d9b640 1 --2- 172.21.15.90:0/1735681627 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f00d4066ec0 0x7f00d4069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:22.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.700+0000 7f00e9d9b640 1 --2- 172.21.15.90:0/1735681627 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f00d4066ec0 0x7f00d4069380 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f00f40c5530 tx=0x7f00dc007d00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:22.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:22.700+0000 7f00eb7fe640 1 -- 172.21.15.90:0/1735681627 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f00e40ef570 con 0x7f00f40c46f0 2024-03-11T20:51:23.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.004+0000 7f00eb7fe640 1 -- 172.21.15.90:0/1735681627 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f00e40eb810 con 0x7f00f40c46f0 2024-03-11T20:51:23.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.180+0000 7f00f89ee640 1 -- 172.21.15.90:0/1735681627 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f00f414e050 con 0x7f00f40c46f0 2024-03-11T20:51:23.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.192+0000 7f00eb7fe640 1 -- 172.21.15.90:0/1735681627 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f00e40b1070 con 0x7f00f40c46f0 2024-03-11T20:51:23.191 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 64 2024-03-11T20:51:23.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.196+0000 7f00f89ee640 1 -- 172.21.15.90:0/1735681627 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f00d4066ec0 msgr2=0x7f00d4069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:23.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.196+0000 7f00f89ee640 1 --2- 172.21.15.90:0/1735681627 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f00d4066ec0 0x7f00d4069380 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f00f40c5530 tx=0x7f00dc007d00 comp rx=0 tx=0).stop 2024-03-11T20:51:23.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.196+0000 7f00f89ee640 1 -- 172.21.15.90:0/1735681627 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f00f40c46f0 msgr2=0x7f00f4152aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:23.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.196+0000 7f00f89ee640 1 --2- 172.21.15.90:0/1735681627 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f00f40c46f0 0x7f00f4152aa0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f00e401a0d0 tx=0x7f00e4019240 comp rx=0 tx=0).stop 2024-03-11T20:51:23.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.196+0000 7f00f89ee640 1 -- 172.21.15.90:0/1735681627 shutdown_connections 2024-03-11T20:51:23.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.196+0000 7f00f89ee640 1 --2- 172.21.15.90:0/1735681627 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f00d4066ec0 0x7f00d4069380 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:23.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.196+0000 7f00f89ee640 1 --2- 172.21.15.90:0/1735681627 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f00f4153fc0 0x7f00f415b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:23.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.196+0000 7f00f89ee640 1 --2- 172.21.15.90:0/1735681627 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f00f40cb840 0x7f00f415b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:23.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.196+0000 7f00f89ee640 1 --2- 172.21.15.90:0/1735681627 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f00f40c46f0 0x7f00f4152aa0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:23.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.196+0000 7f00f89ee640 1 -- 172.21.15.90:0/1735681627 >> 172.21.15.90:0/1735681627 conn(0x7f00f4072730 msgr2=0x7f00f40c5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:23.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.196+0000 7f00f89ee640 1 -- 172.21.15.90:0/1735681627 shutdown_connections 2024-03-11T20:51:23.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.196+0000 7f00f89ee640 1 -- 172.21.15.90:0/1735681627 wait complete. 2024-03-11T20:51:23.211 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-11T20:51:23.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.332+0000 7fd46a66a640 1 Processor -- start 2024-03-11T20:51:23.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.332+0000 7fd46a66a640 1 -- start start 2024-03-11T20:51:23.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.336+0000 7fd46a66a640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd4641509f0 0x7fd464157390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:23.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.336+0000 7fd46a66a640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fd464150dd0 0x7fd4640c28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:23.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.336+0000 7fd46a66a640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4640c5de0 con 0x7fd464070b00 2024-03-11T20:51:23.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.336+0000 7fd46a66a640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd464156910 con 0x7fd464150dd0 2024-03-11T20:51:23.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.336+0000 7fd46a66a640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4641578d0 con 0x7fd4641509f0 2024-03-11T20:51:23.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.336+0000 7fd45b7fe640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fd464150dd0 0x7fd4640c28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:23.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.336+0000 7fd45b7fe640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fd464150dd0 0x7fd4640c28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55992/0 (socket says 172.21.15.90:55992) 2024-03-11T20:51:23.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.336+0000 7fd45b7fe640 1 -- 172.21.15.90:0/512358157 learned_addr learned my addr 172.21.15.90:0/512358157 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:51:23.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.336+0000 7fd463fff640 1 --2- 172.21.15.90:0/512358157 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd4641509f0 0x7fd464157390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:23.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.340+0000 7fd45b7fe640 1 -- 172.21.15.90:0/512358157 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd4641509f0 msgr2=0x7fd464157390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:23.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.340+0000 7fd45b7fe640 1 --2- 172.21.15.90:0/512358157 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd4641509f0 0x7fd464157390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:23.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.340+0000 7fd45b7fe640 1 -- 172.21.15.90:0/512358157 >> v1:172.21.15.92:6789/0 conn(0x7fd464070b00 legacy=0x7fd4640c4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:23.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.340+0000 7fd45b7fe640 1 -- 172.21.15.90:0/512358157 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4640c6030 con 0x7fd464150dd0 2024-03-11T20:51:23.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.344+0000 7fd45b7fe640 1 --2- 172.21.15.90:0/512358157 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd464150dd0 0x7fd4640c28e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd44c009870 tx=0x7fd44c01ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba2da732adc439e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:23.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.348+0000 7fd4637fe640 1 -- 172.21.15.90:0/512358157 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c0a8a50 con 0x7fd464150dd0 2024-03-11T20:51:23.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.348+0000 7fd4637fe640 1 -- 172.21.15.90:0/512358157 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd44c0a8bb0 con 0x7fd464150dd0 2024-03-11T20:51:23.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.348+0000 7fd4637fe640 1 -- 172.21.15.90:0/512358157 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c0b0440 con 0x7fd464150dd0 2024-03-11T20:51:23.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.348+0000 7fd46a66a640 1 -- 172.21.15.90:0/512358157 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd464150dd0 msgr2=0x7fd4640c28e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:23.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.348+0000 7fd46a66a640 1 --2- 172.21.15.90:0/512358157 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd464150dd0 0x7fd4640c28e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd44c009870 tx=0x7fd44c01ed50 comp rx=0 tx=0).stop 2024-03-11T20:51:23.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.348+0000 7fd46a66a640 1 -- 172.21.15.90:0/512358157 shutdown_connections 2024-03-11T20:51:23.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.348+0000 7fd46a66a640 1 --2- 172.21.15.90:0/512358157 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd464150dd0 0x7fd4640c28e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:23.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.348+0000 7fd46a66a640 1 --2- 172.21.15.90:0/512358157 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd4641509f0 0x7fd464157390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:23.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.348+0000 7fd46a66a640 1 -- 172.21.15.90:0/512358157 >> 172.21.15.90:0/512358157 conn(0x7fd46406ac00 msgr2=0x7fd46406b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:23.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.348+0000 7fd46a66a640 1 -- 172.21.15.90:0/512358157 shutdown_connections 2024-03-11T20:51:23.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.348+0000 7fd46a66a640 1 -- 172.21.15.90:0/512358157 wait complete. 2024-03-11T20:51:23.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.352+0000 7fd46a66a640 1 Processor -- start 2024-03-11T20:51:23.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.352+0000 7fd46a66a640 1 -- start start 2024-03-11T20:51:23.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.352+0000 7fd46a66a640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd464070b00 0x7fd4641f0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:23.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.352+0000 7fd46a66a640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd4641509f0 0x7fd46414d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:23.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.352+0000 7fd46a66a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd464150dd0 0x7fd46414eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:23.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.352+0000 7fd46a66a640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd464142250 con 0x7fd464070b00 2024-03-11T20:51:23.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.352+0000 7fd46a66a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4641f0a30 con 0x7fd464150dd0 2024-03-11T20:51:23.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.352+0000 7fd46a66a640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd46413ff00 con 0x7fd4641509f0 2024-03-11T20:51:23.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.352+0000 7fd463fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd464070b00 0x7fd4641f0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:23.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.352+0000 7fd463fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd464070b00 0x7fd4641f0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55994/0 (socket says 172.21.15.90:55994) 2024-03-11T20:51:23.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.352+0000 7fd463fff640 1 -- 172.21.15.90:0/3170478789 learned_addr learned my addr 172.21.15.90:0/3170478789 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:51:23.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.352+0000 7fd45b7fe640 1 --2- 172.21.15.90:0/3170478789 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd4641509f0 0x7fd46414d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:23.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.352+0000 7fd468c07640 1 --2- 172.21.15.90:0/3170478789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd464150dd0 0x7fd46414eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:23.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.356+0000 7fd463fff640 1 -- 172.21.15.90:0/3170478789 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd4641509f0 msgr2=0x7fd46414d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:23.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.356+0000 7fd463fff640 1 --2- 172.21.15.90:0/3170478789 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd4641509f0 0x7fd46414d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:23.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.356+0000 7fd463fff640 1 -- 172.21.15.90:0/3170478789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd464150dd0 msgr2=0x7fd46414eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:23.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.356+0000 7fd463fff640 1 --2- 172.21.15.90:0/3170478789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd464150dd0 0x7fd46414eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:23.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.356+0000 7fd463fff640 1 -- 172.21.15.90:0/3170478789 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd44c0094d0 con 0x7fd464070b00 2024-03-11T20:51:23.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.360+0000 7fd463fff640 1 --2- 172.21.15.90:0/3170478789 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd464070b00 0x7fd4641f0e50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd45400e960 tx=0x7fd45400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:23.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.364+0000 7fd4617fa640 1 -- 172.21.15.90:0/3170478789 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd454009690 con 0x7fd464070b00 2024-03-11T20:51:23.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.364+0000 7fd4617fa640 1 -- 172.21.15.90:0/3170478789 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd45400ca00 con 0x7fd464070b00 2024-03-11T20:51:23.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.364+0000 7fd4617fa640 1 -- 172.21.15.90:0/3170478789 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45400cd70 con 0x7fd464070b00 2024-03-11T20:51:23.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.364+0000 7fd46a66a640 1 -- 172.21.15.90:0/3170478789 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd464140150 con 0x7fd464070b00 2024-03-11T20:51:23.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.364+0000 7fd46a66a640 1 -- 172.21.15.90:0/3170478789 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4641f0370 con 0x7fd464070b00 2024-03-11T20:51:23.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.364+0000 7fd46a66a640 1 -- 172.21.15.90:0/3170478789 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd46404eef0 con 0x7fd464070b00 2024-03-11T20:51:23.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.384+0000 7fd4617fa640 1 -- 172.21.15.90:0/3170478789 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fd454013470 con 0x7fd464070b00 2024-03-11T20:51:23.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.384+0000 7fd4617fa640 1 --2- 172.21.15.90:0/3170478789 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fd444066e70 0x7fd444069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:23.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.384+0000 7fd45b7fe640 1 --2- 172.21.15.90:0/3170478789 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fd444066e70 0x7fd444069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:23.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.384+0000 7fd45b7fe640 1 --2- 172.21.15.90:0/3170478789 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fd444066e70 0x7fd444069330 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd44c0b2000 tx=0x7fd44c0aa070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:23.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.388+0000 7fd4617fa640 1 -- 172.21.15.90:0/3170478789 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fd454055fc0 con 0x7fd464070b00 2024-03-11T20:51:23.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.644+0000 7fd4617fa640 1 -- 172.21.15.90:0/3170478789 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd454052260 con 0x7fd464070b00 2024-03-11T20:51:23.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.820+0000 7fd46a66a640 1 -- 172.21.15.90:0/3170478789 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd4640721d0 con 0x7fd464070b00 2024-03-11T20:51:23.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.848+0000 7fd4617fa640 1 -- 172.21.15.90:0/3170478789 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd4540210a0 con 0x7fd464070b00 2024-03-11T20:51:23.848 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 32 2024-03-11T20:51:23.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.852+0000 7fd46a66a640 1 -- 172.21.15.90:0/3170478789 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fd444066e70 msgr2=0x7fd444069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:23.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.852+0000 7fd46a66a640 1 --2- 172.21.15.90:0/3170478789 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fd444066e70 0x7fd444069330 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd44c0b2000 tx=0x7fd44c0aa070 comp rx=0 tx=0).stop 2024-03-11T20:51:23.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.852+0000 7fd46a66a640 1 -- 172.21.15.90:0/3170478789 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd464070b00 msgr2=0x7fd4641f0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:23.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.852+0000 7fd46a66a640 1 --2- 172.21.15.90:0/3170478789 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd464070b00 0x7fd4641f0e50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd45400e960 tx=0x7fd45400ee20 comp rx=0 tx=0).stop 2024-03-11T20:51:23.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.852+0000 7fd46a66a640 1 -- 172.21.15.90:0/3170478789 shutdown_connections 2024-03-11T20:51:23.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.852+0000 7fd46a66a640 1 --2- 172.21.15.90:0/3170478789 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fd444066e70 0x7fd444069330 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:23.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.852+0000 7fd46a66a640 1 --2- 172.21.15.90:0/3170478789 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd464150dd0 0x7fd46414eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:23.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.852+0000 7fd46a66a640 1 --2- 172.21.15.90:0/3170478789 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd4641509f0 0x7fd46414d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:23.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.852+0000 7fd46a66a640 1 --2- 172.21.15.90:0/3170478789 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd464070b00 0x7fd4641f0e50 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:23.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.852+0000 7fd46a66a640 1 -- 172.21.15.90:0/3170478789 >> 172.21.15.90:0/3170478789 conn(0x7fd46406ac00 msgr2=0x7fd46406e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:23.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.852+0000 7fd46a66a640 1 -- 172.21.15.90:0/3170478789 shutdown_connections 2024-03-11T20:51:23.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.852+0000 7fd46a66a640 1 -- 172.21.15.90:0/3170478789 wait complete. 2024-03-11T20:51:23.869 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-11T20:51:23.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.992+0000 7f6b0c370640 1 Processor -- start 2024-03-11T20:51:23.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.992+0000 7f6b0c370640 1 -- start start 2024-03-11T20:51:23.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.992+0000 7f6b0c370640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f6b040c7f20 0x7f6b04157ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:23.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.992+0000 7f6b0c370640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f6b04066d60 0x7f6b04158000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:23.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.992+0000 7f6b0c370640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b040c6720 con 0x7f6b04151c70 2024-03-11T20:51:23.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.992+0000 7f6b0c370640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b040cc7b0 con 0x7f6b040c7f20 2024-03-11T20:51:23.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.992+0000 7f6b0c370640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b040c5d20 con 0x7f6b04066d60 2024-03-11T20:51:23.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.992+0000 7f6b0a10c640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f6b04151c70 0x7f6b0406bc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:37614/0 (socket says 172.21.15.90:37614) 2024-03-11T20:51:23.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.992+0000 7f6b0a10c640 1 -- 172.21.15.90:0/1354227172 learned_addr learned my addr 172.21.15.90:0/1354227172 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T20:51:23.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.992+0000 7f6b0990b640 1 --2- 172.21.15.90:0/1354227172 >> v2:172.21.15.90:3300/0 conn(0x7f6b040c7f20 0x7f6b04157ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:23.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.996+0000 7f6b0910a640 1 -- 172.21.15.90:0/1354227172 <== mon.0 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2954449929 0 0) 0x7f6b040c6720 con 0x7f6b04151c70 2024-03-11T20:51:23.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:23.996+0000 7f6b0910a640 1 -- 172.21.15.90:0/1354227172 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6af0003530 con 0x7f6b04151c70 2024-03-11T20:51:23.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.000+0000 7f6b0990b640 1 -- 172.21.15.90:0/1354227172 >> v2:172.21.15.92:3300/0 conn(0x7f6b04066d60 msgr2=0x7f6b04158000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:23.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.000+0000 7f6b0990b640 1 --2- 172.21.15.90:0/1354227172 >> v2:172.21.15.92:3300/0 conn(0x7f6b04066d60 0x7f6b04158000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:23.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.000+0000 7f6b0990b640 1 -- 172.21.15.90:0/1354227172 >> v1:172.21.15.90:6789/0 conn(0x7f6b04151c70 legacy=0x7f6b0406bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:23.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.000+0000 7f6b0990b640 1 -- 172.21.15.90:0/1354227172 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b040c5a40 con 0x7f6b040c7f20 2024-03-11T20:51:24.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.008+0000 7f6b0990b640 1 --2- 172.21.15.90:0/1354227172 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6b040c7f20 0x7f6b04157ac0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6b040c8300 tx=0x7f6b0001c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26261d1d07eaebd0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:24.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.012+0000 7f6b0910a640 1 -- 172.21.15.90:0/1354227172 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b000a58d0 con 0x7f6b040c7f20 2024-03-11T20:51:24.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.012+0000 7f6b0910a640 1 -- 172.21.15.90:0/1354227172 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b000a5a30 con 0x7f6b040c7f20 2024-03-11T20:51:24.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.012+0000 7f6b0910a640 1 -- 172.21.15.90:0/1354227172 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b000a5dc0 con 0x7f6b040c7f20 2024-03-11T20:51:24.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.012+0000 7f6b0c370640 1 -- 172.21.15.90:0/1354227172 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6b040c7f20 msgr2=0x7f6b04157ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:24.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.012+0000 7f6b0c370640 1 --2- 172.21.15.90:0/1354227172 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6b040c7f20 0x7f6b04157ac0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6b040c8300 tx=0x7f6b0001c810 comp rx=0 tx=0).stop 2024-03-11T20:51:24.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.012+0000 7f6b0c370640 1 -- 172.21.15.90:0/1354227172 shutdown_connections 2024-03-11T20:51:24.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.012+0000 7f6b0c370640 1 --2- 172.21.15.90:0/1354227172 >> v2:172.21.15.92:3300/0 conn(0x7f6b04066d60 0x7f6b04158000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:24.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.012+0000 7f6b0c370640 1 --2- 172.21.15.90:0/1354227172 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6b040c7f20 0x7f6b04157ac0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:24.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.012+0000 7f6b0c370640 1 -- 172.21.15.90:0/1354227172 >> 172.21.15.90:0/1354227172 conn(0x7f6b040cb880 msgr2=0x7f6b040cbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:24.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.012+0000 7f6b0c370640 1 -- 172.21.15.90:0/1354227172 shutdown_connections 2024-03-11T20:51:24.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.012+0000 7f6b0c370640 1 -- 172.21.15.90:0/1354227172 wait complete. 2024-03-11T20:51:24.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.012+0000 7f6b0c370640 1 Processor -- start 2024-03-11T20:51:24.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.012+0000 7f6b0c370640 1 -- start start 2024-03-11T20:51:24.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.012+0000 7f6b0c370640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b04066d60 0x7f6b04159b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:24.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.012+0000 7f6b0c370640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6b040c7f20 0x7f6b041474b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:24.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.012+0000 7f6b0c370640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6b04151c70 0x7f6b041f0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:24.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.012+0000 7f6b0c370640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b04146990 con 0x7f6b04151c70 2024-03-11T20:51:24.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.012+0000 7f6b0c370640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0415a070 con 0x7f6b04066d60 2024-03-11T20:51:24.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.012+0000 7f6b0c370640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b04158b40 con 0x7f6b040c7f20 2024-03-11T20:51:24.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.016+0000 7f6b0a90d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6b04151c70 0x7f6b041f0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:24.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.016+0000 7f6b0a90d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6b04151c70 0x7f6b041f0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59874/0 (socket says 172.21.15.90:59874) 2024-03-11T20:51:24.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.016+0000 7f6b0a90d640 1 -- 172.21.15.90:0/242919136 learned_addr learned my addr 172.21.15.90:0/242919136 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:51:24.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.020+0000 7f6b0a90d640 1 -- 172.21.15.90:0/242919136 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6b040c7f20 msgr2=0x7f6b041474b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:24.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.020+0000 7f6b0a90d640 1 --2- 172.21.15.90:0/242919136 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6b040c7f20 0x7f6b041474b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:24.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.020+0000 7f6b0a90d640 1 -- 172.21.15.90:0/242919136 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b04066d60 msgr2=0x7f6b04159b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:24.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.020+0000 7f6b0a90d640 1 --2- 172.21.15.90:0/242919136 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b04066d60 0x7f6b04159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:24.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.020+0000 7f6b0a90d640 1 -- 172.21.15.90:0/242919136 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b00009a70 con 0x7f6b04151c70 2024-03-11T20:51:24.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.020+0000 7f6b0a90d640 1 --2- 172.21.15.90:0/242919136 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6b04151c70 0x7f6b041f0e00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6af800b410 tx=0x7f6af800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:24.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.024+0000 7f6af6ffd640 1 -- 172.21.15.90:0/242919136 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af8011830 con 0x7f6b04151c70 2024-03-11T20:51:24.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.024+0000 7f6af6ffd640 1 -- 172.21.15.90:0/242919136 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6af80119b0 con 0x7f6b04151c70 2024-03-11T20:51:24.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.024+0000 7f6af6ffd640 1 -- 172.21.15.90:0/242919136 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af8011d60 con 0x7f6b04151c70 2024-03-11T20:51:24.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.024+0000 7f6b0c370640 1 -- 172.21.15.90:0/242919136 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b04158e10 con 0x7f6b04151c70 2024-03-11T20:51:24.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.024+0000 7f6b0c370640 1 -- 172.21.15.90:0/242919136 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b04146060 con 0x7f6b04151c70 2024-03-11T20:51:24.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.028+0000 7f6b0c370640 1 -- 172.21.15.90:0/242919136 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b0406c2a0 con 0x7f6b04151c70 2024-03-11T20:51:24.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.316+0000 7f6af6ffd640 1 -- 172.21.15.90:0/242919136 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f6af801a400 con 0x7f6b04151c70 2024-03-11T20:51:24.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.316+0000 7f6af6ffd640 1 --2- 172.21.15.90:0/242919136 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f6ae8066e90 0x7f6ae8069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:24.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.316+0000 7f6b0a10c640 1 --2- 172.21.15.90:0/242919136 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f6ae8066e90 0x7f6ae8069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:24.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.316+0000 7f6b0a10c640 1 --2- 172.21.15.90:0/242919136 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f6ae8066e90 0x7f6ae8069350 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6afc007fd0 tx=0x7f6afc007900 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:24.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.348+0000 7f6af6ffd640 1 -- 172.21.15.90:0/242919136 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f6af8055de0 con 0x7f6b04151c70 2024-03-11T20:51:24.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.348+0000 7f6af6ffd640 1 -- 172.21.15.90:0/242919136 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6af8052080 con 0x7f6b04151c70 2024-03-11T20:51:24.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.528+0000 7f6b0c370640 1 -- 172.21.15.90:0/242919136 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6b04146ad0 con 0x7f6b04151c70 2024-03-11T20:51:24.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.536+0000 7f6af6ffd640 1 -- 172.21.15.90:0/242919136 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v30) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6af8015080 con 0x7f6b04151c70 2024-03-11T20:51:24.535 INFO:teuthology.orchestra.run.smithi090.stderr:listed 0 entries 2024-03-11T20:51:24.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.540+0000 7f6b0c370640 1 -- 172.21.15.90:0/242919136 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f6ae8066e90 msgr2=0x7f6ae8069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:24.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.540+0000 7f6b0c370640 1 --2- 172.21.15.90:0/242919136 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f6ae8066e90 0x7f6ae8069350 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6afc007fd0 tx=0x7f6afc007900 comp rx=0 tx=0).stop 2024-03-11T20:51:24.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.540+0000 7f6b0c370640 1 -- 172.21.15.90:0/242919136 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6b04151c70 msgr2=0x7f6b041f0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:24.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.540+0000 7f6b0c370640 1 --2- 172.21.15.90:0/242919136 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6b04151c70 0x7f6b041f0e00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6af800b410 tx=0x7f6af800b8d0 comp rx=0 tx=0).stop 2024-03-11T20:51:24.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.540+0000 7f6b0c370640 1 -- 172.21.15.90:0/242919136 shutdown_connections 2024-03-11T20:51:24.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.540+0000 7f6b0c370640 1 --2- 172.21.15.90:0/242919136 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f6ae8066e90 0x7f6ae8069350 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:24.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.540+0000 7f6b0c370640 1 --2- 172.21.15.90:0/242919136 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6b04151c70 0x7f6b041f0e00 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:24.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.540+0000 7f6b0c370640 1 --2- 172.21.15.90:0/242919136 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f6b040c7f20 0x7f6b041474b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:24.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.540+0000 7f6b0c370640 1 --2- 172.21.15.90:0/242919136 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f6b04066d60 0x7f6b04159b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:24.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.540+0000 7f6b0c370640 1 -- 172.21.15.90:0/242919136 >> 172.21.15.90:0/242919136 conn(0x7f6b040cb880 msgr2=0x7f6b041579e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:24.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.540+0000 7f6b0c370640 1 -- 172.21.15.90:0/242919136 shutdown_connections 2024-03-11T20:51:24.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.540+0000 7f6b0c370640 1 -- 172.21.15.90:0/242919136 wait complete. 2024-03-11T20:51:24.555 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-11T20:51:24.555 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-11T20:51:24.561 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T20:51:24.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.732+0000 7f793783d640 1 Processor -- start 2024-03-11T20:51:24.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.732+0000 7f793783d640 1 -- start start 2024-03-11T20:51:24.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.732+0000 7f793783d640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f7930153cf0 0x7f793015d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:24.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.732+0000 7f793783d640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f79300c4b60 0x7f793015a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:24.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.732+0000 7f793783d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f79300c42f0 0x7f79300ca370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:24.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.732+0000 7f793783d640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f793015e120 con 0x7f7930153cf0 2024-03-11T20:51:24.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.732+0000 7f793783d640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7930159b30 con 0x7f79300c42f0 2024-03-11T20:51:24.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.732+0000 7f793783d640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f793015d660 con 0x7f79300c4b60 2024-03-11T20:51:24.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.732+0000 7f79355d9640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f7930153cf0 0x7f793015d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:24.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.736+0000 7f79355d9640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f7930153cf0 0x7f793015d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59890/0 (socket says 172.21.15.90:59890) 2024-03-11T20:51:24.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.736+0000 7f79355d9640 1 -- 172.21.15.90:0/2393299649 learned_addr learned my addr 172.21.15.90:0/2393299649 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:51:24.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.736+0000 7f7934dd8640 1 --2- 172.21.15.90:0/2393299649 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f79300c4b60 0x7f793015a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:24.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.736+0000 7f7935dda640 1 --2- 172.21.15.90:0/2393299649 >> v2:172.21.15.92:3300/0 conn(0x7f79300c42f0 0x7f79300ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:24.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.736+0000 7f79355d9640 1 -- 172.21.15.90:0/2393299649 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f79300c4b60 msgr2=0x7f793015a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:24.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.736+0000 7f79355d9640 1 --2- 172.21.15.90:0/2393299649 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f79300c4b60 0x7f793015a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:24.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.736+0000 7f79355d9640 1 -- 172.21.15.90:0/2393299649 >> v2:172.21.15.92:3300/0 conn(0x7f79300c42f0 msgr2=0x7f79300ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:24.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.736+0000 7f79355d9640 1 --2- 172.21.15.90:0/2393299649 >> v2:172.21.15.92:3300/0 conn(0x7f79300c42f0 0x7f79300ca370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:24.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.736+0000 7f79355d9640 1 -- 172.21.15.90:0/2393299649 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f793015a0b0 con 0x7f7930153cf0 2024-03-11T20:51:24.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.740+0000 7f79355d9640 1 --2- 172.21.15.90:0/2393299649 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7930153cf0 0x7f793015d120 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f793008caa0 tx=0x7f792c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53779178d0025431 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:24.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.748+0000 7f7927fff640 1 -- 172.21.15.90:0/2393299649 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792c0aa640 con 0x7f7930153cf0 2024-03-11T20:51:24.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.748+0000 7f7927fff640 1 -- 172.21.15.90:0/2393299649 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f792c0aa7a0 con 0x7f7930153cf0 2024-03-11T20:51:24.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.748+0000 7f7927fff640 1 -- 172.21.15.90:0/2393299649 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792c0aab30 con 0x7f7930153cf0 2024-03-11T20:51:24.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.748+0000 7f793783d640 1 -- 172.21.15.90:0/2393299649 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7930153cf0 msgr2=0x7f793015d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:24.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.748+0000 7f793783d640 1 --2- 172.21.15.90:0/2393299649 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7930153cf0 0x7f793015d120 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f793008caa0 tx=0x7f792c01c520 comp rx=0 tx=0).stop 2024-03-11T20:51:24.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.748+0000 7f793783d640 1 -- 172.21.15.90:0/2393299649 shutdown_connections 2024-03-11T20:51:24.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.748+0000 7f793783d640 1 --2- 172.21.15.90:0/2393299649 >> v2:172.21.15.92:3300/0 conn(0x7f79300c42f0 0x7f79300ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:24.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.748+0000 7f793783d640 1 --2- 172.21.15.90:0/2393299649 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f79300c4b60 0x7f793015a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:24.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.748+0000 7f793783d640 1 --2- 172.21.15.90:0/2393299649 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7930153cf0 0x7f793015d120 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:24.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.748+0000 7f793783d640 1 -- 172.21.15.90:0/2393299649 >> 172.21.15.90:0/2393299649 conn(0x7f793006b640 msgr2=0x7f793006ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:24.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.748+0000 7f793783d640 1 -- 172.21.15.90:0/2393299649 shutdown_connections 2024-03-11T20:51:24.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.748+0000 7f793783d640 1 -- 172.21.15.90:0/2393299649 wait complete. 2024-03-11T20:51:24.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.748+0000 7f793783d640 1 Processor -- start 2024-03-11T20:51:24.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.748+0000 7f793783d640 1 -- start start 2024-03-11T20:51:24.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.748+0000 7f793783d640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f79300c42f0 0x7f793014f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:24.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.748+0000 7f793783d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79300c4b60 0x7f79301f6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:24.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.748+0000 7f793783d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7930153cf0 0x7f79300b4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:24.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.748+0000 7f793783d640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793014f840 con 0x7f7930153cf0 2024-03-11T20:51:24.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.748+0000 7f793783d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79301f2810 con 0x7f79300c4b60 2024-03-11T20:51:24.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.748+0000 7f793783d640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79301f37e0 con 0x7f79300c42f0 2024-03-11T20:51:24.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.748+0000 7f7935dda640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7930153cf0 0x7f79300b4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:24.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.752+0000 7f7935dda640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7930153cf0 0x7f79300b4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59894/0 (socket says 172.21.15.90:59894) 2024-03-11T20:51:24.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.752+0000 7f7935dda640 1 -- 172.21.15.90:0/2416110157 learned_addr learned my addr 172.21.15.90:0/2416110157 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:51:24.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.752+0000 7f79355d9640 1 --2- 172.21.15.90:0/2416110157 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f79300c42f0 0x7f793014f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:24.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.752+0000 7f7934dd8640 1 --2- 172.21.15.90:0/2416110157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79300c4b60 0x7f79301f6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:24.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.752+0000 7f7935dda640 1 -- 172.21.15.90:0/2416110157 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f79300c42f0 msgr2=0x7f793014f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:24.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.752+0000 7f7935dda640 1 --2- 172.21.15.90:0/2416110157 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f79300c42f0 0x7f793014f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:24.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.752+0000 7f7935dda640 1 -- 172.21.15.90:0/2416110157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79300c4b60 msgr2=0x7f79301f6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:24.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.752+0000 7f7935dda640 1 --2- 172.21.15.90:0/2416110157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79300c4b60 0x7f79301f6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:24.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.752+0000 7f7935dda640 1 -- 172.21.15.90:0/2416110157 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f792c0094d0 con 0x7f7930153cf0 2024-03-11T20:51:24.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.756+0000 7f7935dda640 1 --2- 172.21.15.90:0/2416110157 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7930153cf0 0x7f79300b4570 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7928007e70 tx=0x7f792800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:24.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.760+0000 7f7925ffb640 1 -- 172.21.15.90:0/2416110157 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7928014620 con 0x7f7930153cf0 2024-03-11T20:51:24.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.760+0000 7f7925ffb640 1 -- 172.21.15.90:0/2416110157 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79280147a0 con 0x7f7930153cf0 2024-03-11T20:51:24.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.760+0000 7f793783d640 1 -- 172.21.15.90:0/2416110157 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79301f6cd0 con 0x7f7930153cf0 2024-03-11T20:51:24.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.760+0000 7f7925ffb640 1 -- 172.21.15.90:0/2416110157 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7928014b50 con 0x7f7930153cf0 2024-03-11T20:51:24.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.760+0000 7f793783d640 1 -- 172.21.15.90:0/2416110157 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f793014fa20 con 0x7f7930153cf0 2024-03-11T20:51:24.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:24.760+0000 7f793783d640 1 -- 172.21.15.90:0/2416110157 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f793004eef0 con 0x7f7930153cf0 2024-03-11T20:51:25.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.044+0000 7f7925ffb640 1 -- 172.21.15.90:0/2416110157 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f7928014cb0 con 0x7f7930153cf0 2024-03-11T20:51:25.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.048+0000 7f7925ffb640 1 --2- 172.21.15.90:0/2416110157 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f7914066ec0 0x7f7914069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:25.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.048+0000 7f79355d9640 1 --2- 172.21.15.90:0/2416110157 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f7914066ec0 0x7f7914069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:25.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.048+0000 7f79355d9640 1 --2- 172.21.15.90:0/2416110157 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f7914066ec0 0x7f7914069380 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f792c0ae000 tx=0x7f792c0b5040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:25.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.124+0000 7f7925ffb640 1 -- 172.21.15.90:0/2416110157 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f7928056bf0 con 0x7f7930153cf0 2024-03-11T20:51:25.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.124+0000 7f7925ffb640 1 -- 172.21.15.90:0/2416110157 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7928052e90 con 0x7f7930153cf0 2024-03-11T20:51:25.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.316+0000 7f793783d640 1 -- 172.21.15.90:0/2416110157 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79301f6cd0 con 0x7f7930153cf0 2024-03-11T20:51:25.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.368+0000 7f7925ffb640 1 -- 172.21.15.90:0/2416110157 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+18819 (secure 0 0 0) 0x7f79301f6cd0 con 0x7f7930153cf0 2024-03-11T20:51:25.368 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T20:51:25.368 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":30,"fsid":"76d56b10-6c29-476a-804b-8482ea90f9e2","created":"2024-03-11T18:34:54.521741+0000","modified":"2024-03-11T18:37:03.877940+0000","last_up_change":"2024-03-11T18:35:16.207500+0000","last_in_change":"2024-03-11T18:35:05.213634+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T18:35:16.193842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T18:35:54.001469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T18:35:55.917321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T18:36:02.210943+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"34decff9-7280-4d84-9665-d6af8ea6de84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6819","nonce":1755667921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6821","nonce":1755667921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6825","nonce":1755667921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6823","nonce":1755667921}]},"public_addr":"172.21.15.90:6819/1755667921","cluster_addr":"172.21.15.90:6821/1755667921","heartbeat_back_addr":"172.21.15.90:6825/1755667921","heartbeat_front_addr":"172.21.15.90:6823/1755667921","state":["exists","up"]},{"osd":1,"uuid":"8ecdce9e-5486-415a-a609-dd16e945440e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6807","nonce":438682081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6809","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6811","nonce":438682081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6817","nonce":438682081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6815","nonce":438682081}]},"public_addr":"172.21.15.90:6807/438682081","cluster_addr":"172.21.15.90:6811/438682081","heartbeat_back_addr":"172.21.15.90:6817/438682081","heartbeat_front_addr":"172.21.15.90:6815/438682081","state":["exists","up"]},{"osd":2,"uuid":"737486d4-0e90-4c82-9e79-bf37c812e5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6827","nonce":2837704738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6829","nonce":2837704738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6833","nonce":2837704738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6831","nonce":2837704738}]},"public_addr":"172.21.15.90:6827/2837704738","cluster_addr":"172.21.15.90:6829/2837704738","heartbeat_back_addr":"172.21.15.90:6833/2837704738","heartbeat_front_addr":"172.21.15.90:6831/2837704738","state":["exists","up"]},{"osd":3,"uuid":"698bd661-bbef-4699-a619-f7bfe23ce12c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6803","nonce":240987971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6805","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6806","nonce":240987971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6813","nonce":240987971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6810","nonce":240987971}]},"public_addr":"172.21.15.90:6803/240987971","cluster_addr":"172.21.15.90:6806/240987971","heartbeat_back_addr":"172.21.15.90:6813/240987971","heartbeat_front_addr":"172.21.15.90:6810/240987971","state":["exists","up"]},{"osd":4,"uuid":"aba9e31f-73d9-43fe-a10b-2dc5832968b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6801","nonce":502685275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6803","nonce":502685275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6807","nonce":502685275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6805","nonce":502685275}]},"public_addr":"172.21.15.92:6801/502685275","cluster_addr":"172.21.15.92:6803/502685275","heartbeat_back_addr":"172.21.15.92:6807/502685275","heartbeat_front_addr":"172.21.15.92:6805/502685275","state":["exists","up"]},{"osd":5,"uuid":"bdd56766-4b28-4007-8de3-b5b08cf335d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6809","nonce":200142070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6811","nonce":200142070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6815","nonce":200142070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6813","nonce":200142070}]},"public_addr":"172.21.15.92:6809/200142070","cluster_addr":"172.21.15.92:6811/200142070","heartbeat_back_addr":"172.21.15.92:6815/200142070","heartbeat_front_addr":"172.21.15.92:6813/200142070","state":["exists","up"]},{"osd":6,"uuid":"6c580a59-610f-455e-a96b-033299dcc118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6817","nonce":1919010204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6819","nonce":1919010204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6823","nonce":1919010204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6821","nonce":1919010204}]},"public_addr":"172.21.15.92:6817/1919010204","cluster_addr":"172.21.15.92:6819/1919010204","heartbeat_back_addr":"172.21.15.92:6823/1919010204","heartbeat_front_addr":"172.21.15.92:6821/1919010204","state":["exists","up"]},{"osd":7,"uuid":"958184cc-aebf-4c5b-91a0-2a3e38a6b42b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6825","nonce":3475925719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6827","nonce":3475925719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6831","nonce":3475925719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6829","nonce":3475925719}]},"public_addr":"172.21.15.92:6825/3475925719","cluster_addr":"172.21.15.92:6827/3475925719","heartbeat_back_addr":"172.21.15.92:6831/3475925719","heartbeat_front_addr":"172.21.15.92:6829/3475925719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.816294+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.048084+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.137975+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.968246+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.016829+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.931676+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.041673+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.079404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.33","mappings":[{"from":3,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T20:51:25.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.372+0000 7f793783d640 1 -- 172.21.15.90:0/2416110157 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f7914066ec0 msgr2=0x7f7914069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:25.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.372+0000 7f793783d640 1 --2- 172.21.15.90:0/2416110157 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f7914066ec0 0x7f7914069380 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f792c0ae000 tx=0x7f792c0b5040 comp rx=0 tx=0).stop 2024-03-11T20:51:25.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.372+0000 7f793783d640 1 -- 172.21.15.90:0/2416110157 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7930153cf0 msgr2=0x7f79300b4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:25.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.372+0000 7f793783d640 1 --2- 172.21.15.90:0/2416110157 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7930153cf0 0x7f79300b4570 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7928007e70 tx=0x7f792800e4b0 comp rx=0 tx=0).stop 2024-03-11T20:51:25.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.372+0000 7f793783d640 1 -- 172.21.15.90:0/2416110157 shutdown_connections 2024-03-11T20:51:25.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.372+0000 7f793783d640 1 --2- 172.21.15.90:0/2416110157 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f7914066ec0 0x7f7914069380 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:25.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.372+0000 7f793783d640 1 --2- 172.21.15.90:0/2416110157 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7930153cf0 0x7f79300b4570 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:25.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.372+0000 7f793783d640 1 --2- 172.21.15.90:0/2416110157 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f79300c4b60 0x7f79301f6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:25.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.372+0000 7f793783d640 1 --2- 172.21.15.90:0/2416110157 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f79300c42f0 0x7f793014f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:25.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.372+0000 7f793783d640 1 -- 172.21.15.90:0/2416110157 >> 172.21.15.90:0/2416110157 conn(0x7f793006b640 msgr2=0x7f79301513f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:25.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.372+0000 7f793783d640 1 -- 172.21.15.90:0/2416110157 shutdown_connections 2024-03-11T20:51:25.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.372+0000 7f793783d640 1 -- 172.21.15.90:0/2416110157 wait complete. 2024-03-11T20:51:25.391 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-11T18:35:16.193842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-11T18:35:54.001469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-11T18:35:55.917321+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-11T18:36:02.210943+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-11T20:51:25.391 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-11T20:51:25.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.520+0000 7f9acbac6640 1 Processor -- start 2024-03-11T20:51:25.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.520+0000 7f9acbac6640 1 -- start start 2024-03-11T20:51:25.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.520+0000 7f9acbac6640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f9ac40cc3a0 0x7f9ac415a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:25.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.520+0000 7f9acbac6640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9ac4066d60 0x7f9ac415a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:25.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.520+0000 7f9acbac6640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ac4159d80 con 0x7f9ac4153fc0 2024-03-11T20:51:25.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.520+0000 7f9acbac6640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ac40c7990 con 0x7f9ac40cc3a0 2024-03-11T20:51:25.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.520+0000 7f9acbac6640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac40c6e10 con 0x7f9ac4066d60 2024-03-11T20:51:25.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.520+0000 7f9ac9862640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f9ac4153fc0 0x7f9ac406bc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:37622/0 (socket says 172.21.15.90:37622) 2024-03-11T20:51:25.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.520+0000 7f9ac9862640 1 -- 172.21.15.90:0/1004130170 learned_addr learned my addr 172.21.15.90:0/1004130170 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T20:51:25.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.520+0000 7f9aca063640 1 --2- 172.21.15.90:0/1004130170 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9ac4066d60 0x7f9ac415a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:25.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.520+0000 7f9ac9061640 1 --2- 172.21.15.90:0/1004130170 >> v2:172.21.15.92:3300/0 conn(0x7f9ac40cc3a0 0x7f9ac415a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:25.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.524+0000 7f9ac8860640 1 -- 172.21.15.90:0/1004130170 <== mon.0 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3385024080 0 0) 0x7f9ac4159d80 con 0x7f9ac4153fc0 2024-03-11T20:51:25.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.524+0000 7f9ac8860640 1 -- 172.21.15.90:0/1004130170 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ab0003530 con 0x7f9ac4153fc0 2024-03-11T20:51:25.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.528+0000 7f9ac8860640 1 -- 172.21.15.90:0/1004130170 <== mon.0 v1:172.21.15.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1659249975 0 0) 0x7f9ab0003530 con 0x7f9ac4153fc0 2024-03-11T20:51:25.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.528+0000 7f9ac8860640 1 -- 172.21.15.90:0/1004130170 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9ac4066d60 msgr2=0x7f9ac415a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:25.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.528+0000 7f9ac8860640 1 --2- 172.21.15.90:0/1004130170 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9ac4066d60 0x7f9ac415a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:25.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.528+0000 7f9ac8860640 1 -- 172.21.15.90:0/1004130170 >> v2:172.21.15.92:3300/0 conn(0x7f9ac40cc3a0 msgr2=0x7f9ac415a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:25.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.528+0000 7f9ac8860640 1 --2- 172.21.15.90:0/1004130170 >> v2:172.21.15.92:3300/0 conn(0x7f9ac40cc3a0 0x7f9ac415a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:25.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.528+0000 7f9ac8860640 1 -- 172.21.15.90:0/1004130170 --> v1:172.21.15.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac40c70f0 con 0x7f9ac4153fc0 2024-03-11T20:51:25.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.528+0000 7f9ac8860640 1 -- 172.21.15.90:0/1004130170 <== mon.0 v1:172.21.15.90:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 691735435 0 0) 0x7f9ac0001680 con 0x7f9ac4153fc0 2024-03-11T20:51:25.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.528+0000 7f9ac8860640 1 -- 172.21.15.90:0/1004130170 >> v1:172.21.15.90:6789/0 conn(0x7f9ac4153fc0 legacy=0x7f9ac406bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:25.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.528+0000 7f9ac8860640 1 --2- 172.21.15.90:0/1004130170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ab0003980 0x7f9ab0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:25.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.528+0000 7f9ac8860640 1 --2- 172.21.15.90:0/1004130170 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ab00047d0 0x7f9ab0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:25.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.528+0000 7f9ac8860640 1 --2- 172.21.15.90:0/1004130170 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9ac4066d60 0x7f9ab0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:25.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.528+0000 7f9ac8860640 1 -- 172.21.15.90:0/1004130170 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac40c7990 con 0x7f9ab00047d0 2024-03-11T20:51:25.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.528+0000 7f9ac8860640 1 -- 172.21.15.90:0/1004130170 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac40c6e10 con 0x7f9ab0003980 2024-03-11T20:51:25.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.528+0000 7f9ac8860640 1 -- 172.21.15.90:0/1004130170 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab000b9c0 con 0x7f9ac4066d60 2024-03-11T20:51:25.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.532+0000 7f9ac9061640 1 --2- 172.21.15.90:0/1004130170 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ab00047d0 0x7f9ab0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:25.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.532+0000 7f9ac9862640 1 --2- 172.21.15.90:0/1004130170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ab0003980 0x7f9ab0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:25.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.536+0000 7f9ac9061640 1 -- 172.21.15.90:0/1004130170 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9ac4066d60 msgr2=0x7f9ab0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:25.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.536+0000 7f9ac9061640 1 --2- 172.21.15.90:0/1004130170 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9ac4066d60 0x7f9ab0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:25.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.536+0000 7f9ac9061640 1 -- 172.21.15.90:0/1004130170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ab0003980 msgr2=0x7f9ab0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:25.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.536+0000 7f9ac9061640 1 --2- 172.21.15.90:0/1004130170 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ab0003980 0x7f9ab0003da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:25.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.536+0000 7f9ac9061640 1 -- 172.21.15.90:0/1004130170 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab0003530 con 0x7f9ab00047d0 2024-03-11T20:51:25.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.540+0000 7f9ac9061640 1 --2- 172.21.15.90:0/1004130170 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ab00047d0 0x7f9ab0004c50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8009ae0 tx=0x7f9ab801cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e126653ebaebf8a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:25.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.544+0000 7f9ac8860640 1 -- 172.21.15.90:0/1004130170 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab80a55c0 con 0x7f9ab00047d0 2024-03-11T20:51:25.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.544+0000 7f9ac8860640 1 -- 172.21.15.90:0/1004130170 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ab80a5720 con 0x7f9ab00047d0 2024-03-11T20:51:25.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.544+0000 7f9ac8860640 1 -- 172.21.15.90:0/1004130170 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab80a5b10 con 0x7f9ab00047d0 2024-03-11T20:51:25.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.544+0000 7f9acbac6640 1 -- 172.21.15.90:0/1004130170 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ab00047d0 msgr2=0x7f9ab0004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:25.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.544+0000 7f9acbac6640 1 --2- 172.21.15.90:0/1004130170 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ab00047d0 0x7f9ab0004c50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8009ae0 tx=0x7f9ab801cab0 comp rx=0 tx=0).stop 2024-03-11T20:51:25.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.544+0000 7f9aca063640 1 -- 172.21.15.90:0/1004130170 reap_dead start 2024-03-11T20:51:25.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.544+0000 7f9acbac6640 1 -- 172.21.15.90:0/1004130170 shutdown_connections 2024-03-11T20:51:25.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.544+0000 7f9acbac6640 1 -- 172.21.15.90:0/1004130170 >> 172.21.15.90:0/1004130170 conn(0x7f9ac40cb880 msgr2=0x7f9ac40cbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:25.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.544+0000 7f9acbac6640 1 -- 172.21.15.90:0/1004130170 shutdown_connections 2024-03-11T20:51:25.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.544+0000 7f9acbac6640 1 -- 172.21.15.90:0/1004130170 wait complete. 2024-03-11T20:51:25.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.544+0000 7f9acbac6640 1 Processor -- start 2024-03-11T20:51:25.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.544+0000 7f9acbac6640 1 -- start start 2024-03-11T20:51:25.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.548+0000 7f9acbac6640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ac414c140 0x7f9ac41f3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:25.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.548+0000 7f9acbac6640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9ac40c44f0 0x7f9ac41ef560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:25.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.548+0000 7f9acbac6640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ac414c520 0x7f9ac4148070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:25.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.548+0000 7f9acbac6640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac41efaa0 con 0x7f9ac414c140 2024-03-11T20:51:25.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.548+0000 7f9acbac6640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac41573a0 con 0x7f9ac414c520 2024-03-11T20:51:25.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.548+0000 7f9acbac6640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac4157640 con 0x7f9ac40c44f0 2024-03-11T20:51:25.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.548+0000 7f9ac9862640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ac414c140 0x7f9ac41f3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:25.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.548+0000 7f9ac9862640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ac414c140 0x7f9ac41f3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59898/0 (socket says 172.21.15.90:59898) 2024-03-11T20:51:25.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.548+0000 7f9ac9862640 1 -- 172.21.15.90:0/2399582861 learned_addr learned my addr 172.21.15.90:0/2399582861 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:51:25.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.548+0000 7f9aca063640 1 --2- 172.21.15.90:0/2399582861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ac414c520 0x7f9ac4148070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:25.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.548+0000 7f9ac9061640 1 --2- 172.21.15.90:0/2399582861 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9ac40c44f0 0x7f9ac41ef560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:25.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.552+0000 7f9ac9862640 1 -- 172.21.15.90:0/2399582861 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9ac40c44f0 msgr2=0x7f9ac41ef560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:25.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.552+0000 7f9ac9862640 1 --2- 172.21.15.90:0/2399582861 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9ac40c44f0 0x7f9ac41ef560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:25.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.552+0000 7f9ac9862640 1 -- 172.21.15.90:0/2399582861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ac414c520 msgr2=0x7f9ac4148070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:25.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.552+0000 7f9ac9862640 1 --2- 172.21.15.90:0/2399582861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ac414c520 0x7f9ac4148070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:25.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.552+0000 7f9ac9862640 1 -- 172.21.15.90:0/2399582861 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac4157920 con 0x7f9ac414c140 2024-03-11T20:51:25.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.556+0000 7f9ac9862640 1 --2- 172.21.15.90:0/2399582861 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ac414c140 0x7f9ac41f3650 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9ac000db10 tx=0x7f9ac0001d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:25.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.560+0000 7f9ab67fc640 1 -- 172.21.15.90:0/2399582861 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0004cc0 con 0x7f9ac414c140 2024-03-11T20:51:25.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.560+0000 7f9ab67fc640 1 -- 172.21.15.90:0/2399582861 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ac0004e20 con 0x7f9ac414c140 2024-03-11T20:51:25.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.560+0000 7f9acbac6640 1 -- 172.21.15.90:0/2399582861 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ac41f2b70 con 0x7f9ac414c140 2024-03-11T20:51:25.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.560+0000 7f9ab67fc640 1 -- 172.21.15.90:0/2399582861 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac00072f0 con 0x7f9ac414c140 2024-03-11T20:51:25.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.560+0000 7f9acbac6640 1 -- 172.21.15.90:0/2399582861 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ac41f3130 con 0x7f9ac414c140 2024-03-11T20:51:25.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.560+0000 7f9acbac6640 1 -- 172.21.15.90:0/2399582861 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ac406a1b0 con 0x7f9ac414c140 2024-03-11T20:51:25.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.844+0000 7f9ab67fc640 1 -- 172.21.15.90:0/2399582861 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f9ac0007450 con 0x7f9ac414c140 2024-03-11T20:51:25.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.844+0000 7f9ab67fc640 1 --2- 172.21.15.90:0/2399582861 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f9aa8066e90 0x7f9aa8069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:25.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.844+0000 7f9ac9061640 1 --2- 172.21.15.90:0/2399582861 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f9aa8066e90 0x7f9aa8069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:25.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.844+0000 7f9ac9061640 1 --2- 172.21.15.90:0/2399582861 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f9aa8066e90 0x7f9aa8069350 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9ab80af000 tx=0x7f9ab80046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:25.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.884+0000 7f9ab67fc640 1 -- 172.21.15.90:0/2399582861 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f9ac0007700 con 0x7f9ac414c140 2024-03-11T20:51:25.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:25.912+0000 7f9ab67fc640 1 -- 172.21.15.90:0/2399582861 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9ac0051a60 con 0x7f9ac414c140 2024-03-11T20:51:26.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.084+0000 7f9acbac6640 1 -- 172.21.15.90:0/2399582861 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9ac404eef0 con 0x7f9ac414c140 2024-03-11T20:51:26.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.092+0000 7f9ab67fc640 1 -- 172.21.15.90:0/2399582861 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9ac0023020 con 0x7f9ac414c140 2024-03-11T20:51:26.091 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 1 2024-03-11T20:51:26.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.096+0000 7f9acbac6640 1 -- 172.21.15.90:0/2399582861 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f9aa8066e90 msgr2=0x7f9aa8069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:26.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.096+0000 7f9acbac6640 1 --2- 172.21.15.90:0/2399582861 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f9aa8066e90 0x7f9aa8069350 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9ab80af000 tx=0x7f9ab80046d0 comp rx=0 tx=0).stop 2024-03-11T20:51:26.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.096+0000 7f9acbac6640 1 -- 172.21.15.90:0/2399582861 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ac414c140 msgr2=0x7f9ac41f3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:26.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.096+0000 7f9acbac6640 1 --2- 172.21.15.90:0/2399582861 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ac414c140 0x7f9ac41f3650 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9ac000db10 tx=0x7f9ac0001d50 comp rx=0 tx=0).stop 2024-03-11T20:51:26.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.096+0000 7f9acbac6640 1 -- 172.21.15.90:0/2399582861 shutdown_connections 2024-03-11T20:51:26.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.096+0000 7f9acbac6640 1 --2- 172.21.15.90:0/2399582861 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f9aa8066e90 0x7f9aa8069350 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:26.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.096+0000 7f9acbac6640 1 --2- 172.21.15.90:0/2399582861 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f9ac414c520 0x7f9ac4148070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:26.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.096+0000 7f9acbac6640 1 --2- 172.21.15.90:0/2399582861 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f9ac40c44f0 0x7f9ac41ef560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:26.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.096+0000 7f9acbac6640 1 --2- 172.21.15.90:0/2399582861 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9ac414c140 0x7f9ac41f3650 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:26.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.096+0000 7f9acbac6640 1 -- 172.21.15.90:0/2399582861 >> 172.21.15.90:0/2399582861 conn(0x7f9ac40cb880 msgr2=0x7f9ac40cbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:26.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.096+0000 7f9acbac6640 1 -- 172.21.15.90:0/2399582861 shutdown_connections 2024-03-11T20:51:26.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.096+0000 7f9acbac6640 1 -- 172.21.15.90:0/2399582861 wait complete. 2024-03-11T20:51:26.112 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-11T20:51:26.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.232+0000 7f42e55b9640 1 Processor -- start 2024-03-11T20:51:26.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.232+0000 7f42e55b9640 1 -- start start 2024-03-11T20:51:26.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.236+0000 7f42e55b9640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f42e0151000 0x7f42e00c2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:26.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.236+0000 7f42e55b9640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f42e0066d60 0x7f42e01613a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:26.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.236+0000 7f42e55b9640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42e00c7980 con 0x7f42e00c5b50 2024-03-11T20:51:26.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.236+0000 7f42e55b9640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42e00cc820 con 0x7f42e0066d60 2024-03-11T20:51:26.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.236+0000 7f42e55b9640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42e00caca0 con 0x7f42e0151000 2024-03-11T20:51:26.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.236+0000 7f42deffd640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f42e0151000 0x7f42e00c2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:26.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.236+0000 7f42d67fc640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f42e00c5b50 0x7f42e00c4ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.90:55012/0 (socket says 172.21.15.90:55012) 2024-03-11T20:51:26.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.236+0000 7f42d67fc640 1 -- 172.21.15.90:0/2502767609 learned_addr learned my addr 172.21.15.90:0/2502767609 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T20:51:26.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.244+0000 7f42deffd640 1 -- 172.21.15.90:0/2502767609 >> v2:172.21.15.90:3300/0 conn(0x7f42e0066d60 msgr2=0x7f42e01613a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:26.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.244+0000 7f42deffd640 1 --2- 172.21.15.90:0/2502767609 >> v2:172.21.15.90:3300/0 conn(0x7f42e0066d60 0x7f42e01613a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:26.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.244+0000 7f42deffd640 1 -- 172.21.15.90:0/2502767609 >> v1:172.21.15.92:6789/0 conn(0x7f42e00c5b50 legacy=0x7f42e00c4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:26.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.244+0000 7f42deffd640 1 -- 172.21.15.90:0/2502767609 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e00c2ad0 con 0x7f42e0151000 2024-03-11T20:51:26.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.260+0000 7f42deffd640 1 --2- 172.21.15.90:0/2502767609 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f42e0151000 0x7f42e00c2d80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f42d00097c0 tx=0x7f42d001c350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4273be746fecbaf server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:26.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.364+0000 7f42de7fc640 1 -- 172.21.15.90:0/2502767609 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d0004230 con 0x7f42e0151000 2024-03-11T20:51:26.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.364+0000 7f42de7fc640 1 -- 172.21.15.90:0/2502767609 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d00a7050 con 0x7f42e0151000 2024-03-11T20:51:26.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.364+0000 7f42de7fc640 1 -- 172.21.15.90:0/2502767609 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d0003620 con 0x7f42e0151000 2024-03-11T20:51:26.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.368+0000 7f42e55b9640 1 -- 172.21.15.90:0/2502767609 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f42e0151000 msgr2=0x7f42e00c2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:26.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.368+0000 7f42e55b9640 1 --2- 172.21.15.90:0/2502767609 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f42e0151000 0x7f42e00c2d80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f42d00097c0 tx=0x7f42d001c350 comp rx=0 tx=0).stop 2024-03-11T20:51:26.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.368+0000 7f42e55b9640 1 -- 172.21.15.90:0/2502767609 shutdown_connections 2024-03-11T20:51:26.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.368+0000 7f42e55b9640 1 --2- 172.21.15.90:0/2502767609 >> v2:172.21.15.90:3300/0 conn(0x7f42e0066d60 0x7f42e01613a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:26.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.368+0000 7f42e55b9640 1 --2- 172.21.15.90:0/2502767609 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f42e0151000 0x7f42e00c2d80 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:26.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.368+0000 7f42e55b9640 1 -- 172.21.15.90:0/2502767609 >> 172.21.15.90:0/2502767609 conn(0x7f42e00cb880 msgr2=0x7f42e00cbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:26.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.368+0000 7f42e55b9640 1 -- 172.21.15.90:0/2502767609 shutdown_connections 2024-03-11T20:51:26.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.368+0000 7f42e55b9640 1 -- 172.21.15.90:0/2502767609 wait complete. 2024-03-11T20:51:26.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.372+0000 7f42e55b9640 1 Processor -- start 2024-03-11T20:51:26.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.372+0000 7f42e55b9640 1 -- start start 2024-03-11T20:51:26.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.372+0000 7f42e55b9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42e0066d60 0x7f42e0160a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:26.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.372+0000 7f42e55b9640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f42e00c5b50 0x7f42e0146120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:26.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.372+0000 7f42e55b9640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f42e0151000 0x7f42e0148d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:26.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.372+0000 7f42e55b9640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e00725c0 con 0x7f42e00c5b50 2024-03-11T20:51:26.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.372+0000 7f42e55b9640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e01f31c0 con 0x7f42e0066d60 2024-03-11T20:51:26.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.372+0000 7f42e55b9640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42e0148880 con 0x7f42e0151000 2024-03-11T20:51:26.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.372+0000 7f42deffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42e0066d60 0x7f42e0160a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:26.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.376+0000 7f42deffd640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42e0066d60 0x7f42e0160a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.90:49370/0 (socket says 172.21.15.90:49370) 2024-03-11T20:51:26.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.376+0000 7f42deffd640 1 -- 172.21.15.90:0/2670735461 learned_addr learned my addr 172.21.15.90:0/2670735461 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:51:26.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.376+0000 7f42df7fe640 1 --2- 172.21.15.90:0/2670735461 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f42e0151000 0x7f42e0148d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:26.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.380+0000 7f42d67fc640 1 --2- 172.21.15.90:0/2670735461 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f42e00c5b50 0x7f42e0146120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:26.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.384+0000 7f42deffd640 1 -- 172.21.15.90:0/2670735461 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f42e0151000 msgr2=0x7f42e0148d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:26.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.384+0000 7f42deffd640 1 --2- 172.21.15.90:0/2670735461 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f42e0151000 0x7f42e0148d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:26.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.384+0000 7f42deffd640 1 -- 172.21.15.90:0/2670735461 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f42e00c5b50 msgr2=0x7f42e0146120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:26.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.384+0000 7f42deffd640 1 --2- 172.21.15.90:0/2670735461 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f42e00c5b50 0x7f42e0146120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:26.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.384+0000 7f42deffd640 1 -- 172.21.15.90:0/2670735461 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d0009420 con 0x7f42e0066d60 2024-03-11T20:51:26.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.392+0000 7f42deffd640 1 --2- 172.21.15.90:0/2670735461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42e0066d60 0x7f42e0160a20 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f42d0019f80 tx=0x7f42d00190f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:26.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.500+0000 7f42d7fff640 1 -- 172.21.15.90:0/2670735461 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d0003d60 con 0x7f42e0066d60 2024-03-11T20:51:26.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.500+0000 7f42d7fff640 1 -- 172.21.15.90:0/2670735461 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d00ab440 con 0x7f42e0066d60 2024-03-11T20:51:26.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.500+0000 7f42e55b9640 1 -- 172.21.15.90:0/2670735461 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42e014a150 con 0x7f42e0066d60 2024-03-11T20:51:26.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.500+0000 7f42d7fff640 1 -- 172.21.15.90:0/2670735461 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d00ab7f0 con 0x7f42e0066d60 2024-03-11T20:51:26.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.500+0000 7f42e55b9640 1 -- 172.21.15.90:0/2670735461 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42e0072af0 con 0x7f42e0066d60 2024-03-11T20:51:26.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.504+0000 7f42e55b9640 1 -- 172.21.15.90:0/2670735461 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42e004eef0 con 0x7f42e0066d60 2024-03-11T20:51:26.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.812+0000 7f42d7fff640 1 -- 172.21.15.90:0/2670735461 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f42d00b2050 con 0x7f42e0066d60 2024-03-11T20:51:26.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.812+0000 7f42d7fff640 1 --2- 172.21.15.90:0/2670735461 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f42c0066e90 0x7f42c0069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:26.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.812+0000 7f42d67fc640 1 --2- 172.21.15.90:0/2670735461 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f42c0066e90 0x7f42c0069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:26.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.812+0000 7f42d67fc640 1 --2- 172.21.15.90:0/2670735461 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f42c0066e90 0x7f42c0069350 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f42e0148600 tx=0x7f42c8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:26.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.816+0000 7f42d7fff640 1 -- 172.21.15.90:0/2670735461 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f42d00f0680 con 0x7f42e0066d60 2024-03-11T20:51:26.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:26.848+0000 7f42d7fff640 1 -- 172.21.15.90:0/2670735461 <== mon.1 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f42d00ec220 con 0x7f42e0066d60 2024-03-11T20:51:27.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.036+0000 7f42e55b9640 1 -- 172.21.15.90:0/2670735461 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f42e006cf10 con 0x7f42e0066d60 2024-03-11T20:51:27.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.044+0000 7f42d7fff640 1 -- 172.21.15.90:0/2670735461 <== mon.1 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f42d00ef8e0 con 0x7f42e0066d60 2024-03-11T20:51:27.046 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 32 2024-03-11T20:51:27.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.044+0000 7f42e55b9640 1 -- 172.21.15.90:0/2670735461 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f42c0066e90 msgr2=0x7f42c0069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:27.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.044+0000 7f42e55b9640 1 --2- 172.21.15.90:0/2670735461 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f42c0066e90 0x7f42c0069350 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f42e0148600 tx=0x7f42c8009040 comp rx=0 tx=0).stop 2024-03-11T20:51:27.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.044+0000 7f42e55b9640 1 -- 172.21.15.90:0/2670735461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42e0066d60 msgr2=0x7f42e0160a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:27.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.044+0000 7f42e55b9640 1 --2- 172.21.15.90:0/2670735461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42e0066d60 0x7f42e0160a20 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f42d0019f80 tx=0x7f42d00190f0 comp rx=0 tx=0).stop 2024-03-11T20:51:27.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.044+0000 7f42e55b9640 1 -- 172.21.15.90:0/2670735461 shutdown_connections 2024-03-11T20:51:27.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.044+0000 7f42e55b9640 1 --2- 172.21.15.90:0/2670735461 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f42c0066e90 0x7f42c0069350 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:27.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.044+0000 7f42e55b9640 1 --2- 172.21.15.90:0/2670735461 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f42e0151000 0x7f42e0148d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:27.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.044+0000 7f42e55b9640 1 --2- 172.21.15.90:0/2670735461 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f42e00c5b50 0x7f42e0146120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:27.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.044+0000 7f42e55b9640 1 --2- 172.21.15.90:0/2670735461 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f42e0066d60 0x7f42e0160a20 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:27.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.044+0000 7f42e55b9640 1 -- 172.21.15.90:0/2670735461 >> 172.21.15.90:0/2670735461 conn(0x7f42e00cb880 msgr2=0x7f42e00c6910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:27.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.048+0000 7f42e55b9640 1 -- 172.21.15.90:0/2670735461 shutdown_connections 2024-03-11T20:51:27.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.048+0000 7f42e55b9640 1 -- 172.21.15.90:0/2670735461 wait complete. 2024-03-11T20:51:27.065 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-11T20:51:27.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.192+0000 7ffb34f3f640 1 Processor -- start 2024-03-11T20:51:27.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.192+0000 7ffb34f3f640 1 -- start start 2024-03-11T20:51:27.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.192+0000 7ffb34f3f640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7ffb3006e9b0 0x7ffb3014fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:27.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.192+0000 7ffb34f3f640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb30070b00 con 0x7ffb3006f220 2024-03-11T20:51:27.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.192+0000 7ffb34f3f640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb30071df0 con 0x7ffb301598c0 2024-03-11T20:51:27.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.192+0000 7ffb34f3f640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb3015a3b0 con 0x7ffb3006e9b0 2024-03-11T20:51:27.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.192+0000 7ffb25d9b640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7ffb3006f220 0x7ffb300c3d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:37636/0 (socket says 172.21.15.90:37636) 2024-03-11T20:51:27.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.192+0000 7ffb25d9b640 1 -- 172.21.15.90:0/1088958149 learned_addr learned my addr 172.21.15.90:0/1088958149 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T20:51:27.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.192+0000 7ffb2ed9d640 1 --2- 172.21.15.90:0/1088958149 >> v2:172.21.15.90:3300/0 conn(0x7ffb3006e9b0 0x7ffb3014fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:27.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.196+0000 7ffb2ed9d640 1 -- 172.21.15.90:0/1088958149 >> v1:172.21.15.92:6789/0 conn(0x7ffb301598c0 legacy=0x7ffb30159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:27.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.196+0000 7ffb2ed9d640 1 -- 172.21.15.90:0/1088958149 >> v1:172.21.15.90:6789/0 conn(0x7ffb3006f220 legacy=0x7ffb300c3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:27.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.196+0000 7ffb2ed9d640 1 -- 172.21.15.90:0/1088958149 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb30070d50 con 0x7ffb3006e9b0 2024-03-11T20:51:27.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.200+0000 7ffb2ed9d640 1 --2- 172.21.15.90:0/1088958149 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb3006e9b0 0x7ffb3014fe20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ffb28009c80 tx=0x7ffb28020270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28fe7ed694791ada server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:27.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.204+0000 7ffb2dd9b640 1 -- 172.21.15.90:0/1088958149 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb280b0600 con 0x7ffb3006e9b0 2024-03-11T20:51:27.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.204+0000 7ffb2dd9b640 1 -- 172.21.15.90:0/1088958149 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb280ac050 con 0x7ffb3006e9b0 2024-03-11T20:51:27.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.204+0000 7ffb2dd9b640 1 -- 172.21.15.90:0/1088958149 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb280b0980 con 0x7ffb3006e9b0 2024-03-11T20:51:27.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.204+0000 7ffb34f3f640 1 -- 172.21.15.90:0/1088958149 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb3006e9b0 msgr2=0x7ffb3014fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:27.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.204+0000 7ffb34f3f640 1 --2- 172.21.15.90:0/1088958149 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb3006e9b0 0x7ffb3014fe20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ffb28009c80 tx=0x7ffb28020270 comp rx=0 tx=0).stop 2024-03-11T20:51:27.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.204+0000 7ffb34f3f640 1 -- 172.21.15.90:0/1088958149 shutdown_connections 2024-03-11T20:51:27.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.204+0000 7ffb34f3f640 1 --2- 172.21.15.90:0/1088958149 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb3006e9b0 0x7ffb3014fe20 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:27.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.204+0000 7ffb34f3f640 1 -- 172.21.15.90:0/1088958149 >> 172.21.15.90:0/1088958149 conn(0x7ffb3006ac00 msgr2=0x7ffb3006b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:27.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.204+0000 7ffb34f3f640 1 -- 172.21.15.90:0/1088958149 shutdown_connections 2024-03-11T20:51:27.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.204+0000 7ffb34f3f640 1 -- 172.21.15.90:0/1088958149 wait complete. 2024-03-11T20:51:27.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.208+0000 7ffb34f3f640 1 Processor -- start 2024-03-11T20:51:27.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.208+0000 7ffb34f3f640 1 -- start start 2024-03-11T20:51:27.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.208+0000 7ffb34f3f640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ffb3006e9b0 0x7ffb300c45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:27.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.208+0000 7ffb34f3f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb3006f220 0x7ffb301ef560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:27.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.208+0000 7ffb34f3f640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb301598c0 0x7ffb30145ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:27.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.208+0000 7ffb34f3f640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb301f32f0 con 0x7ffb301598c0 2024-03-11T20:51:27.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.208+0000 7ffb34f3f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb301ef140 con 0x7ffb3006f220 2024-03-11T20:51:27.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.208+0000 7ffb34f3f640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb301e0cc0 con 0x7ffb3006e9b0 2024-03-11T20:51:27.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.208+0000 7ffb2ed9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb301598c0 0x7ffb30145ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:27.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.208+0000 7ffb2ed9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb301598c0 0x7ffb30145ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59948/0 (socket says 172.21.15.90:59948) 2024-03-11T20:51:27.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.208+0000 7ffb2ed9d640 1 -- 172.21.15.90:0/1070348948 learned_addr learned my addr 172.21.15.90:0/1070348948 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:51:27.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.208+0000 7ffb2e59c640 1 --2- 172.21.15.90:0/1070348948 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ffb3006e9b0 0x7ffb300c45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:27.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.208+0000 7ffb25d9b640 1 --2- 172.21.15.90:0/1070348948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb3006f220 0x7ffb301ef560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:27.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.212+0000 7ffb2ed9d640 1 -- 172.21.15.90:0/1070348948 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ffb3006e9b0 msgr2=0x7ffb300c45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:27.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.212+0000 7ffb2ed9d640 1 --2- 172.21.15.90:0/1070348948 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ffb3006e9b0 0x7ffb300c45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:27.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.212+0000 7ffb2ed9d640 1 -- 172.21.15.90:0/1070348948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb3006f220 msgr2=0x7ffb301ef560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:27.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.212+0000 7ffb2ed9d640 1 --2- 172.21.15.90:0/1070348948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb3006f220 0x7ffb301ef560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:27.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.212+0000 7ffb2ed9d640 1 -- 172.21.15.90:0/1070348948 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb2800d040 con 0x7ffb301598c0 2024-03-11T20:51:27.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.216+0000 7ffb2ed9d640 1 --2- 172.21.15.90:0/1070348948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb301598c0 0x7ffb30145ff0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ffb28009c80 tx=0x7ffb2801d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:27.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.220+0000 7ffb277fe640 1 -- 172.21.15.90:0/1070348948 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb280ba8e0 con 0x7ffb301598c0 2024-03-11T20:51:27.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.220+0000 7ffb277fe640 1 -- 172.21.15.90:0/1070348948 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb280baa40 con 0x7ffb301598c0 2024-03-11T20:51:27.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.220+0000 7ffb277fe640 1 -- 172.21.15.90:0/1070348948 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb280badf0 con 0x7ffb301598c0 2024-03-11T20:51:27.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.220+0000 7ffb34f3f640 1 -- 172.21.15.90:0/1070348948 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb3014f650 con 0x7ffb301598c0 2024-03-11T20:51:27.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.220+0000 7ffb34f3f640 1 -- 172.21.15.90:0/1070348948 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb301f2a50 con 0x7ffb301598c0 2024-03-11T20:51:27.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.220+0000 7ffb34f3f640 1 -- 172.21.15.90:0/1070348948 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb301ef280 con 0x7ffb301598c0 2024-03-11T20:51:27.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.504+0000 7ffb277fe640 1 -- 172.21.15.90:0/1070348948 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7ffb280b7070 con 0x7ffb301598c0 2024-03-11T20:51:27.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.508+0000 7ffb277fe640 1 --2- 172.21.15.90:0/1070348948 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7ffb10066e70 0x7ffb10069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:27.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.508+0000 7ffb2e59c640 1 --2- 172.21.15.90:0/1070348948 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7ffb10066e70 0x7ffb10069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:27.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.508+0000 7ffb2e59c640 1 --2- 172.21.15.90:0/1070348948 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7ffb10066e70 0x7ffb10069330 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ffb20007fd0 tx=0x7ffb20009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:27.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.548+0000 7ffb277fe640 1 -- 172.21.15.90:0/1070348948 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7ffb280f39c0 con 0x7ffb301598c0 2024-03-11T20:51:27.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.556+0000 7ffb277fe640 1 -- 172.21.15.90:0/1070348948 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffb280efc60 con 0x7ffb301598c0 2024-03-11T20:51:27.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.744+0000 7ffb34f3f640 1 -- 172.21.15.90:0/1070348948 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ffb301ef280 con 0x7ffb301598c0 2024-03-11T20:51:27.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.760+0000 7ffb277fe640 1 -- 172.21.15.90:0/1070348948 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7ffb301ef280 con 0x7ffb301598c0 2024-03-11T20:51:27.758 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 64 2024-03-11T20:51:27.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.760+0000 7ffb34f3f640 1 -- 172.21.15.90:0/1070348948 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7ffb10066e70 msgr2=0x7ffb10069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:27.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.760+0000 7ffb34f3f640 1 --2- 172.21.15.90:0/1070348948 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7ffb10066e70 0x7ffb10069330 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ffb20007fd0 tx=0x7ffb20009040 comp rx=0 tx=0).stop 2024-03-11T20:51:27.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.760+0000 7ffb34f3f640 1 -- 172.21.15.90:0/1070348948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb301598c0 msgr2=0x7ffb30145ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:27.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.760+0000 7ffb34f3f640 1 --2- 172.21.15.90:0/1070348948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb301598c0 0x7ffb30145ff0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ffb28009c80 tx=0x7ffb2801d010 comp rx=0 tx=0).stop 2024-03-11T20:51:27.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.760+0000 7ffb34f3f640 1 -- 172.21.15.90:0/1070348948 shutdown_connections 2024-03-11T20:51:27.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.760+0000 7ffb34f3f640 1 --2- 172.21.15.90:0/1070348948 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7ffb10066e70 0x7ffb10069330 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:27.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.760+0000 7ffb34f3f640 1 --2- 172.21.15.90:0/1070348948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb301598c0 0x7ffb30145ff0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:27.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.760+0000 7ffb34f3f640 1 --2- 172.21.15.90:0/1070348948 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb3006f220 0x7ffb301ef560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:27.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.760+0000 7ffb34f3f640 1 --2- 172.21.15.90:0/1070348948 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ffb3006e9b0 0x7ffb300c45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:27.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.764+0000 7ffb34f3f640 1 -- 172.21.15.90:0/1070348948 >> 172.21.15.90:0/1070348948 conn(0x7ffb3006ac00 msgr2=0x7ffb30159ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:27.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.764+0000 7ffb34f3f640 1 -- 172.21.15.90:0/1070348948 shutdown_connections 2024-03-11T20:51:27.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.764+0000 7ffb34f3f640 1 -- 172.21.15.90:0/1070348948 wait complete. 2024-03-11T20:51:27.779 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-11T20:51:27.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.900+0000 7fbb66391640 1 Processor -- start 2024-03-11T20:51:27.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.900+0000 7fbb66391640 1 -- start start 2024-03-11T20:51:27.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.904+0000 7fbb66391640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbb60153fe0 0x7fbb600ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:27.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.904+0000 7fbb66391640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fbb6006bdb0 0x7fbb6006c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:27.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.904+0000 7fbb66391640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb6006d000 con 0x7fbb600cb7d0 2024-03-11T20:51:27.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.904+0000 7fbb66391640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb601543c0 con 0x7fbb6006bdb0 2024-03-11T20:51:27.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.904+0000 7fbb66391640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb6015ab90 con 0x7fbb60153fe0 2024-03-11T20:51:27.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.904+0000 7fbb6492e640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fbb6006bdb0 0x7fbb6006c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:27.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.904+0000 7fbb5f7fe640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7fbb600cb7d0 0x7fbb600cf800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:37648/0 (socket says 172.21.15.90:37648) 2024-03-11T20:51:27.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.904+0000 7fbb5f7fe640 1 -- 172.21.15.90:0/771895645 learned_addr learned my addr 172.21.15.90:0/771895645 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T20:51:27.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.908+0000 7fbb5effd640 1 -- 172.21.15.90:0/771895645 <== mon.0 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 195764835 0 0) 0x7fbb6006d000 con 0x7fbb600cb7d0 2024-03-11T20:51:27.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.908+0000 7fbb5effd640 1 -- 172.21.15.90:0/771895645 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb48003530 con 0x7fbb600cb7d0 2024-03-11T20:51:27.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.912+0000 7fbb6492e640 1 -- 172.21.15.90:0/771895645 >> v2:172.21.15.92:3300/0 conn(0x7fbb60153fe0 msgr2=0x7fbb600ca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:27.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.912+0000 7fbb6492e640 1 --2- 172.21.15.90:0/771895645 >> v2:172.21.15.92:3300/0 conn(0x7fbb60153fe0 0x7fbb600ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:27.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.912+0000 7fbb6492e640 1 -- 172.21.15.90:0/771895645 >> v1:172.21.15.90:6789/0 conn(0x7fbb600cb7d0 legacy=0x7fbb600cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:27.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.912+0000 7fbb6492e640 1 -- 172.21.15.90:0/771895645 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb60159ca0 con 0x7fbb6006bdb0 2024-03-11T20:51:27.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.916+0000 7fbb6492e640 1 --2- 172.21.15.90:0/771895645 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb6006bdb0 0x7fbb6006c1c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbb6006ce20 tx=0x7fbb5801c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25cdc583398fd83 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:27.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.924+0000 7fbb5effd640 1 -- 172.21.15.90:0/771895645 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb580a59c0 con 0x7fbb6006bdb0 2024-03-11T20:51:27.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.924+0000 7fbb5effd640 1 -- 172.21.15.90:0/771895645 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb580a5b20 con 0x7fbb6006bdb0 2024-03-11T20:51:27.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.924+0000 7fbb5effd640 1 -- 172.21.15.90:0/771895645 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb580ad440 con 0x7fbb6006bdb0 2024-03-11T20:51:27.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.924+0000 7fbb66391640 1 -- 172.21.15.90:0/771895645 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb6006bdb0 msgr2=0x7fbb6006c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:27.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.924+0000 7fbb66391640 1 --2- 172.21.15.90:0/771895645 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb6006bdb0 0x7fbb6006c1c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbb6006ce20 tx=0x7fbb5801c900 comp rx=0 tx=0).stop 2024-03-11T20:51:27.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.924+0000 7fbb66391640 1 -- 172.21.15.90:0/771895645 shutdown_connections 2024-03-11T20:51:27.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.924+0000 7fbb66391640 1 --2- 172.21.15.90:0/771895645 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb6006bdb0 0x7fbb6006c1c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:27.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.924+0000 7fbb66391640 1 --2- 172.21.15.90:0/771895645 >> v2:172.21.15.92:3300/0 conn(0x7fbb60153fe0 0x7fbb600ca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:27.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.924+0000 7fbb66391640 1 -- 172.21.15.90:0/771895645 >> 172.21.15.90:0/771895645 conn(0x7fbb60072500 msgr2=0x7fbb60072910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:27.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.924+0000 7fbb66391640 1 -- 172.21.15.90:0/771895645 shutdown_connections 2024-03-11T20:51:27.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.924+0000 7fbb66391640 1 -- 172.21.15.90:0/771895645 wait complete. 2024-03-11T20:51:27.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.924+0000 7fbb66391640 1 Processor -- start 2024-03-11T20:51:27.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.924+0000 7fbb66391640 1 -- start start 2024-03-11T20:51:27.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.928+0000 7fbb66391640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbb6006bdb0 0x7fbb60159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:27.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.928+0000 7fbb66391640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb600cb7d0 0x7fbb60143b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:27.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.928+0000 7fbb66391640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb60153fe0 0x7fbb601440d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:27.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.928+0000 7fbb66391640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb601f2b00 con 0x7fbb60153fe0 2024-03-11T20:51:27.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.928+0000 7fbb66391640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb60159470 con 0x7fbb600cb7d0 2024-03-11T20:51:27.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.928+0000 7fbb66391640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb60142af0 con 0x7fbb6006bdb0 2024-03-11T20:51:27.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.928+0000 7fbb6492e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb60153fe0 0x7fbb601440d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:27.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.928+0000 7fbb6492e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb60153fe0 0x7fbb601440d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59962/0 (socket says 172.21.15.90:59962) 2024-03-11T20:51:27.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.928+0000 7fbb6492e640 1 -- 172.21.15.90:0/1686982986 learned_addr learned my addr 172.21.15.90:0/1686982986 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:51:27.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.928+0000 7fbb5ffff640 1 --2- 172.21.15.90:0/1686982986 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbb6006bdb0 0x7fbb60159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:27.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.932+0000 7fbb6492e640 1 -- 172.21.15.90:0/1686982986 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbb6006bdb0 msgr2=0x7fbb60159690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:27.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.932+0000 7fbb6492e640 1 --2- 172.21.15.90:0/1686982986 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbb6006bdb0 0x7fbb60159690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:27.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.932+0000 7fbb6492e640 1 -- 172.21.15.90:0/1686982986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb600cb7d0 msgr2=0x7fbb60143b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:27.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.932+0000 7fbb6492e640 1 --2- 172.21.15.90:0/1686982986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb600cb7d0 0x7fbb60143b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:27.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.932+0000 7fbb6492e640 1 -- 172.21.15.90:0/1686982986 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb58009a70 con 0x7fbb60153fe0 2024-03-11T20:51:27.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.936+0000 7fbb6492e640 1 --2- 172.21.15.90:0/1686982986 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb60153fe0 0x7fbb601440d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbb58019510 tx=0x7fbb580196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:27.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.940+0000 7fbb5cff9640 1 -- 172.21.15.90:0/1686982986 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb580aec60 con 0x7fbb60153fe0 2024-03-11T20:51:27.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.940+0000 7fbb5cff9640 1 -- 172.21.15.90:0/1686982986 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb580aedc0 con 0x7fbb60153fe0 2024-03-11T20:51:27.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.940+0000 7fbb5cff9640 1 -- 172.21.15.90:0/1686982986 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb580b7650 con 0x7fbb60153fe0 2024-03-11T20:51:27.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.940+0000 7fbb66391640 1 -- 172.21.15.90:0/1686982986 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb60142dd0 con 0x7fbb60153fe0 2024-03-11T20:51:27.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.940+0000 7fbb66391640 1 -- 172.21.15.90:0/1686982986 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb60143390 con 0x7fbb60153fe0 2024-03-11T20:51:27.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:27.940+0000 7fbb66391640 1 -- 172.21.15.90:0/1686982986 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb6004eef0 con 0x7fbb60153fe0 2024-03-11T20:51:28.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.228+0000 7fbb5cff9640 1 -- 172.21.15.90:0/1686982986 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fbb580b4050 con 0x7fbb60153fe0 2024-03-11T20:51:28.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.228+0000 7fbb5cff9640 1 --2- 172.21.15.90:0/1686982986 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fbb4c066e90 0x7fbb4c069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:28.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.228+0000 7fbb5ffff640 1 --2- 172.21.15.90:0/1686982986 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fbb4c066e90 0x7fbb4c069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:28.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.228+0000 7fbb5ffff640 1 --2- 172.21.15.90:0/1686982986 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fbb4c066e90 0x7fbb4c069350 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fbb500098f0 tx=0x7fbb50006f90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:28.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.264+0000 7fbb5cff9640 1 -- 172.21.15.90:0/1686982986 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fbb580f1620 con 0x7fbb60153fe0 2024-03-11T20:51:28.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.264+0000 7fbb5cff9640 1 -- 172.21.15.90:0/1686982986 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbb580ed1c0 con 0x7fbb60153fe0 2024-03-11T20:51:28.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.448+0000 7fbb66391640 1 -- 172.21.15.90:0/1686982986 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fbb6006cb60 con 0x7fbb60153fe0 2024-03-11T20:51:28.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.464+0000 7fbb5cff9640 1 -- 172.21.15.90:0/1686982986 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7fbb580b2070 con 0x7fbb60153fe0 2024-03-11T20:51:28.462 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 32 2024-03-11T20:51:28.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.464+0000 7fbb66391640 1 -- 172.21.15.90:0/1686982986 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fbb4c066e90 msgr2=0x7fbb4c069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:28.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.464+0000 7fbb66391640 1 --2- 172.21.15.90:0/1686982986 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fbb4c066e90 0x7fbb4c069350 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fbb500098f0 tx=0x7fbb50006f90 comp rx=0 tx=0).stop 2024-03-11T20:51:28.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.464+0000 7fbb66391640 1 -- 172.21.15.90:0/1686982986 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb60153fe0 msgr2=0x7fbb601440d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:28.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.464+0000 7fbb66391640 1 --2- 172.21.15.90:0/1686982986 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb60153fe0 0x7fbb601440d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbb58019510 tx=0x7fbb580196a0 comp rx=0 tx=0).stop 2024-03-11T20:51:28.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.464+0000 7fbb66391640 1 -- 172.21.15.90:0/1686982986 shutdown_connections 2024-03-11T20:51:28.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.464+0000 7fbb66391640 1 --2- 172.21.15.90:0/1686982986 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fbb4c066e90 0x7fbb4c069350 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:28.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.464+0000 7fbb66391640 1 --2- 172.21.15.90:0/1686982986 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb60153fe0 0x7fbb601440d0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:28.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.464+0000 7fbb66391640 1 --2- 172.21.15.90:0/1686982986 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb600cb7d0 0x7fbb60143b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:28.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.464+0000 7fbb66391640 1 --2- 172.21.15.90:0/1686982986 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbb6006bdb0 0x7fbb60159690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:28.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.464+0000 7fbb66391640 1 -- 172.21.15.90:0/1686982986 >> 172.21.15.90:0/1686982986 conn(0x7fbb60072500 msgr2=0x7fbb600cdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:28.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.468+0000 7fbb66391640 1 -- 172.21.15.90:0/1686982986 shutdown_connections 2024-03-11T20:51:28.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.468+0000 7fbb66391640 1 -- 172.21.15.90:0/1686982986 wait complete. 2024-03-11T20:51:28.483 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-11T20:51:28.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.608+0000 7fec41a2e640 1 Processor -- start 2024-03-11T20:51:28.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.608+0000 7fec41a2e640 1 -- start start 2024-03-11T20:51:28.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.612+0000 7fec41a2e640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fec3c153fc0 0x7fec3c159be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:28.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.612+0000 7fec41a2e640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fec3c0cb840 0x7fec3c06c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:28.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.612+0000 7fec41a2e640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fec3c0c46f0 0x7fec3c0cf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:28.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.612+0000 7fec41a2e640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec3c0b2dc0 con 0x7fec3c0cb840 2024-03-11T20:51:28.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.612+0000 7fec41a2e640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec3c1543a0 con 0x7fec3c0c46f0 2024-03-11T20:51:28.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.612+0000 7fec41a2e640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c15ab60 con 0x7fec3c153fc0 2024-03-11T20:51:28.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.612+0000 7fec3a7fc640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fec3c0cb840 0x7fec3c06c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:28.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.612+0000 7fec3a7fc640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fec3c0cb840 0x7fec3c06c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59978/0 (socket says 172.21.15.90:59978) 2024-03-11T20:51:28.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.612+0000 7fec3a7fc640 1 -- 172.21.15.90:0/3645836727 learned_addr learned my addr 172.21.15.90:0/3645836727 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:51:28.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.612+0000 7fec3b7fe640 1 --2- 172.21.15.90:0/3645836727 >> v2:172.21.15.92:3300/0 conn(0x7fec3c0c46f0 0x7fec3c0cf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:28.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.612+0000 7fec3affd640 1 --2- 172.21.15.90:0/3645836727 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fec3c153fc0 0x7fec3c159be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:28.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.616+0000 7fec3a7fc640 1 -- 172.21.15.90:0/3645836727 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fec3c153fc0 msgr2=0x7fec3c159be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:28.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.616+0000 7fec3a7fc640 1 --2- 172.21.15.90:0/3645836727 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fec3c153fc0 0x7fec3c159be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:28.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.616+0000 7fec3a7fc640 1 -- 172.21.15.90:0/3645836727 >> v2:172.21.15.92:3300/0 conn(0x7fec3c0c46f0 msgr2=0x7fec3c0cf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:28.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.616+0000 7fec3a7fc640 1 --2- 172.21.15.90:0/3645836727 >> v2:172.21.15.92:3300/0 conn(0x7fec3c0c46f0 0x7fec3c0cf960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:28.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.616+0000 7fec3a7fc640 1 -- 172.21.15.90:0/3645836727 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec3c15a2e0 con 0x7fec3c0cb840 2024-03-11T20:51:28.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.620+0000 7fec3a7fc640 1 --2- 172.21.15.90:0/3645836727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec3c0cb840 0x7fec3c06c3b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fec3c0747c0 tx=0x7fec3001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=204dffdbc079ec62 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:28.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.624+0000 7fec39ffb640 1 -- 172.21.15.90:0/3645836727 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec300ab5f0 con 0x7fec3c0cb840 2024-03-11T20:51:28.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.624+0000 7fec39ffb640 1 -- 172.21.15.90:0/3645836727 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec300ab750 con 0x7fec3c0cb840 2024-03-11T20:51:28.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.624+0000 7fec39ffb640 1 -- 172.21.15.90:0/3645836727 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec300abae0 con 0x7fec3c0cb840 2024-03-11T20:51:28.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.624+0000 7fec41a2e640 1 -- 172.21.15.90:0/3645836727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec3c0cb840 msgr2=0x7fec3c06c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:28.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.624+0000 7fec41a2e640 1 --2- 172.21.15.90:0/3645836727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec3c0cb840 0x7fec3c06c3b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fec3c0747c0 tx=0x7fec3001c520 comp rx=0 tx=0).stop 2024-03-11T20:51:28.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.624+0000 7fec41a2e640 1 -- 172.21.15.90:0/3645836727 shutdown_connections 2024-03-11T20:51:28.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.624+0000 7fec41a2e640 1 --2- 172.21.15.90:0/3645836727 >> v2:172.21.15.92:3300/0 conn(0x7fec3c0c46f0 0x7fec3c0cf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:28.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.624+0000 7fec41a2e640 1 --2- 172.21.15.90:0/3645836727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec3c0cb840 0x7fec3c06c3b0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:28.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.624+0000 7fec41a2e640 1 --2- 172.21.15.90:0/3645836727 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fec3c153fc0 0x7fec3c159be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:28.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.624+0000 7fec41a2e640 1 -- 172.21.15.90:0/3645836727 >> 172.21.15.90:0/3645836727 conn(0x7fec3c072730 msgr2=0x7fec3c072b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:28.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.624+0000 7fec41a2e640 1 -- 172.21.15.90:0/3645836727 shutdown_connections 2024-03-11T20:51:28.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.624+0000 7fec41a2e640 1 -- 172.21.15.90:0/3645836727 wait complete. 2024-03-11T20:51:28.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.624+0000 7fec41a2e640 1 Processor -- start 2024-03-11T20:51:28.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.624+0000 7fec41a2e640 1 -- start start 2024-03-11T20:51:28.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.628+0000 7fec41a2e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec3c0c46f0 0x7fec3c14b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:28.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.628+0000 7fec41a2e640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fec3c0cb840 0x7fec3c14c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:28.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.628+0000 7fec41a2e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec3c153fc0 0x7fec3c1f5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:28.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.628+0000 7fec41a2e640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c14bcd0 con 0x7fec3c0c46f0 2024-03-11T20:51:28.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.628+0000 7fec41a2e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c14a180 con 0x7fec3c153fc0 2024-03-11T20:51:28.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.628+0000 7fec41a2e640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c14d3e0 con 0x7fec3c0cb840 2024-03-11T20:51:28.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.628+0000 7fec3affd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec3c0c46f0 0x7fec3c14b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:28.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.628+0000 7fec3affd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec3c0c46f0 0x7fec3c14b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59994/0 (socket says 172.21.15.90:59994) 2024-03-11T20:51:28.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.628+0000 7fec3affd640 1 -- 172.21.15.90:0/2666505673 learned_addr learned my addr 172.21.15.90:0/2666505673 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:51:28.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.628+0000 7fec3a7fc640 1 --2- 172.21.15.90:0/2666505673 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fec3c0cb840 0x7fec3c14c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:28.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.628+0000 7fec3b7fe640 1 --2- 172.21.15.90:0/2666505673 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec3c153fc0 0x7fec3c1f5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:28.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.632+0000 7fec3affd640 1 -- 172.21.15.90:0/2666505673 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fec3c0cb840 msgr2=0x7fec3c14c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:28.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.632+0000 7fec3affd640 1 --2- 172.21.15.90:0/2666505673 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fec3c0cb840 0x7fec3c14c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:28.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.632+0000 7fec3affd640 1 -- 172.21.15.90:0/2666505673 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec3c153fc0 msgr2=0x7fec3c1f5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:28.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.632+0000 7fec3affd640 1 --2- 172.21.15.90:0/2666505673 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec3c153fc0 0x7fec3c1f5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:28.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.632+0000 7fec3affd640 1 -- 172.21.15.90:0/2666505673 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec300094d0 con 0x7fec3c0c46f0 2024-03-11T20:51:28.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.632+0000 7fec3affd640 1 --2- 172.21.15.90:0/2666505673 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec3c0c46f0 0x7fec3c14b790 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fec2c00d830 tx=0x7fec2c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:28.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.636+0000 7fec23fff640 1 -- 172.21.15.90:0/2666505673 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2c00b6c0 con 0x7fec3c0c46f0 2024-03-11T20:51:28.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.636+0000 7fec23fff640 1 -- 172.21.15.90:0/2666505673 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec2c00b820 con 0x7fec3c0c46f0 2024-03-11T20:51:28.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.636+0000 7fec23fff640 1 -- 172.21.15.90:0/2666505673 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2c00bb90 con 0x7fec3c0c46f0 2024-03-11T20:51:28.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.636+0000 7fec41a2e640 1 -- 172.21.15.90:0/2666505673 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec3c14ce60 con 0x7fec3c0c46f0 2024-03-11T20:51:28.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.636+0000 7fec41a2e640 1 -- 172.21.15.90:0/2666505673 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec3c14a380 con 0x7fec3c0c46f0 2024-03-11T20:51:28.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.636+0000 7fec41a2e640 1 -- 172.21.15.90:0/2666505673 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec3c04eef0 con 0x7fec3c0c46f0 2024-03-11T20:51:28.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.928+0000 7fec23fff640 1 -- 172.21.15.90:0/2666505673 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fec2c01e460 con 0x7fec3c0c46f0 2024-03-11T20:51:28.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.928+0000 7fec23fff640 1 --2- 172.21.15.90:0/2666505673 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fec28066e90 0x7fec28069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:28.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.928+0000 7fec3a7fc640 1 --2- 172.21.15.90:0/2666505673 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fec28066e90 0x7fec28069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:28.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.932+0000 7fec3a7fc640 1 --2- 172.21.15.90:0/2666505673 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fec28066e90 0x7fec28069350 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fec300af000 tx=0x7fec300b6040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:28.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.968+0000 7fec23fff640 1 -- 172.21.15.90:0/2666505673 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fec2c0566d0 con 0x7fec3c0c46f0 2024-03-11T20:51:28.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:28.968+0000 7fec23fff640 1 -- 172.21.15.90:0/2666505673 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fec2c052970 con 0x7fec3c0c46f0 2024-03-11T20:51:29.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:29.156+0000 7fec41a2e640 1 -- 172.21.15.90:0/2666505673 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fec3c0caf30 con 0x7fec3c0c46f0 2024-03-11T20:51:29.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:29.164+0000 7fec23fff640 1 -- 172.21.15.90:0/2666505673 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v30) v1 ==== 80+0+0 (secure 0 0 0) 0x7fec2c056030 con 0x7fec3c0c46f0 2024-03-11T20:51:29.164 INFO:teuthology.orchestra.run.smithi090.stderr:listed 0 entries 2024-03-11T20:51:29.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:29.168+0000 7fec41a2e640 1 -- 172.21.15.90:0/2666505673 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fec28066e90 msgr2=0x7fec28069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:29.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:29.168+0000 7fec41a2e640 1 --2- 172.21.15.90:0/2666505673 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fec28066e90 0x7fec28069350 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fec300af000 tx=0x7fec300b6040 comp rx=0 tx=0).stop 2024-03-11T20:51:29.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:29.168+0000 7fec41a2e640 1 -- 172.21.15.90:0/2666505673 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec3c0c46f0 msgr2=0x7fec3c14b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:29.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:29.168+0000 7fec41a2e640 1 --2- 172.21.15.90:0/2666505673 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec3c0c46f0 0x7fec3c14b790 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fec2c00d830 tx=0x7fec2c00dcf0 comp rx=0 tx=0).stop 2024-03-11T20:51:29.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:29.168+0000 7fec41a2e640 1 -- 172.21.15.90:0/2666505673 shutdown_connections 2024-03-11T20:51:29.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:29.168+0000 7fec41a2e640 1 --2- 172.21.15.90:0/2666505673 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fec28066e90 0x7fec28069350 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:29.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:29.168+0000 7fec41a2e640 1 --2- 172.21.15.90:0/2666505673 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fec3c153fc0 0x7fec3c1f5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:29.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:29.168+0000 7fec41a2e640 1 --2- 172.21.15.90:0/2666505673 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fec3c0cb840 0x7fec3c14c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:29.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:29.168+0000 7fec41a2e640 1 --2- 172.21.15.90:0/2666505673 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec3c0c46f0 0x7fec3c14b790 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:29.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:29.168+0000 7fec41a2e640 1 -- 172.21.15.90:0/2666505673 >> 172.21.15.90:0/2666505673 conn(0x7fec3c072730 msgr2=0x7fec3c0b2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:29.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:29.168+0000 7fec41a2e640 1 -- 172.21.15.90:0/2666505673 shutdown_connections 2024-03-11T20:51:29.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:29.168+0000 7fec41a2e640 1 -- 172.21.15.90:0/2666505673 wait complete. 2024-03-11T20:51:29.185 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi090.front.sepia.ceph.com... 2024-03-11T20:51:29.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T20:51:29.185 DEBUG:teuthology.orchestra.run.smithi090:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-11T20:51:29.196 INFO:teuthology.orchestra.run:waiting for 300 2024-03-11T20:51:29.236 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi090.stderr:ceph-fuse[36299]: fuse finished with error 0 and tester_r 0 2024-03-11T20:51:30.380 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi090.stderr:==00:02:14:21.418 36299== 2024-03-11T20:51:30.381 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi090.stderr:==00:02:14:21.418 36299== Exit program on first error (--exit-on-first-error=yes) 2024-03-11T20:51:30.598 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi090.stderr:daemon-helper: command failed with exit status 42 2024-03-11T20:51:31.655 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:51:31.658 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:51:35.298 DEBUG:teuthology.orchestra.run:got remote process result: 42 2024-03-11T20:51:35.298 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi090.front.sepia.ceph.com 2024-03-11T20:51:35.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T20:51:35.299 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-11T20:51:35.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T20:51:35.306 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list 2024-03-11T20:51:35.354 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-11T20:51:35.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T20:51:35.354 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-11T20:51:35.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-11T20:51:35.411 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link delete ceph-brx 2024-03-11T20:51:35.607 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-03-11T20:51:35.617 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-11T20:51:35.617 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-11T20:51:35.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:35.744+0000 7fe9f3484640 1 Processor -- start 2024-03-11T20:51:35.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:35.744+0000 7fe9f3484640 1 -- start start 2024-03-11T20:51:35.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:35.744+0000 7fe9f3484640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe9ec0c6cf0 0x7fe9ec165590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:35.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:35.744+0000 7fe9f3484640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9ec0c7d50 con 0x7fe9ec06f400 2024-03-11T20:51:35.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:35.744+0000 7fe9f3484640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9ec0703f0 con 0x7fe9ec1540c0 2024-03-11T20:51:35.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:35.744+0000 7fe9f3484640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9ec15d900 con 0x7fe9ec0c6cf0 2024-03-11T20:51:35.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:35.744+0000 7fe9f1a21640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe9ec0c6cf0 0x7fe9ec165590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:35.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:35.748+0000 7fe9f1a21640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fe9ec0c6cf0 0x7fe9ec165590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.90:50078/0 (socket says 172.21.15.90:50078) 2024-03-11T20:51:35.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:35.748+0000 7fe9f1a21640 1 -- 172.21.15.90:0/949806688 learned_addr learned my addr 172.21.15.90:0/949806688 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:51:35.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:35.752+0000 7fe9f1a21640 1 -- 172.21.15.90:0/949806688 >> v1:172.21.15.92:6789/0 conn(0x7fe9ec1540c0 legacy=0x7fe9ec15d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:35.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:35.752+0000 7fe9f1a21640 1 -- 172.21.15.90:0/949806688 >> v1:172.21.15.90:6789/0 conn(0x7fe9ec06f400 legacy=0x7fe9ec0cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:35.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:35.752+0000 7fe9f1a21640 1 -- 172.21.15.90:0/949806688 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9ec06b480 con 0x7fe9ec0c6cf0 2024-03-11T20:51:35.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:35.764+0000 7fe9f1a21640 1 --2- 172.21.15.90:0/949806688 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9ec0c6cf0 0x7fe9ec165590 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4009870 tx=0x7fe9e401c4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de7f0c3fd401bf37 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:35.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:35.780+0000 7fe9e3fff640 1 -- 172.21.15.90:0/949806688 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e40aa650 con 0x7fe9ec0c6cf0 2024-03-11T20:51:35.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:35.780+0000 7fe9e3fff640 1 -- 172.21.15.90:0/949806688 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9e40aa7b0 con 0x7fe9ec0c6cf0 2024-03-11T20:51:35.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:35.780+0000 7fe9e3fff640 1 -- 172.21.15.90:0/949806688 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e40aab40 con 0x7fe9ec0c6cf0 2024-03-11T20:51:35.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:35.780+0000 7fe9f3484640 1 -- 172.21.15.90:0/949806688 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9ec0c6cf0 msgr2=0x7fe9ec165590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:35.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:35.780+0000 7fe9f3484640 1 --2- 172.21.15.90:0/949806688 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9ec0c6cf0 0x7fe9ec165590 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4009870 tx=0x7fe9e401c4f0 comp rx=0 tx=0).stop 2024-03-11T20:51:35.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:35.780+0000 7fe9f3484640 1 -- 172.21.15.90:0/949806688 shutdown_connections 2024-03-11T20:51:35.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:35.780+0000 7fe9f3484640 1 --2- 172.21.15.90:0/949806688 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9ec0c6cf0 0x7fe9ec165590 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:35.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:35.780+0000 7fe9f3484640 1 -- 172.21.15.90:0/949806688 >> 172.21.15.90:0/949806688 conn(0x7fe9ec071aa0 msgr2=0x7fe9ec071eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:35.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:35.780+0000 7fe9f3484640 1 -- 172.21.15.90:0/949806688 shutdown_connections 2024-03-11T20:51:35.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:35.780+0000 7fe9f3484640 1 -- 172.21.15.90:0/949806688 wait complete. 2024-03-11T20:51:35.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:35.784+0000 7fe9f3484640 1 Processor -- start 2024-03-11T20:51:35.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:35.784+0000 7fe9f3484640 1 -- start start 2024-03-11T20:51:35.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:35.784+0000 7fe9f3484640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9ec06f400 0x7fe9ec1f3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:35.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:35.784+0000 7fe9f3484640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe9ec0c6cf0 0x7fe9ec0cb120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:35.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:35.784+0000 7fe9f3484640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9ec1540c0 0x7fe9ec1454d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:35.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:35.784+0000 7fe9f3484640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ec0cc820 con 0x7fe9ec1540c0 2024-03-11T20:51:35.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:35.784+0000 7fe9f3484640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ec0cac90 con 0x7fe9ec06f400 2024-03-11T20:51:35.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:35.784+0000 7fe9f3484640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ec14cef0 con 0x7fe9ec0c6cf0 2024-03-11T20:51:35.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:35.784+0000 7fe9f1a21640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9ec1540c0 0x7fe9ec1454d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:35.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:35.784+0000 7fe9f1a21640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9ec1540c0 0x7fe9ec1454d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58476/0 (socket says 172.21.15.90:58476) 2024-03-11T20:51:35.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:35.784+0000 7fe9f1a21640 1 -- 172.21.15.90:0/1099493336 learned_addr learned my addr 172.21.15.90:0/1099493336 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:51:35.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:35.788+0000 7fe9f1a21640 1 -- 172.21.15.90:0/1099493336 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe9ec0c6cf0 msgr2=0x7fe9ec0cb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:35.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:35.788+0000 7fe9f1a21640 1 --2- 172.21.15.90:0/1099493336 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe9ec0c6cf0 0x7fe9ec0cb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:35.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:35.788+0000 7fe9f1a21640 1 -- 172.21.15.90:0/1099493336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9ec06f400 msgr2=0x7fe9ec1f3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:35.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:35.788+0000 7fe9f1a21640 1 --2- 172.21.15.90:0/1099493336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9ec06f400 0x7fe9ec1f3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:35.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:35.788+0000 7fe9f1a21640 1 -- 172.21.15.90:0/1099493336 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e40094d0 con 0x7fe9ec1540c0 2024-03-11T20:51:35.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:35.792+0000 7fe9f1a21640 1 --2- 172.21.15.90:0/1099493336 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9ec1540c0 0x7fe9ec1454d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe9e40ac040 tx=0x7fe9e401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:35.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:35.796+0000 7fe9e1ffb640 1 -- 172.21.15.90:0/1099493336 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e40aa650 con 0x7fe9ec1540c0 2024-03-11T20:51:35.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:35.796+0000 7fe9e1ffb640 1 -- 172.21.15.90:0/1099493336 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9e40b3800 con 0x7fe9ec1540c0 2024-03-11T20:51:35.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:35.796+0000 7fe9f3484640 1 -- 172.21.15.90:0/1099493336 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9ec0c8360 con 0x7fe9ec1540c0 2024-03-11T20:51:35.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:35.796+0000 7fe9e1ffb640 1 -- 172.21.15.90:0/1099493336 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e40b3bb0 con 0x7fe9ec1540c0 2024-03-11T20:51:35.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:35.796+0000 7fe9f3484640 1 -- 172.21.15.90:0/1099493336 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9ec14d430 con 0x7fe9ec1540c0 2024-03-11T20:51:35.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:35.796+0000 7fe9f3484640 1 -- 172.21.15.90:0/1099493336 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9ec04eef0 con 0x7fe9ec1540c0 2024-03-11T20:51:35.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:35.816+0000 7fe9e1ffb640 1 -- 172.21.15.90:0/1099493336 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fe9e40b3d10 con 0x7fe9ec1540c0 2024-03-11T20:51:35.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:35.816+0000 7fe9e1ffb640 1 --2- 172.21.15.90:0/1099493336 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe9d0066e90 0x7fe9d0069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:35.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:35.816+0000 7fe9f1220640 1 --2- 172.21.15.90:0/1099493336 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe9d0066e90 0x7fe9d0069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:35.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:35.820+0000 7fe9f1220640 1 --2- 172.21.15.90:0/1099493336 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe9d0066e90 0x7fe9d0069350 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe9e8007fd0 tx=0x7fe9e8007d40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:35.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:35.820+0000 7fe9e1ffb640 1 -- 172.21.15.90:0/1099493336 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fe9e40f1360 con 0x7fe9ec1540c0 2024-03-11T20:51:36.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.096+0000 7fe9e1ffb640 1 -- 172.21.15.90:0/1099493336 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe9e40ecf00 con 0x7fe9ec1540c0 2024-03-11T20:51:36.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.260+0000 7fe9f3484640 1 -- 172.21.15.90:0/1099493336 --> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe9ec06ba40 con 0x7fe9d0066e90 2024-03-11T20:51:36.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.268+0000 7fe9e1ffb640 1 -- 172.21.15.90:0/1099493336 <== mgr.4101 v2:172.21.15.90:6800/1999720161 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+334701 (secure 0 0 0) 0x7fe9e8004960 con 0x7fe9d0066e90 2024-03-11T20:51:36.273 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T20:51:36.275 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-03-11T20:51:36.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.276+0000 7fe9f3484640 1 -- 172.21.15.90:0/1099493336 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe9d0066e90 msgr2=0x7fe9d0069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:36.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.276+0000 7fe9f3484640 1 --2- 172.21.15.90:0/1099493336 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe9d0066e90 0x7fe9d0069350 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe9e8007fd0 tx=0x7fe9e8007d40 comp rx=0 tx=0).stop 2024-03-11T20:51:36.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.276+0000 7fe9f3484640 1 -- 172.21.15.90:0/1099493336 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9ec1540c0 msgr2=0x7fe9ec1454d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:36.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.276+0000 7fe9f3484640 1 --2- 172.21.15.90:0/1099493336 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9ec1540c0 0x7fe9ec1454d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe9e40ac040 tx=0x7fe9e401ad30 comp rx=0 tx=0).stop 2024-03-11T20:51:36.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.276+0000 7fe9f3484640 1 -- 172.21.15.90:0/1099493336 shutdown_connections 2024-03-11T20:51:36.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.276+0000 7fe9f3484640 1 --2- 172.21.15.90:0/1099493336 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fe9d0066e90 0x7fe9d0069350 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:36.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.276+0000 7fe9f3484640 1 --2- 172.21.15.90:0/1099493336 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9ec1540c0 0x7fe9ec1454d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:36.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.276+0000 7fe9f3484640 1 --2- 172.21.15.90:0/1099493336 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fe9ec0c6cf0 0x7fe9ec0cb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:36.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.276+0000 7fe9f3484640 1 --2- 172.21.15.90:0/1099493336 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fe9ec06f400 0x7fe9ec1f3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:36.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.276+0000 7fe9f3484640 1 -- 172.21.15.90:0/1099493336 >> 172.21.15.90:0/1099493336 conn(0x7fe9ec071aa0 msgr2=0x7fe9ec15c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:36.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.276+0000 7fe9f3484640 1 -- 172.21.15.90:0/1099493336 shutdown_connections 2024-03-11T20:51:36.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.276+0000 7fe9f3484640 1 -- 172.21.15.90:0/1099493336 wait complete. 2024-03-11T20:51:36.292 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":4115,"stamp":"2024-03-11T20:51:35.451217+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":211048067,"num_objects":150,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":150,"num_whiteouts":0,"num_read":7774,"num_read_kb":1584288,"num_write":72048,"num_write_kb":4420358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76413,"ondisk_log_size":76413,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1577152,"kb_used_data":414912,"kb_used_omap":12,"kb_used_meta":1162163,"kb_avail":753397568,"statfs":{"total":773094113280,"available":771479109632,"internally_reserved":0,"allocated":424869888,"data_stored":423141350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":1190055476},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7707,"num_objects":-1,"num_object_clones":0,"num_object_copies":-4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":25,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005057"},"pg_stats":[{"pgid":"2.1f","version":"30'681","reported_seq":777,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:18.510855+0000","last_change":"2024-03-11T18:35:56.290140+0000","last_active":"2024-03-11T20:51:18.510855+0000","last_peered":"2024-03-11T20:51:18.510855+0000","last_clean":"2024-03-11T20:51:18.510855+0000","last_became_active":"2024-03-11T18:35:56.289178+0000","last_became_peered":"2024-03-11T18:35:56.289178+0000","last_unstale":"2024-03-11T20:51:18.510855+0000","last_undegraded":"2024-03-11T20:51:18.510855+0000","last_fullsized":"2024-03-11T20:51:18.510855+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":681,"log_dups_size":0,"ondisk_log_size":681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:59:46.211033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389280,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":643,"num_write_kb":8491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"30'202","reported_seq":298,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.761828+0000","last_change":"2024-03-11T18:35:57.676047+0000","last_active":"2024-03-11T20:51:29.761828+0000","last_peered":"2024-03-11T20:51:29.761828+0000","last_clean":"2024-03-11T20:51:29.761828+0000","last_became_active":"2024-03-11T18:35:57.675105+0000","last_became_peered":"2024-03-11T18:35:57.675105+0000","last_unstale":"2024-03-11T20:51:29.761828+0000","last_undegraded":"2024-03-11T20:51:29.761828+0000","last_fullsized":"2024-03-11T20:51:29.761828+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:45:39.057074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"30'811","reported_seq":1003,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:10.788625+0000","last_change":"2024-03-11T18:36:04.607351+0000","last_active":"2024-03-11T20:51:10.788625+0000","last_peered":"2024-03-11T20:51:10.788625+0000","last_clean":"2024-03-11T20:51:10.788625+0000","last_became_active":"2024-03-11T18:36:04.599562+0000","last_became_peered":"2024-03-11T18:36:04.599562+0000","last_unstale":"2024-03-11T20:51:10.788625+0000","last_undegraded":"2024-03-11T20:51:10.788625+0000","last_fullsized":"2024-03-11T20:51:10.788625+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":811,"log_dups_size":0,"ondisk_log_size":811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T18:57:51.134410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":119,"num_read_kb":64172,"num_write":795,"num_write_kb":180282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"30'654","reported_seq":752,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:49:28.711104+0000","last_change":"2024-03-11T18:35:56.291253+0000","last_active":"2024-03-11T20:49:28.711104+0000","last_peered":"2024-03-11T20:49:28.711104+0000","last_clean":"2024-03-11T20:49:28.711104+0000","last_became_active":"2024-03-11T18:35:56.290862+0000","last_became_peered":"2024-03-11T18:35:56.290862+0000","last_unstale":"2024-03-11T20:49:28.711104+0000","last_undegraded":"2024-03-11T20:49:28.711104+0000","last_fullsized":"2024-03-11T20:49:28.711104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":654,"log_dups_size":0,"ondisk_log_size":654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:36:17.809744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":603,"num_write_kb":8489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"30'187","reported_seq":286,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.887083+0000","last_change":"2024-03-11T18:37:04.143178+0000","last_active":"2024-03-11T20:51:29.887083+0000","last_peered":"2024-03-11T20:51:29.887083+0000","last_clean":"2024-03-11T20:51:29.887083+0000","last_became_active":"2024-03-11T18:37:04.142880+0000","last_became_peered":"2024-03-11T18:37:04.142880+0000","last_unstale":"2024-03-11T20:51:29.887083+0000","last_undegraded":"2024-03-11T20:51:29.887083+0000","last_fullsized":"2024-03-11T20:51:29.887083+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:58:36.259380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"30'785","reported_seq":972,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:17.717779+0000","last_change":"2024-03-11T18:37:04.274577+0000","last_active":"2024-03-11T20:51:17.717779+0000","last_peered":"2024-03-11T20:51:17.717779+0000","last_clean":"2024-03-11T20:51:17.717779+0000","last_became_active":"2024-03-11T18:37:04.274018+0000","last_became_peered":"2024-03-11T18:37:04.274018+0000","last_unstale":"2024-03-11T20:51:17.717779+0000","last_undegraded":"2024-03-11T20:51:17.717779+0000","last_fullsized":"2024-03-11T20:51:17.717779+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":785,"log_dups_size":0,"ondisk_log_size":785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:51:28.017349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":106,"num_read_kb":55696,"num_write":774,"num_write_kb":160695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2,3],"acting":[4,6,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"30'1734","reported_seq":1819,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:50:57.274928+0000","last_change":"2024-03-11T18:35:56.355147+0000","last_active":"2024-03-11T20:50:57.274928+0000","last_peered":"2024-03-11T20:50:57.274928+0000","last_clean":"2024-03-11T20:50:57.274928+0000","last_became_active":"2024-03-11T18:35:56.354388+0000","last_became_peered":"2024-03-11T18:35:56.354388+0000","last_unstale":"2024-03-11T20:50:57.274928+0000","last_undegraded":"2024-03-11T20:50:57.274928+0000","last_fullsized":"2024-03-11T20:50:57.274928+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1734,"log_dups_size":0,"ondisk_log_size":1734,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:03:55.046045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1669,"num_write_kb":17224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"30'190","reported_seq":273,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.629989+0000","last_change":"2024-03-11T18:35:57.641486+0000","last_active":"2024-03-11T20:51:29.629989+0000","last_peered":"2024-03-11T20:51:29.629989+0000","last_clean":"2024-03-11T20:51:29.629989+0000","last_became_active":"2024-03-11T18:35:57.641052+0000","last_became_peered":"2024-03-11T18:35:57.641052+0000","last_unstale":"2024-03-11T20:51:29.629989+0000","last_undegraded":"2024-03-11T20:51:29.629989+0000","last_fullsized":"2024-03-11T20:51:29.629989+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:29:45.662029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"30'808","reported_seq":973,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:08.067327+0000","last_change":"2024-03-11T18:36:04.629291+0000","last_active":"2024-03-11T20:51:08.067327+0000","last_peered":"2024-03-11T20:51:08.067327+0000","last_clean":"2024-03-11T20:51:08.067327+0000","last_became_active":"2024-03-11T18:36:04.624110+0000","last_became_peered":"2024-03-11T18:36:04.624110+0000","last_unstale":"2024-03-11T20:51:08.067327+0000","last_undegraded":"2024-03-11T20:51:08.067327+0000","last_fullsized":"2024-03-11T20:51:08.067327+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":808,"log_dups_size":0,"ondisk_log_size":808,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:04:03.368322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":93,"num_read_kb":49009,"num_write":797,"num_write_kb":196902,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"30'832","reported_seq":917,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:50:57.274905+0000","last_change":"2024-03-11T18:35:56.298903+0000","last_active":"2024-03-11T20:50:57.274905+0000","last_peered":"2024-03-11T20:50:57.274905+0000","last_clean":"2024-03-11T20:50:57.274905+0000","last_became_active":"2024-03-11T18:35:56.297582+0000","last_became_peered":"2024-03-11T18:35:56.297582+0000","last_unstale":"2024-03-11T20:50:57.274905+0000","last_undegraded":"2024-03-11T20:50:57.274905+0000","last_fullsized":"2024-03-11T20:50:57.274905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":832,"log_dups_size":0,"ondisk_log_size":832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:53:25.278916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":777,"num_write_kb":4502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"30'203","reported_seq":286,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.623592+0000","last_change":"2024-03-11T18:35:57.640122+0000","last_active":"2024-03-11T20:51:29.623592+0000","last_peered":"2024-03-11T20:51:29.623592+0000","last_clean":"2024-03-11T20:51:29.623592+0000","last_became_active":"2024-03-11T18:35:57.639770+0000","last_became_peered":"2024-03-11T18:35:57.639770+0000","last_unstale":"2024-03-11T20:51:29.623592+0000","last_undegraded":"2024-03-11T20:51:29.623592+0000","last_fullsized":"2024-03-11T20:51:29.623592+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:39:25.767184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"30'729","reported_seq":825,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:08.238310+0000","last_change":"2024-03-11T18:36:04.627149+0000","last_active":"2024-03-11T20:51:08.238310+0000","last_peered":"2024-03-11T20:51:08.238310+0000","last_clean":"2024-03-11T20:51:08.238310+0000","last_became_active":"2024-03-11T18:36:04.626108+0000","last_became_peered":"2024-03-11T18:36:04.626108+0000","last_unstale":"2024-03-11T20:51:08.238310+0000","last_undegraded":"2024-03-11T20:51:08.238310+0000","last_fullsized":"2024-03-11T20:51:08.238310+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":729,"log_dups_size":0,"ondisk_log_size":729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:03:40.619618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":20804,"num_write":712,"num_write_kb":76413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"30'216","reported_seq":307,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.960165+0000","last_change":"2024-03-11T18:35:57.616739+0000","last_active":"2024-03-11T20:51:29.960165+0000","last_peered":"2024-03-11T20:51:29.960165+0000","last_clean":"2024-03-11T20:51:29.960165+0000","last_became_active":"2024-03-11T18:35:57.616398+0000","last_became_peered":"2024-03-11T18:35:57.616398+0000","last_unstale":"2024-03-11T20:51:29.960165+0000","last_undegraded":"2024-03-11T20:51:29.960165+0000","last_fullsized":"2024-03-11T20:51:29.960165+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:53:30.324898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"30'1448","reported_seq":1539,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:49:55.997509+0000","last_change":"2024-03-11T18:35:56.304762+0000","last_active":"2024-03-11T20:49:55.997509+0000","last_peered":"2024-03-11T20:49:55.997509+0000","last_clean":"2024-03-11T20:49:55.997509+0000","last_became_active":"2024-03-11T18:35:56.302803+0000","last_became_peered":"2024-03-11T18:35:56.302803+0000","last_unstale":"2024-03-11T20:49:55.997509+0000","last_undegraded":"2024-03-11T20:49:55.997509+0000","last_fullsized":"2024-03-11T20:49:55.997509+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1448,"log_dups_size":0,"ondisk_log_size":1448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:31:18.726996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1389,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"30'793","reported_seq":965,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:50:56.185068+0000","last_change":"2024-03-11T18:36:04.626968+0000","last_active":"2024-03-11T20:50:56.185068+0000","last_peered":"2024-03-11T20:50:56.185068+0000","last_clean":"2024-03-11T20:50:56.185068+0000","last_became_active":"2024-03-11T18:36:04.626337+0000","last_became_peered":"2024-03-11T18:36:04.626337+0000","last_unstale":"2024-03-11T20:50:56.185068+0000","last_undegraded":"2024-03-11T20:50:56.185068+0000","last_fullsized":"2024-03-11T20:50:56.185068+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":793,"log_dups_size":0,"ondisk_log_size":793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:53:37.432826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":94,"num_read_kb":46134,"num_write":781,"num_write_kb":151379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"30'211","reported_seq":300,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.763712+0000","last_change":"2024-03-11T18:35:57.689835+0000","last_active":"2024-03-11T20:51:29.763712+0000","last_peered":"2024-03-11T20:51:29.763712+0000","last_clean":"2024-03-11T20:51:29.763712+0000","last_became_active":"2024-03-11T18:35:57.689039+0000","last_became_peered":"2024-03-11T18:35:57.689039+0000","last_unstale":"2024-03-11T20:51:29.763712+0000","last_undegraded":"2024-03-11T20:51:29.763712+0000","last_fullsized":"2024-03-11T20:51:29.763712+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:38:27.733669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"30'1235","reported_seq":1328,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:49:27.679855+0000","last_change":"2024-03-11T18:35:56.291070+0000","last_active":"2024-03-11T20:49:27.679855+0000","last_peered":"2024-03-11T20:49:27.679855+0000","last_clean":"2024-03-11T20:49:27.679855+0000","last_became_active":"2024-03-11T18:35:56.290504+0000","last_became_peered":"2024-03-11T18:35:56.290504+0000","last_unstale":"2024-03-11T20:49:27.679855+0000","last_undegraded":"2024-03-11T20:49:27.679855+0000","last_fullsized":"2024-03-11T20:49:27.679855+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1235,"log_dups_size":0,"ondisk_log_size":1235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:29:31.660243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1168,"num_write_kb":8796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"30'848","reported_seq":1062,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:12.395257+0000","last_change":"2024-03-11T18:36:04.626150+0000","last_active":"2024-03-11T20:51:12.395257+0000","last_peered":"2024-03-11T20:51:12.395257+0000","last_clean":"2024-03-11T20:51:12.395257+0000","last_became_active":"2024-03-11T18:36:04.625084+0000","last_became_peered":"2024-03-11T18:36:04.625084+0000","last_unstale":"2024-03-11T20:51:12.395257+0000","last_undegraded":"2024-03-11T20:51:12.395257+0000","last_fullsized":"2024-03-11T20:51:12.395257+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":848,"log_dups_size":0,"ondisk_log_size":848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:28:58.373219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":136,"num_read_kb":82099,"num_write":832,"num_write_kb":151378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"30'209","reported_seq":300,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.771259+0000","last_change":"2024-03-11T18:35:57.684535+0000","last_active":"2024-03-11T20:51:29.771259+0000","last_peered":"2024-03-11T20:51:29.771259+0000","last_clean":"2024-03-11T20:51:29.771259+0000","last_became_active":"2024-03-11T18:35:57.681098+0000","last_became_peered":"2024-03-11T18:35:57.681098+0000","last_unstale":"2024-03-11T20:51:29.771259+0000","last_undegraded":"2024-03-11T20:51:29.771259+0000","last_fullsized":"2024-03-11T20:51:29.771259+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:25:49.707568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"30'483","reported_seq":576,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:49:29.907364+0000","last_change":"2024-03-11T18:35:56.290745+0000","last_active":"2024-03-11T20:49:29.907364+0000","last_peered":"2024-03-11T20:49:29.907364+0000","last_clean":"2024-03-11T20:49:29.907364+0000","last_became_active":"2024-03-11T18:35:56.290006+0000","last_became_peered":"2024-03-11T18:35:56.290006+0000","last_unstale":"2024-03-11T20:49:29.907364+0000","last_undegraded":"2024-03-11T20:49:29.907364+0000","last_fullsized":"2024-03-11T20:49:29.907364+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:06:02.844765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194394,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":433,"num_write_kb":4314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"30'749","reported_seq":894,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:08.589998+0000","last_change":"2024-03-11T18:36:04.620689+0000","last_active":"2024-03-11T20:51:08.589998+0000","last_peered":"2024-03-11T20:51:08.589998+0000","last_clean":"2024-03-11T20:51:08.589998+0000","last_became_active":"2024-03-11T18:36:04.620282+0000","last_became_peered":"2024-03-11T18:36:04.620282+0000","last_unstale":"2024-03-11T20:51:08.589998+0000","last_undegraded":"2024-03-11T20:51:08.589998+0000","last_fullsized":"2024-03-11T20:51:08.589998+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":749,"log_dups_size":0,"ondisk_log_size":749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:54:16.807299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":68,"num_read_kb":34102,"num_write":731,"num_write_kb":109760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"30'224","reported_seq":315,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.532383+0000","last_change":"2024-03-11T18:35:57.615234+0000","last_active":"2024-03-11T20:51:29.532383+0000","last_peered":"2024-03-11T20:51:29.532383+0000","last_clean":"2024-03-11T20:51:29.532383+0000","last_became_active":"2024-03-11T18:35:57.614885+0000","last_became_peered":"2024-03-11T18:35:57.614885+0000","last_unstale":"2024-03-11T20:51:29.532383+0000","last_undegraded":"2024-03-11T20:51:29.532383+0000","last_fullsized":"2024-03-11T20:51:29.532383+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":224,"log_dups_size":0,"ondisk_log_size":224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:22:13.730630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"30'348","reported_seq":441,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:50:17.268278+0000","last_change":"2024-03-11T18:35:56.304617+0000","last_active":"2024-03-11T20:50:17.268278+0000","last_peered":"2024-03-11T20:50:17.268278+0000","last_clean":"2024-03-11T20:50:17.268278+0000","last_became_active":"2024-03-11T18:35:56.301698+0000","last_became_peered":"2024-03-11T18:35:56.301698+0000","last_unstale":"2024-03-11T20:50:17.268278+0000","last_undegraded":"2024-03-11T20:50:17.268278+0000","last_fullsized":"2024-03-11T20:50:17.268278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:29:31.660243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":289,"num_write_kb":4232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"30'714","reported_seq":907,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:15.480325+0000","last_change":"2024-03-11T18:37:04.272234+0000","last_active":"2024-03-11T20:51:15.480325+0000","last_peered":"2024-03-11T20:51:15.480325+0000","last_clean":"2024-03-11T20:51:15.480325+0000","last_became_active":"2024-03-11T18:37:04.271646+0000","last_became_peered":"2024-03-11T18:37:04.271646+0000","last_unstale":"2024-03-11T20:51:15.480325+0000","last_undegraded":"2024-03-11T20:51:15.480325+0000","last_fullsized":"2024-03-11T20:51:15.480325+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:51:13.205384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":108,"num_read_kb":62192,"num_write":690,"num_write_kb":100980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,2],"acting":[4,3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"30'216","reported_seq":307,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.862313+0000","last_change":"2024-03-11T18:35:57.506963+0000","last_active":"2024-03-11T20:51:29.862313+0000","last_peered":"2024-03-11T20:51:29.862313+0000","last_clean":"2024-03-11T20:51:29.862313+0000","last_became_active":"2024-03-11T18:35:57.506038+0000","last_became_peered":"2024-03-11T18:35:57.506038+0000","last_unstale":"2024-03-11T20:51:29.862313+0000","last_undegraded":"2024-03-11T20:51:29.862313+0000","last_fullsized":"2024-03-11T20:51:29.862313+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:52:06.024245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"30'413","reported_seq":506,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:50:17.304480+0000","last_change":"2024-03-11T18:35:56.364512+0000","last_active":"2024-03-11T20:50:17.304480+0000","last_peered":"2024-03-11T20:50:17.304480+0000","last_clean":"2024-03-11T20:50:17.304480+0000","last_became_active":"2024-03-11T18:35:56.363731+0000","last_became_peered":"2024-03-11T18:35:56.363731+0000","last_unstale":"2024-03-11T20:50:17.304480+0000","last_undegraded":"2024-03-11T20:50:17.304480+0000","last_fullsized":"2024-03-11T20:50:17.304480+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:36:17.809744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":342,"num_write_kb":4261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"30'829","reported_seq":1056,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:14.415675+0000","last_change":"2024-03-11T18:36:04.643782+0000","last_active":"2024-03-11T20:51:14.415675+0000","last_peered":"2024-03-11T20:51:14.415675+0000","last_clean":"2024-03-11T20:51:14.415675+0000","last_became_active":"2024-03-11T18:36:04.642133+0000","last_became_peered":"2024-03-11T18:36:04.642133+0000","last_unstale":"2024-03-11T20:51:14.415675+0000","last_undegraded":"2024-03-11T20:51:14.415675+0000","last_fullsized":"2024-03-11T20:51:14.415675+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":829,"log_dups_size":0,"ondisk_log_size":829,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:53:33.601193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":144,"num_read_kb":78357,"num_write":819,"num_write_kb":238245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"30'168","reported_seq":259,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.764491+0000","last_change":"2024-03-11T18:35:57.675207+0000","last_active":"2024-03-11T20:51:29.764491+0000","last_peered":"2024-03-11T20:51:29.764491+0000","last_clean":"2024-03-11T20:51:29.764491+0000","last_became_active":"2024-03-11T18:35:57.674472+0000","last_became_peered":"2024-03-11T18:35:57.674472+0000","last_unstale":"2024-03-11T20:51:29.764491+0000","last_undegraded":"2024-03-11T20:51:29.764491+0000","last_fullsized":"2024-03-11T20:51:29.764491+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:29:45.662029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"30'6434","reported_seq":6527,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:22.287372+0000","last_change":"2024-03-11T18:35:56.348479+0000","last_active":"2024-03-11T20:51:22.287372+0000","last_peered":"2024-03-11T20:51:22.287372+0000","last_clean":"2024-03-11T20:51:22.287372+0000","last_became_active":"2024-03-11T18:35:56.348040+0000","last_became_peered":"2024-03-11T18:35:56.348040+0000","last_unstale":"2024-03-11T20:51:22.287372+0000","last_undegraded":"2024-03-11T20:51:22.287372+0000","last_fullsized":"2024-03-11T20:51:22.287372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":6434,"log_dups_size":0,"ondisk_log_size":6434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:27:26.471550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17421,"num_write_kb":6249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"30'799","reported_seq":995,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:50:55.170434+0000","last_change":"2024-03-11T18:36:04.599805+0000","last_active":"2024-03-11T20:50:55.170434+0000","last_peered":"2024-03-11T20:50:55.170434+0000","last_clean":"2024-03-11T20:50:55.170434+0000","last_became_active":"2024-03-11T18:36:04.599065+0000","last_became_peered":"2024-03-11T18:36:04.599065+0000","last_unstale":"2024-03-11T20:50:55.170434+0000","last_undegraded":"2024-03-11T20:50:55.170434+0000","last_fullsized":"2024-03-11T20:50:55.170434+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":799,"log_dups_size":0,"ondisk_log_size":799,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:36:07.187003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":111,"num_read_kb":58643,"num_write":786,"num_write_kb":122786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"30'199","reported_seq":293,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.664765+0000","last_change":"2024-03-11T18:35:57.538551+0000","last_active":"2024-03-11T20:51:29.664765+0000","last_peered":"2024-03-11T20:51:29.664765+0000","last_clean":"2024-03-11T20:51:29.664765+0000","last_became_active":"2024-03-11T18:35:57.537706+0000","last_became_peered":"2024-03-11T18:35:57.537706+0000","last_unstale":"2024-03-11T20:51:29.664765+0000","last_undegraded":"2024-03-11T20:51:29.664765+0000","last_fullsized":"2024-03-11T20:51:29.664765+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:03:56.260394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"30'985","reported_seq":1080,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:49:47.414091+0000","last_change":"2024-03-11T18:35:56.302575+0000","last_active":"2024-03-11T20:49:47.414091+0000","last_peered":"2024-03-11T20:49:47.414091+0000","last_clean":"2024-03-11T20:49:47.414091+0000","last_became_active":"2024-03-11T18:35:56.301819+0000","last_became_peered":"2024-03-11T18:35:56.301819+0000","last_unstale":"2024-03-11T20:49:47.414091+0000","last_undegraded":"2024-03-11T20:49:47.414091+0000","last_fullsized":"2024-03-11T20:49:47.414091+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":985,"log_dups_size":0,"ondisk_log_size":985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:38:26.519320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":935,"num_write_kb":8676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"30'751","reported_seq":913,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:14.544182+0000","last_change":"2024-03-11T18:36:04.609282+0000","last_active":"2024-03-11T20:51:14.544182+0000","last_peered":"2024-03-11T20:51:14.544182+0000","last_clean":"2024-03-11T20:51:14.544182+0000","last_became_active":"2024-03-11T18:36:04.608260+0000","last_became_peered":"2024-03-11T18:36:04.608260+0000","last_unstale":"2024-03-11T20:51:14.544182+0000","last_undegraded":"2024-03-11T20:51:14.544182+0000","last_fullsized":"2024-03-11T20:51:14.544182+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":751,"log_dups_size":0,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:49:28.143755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":79,"num_read_kb":54998,"num_write":738,"num_write_kb":142673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"30'218","reported_seq":311,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.614589+0000","last_change":"2024-03-11T18:35:57.688727+0000","last_active":"2024-03-11T20:51:29.614589+0000","last_peered":"2024-03-11T20:51:29.614589+0000","last_clean":"2024-03-11T20:51:29.614589+0000","last_became_active":"2024-03-11T18:35:57.688209+0000","last_became_peered":"2024-03-11T18:35:57.688209+0000","last_unstale":"2024-03-11T20:51:29.614589+0000","last_undegraded":"2024-03-11T20:51:29.614589+0000","last_fullsized":"2024-03-11T20:51:29.614589+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:58:13.389498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"30'373","reported_seq":469,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:49:53.793880+0000","last_change":"2024-03-11T18:35:56.293842+0000","last_active":"2024-03-11T20:49:53.793880+0000","last_peered":"2024-03-11T20:49:53.793880+0000","last_clean":"2024-03-11T20:49:53.793880+0000","last_became_active":"2024-03-11T18:35:56.293276+0000","last_became_peered":"2024-03-11T18:35:56.293276+0000","last_unstale":"2024-03-11T20:49:53.793880+0000","last_undegraded":"2024-03-11T20:49:53.793880+0000","last_fullsized":"2024-03-11T20:49:53.793880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:04:13.101160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":309,"num_write_kb":4248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"30'742","reported_seq":881,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:15.435885+0000","last_change":"2024-03-11T18:36:04.626320+0000","last_active":"2024-03-11T20:51:15.435885+0000","last_peered":"2024-03-11T20:51:15.435885+0000","last_clean":"2024-03-11T20:51:15.435885+0000","last_became_active":"2024-03-11T18:36:04.625663+0000","last_became_peered":"2024-03-11T18:36:04.625663+0000","last_unstale":"2024-03-11T20:51:15.435885+0000","last_undegraded":"2024-03-11T20:51:15.435885+0000","last_fullsized":"2024-03-11T20:51:15.435885+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":742,"log_dups_size":0,"ondisk_log_size":742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:39:32.875112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":56,"num_read_kb":20684,"num_write":725,"num_write_kb":55759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"30'199","reported_seq":293,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.548835+0000","last_change":"2024-03-11T18:35:57.612242+0000","last_active":"2024-03-11T20:51:29.548835+0000","last_peered":"2024-03-11T20:51:29.548835+0000","last_clean":"2024-03-11T20:51:29.548835+0000","last_became_active":"2024-03-11T18:35:57.611509+0000","last_became_peered":"2024-03-11T18:35:57.611509+0000","last_unstale":"2024-03-11T20:51:29.548835+0000","last_undegraded":"2024-03-11T20:51:29.548835+0000","last_fullsized":"2024-03-11T20:51:29.548835+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:45:28.543623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"30'1190","reported_seq":1286,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:49:17.484772+0000","last_change":"2024-03-11T18:35:56.298766+0000","last_active":"2024-03-11T20:49:17.484772+0000","last_peered":"2024-03-11T20:49:17.484772+0000","last_clean":"2024-03-11T20:49:17.484772+0000","last_became_active":"2024-03-11T18:35:56.297747+0000","last_became_peered":"2024-03-11T18:35:56.297747+0000","last_unstale":"2024-03-11T20:49:17.484772+0000","last_undegraded":"2024-03-11T20:49:17.484772+0000","last_fullsized":"2024-03-11T20:49:17.484772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1190,"log_dups_size":0,"ondisk_log_size":1190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:13:39.177689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389196,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1123,"num_write_kb":8732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"30'749","reported_seq":963,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:17.573144+0000","last_change":"2024-03-11T18:36:04.635773+0000","last_active":"2024-03-11T20:51:17.573144+0000","last_peered":"2024-03-11T20:51:17.573144+0000","last_clean":"2024-03-11T20:51:17.573144+0000","last_became_active":"2024-03-11T18:36:04.635261+0000","last_became_peered":"2024-03-11T18:36:04.635261+0000","last_unstale":"2024-03-11T20:51:17.573144+0000","last_undegraded":"2024-03-11T20:51:17.573144+0000","last_fullsized":"2024-03-11T20:51:17.573144+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":749,"log_dups_size":0,"ondisk_log_size":749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:44:18.985469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":131,"num_read_kb":65646,"num_write":733,"num_write_kb":179820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"30'211","reported_seq":307,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.668044+0000","last_change":"2024-03-11T18:35:57.538552+0000","last_active":"2024-03-11T20:51:29.668044+0000","last_peered":"2024-03-11T20:51:29.668044+0000","last_clean":"2024-03-11T20:51:29.668044+0000","last_became_active":"2024-03-11T18:35:57.537717+0000","last_became_peered":"2024-03-11T18:35:57.537717+0000","last_unstale":"2024-03-11T20:51:29.668044+0000","last_undegraded":"2024-03-11T20:51:29.668044+0000","last_fullsized":"2024-03-11T20:51:29.668044+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:53:26.493265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"30'4762","reported_seq":9555,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:24.467774+0000","last_change":"2024-03-11T18:35:56.294572+0000","last_active":"2024-03-11T20:51:24.467774+0000","last_peered":"2024-03-11T20:51:24.467774+0000","last_clean":"2024-03-11T20:51:24.467774+0000","last_became_active":"2024-03-11T18:35:56.294096+0000","last_became_peered":"2024-03-11T18:35:56.294096+0000","last_unstale":"2024-03-11T20:51:24.467774+0000","last_undegraded":"2024-03-11T20:51:24.467774+0000","last_fullsized":"2024-03-11T20:51:24.467774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":4762,"log_dups_size":0,"ondisk_log_size":4762,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:22:12.516281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1555811,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4697,"num_read_kb":5145,"num_write":4704,"num_write_kb":5149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"30'682","reported_seq":802,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:08.234604+0000","last_change":"2024-03-11T18:36:04.620762+0000","last_active":"2024-03-11T20:51:08.234604+0000","last_peered":"2024-03-11T20:51:08.234604+0000","last_clean":"2024-03-11T20:51:08.234604+0000","last_became_active":"2024-03-11T18:36:04.620356+0000","last_became_peered":"2024-03-11T18:36:04.620356+0000","last_unstale":"2024-03-11T20:51:08.234604+0000","last_undegraded":"2024-03-11T20:51:08.234604+0000","last_fullsized":"2024-03-11T20:51:08.234604+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":682,"log_dups_size":0,"ondisk_log_size":682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:58:20.497426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":17597,"num_write":672,"num_write_kb":47486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"30'198","reported_seq":292,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.887154+0000","last_change":"2024-03-11T18:35:57.602773+0000","last_active":"2024-03-11T20:51:29.887154+0000","last_peered":"2024-03-11T20:51:29.887154+0000","last_clean":"2024-03-11T20:51:29.887154+0000","last_became_active":"2024-03-11T18:35:57.602060+0000","last_became_peered":"2024-03-11T18:35:57.602060+0000","last_unstale":"2024-03-11T20:51:29.887154+0000","last_undegraded":"2024-03-11T20:51:29.887154+0000","last_fullsized":"2024-03-11T20:51:29.887154+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:36:22.201707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"30'1021","reported_seq":1119,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:50:32.270816+0000","last_change":"2024-03-11T18:35:56.305283+0000","last_active":"2024-03-11T20:50:32.270816+0000","last_peered":"2024-03-11T20:50:32.270816+0000","last_clean":"2024-03-11T20:50:32.270816+0000","last_became_active":"2024-03-11T18:35:56.303063+0000","last_became_peered":"2024-03-11T18:35:56.303063+0000","last_unstale":"2024-03-11T20:50:32.270816+0000","last_undegraded":"2024-03-11T20:50:32.270816+0000","last_fullsized":"2024-03-11T20:50:32.270816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1021,"log_dups_size":0,"ondisk_log_size":1021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T18:51:00.795706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":963,"num_write_kb":12732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"30'852","reported_seq":1017,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:15.381877+0000","last_change":"2024-03-11T18:36:04.617247+0000","last_active":"2024-03-11T20:51:15.381877+0000","last_peered":"2024-03-11T20:51:15.381877+0000","last_clean":"2024-03-11T20:51:15.381877+0000","last_became_active":"2024-03-11T18:36:04.610765+0000","last_became_peered":"2024-03-11T18:36:04.610765+0000","last_unstale":"2024-03-11T20:51:15.381877+0000","last_undegraded":"2024-03-11T20:51:15.381877+0000","last_fullsized":"2024-03-11T20:51:15.381877+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:22:19.187685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":85,"num_read_kb":37203,"num_write":843,"num_write_kb":103063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"30'185","reported_seq":279,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.764151+0000","last_change":"2024-03-11T18:35:57.684207+0000","last_active":"2024-03-11T20:51:29.764151+0000","last_peered":"2024-03-11T20:51:29.764151+0000","last_clean":"2024-03-11T20:51:29.764151+0000","last_became_active":"2024-03-11T18:35:57.682653+0000","last_became_peered":"2024-03-11T18:35:57.682653+0000","last_unstale":"2024-03-11T20:51:29.764151+0000","last_undegraded":"2024-03-11T20:51:29.764151+0000","last_fullsized":"2024-03-11T20:51:29.764151+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":185,"log_dups_size":0,"ondisk_log_size":185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:28:51.265291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"30'371","reported_seq":467,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:08.599939+0000","last_change":"2024-03-11T18:35:56.297078+0000","last_active":"2024-03-11T20:51:08.599939+0000","last_peered":"2024-03-11T20:51:08.599939+0000","last_clean":"2024-03-11T20:51:08.599939+0000","last_became_active":"2024-03-11T18:35:56.296705+0000","last_became_peered":"2024-03-11T18:35:56.296705+0000","last_unstale":"2024-03-11T20:51:08.599939+0000","last_undegraded":"2024-03-11T20:51:08.599939+0000","last_fullsized":"2024-03-11T20:51:08.599939+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:53:25.278916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194372,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":305,"num_write_kb":4243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":253,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:47:47.393591+0000","last_change":"2024-03-11T18:35:18.199528+0000","last_active":"2024-03-11T20:47:47.393591+0000","last_peered":"2024-03-11T20:47:47.393591+0000","last_clean":"2024-03-11T20:47:47.393591+0000","last_became_active":"2024-03-11T18:35:18.197797+0000","last_became_peered":"2024-03-11T18:35:18.197797+0000","last_unstale":"2024-03-11T20:47:47.393591+0000","last_undegraded":"2024-03-11T20:47:47.393591+0000","last_fullsized":"2024-03-11T20:47:47.393591+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:16.207500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:16.207500+0000","last_clean_scrub_stamp":"2024-03-11T18:35:16.207500+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:29:00.357496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"30'785","reported_seq":953,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:10.792501+0000","last_change":"2024-03-11T18:36:04.599815+0000","last_active":"2024-03-11T20:51:10.792501+0000","last_peered":"2024-03-11T20:51:10.792501+0000","last_clean":"2024-03-11T20:51:10.792501+0000","last_became_active":"2024-03-11T18:36:04.598777+0000","last_became_peered":"2024-03-11T18:36:04.598777+0000","last_unstale":"2024-03-11T20:51:10.792501+0000","last_undegraded":"2024-03-11T20:51:10.792501+0000","last_fullsized":"2024-03-11T20:51:10.792501+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":785,"log_dups_size":0,"ondisk_log_size":785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:39:32.875112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":86,"num_read_kb":36797,"num_write":762,"num_write_kb":114420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"30'207","reported_seq":276,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.794305+0000","last_change":"2024-03-11T18:37:04.141957+0000","last_active":"2024-03-11T20:51:29.794305+0000","last_peered":"2024-03-11T20:51:29.794305+0000","last_clean":"2024-03-11T20:51:29.794305+0000","last_became_active":"2024-03-11T18:37:04.141663+0000","last_became_peered":"2024-03-11T18:37:04.141663+0000","last_unstale":"2024-03-11T20:51:29.794305+0000","last_undegraded":"2024-03-11T20:51:29.794305+0000","last_fullsized":"2024-03-11T20:51:29.794305+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:25:52.716744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"30'286","reported_seq":377,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:03.112535+0000","last_change":"2024-03-11T18:35:56.294626+0000","last_active":"2024-03-11T20:51:03.112535+0000","last_peered":"2024-03-11T20:51:03.112535+0000","last_clean":"2024-03-11T20:51:03.112535+0000","last_became_active":"2024-03-11T18:35:56.293961+0000","last_became_peered":"2024-03-11T18:35:56.293961+0000","last_unstale":"2024-03-11T20:51:03.112535+0000","last_undegraded":"2024-03-11T20:51:03.112535+0000","last_fullsized":"2024-03-11T20:51:03.112535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:50:38.709532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4196056,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":223,"num_write_kb":4226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"30'822","reported_seq":987,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:14.297709+0000","last_change":"2024-03-11T18:36:04.619862+0000","last_active":"2024-03-11T20:51:14.297709+0000","last_peered":"2024-03-11T20:51:14.297709+0000","last_clean":"2024-03-11T20:51:14.297709+0000","last_became_active":"2024-03-11T18:36:04.618908+0000","last_became_peered":"2024-03-11T18:36:04.618908+0000","last_unstale":"2024-03-11T20:51:14.297709+0000","last_undegraded":"2024-03-11T20:51:14.297709+0000","last_fullsized":"2024-03-11T20:51:14.297709+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":822,"log_dups_size":0,"ondisk_log_size":822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:34:46.433500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":85,"num_read_kb":42087,"num_write":807,"num_write_kb":152305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"30'221","reported_seq":310,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.603226+0000","last_change":"2024-03-11T18:35:57.686416+0000","last_active":"2024-03-11T20:51:29.603226+0000","last_peered":"2024-03-11T20:51:29.603226+0000","last_clean":"2024-03-11T20:51:29.603226+0000","last_became_active":"2024-03-11T18:35:57.685828+0000","last_became_peered":"2024-03-11T18:35:57.685828+0000","last_unstale":"2024-03-11T20:51:29.603226+0000","last_undegraded":"2024-03-11T20:51:29.603226+0000","last_fullsized":"2024-03-11T20:51:29.603226+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:41:42.263652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"30'406","reported_seq":497,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:49:30.772686+0000","last_change":"2024-03-11T18:35:56.295539+0000","last_active":"2024-03-11T20:49:30.772686+0000","last_peered":"2024-03-11T20:49:30.772686+0000","last_clean":"2024-03-11T20:49:30.772686+0000","last_became_active":"2024-03-11T18:35:56.295123+0000","last_became_peered":"2024-03-11T18:35:56.295123+0000","last_unstale":"2024-03-11T20:49:30.772686+0000","last_undegraded":"2024-03-11T20:49:30.772686+0000","last_fullsized":"2024-03-11T20:49:30.772686+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:03:06.585728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":350,"num_write_kb":4273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"30'821","reported_seq":1021,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:04.302500+0000","last_change":"2024-03-11T18:36:04.616750+0000","last_active":"2024-03-11T20:51:04.302500+0000","last_peered":"2024-03-11T20:51:04.302500+0000","last_clean":"2024-03-11T20:51:04.302500+0000","last_became_active":"2024-03-11T18:36:04.616338+0000","last_became_peered":"2024-03-11T18:36:04.616338+0000","last_unstale":"2024-03-11T20:51:04.302500+0000","last_undegraded":"2024-03-11T20:51:04.302500+0000","last_fullsized":"2024-03-11T20:51:04.302500+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":821,"log_dups_size":0,"ondisk_log_size":821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:14:42.558491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":122,"num_read_kb":70346,"num_write":827,"num_write_kb":188969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"30'199","reported_seq":288,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.638812+0000","last_change":"2024-03-11T18:35:57.689888+0000","last_active":"2024-03-11T20:51:29.638812+0000","last_peered":"2024-03-11T20:51:29.638812+0000","last_clean":"2024-03-11T20:51:29.638812+0000","last_became_active":"2024-03-11T18:35:57.689424+0000","last_became_peered":"2024-03-11T18:35:57.689424+0000","last_unstale":"2024-03-11T20:51:29.638812+0000","last_undegraded":"2024-03-11T20:51:29.638812+0000","last_fullsized":"2024-03-11T20:51:29.638812+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:25:16.564261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"30'861","reported_seq":952,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:49:04.742266+0000","last_change":"2024-03-11T18:35:56.293862+0000","last_active":"2024-03-11T20:49:04.742266+0000","last_peered":"2024-03-11T20:49:04.742266+0000","last_clean":"2024-03-11T20:49:04.742266+0000","last_became_active":"2024-03-11T18:35:56.293585+0000","last_became_peered":"2024-03-11T18:35:56.293585+0000","last_unstale":"2024-03-11T20:49:04.742266+0000","last_undegraded":"2024-03-11T20:49:04.742266+0000","last_fullsized":"2024-03-11T20:49:04.742266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":861,"log_dups_size":0,"ondisk_log_size":861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:45:27.329274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":804,"num_write_kb":4496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"30'804","reported_seq":982,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:17.590386+0000","last_change":"2024-03-11T18:36:04.627729+0000","last_active":"2024-03-11T20:51:17.590386+0000","last_peered":"2024-03-11T20:51:17.590386+0000","last_clean":"2024-03-11T20:51:17.590386+0000","last_became_active":"2024-03-11T18:36:04.626596+0000","last_became_peered":"2024-03-11T18:36:04.626596+0000","last_unstale":"2024-03-11T20:51:17.590386+0000","last_undegraded":"2024-03-11T20:51:17.590386+0000","last_fullsized":"2024-03-11T20:51:17.590386+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:51:28.017349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":98,"num_read_kb":46229,"num_write":782,"num_write_kb":114631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"30'184","reported_seq":273,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.619621+0000","last_change":"2024-03-11T18:35:57.690506+0000","last_active":"2024-03-11T20:51:29.619621+0000","last_peered":"2024-03-11T20:51:29.619621+0000","last_clean":"2024-03-11T20:51:29.619621+0000","last_became_active":"2024-03-11T18:35:57.690223+0000","last_became_peered":"2024-03-11T18:35:57.690223+0000","last_unstale":"2024-03-11T20:51:29.619621+0000","last_undegraded":"2024-03-11T20:51:29.619621+0000","last_fullsized":"2024-03-11T20:51:29.619621+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:58:13.389498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"30'815","reported_seq":905,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:50:41.080749+0000","last_change":"2024-03-11T18:35:56.257209+0000","last_active":"2024-03-11T20:50:41.080749+0000","last_peered":"2024-03-11T20:50:41.080749+0000","last_clean":"2024-03-11T20:50:41.080749+0000","last_became_active":"2024-03-11T18:35:56.256454+0000","last_became_peered":"2024-03-11T18:35:56.256454+0000","last_unstale":"2024-03-11T20:50:41.080749+0000","last_undegraded":"2024-03-11T20:50:41.080749+0000","last_fullsized":"2024-03-11T20:50:41.080749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":815,"log_dups_size":0,"ondisk_log_size":815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:32:06.369248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":757,"num_write_kb":4472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"30'833","reported_seq":1020,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:08.316459+0000","last_change":"2024-03-11T18:36:04.644184+0000","last_active":"2024-03-11T20:51:08.316459+0000","last_peered":"2024-03-11T20:51:08.316459+0000","last_clean":"2024-03-11T20:51:08.316459+0000","last_became_active":"2024-03-11T18:36:04.642079+0000","last_became_peered":"2024-03-11T18:36:04.642079+0000","last_unstale":"2024-03-11T20:51:08.316459+0000","last_undegraded":"2024-03-11T20:51:08.316459+0000","last_fullsized":"2024-03-11T20:51:08.316459+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":833,"log_dups_size":0,"ondisk_log_size":833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:36:26.132020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":107,"num_read_kb":63565,"num_write":823,"num_write_kb":172061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"30'239","reported_seq":326,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.606890+0000","last_change":"2024-03-11T18:35:57.687311+0000","last_active":"2024-03-11T20:51:29.606890+0000","last_peered":"2024-03-11T20:51:29.606890+0000","last_clean":"2024-03-11T20:51:29.606890+0000","last_became_active":"2024-03-11T18:35:57.686850+0000","last_became_peered":"2024-03-11T18:35:57.686850+0000","last_unstale":"2024-03-11T20:51:29.606890+0000","last_undegraded":"2024-03-11T20:51:29.606890+0000","last_fullsized":"2024-03-11T20:51:29.606890+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:29:45.662029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"30'899","reported_seq":988,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:16.314784+0000","last_change":"2024-03-11T18:35:56.291232+0000","last_active":"2024-03-11T20:51:16.314784+0000","last_peered":"2024-03-11T20:51:16.314784+0000","last_clean":"2024-03-11T20:51:16.314784+0000","last_became_active":"2024-03-11T18:35:56.290944+0000","last_became_peered":"2024-03-11T18:35:56.290944+0000","last_unstale":"2024-03-11T20:51:16.314784+0000","last_undegraded":"2024-03-11T20:51:16.314784+0000","last_fullsized":"2024-03-11T20:51:16.314784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":899,"log_dups_size":0,"ondisk_log_size":899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:49:27.827765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":846,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"30'791","reported_seq":968,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:18.444965+0000","last_change":"2024-03-11T18:36:04.602142+0000","last_active":"2024-03-11T20:51:18.444965+0000","last_peered":"2024-03-11T20:51:18.444965+0000","last_clean":"2024-03-11T20:51:18.444965+0000","last_became_active":"2024-03-11T18:36:04.601224+0000","last_became_peered":"2024-03-11T18:36:04.601224+0000","last_unstale":"2024-03-11T20:51:18.444965+0000","last_undegraded":"2024-03-11T20:51:18.444965+0000","last_fullsized":"2024-03-11T20:51:18.444965+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:17:42.075355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":59304,"num_write":781,"num_write_kb":152027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"30'219","reported_seq":306,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.941879+0000","last_change":"2024-03-11T18:35:57.611305+0000","last_active":"2024-03-11T20:51:29.941879+0000","last_peered":"2024-03-11T20:51:29.941879+0000","last_clean":"2024-03-11T20:51:29.941879+0000","last_became_active":"2024-03-11T18:35:57.610955+0000","last_became_peered":"2024-03-11T18:35:57.610955+0000","last_unstale":"2024-03-11T20:51:29.941879+0000","last_undegraded":"2024-03-11T20:51:29.941879+0000","last_fullsized":"2024-03-11T20:51:29.941879+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:31:19.941345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"30'60","reported_seq":149,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:17.314474+0000","last_change":"2024-03-11T18:35:56.361952+0000","last_active":"2024-03-11T20:51:17.314474+0000","last_peered":"2024-03-11T20:51:17.314474+0000","last_clean":"2024-03-11T20:51:17.314474+0000","last_became_active":"2024-03-11T18:35:56.361141+0000","last_became_peered":"2024-03-11T18:35:56.361141+0000","last_unstale":"2024-03-11T20:51:17.314474+0000","last_undegraded":"2024-03-11T20:51:17.314474+0000","last_fullsized":"2024-03-11T20:51:17.314474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:04:13.101160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"30'743","reported_seq":894,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:15.483428+0000","last_change":"2024-03-11T18:36:04.593604+0000","last_active":"2024-03-11T20:51:15.483428+0000","last_peered":"2024-03-11T20:51:15.483428+0000","last_clean":"2024-03-11T20:51:15.483428+0000","last_became_active":"2024-03-11T18:36:04.592304+0000","last_became_peered":"2024-03-11T18:36:04.592304+0000","last_unstale":"2024-03-11T20:51:15.483428+0000","last_undegraded":"2024-03-11T20:51:15.483428+0000","last_fullsized":"2024-03-11T20:51:15.483428+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":743,"log_dups_size":0,"ondisk_log_size":743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:31:04.738940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":77,"num_read_kb":52017,"num_write":733,"num_write_kb":82372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"30'206","reported_seq":293,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.637397+0000","last_change":"2024-03-11T18:35:57.689404+0000","last_active":"2024-03-11T20:51:29.637397+0000","last_peered":"2024-03-11T20:51:29.637397+0000","last_clean":"2024-03-11T20:51:29.637397+0000","last_became_active":"2024-03-11T18:35:57.689139+0000","last_became_peered":"2024-03-11T18:35:57.689139+0000","last_unstale":"2024-03-11T20:51:29.637397+0000","last_undegraded":"2024-03-11T20:51:29.637397+0000","last_fullsized":"2024-03-11T20:51:29.637397+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:54:09.699371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"30'60","reported_seq":149,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:19.415920+0000","last_change":"2024-03-11T18:35:56.297682+0000","last_active":"2024-03-11T20:51:19.415920+0000","last_peered":"2024-03-11T20:51:19.415920+0000","last_clean":"2024-03-11T20:51:19.415920+0000","last_became_active":"2024-03-11T18:35:56.296426+0000","last_became_peered":"2024-03-11T18:35:56.296426+0000","last_unstale":"2024-03-11T20:51:19.415920+0000","last_undegraded":"2024-03-11T20:51:19.415920+0000","last_fullsized":"2024-03-11T20:51:19.415920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:28:50.050942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"30'724","reported_seq":923,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:03.154615+0000","last_change":"2024-03-11T18:36:04.590780+0000","last_active":"2024-03-11T20:51:03.154615+0000","last_peered":"2024-03-11T20:51:03.154615+0000","last_clean":"2024-03-11T20:51:03.154615+0000","last_became_active":"2024-03-11T18:36:04.590014+0000","last_became_peered":"2024-03-11T18:36:04.590014+0000","last_unstale":"2024-03-11T20:51:03.154615+0000","last_undegraded":"2024-03-11T20:51:03.154615+0000","last_fullsized":"2024-03-11T20:51:03.154615+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":724,"log_dups_size":0,"ondisk_log_size":724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:51:13.205384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":124,"num_read_kb":69166,"num_write":714,"num_write_kb":127024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"30'208","reported_seq":295,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.875165+0000","last_change":"2024-03-11T18:35:57.671638+0000","last_active":"2024-03-11T20:51:29.875165+0000","last_peered":"2024-03-11T20:51:29.875165+0000","last_clean":"2024-03-11T20:51:29.875165+0000","last_became_active":"2024-03-11T18:35:57.670893+0000","last_became_peered":"2024-03-11T18:35:57.670893+0000","last_unstale":"2024-03-11T20:51:29.875165+0000","last_undegraded":"2024-03-11T20:51:29.875165+0000","last_fullsized":"2024-03-11T20:51:29.875165+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:31:21.395133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"30'592","reported_seq":681,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:49:43.697234+0000","last_change":"2024-03-11T18:35:56.251580+0000","last_active":"2024-03-11T20:49:43.697234+0000","last_peered":"2024-03-11T20:49:43.697234+0000","last_clean":"2024-03-11T20:49:43.697234+0000","last_became_active":"2024-03-11T18:35:56.251287+0000","last_became_peered":"2024-03-11T18:35:56.251287+0000","last_unstale":"2024-03-11T20:49:43.697234+0000","last_undegraded":"2024-03-11T20:49:43.697234+0000","last_fullsized":"2024-03-11T20:49:43.697234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":592,"log_dups_size":0,"ondisk_log_size":592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:30:39.907360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388676,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":533,"num_write_kb":8445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"30'738","reported_seq":846,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:12.396881+0000","last_change":"2024-03-11T18:36:04.625563+0000","last_active":"2024-03-11T20:51:12.396881+0000","last_peered":"2024-03-11T20:51:12.396881+0000","last_clean":"2024-03-11T20:51:12.396881+0000","last_became_active":"2024-03-11T18:36:04.618911+0000","last_became_peered":"2024-03-11T18:36:04.618911+0000","last_unstale":"2024-03-11T20:51:12.396881+0000","last_undegraded":"2024-03-11T20:51:12.396881+0000","last_fullsized":"2024-03-11T20:51:12.396881+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":738,"log_dups_size":0,"ondisk_log_size":738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:31:04.738940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":10131,"num_write":722,"num_write_kb":109498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"30'204","reported_seq":289,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.606383+0000","last_change":"2024-03-11T18:35:57.697641+0000","last_active":"2024-03-11T20:51:29.606383+0000","last_peered":"2024-03-11T20:51:29.606383+0000","last_clean":"2024-03-11T20:51:29.606383+0000","last_became_active":"2024-03-11T18:35:57.697263+0000","last_became_peered":"2024-03-11T18:35:57.697263+0000","last_unstale":"2024-03-11T20:51:29.606383+0000","last_undegraded":"2024-03-11T20:51:29.606383+0000","last_fullsized":"2024-03-11T20:51:29.606383+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:34:39.325572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"30'2202","reported_seq":2290,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:01.825523+0000","last_change":"2024-03-11T18:35:56.355612+0000","last_active":"2024-03-11T20:51:01.825523+0000","last_peered":"2024-03-11T20:51:01.825523+0000","last_clean":"2024-03-11T20:51:01.825523+0000","last_became_active":"2024-03-11T18:35:56.354827+0000","last_became_peered":"2024-03-11T18:35:56.354827+0000","last_unstale":"2024-03-11T20:51:01.825523+0000","last_undegraded":"2024-03-11T20:51:01.825523+0000","last_fullsized":"2024-03-11T20:51:01.825523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":2202,"log_dups_size":0,"ondisk_log_size":2202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:06:02.844765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583584,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2157,"num_write_kb":13630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"30'771","reported_seq":924,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:14.409334+0000","last_change":"2024-03-11T18:37:04.259892+0000","last_active":"2024-03-11T20:51:14.409334+0000","last_peered":"2024-03-11T20:51:14.409334+0000","last_clean":"2024-03-11T20:51:14.409334+0000","last_became_active":"2024-03-11T18:37:04.259051+0000","last_became_peered":"2024-03-11T18:37:04.259051+0000","last_unstale":"2024-03-11T20:51:14.409334+0000","last_undegraded":"2024-03-11T20:51:14.409334+0000","last_fullsized":"2024-03-11T20:51:14.409334+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":771,"log_dups_size":0,"ondisk_log_size":771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:36:51.665733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":84,"num_read_kb":37526,"num_write":755,"num_write_kb":85949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2,0],"acting":[5,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"30'201","reported_seq":287,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.590814+0000","last_change":"2024-03-11T18:35:57.679035+0000","last_active":"2024-03-11T20:51:29.590814+0000","last_peered":"2024-03-11T20:51:29.590814+0000","last_clean":"2024-03-11T20:51:29.590814+0000","last_became_active":"2024-03-11T18:35:57.678565+0000","last_became_peered":"2024-03-11T18:35:57.678565+0000","last_unstale":"2024-03-11T20:51:29.590814+0000","last_undegraded":"2024-03-11T20:51:29.590814+0000","last_fullsized":"2024-03-11T20:51:29.590814+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:18:05.980207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"30'3437","reported_seq":3524,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.294958+0000","last_change":"2024-03-11T18:35:56.288132+0000","last_active":"2024-03-11T20:51:29.294958+0000","last_peered":"2024-03-11T20:51:29.294958+0000","last_clean":"2024-03-11T20:51:29.294958+0000","last_became_active":"2024-03-11T18:35:56.287846+0000","last_became_peered":"2024-03-11T18:35:56.287846+0000","last_unstale":"2024-03-11T20:51:29.294958+0000","last_undegraded":"2024-03-11T20:51:29.294958+0000","last_fullsized":"2024-03-11T20:51:29.294958+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":3437,"log_dups_size":0,"ondisk_log_size":3437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T18:51:00.795706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7567240,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3391,"num_write_kb":10463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"30'736","reported_seq":902,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:10.794831+0000","last_change":"2024-03-11T18:36:04.643164+0000","last_active":"2024-03-11T20:51:10.794831+0000","last_peered":"2024-03-11T20:51:10.794831+0000","last_clean":"2024-03-11T20:51:10.794831+0000","last_became_active":"2024-03-11T18:36:04.642374+0000","last_became_peered":"2024-03-11T18:36:04.642374+0000","last_unstale":"2024-03-11T20:51:10.794831+0000","last_undegraded":"2024-03-11T20:51:10.794831+0000","last_fullsized":"2024-03-11T20:51:10.794831+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":736,"log_dups_size":0,"ondisk_log_size":736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:25:56.815496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":40504,"num_write":721,"num_write_kb":107102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"30'197","reported_seq":282,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.866466+0000","last_change":"2024-03-11T18:35:57.636550+0000","last_active":"2024-03-11T20:51:29.866466+0000","last_peered":"2024-03-11T20:51:29.866466+0000","last_clean":"2024-03-11T20:51:29.866466+0000","last_became_active":"2024-03-11T18:35:57.636093+0000","last_became_peered":"2024-03-11T18:35:57.636093+0000","last_unstale":"2024-03-11T20:51:29.866466+0000","last_undegraded":"2024-03-11T20:51:29.866466+0000","last_fullsized":"2024-03-11T20:51:29.866466+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:33:31.344465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"30'1238","reported_seq":1325,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:50:53.074114+0000","last_change":"2024-03-11T18:35:56.355480+0000","last_active":"2024-03-11T20:50:53.074114+0000","last_peered":"2024-03-11T20:50:53.074114+0000","last_clean":"2024-03-11T20:50:53.074114+0000","last_became_active":"2024-03-11T18:35:56.354759+0000","last_became_peered":"2024-03-11T18:35:56.354759+0000","last_unstale":"2024-03-11T20:50:53.074114+0000","last_undegraded":"2024-03-11T20:50:53.074114+0000","last_fullsized":"2024-03-11T20:50:53.074114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1238,"log_dups_size":0,"ondisk_log_size":1238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:45:37.842725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1180,"num_write_kb":8784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"30'803","reported_seq":939,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:12.607404+0000","last_change":"2024-03-11T18:36:04.617364+0000","last_active":"2024-03-11T20:51:12.607404+0000","last_peered":"2024-03-11T20:51:12.607404+0000","last_clean":"2024-03-11T20:51:12.607404+0000","last_became_active":"2024-03-11T18:36:04.613159+0000","last_became_peered":"2024-03-11T18:36:04.613159+0000","last_unstale":"2024-03-11T20:51:12.607404+0000","last_undegraded":"2024-03-11T20:51:12.607404+0000","last_fullsized":"2024-03-11T20:51:12.607404+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":803,"log_dups_size":0,"ondisk_log_size":803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:25:23.672189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":38200,"num_write":794,"num_write_kb":117163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"30'218","reported_seq":303,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.626148+0000","last_change":"2024-03-11T18:35:57.683426+0000","last_active":"2024-03-11T20:51:29.626148+0000","last_peered":"2024-03-11T20:51:29.626148+0000","last_clean":"2024-03-11T20:51:29.626148+0000","last_became_active":"2024-03-11T18:35:57.683150+0000","last_became_peered":"2024-03-11T18:35:57.683150+0000","last_unstale":"2024-03-11T20:51:29.626148+0000","last_undegraded":"2024-03-11T20:51:29.626148+0000","last_fullsized":"2024-03-11T20:51:29.626148+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:36:00.079075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"30'63","reported_seq":150,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.294390+0000","last_change":"2024-03-11T18:35:56.361613+0000","last_active":"2024-03-11T20:51:29.294390+0000","last_peered":"2024-03-11T20:51:29.294390+0000","last_clean":"2024-03-11T20:51:29.294390+0000","last_became_active":"2024-03-11T18:35:56.359629+0000","last_became_peered":"2024-03-11T18:35:56.359629+0000","last_unstale":"2024-03-11T20:51:29.294390+0000","last_undegraded":"2024-03-11T20:51:29.294390+0000","last_fullsized":"2024-03-11T20:51:29.294390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:03:06.585728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"30'836","reported_seq":1036,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:12.888024+0000","last_change":"2024-03-11T18:36:04.608168+0000","last_active":"2024-03-11T20:51:12.888024+0000","last_peered":"2024-03-11T20:51:12.888024+0000","last_clean":"2024-03-11T20:51:12.888024+0000","last_became_active":"2024-03-11T18:36:04.607063+0000","last_became_peered":"2024-03-11T18:36:04.607063+0000","last_unstale":"2024-03-11T20:51:12.888024+0000","last_undegraded":"2024-03-11T20:51:12.888024+0000","last_fullsized":"2024-03-11T20:51:12.888024+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":836,"log_dups_size":0,"ondisk_log_size":836,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:53:41.996024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":124,"num_read_kb":69905,"num_write":818,"num_write_kb":155514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"30'815","reported_seq":1016,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:08.312913+0000","last_change":"2024-03-11T18:36:04.607992+0000","last_active":"2024-03-11T20:51:08.312913+0000","last_peered":"2024-03-11T20:51:08.312913+0000","last_clean":"2024-03-11T20:51:08.312913+0000","last_became_active":"2024-03-11T18:36:04.607109+0000","last_became_peered":"2024-03-11T18:36:04.607109+0000","last_unstale":"2024-03-11T20:51:08.312913+0000","last_undegraded":"2024-03-11T20:51:08.312913+0000","last_fullsized":"2024-03-11T20:51:08.312913+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":815,"log_dups_size":0,"ondisk_log_size":815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:14:37.678854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":116,"num_read_kb":66679,"num_write":801,"num_write_kb":181203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"30'218","reported_seq":302,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.672110+0000","last_change":"2024-03-11T18:35:57.680534+0000","last_active":"2024-03-11T20:51:29.672110+0000","last_peered":"2024-03-11T20:51:29.672110+0000","last_clean":"2024-03-11T20:51:29.672110+0000","last_became_active":"2024-03-11T18:35:57.679642+0000","last_became_peered":"2024-03-11T18:35:57.679642+0000","last_unstale":"2024-03-11T20:51:29.672110+0000","last_undegraded":"2024-03-11T20:51:29.672110+0000","last_fullsized":"2024-03-11T20:51:29.672110+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:25:49.707568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"30'1317","reported_seq":1404,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:17.349647+0000","last_change":"2024-03-11T18:35:56.303510+0000","last_active":"2024-03-11T20:51:17.349647+0000","last_peered":"2024-03-11T20:51:17.349647+0000","last_clean":"2024-03-11T20:51:17.349647+0000","last_became_active":"2024-03-11T18:35:56.302758+0000","last_became_peered":"2024-03-11T18:35:56.302758+0000","last_unstale":"2024-03-11T20:51:17.349647+0000","last_undegraded":"2024-03-11T20:51:17.349647+0000","last_fullsized":"2024-03-11T20:51:17.349647+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1317,"log_dups_size":0,"ondisk_log_size":1317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:03:32.297342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583494,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1254,"num_write_kb":12926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"30'766","reported_seq":981,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:50:55.156176+0000","last_change":"2024-03-11T18:36:04.618818+0000","last_active":"2024-03-11T20:50:55.156176+0000","last_peered":"2024-03-11T20:50:55.156176+0000","last_clean":"2024-03-11T20:50:55.156176+0000","last_became_active":"2024-03-11T18:36:04.617868+0000","last_became_peered":"2024-03-11T18:36:04.617868+0000","last_unstale":"2024-03-11T20:50:55.156176+0000","last_undegraded":"2024-03-11T20:50:55.156176+0000","last_fullsized":"2024-03-11T20:50:55.156176+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":766,"log_dups_size":0,"ondisk_log_size":766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:16:10.508913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":132,"num_read_kb":68723,"num_write":745,"num_write_kb":131559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"30'225","reported_seq":310,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.771467+0000","last_change":"2024-03-11T18:35:57.690347+0000","last_active":"2024-03-11T20:51:29.771467+0000","last_peered":"2024-03-11T20:51:29.771467+0000","last_clean":"2024-03-11T20:51:29.771467+0000","last_became_active":"2024-03-11T18:35:57.689146+0000","last_became_peered":"2024-03-11T18:35:57.689146+0000","last_unstale":"2024-03-11T20:51:29.771467+0000","last_undegraded":"2024-03-11T20:51:29.771467+0000","last_fullsized":"2024-03-11T20:51:29.771467+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:03:56.260394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"30'834","reported_seq":920,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:49:31.476898+0000","last_change":"2024-03-11T18:35:56.348355+0000","last_active":"2024-03-11T20:49:31.476898+0000","last_peered":"2024-03-11T20:49:31.476898+0000","last_clean":"2024-03-11T20:49:31.476898+0000","last_became_active":"2024-03-11T18:35:56.347631+0000","last_became_peered":"2024-03-11T18:35:56.347631+0000","last_unstale":"2024-03-11T20:49:31.476898+0000","last_undegraded":"2024-03-11T20:49:31.476898+0000","last_fullsized":"2024-03-11T20:49:31.476898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":834,"log_dups_size":0,"ondisk_log_size":834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:06:02.844765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":793,"num_write_kb":4496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"30'1909","reported_seq":1994,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:49:42.823141+0000","last_change":"2024-03-11T18:35:56.361254+0000","last_active":"2024-03-11T20:49:42.823141+0000","last_peered":"2024-03-11T20:49:42.823141+0000","last_clean":"2024-03-11T20:49:42.823141+0000","last_became_active":"2024-03-11T18:35:56.360580+0000","last_became_peered":"2024-03-11T18:35:56.360580+0000","last_unstale":"2024-03-11T20:49:42.823141+0000","last_undegraded":"2024-03-11T20:49:42.823141+0000","last_fullsized":"2024-03-11T20:49:42.823141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1909,"log_dups_size":0,"ondisk_log_size":1909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:27:26.471550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1850,"num_write_kb":17317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"30'183","reported_seq":266,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.874784+0000","last_change":"2024-03-11T18:35:57.602778+0000","last_active":"2024-03-11T20:51:29.874784+0000","last_peered":"2024-03-11T20:51:29.874784+0000","last_clean":"2024-03-11T20:51:29.874784+0000","last_became_active":"2024-03-11T18:35:57.601990+0000","last_became_peered":"2024-03-11T18:35:57.601990+0000","last_unstale":"2024-03-11T20:51:29.874784+0000","last_undegraded":"2024-03-11T20:51:29.874784+0000","last_fullsized":"2024-03-11T20:51:29.874784+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:56:59.014871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"30'692","reported_seq":830,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:08.239134+0000","last_change":"2024-03-11T18:36:04.606098+0000","last_active":"2024-03-11T20:51:08.239134+0000","last_peered":"2024-03-11T20:51:08.239134+0000","last_clean":"2024-03-11T20:51:08.239134+0000","last_became_active":"2024-03-11T18:36:04.604823+0000","last_became_peered":"2024-03-11T18:36:04.604823+0000","last_unstale":"2024-03-11T20:51:08.239134+0000","last_undegraded":"2024-03-11T20:51:08.239134+0000","last_fullsized":"2024-03-11T20:51:08.239134+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:52:04.862452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":66,"num_read_kb":37267,"num_write":684,"num_write_kb":109032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"30'332","reported_seq":417,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:50:56.328124+0000","last_change":"2024-03-11T18:35:56.296312+0000","last_active":"2024-03-11T20:50:56.328124+0000","last_peered":"2024-03-11T20:50:56.328124+0000","last_clean":"2024-03-11T20:50:56.328124+0000","last_became_active":"2024-03-11T18:35:56.296090+0000","last_became_peered":"2024-03-11T18:35:56.296090+0000","last_unstale":"2024-03-11T20:50:56.328124+0000","last_undegraded":"2024-03-11T20:50:56.328124+0000","last_fullsized":"2024-03-11T20:50:56.328124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:03:06.585728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4195508,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":278,"num_write_kb":4222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"30'199","reported_seq":282,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.579273+0000","last_change":"2024-03-11T18:35:57.611809+0000","last_active":"2024-03-11T20:51:29.579273+0000","last_peered":"2024-03-11T20:51:29.579273+0000","last_clean":"2024-03-11T20:51:29.579273+0000","last_became_active":"2024-03-11T18:35:57.611097+0000","last_became_peered":"2024-03-11T18:35:57.611097+0000","last_unstale":"2024-03-11T20:51:29.579273+0000","last_undegraded":"2024-03-11T20:51:29.579273+0000","last_fullsized":"2024-03-11T20:51:29.579273+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:49:29.042114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"30'703","reported_seq":834,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:06.329767+0000","last_change":"2024-03-11T18:36:04.590064+0000","last_active":"2024-03-11T20:51:06.329767+0000","last_peered":"2024-03-11T20:51:06.329767+0000","last_clean":"2024-03-11T20:51:06.329767+0000","last_became_active":"2024-03-11T18:36:04.589037+0000","last_became_peered":"2024-03-11T18:36:04.589037+0000","last_unstale":"2024-03-11T20:51:06.329767+0000","last_undegraded":"2024-03-11T20:51:06.329767+0000","last_fullsized":"2024-03-11T20:51:06.329767+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:51:28.017349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":59,"num_read_kb":23261,"num_write":682,"num_write_kb":69804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"30'217","reported_seq":311,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.960628+0000","last_change":"2024-03-11T18:35:57.614944+0000","last_active":"2024-03-11T20:51:29.960628+0000","last_peered":"2024-03-11T20:51:29.960628+0000","last_clean":"2024-03-11T20:51:29.960628+0000","last_became_active":"2024-03-11T18:35:57.614144+0000","last_became_peered":"2024-03-11T18:35:57.614144+0000","last_unstale":"2024-03-11T20:51:29.960628+0000","last_undegraded":"2024-03-11T20:51:29.960628+0000","last_fullsized":"2024-03-11T20:51:29.960628+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:53:26.493265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"30'222","reported_seq":316,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.797673+0000","last_change":"2024-03-11T18:35:57.630211+0000","last_active":"2024-03-11T20:51:29.797673+0000","last_peered":"2024-03-11T20:51:29.797673+0000","last_clean":"2024-03-11T20:51:29.797673+0000","last_became_active":"2024-03-11T18:35:57.629599+0000","last_became_peered":"2024-03-11T18:35:57.629599+0000","last_unstale":"2024-03-11T20:51:29.797673+0000","last_undegraded":"2024-03-11T20:51:29.797673+0000","last_fullsized":"2024-03-11T20:51:29.797673+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":222,"log_dups_size":0,"ondisk_log_size":222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:18:05.980207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"30'236","reported_seq":330,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.602616+0000","last_change":"2024-03-11T18:35:57.688700+0000","last_active":"2024-03-11T20:51:29.602616+0000","last_peered":"2024-03-11T20:51:29.602616+0000","last_clean":"2024-03-11T20:51:29.602616+0000","last_became_active":"2024-03-11T18:35:57.687968+0000","last_became_peered":"2024-03-11T18:35:57.687968+0000","last_unstale":"2024-03-11T20:51:29.602616+0000","last_undegraded":"2024-03-11T20:51:29.602616+0000","last_fullsized":"2024-03-11T20:51:29.602616+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":236,"log_dups_size":0,"ondisk_log_size":236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:25:16.564261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"30'203","reported_seq":296,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.872107+0000","last_change":"2024-03-11T18:35:57.680730+0000","last_active":"2024-03-11T20:51:29.872107+0000","last_peered":"2024-03-11T20:51:29.872107+0000","last_clean":"2024-03-11T20:51:29.872107+0000","last_became_active":"2024-03-11T18:35:57.679935+0000","last_became_peered":"2024-03-11T18:35:57.679935+0000","last_unstale":"2024-03-11T20:51:29.872107+0000","last_undegraded":"2024-03-11T20:51:29.872107+0000","last_fullsized":"2024-03-11T20:51:29.872107+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:21:21.894805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"30'184","reported_seq":275,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.764136+0000","last_change":"2024-03-11T18:35:57.690799+0000","last_active":"2024-03-11T20:51:29.764136+0000","last_peered":"2024-03-11T20:51:29.764136+0000","last_clean":"2024-03-11T20:51:29.764136+0000","last_became_active":"2024-03-11T18:35:57.686623+0000","last_became_peered":"2024-03-11T18:35:57.686623+0000","last_unstale":"2024-03-11T20:51:29.764136+0000","last_undegraded":"2024-03-11T20:51:29.764136+0000","last_fullsized":"2024-03-11T20:51:29.764136+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:53:30.324898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"30'208","reported_seq":299,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.659529+0000","last_change":"2024-03-11T18:35:57.691955+0000","last_active":"2024-03-11T20:51:29.659529+0000","last_peered":"2024-03-11T20:51:29.659529+0000","last_clean":"2024-03-11T20:51:29.659529+0000","last_became_active":"2024-03-11T18:35:57.691413+0000","last_became_peered":"2024-03-11T18:35:57.691413+0000","last_unstale":"2024-03-11T20:51:29.659529+0000","last_undegraded":"2024-03-11T20:51:29.659529+0000","last_fullsized":"2024-03-11T20:51:29.659529+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:31:19.941345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"30'244","reported_seq":335,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.963800+0000","last_change":"2024-03-11T18:35:57.684695+0000","last_active":"2024-03-11T20:51:29.963800+0000","last_peered":"2024-03-11T20:51:29.963800+0000","last_clean":"2024-03-11T20:51:29.963800+0000","last_became_active":"2024-03-11T18:35:57.684227+0000","last_became_peered":"2024-03-11T18:35:57.684227+0000","last_unstale":"2024-03-11T20:51:29.963800+0000","last_undegraded":"2024-03-11T20:51:29.963800+0000","last_fullsized":"2024-03-11T20:51:29.963800+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":244,"log_dups_size":0,"ondisk_log_size":244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:28:51.265291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"30'193","reported_seq":284,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.760629+0000","last_change":"2024-03-11T18:35:57.683549+0000","last_active":"2024-03-11T20:51:29.760629+0000","last_peered":"2024-03-11T20:51:29.760629+0000","last_clean":"2024-03-11T20:51:29.760629+0000","last_became_active":"2024-03-11T18:35:57.680610+0000","last_became_peered":"2024-03-11T18:35:57.680610+0000","last_unstale":"2024-03-11T20:51:29.760629+0000","last_undegraded":"2024-03-11T20:51:29.760629+0000","last_fullsized":"2024-03-11T20:51:29.760629+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:25:52.716744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"30'218","reported_seq":289,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.791145+0000","last_change":"2024-03-11T18:37:04.264086+0000","last_active":"2024-03-11T20:51:29.791145+0000","last_peered":"2024-03-11T20:51:29.791145+0000","last_clean":"2024-03-11T20:51:29.791145+0000","last_became_active":"2024-03-11T18:37:04.263764+0000","last_became_peered":"2024-03-11T18:37:04.263764+0000","last_unstale":"2024-03-11T20:51:29.791145+0000","last_undegraded":"2024-03-11T20:51:29.791145+0000","last_fullsized":"2024-03-11T20:51:29.791145+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:34:39.325572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"30'205","reported_seq":294,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.764435+0000","last_change":"2024-03-11T18:35:57.681630+0000","last_active":"2024-03-11T20:51:29.764435+0000","last_peered":"2024-03-11T20:51:29.764435+0000","last_clean":"2024-03-11T20:51:29.764435+0000","last_became_active":"2024-03-11T18:35:57.678876+0000","last_became_peered":"2024-03-11T18:35:57.678876+0000","last_unstale":"2024-03-11T20:51:29.764435+0000","last_undegraded":"2024-03-11T20:51:29.764435+0000","last_fullsized":"2024-03-11T20:51:29.764435+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:03:33.511691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"30'208","reported_seq":297,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.623113+0000","last_change":"2024-03-11T18:35:57.679589+0000","last_active":"2024-03-11T20:51:29.623113+0000","last_peered":"2024-03-11T20:51:29.623113+0000","last_clean":"2024-03-11T20:51:29.623113+0000","last_became_active":"2024-03-11T18:35:57.678838+0000","last_became_peered":"2024-03-11T18:35:57.678838+0000","last_unstale":"2024-03-11T20:51:29.623113+0000","last_undegraded":"2024-03-11T20:51:29.623113+0000","last_fullsized":"2024-03-11T20:51:29.623113+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:54:09.699371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"30'213","reported_seq":302,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.872622+0000","last_change":"2024-03-11T18:35:57.674571+0000","last_active":"2024-03-11T20:51:29.872622+0000","last_peered":"2024-03-11T20:51:29.872622+0000","last_clean":"2024-03-11T20:51:29.872622+0000","last_became_active":"2024-03-11T18:35:57.671573+0000","last_became_peered":"2024-03-11T18:35:57.671573+0000","last_unstale":"2024-03-11T20:51:29.872622+0000","last_undegraded":"2024-03-11T20:51:29.872622+0000","last_fullsized":"2024-03-11T20:51:29.872622+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:35:10.403666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"30'221","reported_seq":310,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.959781+0000","last_change":"2024-03-11T18:35:57.615939+0000","last_active":"2024-03-11T20:51:29.959781+0000","last_peered":"2024-03-11T20:51:29.959781+0000","last_clean":"2024-03-11T20:51:29.959781+0000","last_became_active":"2024-03-11T18:35:57.615579+0000","last_became_peered":"2024-03-11T18:35:57.615579+0000","last_unstale":"2024-03-11T20:51:29.959781+0000","last_undegraded":"2024-03-11T20:51:29.959781+0000","last_fullsized":"2024-03-11T20:51:29.959781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:03:33.511691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"30'206","reported_seq":295,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.632447+0000","last_change":"2024-03-11T18:35:57.688955+0000","last_active":"2024-03-11T20:51:29.632447+0000","last_peered":"2024-03-11T20:51:29.632447+0000","last_clean":"2024-03-11T20:51:29.632447+0000","last_became_active":"2024-03-11T18:35:57.688365+0000","last_became_peered":"2024-03-11T18:35:57.688365+0000","last_unstale":"2024-03-11T20:51:29.632447+0000","last_undegraded":"2024-03-11T20:51:29.632447+0000","last_fullsized":"2024-03-11T20:51:29.632447+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:44:11.877541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"30'205","reported_seq":292,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.795195+0000","last_change":"2024-03-11T18:35:57.602044+0000","last_active":"2024-03-11T20:51:29.795195+0000","last_peered":"2024-03-11T20:51:29.795195+0000","last_clean":"2024-03-11T20:51:29.795195+0000","last_became_active":"2024-03-11T18:35:57.601334+0000","last_became_peered":"2024-03-11T18:35:57.601334+0000","last_unstale":"2024-03-11T20:51:29.795195+0000","last_undegraded":"2024-03-11T20:51:29.795195+0000","last_fullsized":"2024-03-11T20:51:29.795195+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:39:12.693619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"30'206","reported_seq":293,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.612090+0000","last_change":"2024-03-11T18:35:57.677114+0000","last_active":"2024-03-11T20:51:29.612090+0000","last_peered":"2024-03-11T20:51:29.612090+0000","last_clean":"2024-03-11T20:51:29.612090+0000","last_became_active":"2024-03-11T18:35:57.676075+0000","last_became_peered":"2024-03-11T18:35:57.676075+0000","last_unstale":"2024-03-11T20:51:29.612090+0000","last_undegraded":"2024-03-11T20:51:29.612090+0000","last_fullsized":"2024-03-11T20:51:29.612090+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:44:11.877541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"30'219","reported_seq":306,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.614892+0000","last_change":"2024-03-11T18:35:57.641392+0000","last_active":"2024-03-11T20:51:29.614892+0000","last_peered":"2024-03-11T20:51:29.614892+0000","last_clean":"2024-03-11T20:51:29.614892+0000","last_became_active":"2024-03-11T18:35:57.640934+0000","last_became_peered":"2024-03-11T18:35:57.640934+0000","last_unstale":"2024-03-11T20:51:29.614892+0000","last_undegraded":"2024-03-11T20:51:29.614892+0000","last_fullsized":"2024-03-11T20:51:29.614892+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:25:52.716744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"30'213","reported_seq":300,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.664702+0000","last_change":"2024-03-11T18:35:57.605291+0000","last_active":"2024-03-11T20:51:29.664702+0000","last_peered":"2024-03-11T20:51:29.664702+0000","last_clean":"2024-03-11T20:51:29.664702+0000","last_became_active":"2024-03-11T18:35:57.604645+0000","last_became_peered":"2024-03-11T18:35:57.604645+0000","last_unstale":"2024-03-11T20:51:29.664702+0000","last_undegraded":"2024-03-11T20:51:29.664702+0000","last_fullsized":"2024-03-11T20:51:29.664702+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:03:33.511691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"30'210","reported_seq":296,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.888659+0000","last_change":"2024-03-11T18:35:57.503994+0000","last_active":"2024-03-11T20:51:29.888659+0000","last_peered":"2024-03-11T20:51:29.888659+0000","last_clean":"2024-03-11T20:51:29.888659+0000","last_became_active":"2024-03-11T18:35:57.503362+0000","last_became_peered":"2024-03-11T18:35:57.503362+0000","last_unstale":"2024-03-11T20:51:29.888659+0000","last_undegraded":"2024-03-11T20:51:29.888659+0000","last_fullsized":"2024-03-11T20:51:29.888659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:03:58.875164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"30'195","reported_seq":260,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.789716+0000","last_change":"2024-03-11T18:37:04.252064+0000","last_active":"2024-03-11T20:51:29.789716+0000","last_peered":"2024-03-11T20:51:29.789716+0000","last_clean":"2024-03-11T20:51:29.789716+0000","last_became_active":"2024-03-11T18:37:04.251092+0000","last_became_peered":"2024-03-11T18:37:04.251092+0000","last_unstale":"2024-03-11T20:51:29.789716+0000","last_undegraded":"2024-03-11T20:51:29.789716+0000","last_fullsized":"2024-03-11T20:51:29.789716+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:29:45.662029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"30'181","reported_seq":266,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.575251+0000","last_change":"2024-03-11T18:35:57.613374+0000","last_active":"2024-03-11T20:51:29.575251+0000","last_peered":"2024-03-11T20:51:29.575251+0000","last_clean":"2024-03-11T20:51:29.575251+0000","last_became_active":"2024-03-11T18:35:57.613050+0000","last_became_peered":"2024-03-11T18:35:57.613050+0000","last_unstale":"2024-03-11T20:51:29.575251+0000","last_undegraded":"2024-03-11T20:51:29.575251+0000","last_fullsized":"2024-03-11T20:51:29.575251+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:50:39.923881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"30'186","reported_seq":271,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.959316+0000","last_change":"2024-03-11T18:35:57.636565+0000","last_active":"2024-03-11T20:51:29.959316+0000","last_peered":"2024-03-11T20:51:29.959316+0000","last_clean":"2024-03-11T20:51:29.959316+0000","last_became_active":"2024-03-11T18:35:57.635978+0000","last_became_peered":"2024-03-11T18:35:57.635978+0000","last_unstale":"2024-03-11T20:51:29.959316+0000","last_undegraded":"2024-03-11T20:51:29.959316+0000","last_fullsized":"2024-03-11T20:51:29.959316+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:03:56.260394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"30'206","reported_seq":291,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.964837+0000","last_change":"2024-03-11T18:35:57.641714+0000","last_active":"2024-03-11T20:51:29.964837+0000","last_peered":"2024-03-11T20:51:29.964837+0000","last_clean":"2024-03-11T20:51:29.964837+0000","last_became_active":"2024-03-11T18:35:57.641417+0000","last_became_peered":"2024-03-11T18:35:57.641417+0000","last_unstale":"2024-03-11T20:51:29.964837+0000","last_undegraded":"2024-03-11T20:51:29.964837+0000","last_fullsized":"2024-03-11T20:51:29.964837+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:25:49.707568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"30'210","reported_seq":294,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.626373+0000","last_change":"2024-03-11T18:35:57.689703+0000","last_active":"2024-03-11T20:51:29.626373+0000","last_peered":"2024-03-11T20:51:29.626373+0000","last_clean":"2024-03-11T20:51:29.626373+0000","last_became_active":"2024-03-11T18:35:57.689056+0000","last_became_peered":"2024-03-11T18:35:57.689056+0000","last_unstale":"2024-03-11T20:51:29.626373+0000","last_undegraded":"2024-03-11T20:51:29.626373+0000","last_fullsized":"2024-03-11T20:51:29.626373+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:41:42.263652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"30'194","reported_seq":277,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.772230+0000","last_change":"2024-03-11T18:35:57.684305+0000","last_active":"2024-03-11T20:51:29.772230+0000","last_peered":"2024-03-11T20:51:29.772230+0000","last_clean":"2024-03-11T20:51:29.772230+0000","last_became_active":"2024-03-11T18:35:57.682947+0000","last_became_peered":"2024-03-11T18:35:57.682947+0000","last_unstale":"2024-03-11T20:51:29.772230+0000","last_undegraded":"2024-03-11T20:51:29.772230+0000","last_fullsized":"2024-03-11T20:51:29.772230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:53:26.493265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"30'210","reported_seq":293,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.585163+0000","last_change":"2024-03-11T18:35:57.692717+0000","last_active":"2024-03-11T20:51:29.585163+0000","last_peered":"2024-03-11T20:51:29.585163+0000","last_clean":"2024-03-11T20:51:29.585163+0000","last_became_active":"2024-03-11T18:35:57.692504+0000","last_became_peered":"2024-03-11T18:35:57.692504+0000","last_unstale":"2024-03-11T20:51:29.585163+0000","last_undegraded":"2024-03-11T20:51:29.585163+0000","last_fullsized":"2024-03-11T20:51:29.585163+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:25:52.716744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"30'194","reported_seq":277,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.612590+0000","last_change":"2024-03-11T18:35:57.640483+0000","last_active":"2024-03-11T20:51:29.612590+0000","last_peered":"2024-03-11T20:51:29.612590+0000","last_clean":"2024-03-11T20:51:29.612590+0000","last_became_active":"2024-03-11T18:35:57.639886+0000","last_became_peered":"2024-03-11T18:35:57.639886+0000","last_unstale":"2024-03-11T20:51:29.612590+0000","last_undegraded":"2024-03-11T20:51:29.612590+0000","last_fullsized":"2024-03-11T20:51:29.612590+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:34:39.325572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"30'212","reported_seq":295,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.603982+0000","last_change":"2024-03-11T18:35:57.687096+0000","last_active":"2024-03-11T20:51:29.603982+0000","last_peered":"2024-03-11T20:51:29.603982+0000","last_clean":"2024-03-11T20:51:29.603982+0000","last_became_active":"2024-03-11T18:35:57.686771+0000","last_became_peered":"2024-03-11T18:35:57.686771+0000","last_unstale":"2024-03-11T20:51:29.603982+0000","last_undegraded":"2024-03-11T20:51:29.603982+0000","last_fullsized":"2024-03-11T20:51:29.603982+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:16:03.400985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"30'196","reported_seq":292,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.628723+0000","last_change":"2024-03-11T18:35:57.694707+0000","last_active":"2024-03-11T20:51:29.628723+0000","last_peered":"2024-03-11T20:51:29.628723+0000","last_clean":"2024-03-11T20:51:29.628723+0000","last_became_active":"2024-03-11T18:35:57.694104+0000","last_became_peered":"2024-03-11T18:35:57.694104+0000","last_unstale":"2024-03-11T20:51:29.628723+0000","last_undegraded":"2024-03-11T20:51:29.628723+0000","last_fullsized":"2024-03-11T20:51:29.628723+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":196,"log_dups_size":0,"ondisk_log_size":196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:47:28.579467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"30'191","reported_seq":285,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.874739+0000","last_change":"2024-03-11T18:35:57.538548+0000","last_active":"2024-03-11T20:51:29.874739+0000","last_peered":"2024-03-11T20:51:29.874739+0000","last_clean":"2024-03-11T20:51:29.874739+0000","last_became_active":"2024-03-11T18:35:57.537704+0000","last_became_peered":"2024-03-11T18:35:57.537704+0000","last_unstale":"2024-03-11T20:51:29.874739+0000","last_undegraded":"2024-03-11T20:51:29.874739+0000","last_fullsized":"2024-03-11T20:51:29.874739+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:40:52.251981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"30'214","reported_seq":308,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.659184+0000","last_change":"2024-03-11T18:35:57.507126+0000","last_active":"2024-03-11T20:51:29.659184+0000","last_peered":"2024-03-11T20:51:29.659184+0000","last_clean":"2024-03-11T20:51:29.659184+0000","last_became_active":"2024-03-11T18:35:57.506429+0000","last_became_peered":"2024-03-11T18:35:57.506429+0000","last_unstale":"2024-03-11T20:51:29.659184+0000","last_undegraded":"2024-03-11T20:51:29.659184+0000","last_fullsized":"2024-03-11T20:51:29.659184+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:28:51.265291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"30'213","reported_seq":307,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.608987+0000","last_change":"2024-03-11T18:35:57.687817+0000","last_active":"2024-03-11T20:51:29.608987+0000","last_peered":"2024-03-11T20:51:29.608987+0000","last_clean":"2024-03-11T20:51:29.608987+0000","last_became_active":"2024-03-11T18:35:57.687532+0000","last_became_peered":"2024-03-11T18:35:57.687532+0000","last_unstale":"2024-03-11T20:51:29.608987+0000","last_undegraded":"2024-03-11T20:51:29.608987+0000","last_fullsized":"2024-03-11T20:51:29.608987+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:47:28.579467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2961,"num_read_kb":1579043,"num_write":24379,"num_write_kb":4188254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24824,"ondisk_log_size":24824,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13197,"ondisk_log_size":13197,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":210457699,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":4697,"num_read_kb":5145,"num_write":47510,"num_write_kb":229688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":421158912,"data_stored":420915398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38275,"ondisk_log_size":38275,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719478378,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":200100,"kb_used_data":78748,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94171740,"statfs":{"total":96636764160,"available":96431861760,"internally_reserved":0,"allocated":80637952,"data_stored":80415478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":124254666},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719478378,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":245524,"kb_used_data":61772,"kb_used_omap":1,"kb_used_meta":183742,"kb_avail":94126316,"statfs":{"total":96636764160,"available":96385347584,"internally_reserved":0,"allocated":63254528,"data_stored":63047256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":188152262},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719478376,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":200588,"kb_used_data":48708,"kb_used_omap":1,"kb_used_meta":151870,"kb_avail":94171252,"statfs":{"total":96636764160,"available":96431362048,"internally_reserved":0,"allocated":49876992,"data_stored":49644774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":155515334},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719478377,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":183188,"kb_used_data":57676,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94188652,"statfs":{"total":96636764160,"available":96449179648,"internally_reserved":0,"allocated":59060224,"data_stored":58852820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424511080,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":218248,"kb_used_data":42816,"kb_used_omap":1,"kb_used_meta":175422,"kb_avail":94153592,"statfs":{"total":96636764160,"available":96413278208,"internally_reserved":0,"allocated":43843584,"data_stored":43631469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":179632583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424511080,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":152012,"kb_used_data":22340,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94219828,"statfs":{"total":96636764160,"available":96481103872,"internally_reserved":0,"allocated":22876160,"data_stored":22659915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424511081,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":228216,"kb_used_data":56880,"kb_used_omap":1,"kb_used_meta":171326,"kb_avail":94143624,"statfs":{"total":96636764160,"available":96403070976,"internally_reserved":0,"allocated":58245120,"data_stored":58030400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":175438280},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129543783,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149276,"kb_used_data":45972,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94222564,"statfs":{"total":96636764160,"available":96483905536,"internally_reserved":0,"allocated":47075328,"data_stored":46859238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":46166016,"data_stored":46138218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57929728,"data_stored":57899748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":22560768,"data_stored":22529263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":43528192,"data_stored":43500817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":58744832,"data_stored":58722168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49561600,"data_stored":49514122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":62939136,"data_stored":62916604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":79728640,"data_stored":79694458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-11T20:51:36.297 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-11T20:51:36.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.448+0000 7ff333c80640 1 Processor -- start 2024-03-11T20:51:36.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.452+0000 7ff333c80640 1 -- start start 2024-03-11T20:51:36.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.452+0000 7ff333c80640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7ff32c150dd0 0x7ff32c0c28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:36.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.452+0000 7ff333c80640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ff32c070b00 0x7ff32c0c4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:36.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.452+0000 7ff333c80640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff32c0c5de0 con 0x7ff32c1509f0 2024-03-11T20:51:36.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.452+0000 7ff333c80640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff32c156910 con 0x7ff32c150dd0 2024-03-11T20:51:36.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.452+0000 7ff333c80640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff32c0c6030 con 0x7ff32c070b00 2024-03-11T20:51:36.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.452+0000 7ff33121b640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7ff32c150dd0 0x7ff32c0c28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:36.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.452+0000 7ff33121b640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7ff32c150dd0 0x7ff32c0c28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58492/0 (socket says 172.21.15.90:58492) 2024-03-11T20:51:36.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.452+0000 7ff33121b640 1 -- 172.21.15.90:0/3448779432 learned_addr learned my addr 172.21.15.90:0/3448779432 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:51:36.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.452+0000 7ff33221d640 1 --2- 172.21.15.90:0/3448779432 >> v2:172.21.15.92:3300/0 conn(0x7ff32c070b00 0x7ff32c0c4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:36.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.456+0000 7ff33121b640 1 -- 172.21.15.90:0/3448779432 >> v2:172.21.15.92:3300/0 conn(0x7ff32c070b00 msgr2=0x7ff32c0c4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:36.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.456+0000 7ff33121b640 1 --2- 172.21.15.90:0/3448779432 >> v2:172.21.15.92:3300/0 conn(0x7ff32c070b00 0x7ff32c0c4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:36.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.456+0000 7ff33121b640 1 -- 172.21.15.90:0/3448779432 >> v1:172.21.15.92:6789/0 conn(0x7ff32c1509f0 legacy=0x7ff32c157390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:36.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.456+0000 7ff33121b640 1 -- 172.21.15.90:0/3448779432 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff32c0c6310 con 0x7ff32c150dd0 2024-03-11T20:51:36.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.460+0000 7ff33121b640 1 --2- 172.21.15.90:0/3448779432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff32c150dd0 0x7ff32c0c28e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff32c15afa0 tx=0x7ff31c01ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2db7f5c65a29f6d server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:36.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.468+0000 7ff330a1a640 1 -- 172.21.15.90:0/3448779432 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31c0a89d0 con 0x7ff32c150dd0 2024-03-11T20:51:36.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.468+0000 7ff330a1a640 1 -- 172.21.15.90:0/3448779432 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff31c0a8b30 con 0x7ff32c150dd0 2024-03-11T20:51:36.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.468+0000 7ff330a1a640 1 -- 172.21.15.90:0/3448779432 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31c0b0440 con 0x7ff32c150dd0 2024-03-11T20:51:36.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.468+0000 7ff333c80640 1 -- 172.21.15.90:0/3448779432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff32c150dd0 msgr2=0x7ff32c0c28e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:36.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.468+0000 7ff333c80640 1 --2- 172.21.15.90:0/3448779432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff32c150dd0 0x7ff32c0c28e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff32c15afa0 tx=0x7ff31c01ecd0 comp rx=0 tx=0).stop 2024-03-11T20:51:36.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.468+0000 7ff333c80640 1 -- 172.21.15.90:0/3448779432 shutdown_connections 2024-03-11T20:51:36.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.468+0000 7ff333c80640 1 --2- 172.21.15.90:0/3448779432 >> v2:172.21.15.92:3300/0 conn(0x7ff32c070b00 0x7ff32c0c4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:36.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.468+0000 7ff333c80640 1 --2- 172.21.15.90:0/3448779432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff32c150dd0 0x7ff32c0c28e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:36.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.468+0000 7ff333c80640 1 -- 172.21.15.90:0/3448779432 >> 172.21.15.90:0/3448779432 conn(0x7ff32c06ac00 msgr2=0x7ff32c06b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:36.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.468+0000 7ff333c80640 1 -- 172.21.15.90:0/3448779432 shutdown_connections 2024-03-11T20:51:36.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.468+0000 7ff333c80640 1 -- 172.21.15.90:0/3448779432 wait complete. 2024-03-11T20:51:36.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.468+0000 7ff333c80640 1 Processor -- start 2024-03-11T20:51:36.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.468+0000 7ff333c80640 1 -- start start 2024-03-11T20:51:36.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.472+0000 7ff333c80640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff32c070b00 0x7ff32c139800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:36.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.472+0000 7ff333c80640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff32c1509f0 0x7ff32c1545f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:36.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.472+0000 7ff333c80640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff32c150dd0 0x7ff32c154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:36.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.472+0000 7ff333c80640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c13e8e0 con 0x7ff32c1509f0 2024-03-11T20:51:36.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.472+0000 7ff333c80640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c1393b0 con 0x7ff32c070b00 2024-03-11T20:51:36.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.472+0000 7ff333c80640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c156210 con 0x7ff32c150dd0 2024-03-11T20:51:36.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.472+0000 7ff33121b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff32c1509f0 0x7ff32c1545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:36.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.472+0000 7ff33121b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff32c1509f0 0x7ff32c1545f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58500/0 (socket says 172.21.15.90:58500) 2024-03-11T20:51:36.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.472+0000 7ff33121b640 1 -- 172.21.15.90:0/1351084454 learned_addr learned my addr 172.21.15.90:0/1351084454 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:51:36.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.472+0000 7ff331a1c640 1 --2- 172.21.15.90:0/1351084454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff32c070b00 0x7ff32c139800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:36.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.472+0000 7ff33221d640 1 --2- 172.21.15.90:0/1351084454 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff32c150dd0 0x7ff32c154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:36.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.476+0000 7ff33121b640 1 -- 172.21.15.90:0/1351084454 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff32c150dd0 msgr2=0x7ff32c154b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:36.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.476+0000 7ff33121b640 1 --2- 172.21.15.90:0/1351084454 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff32c150dd0 0x7ff32c154b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:36.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.476+0000 7ff33121b640 1 -- 172.21.15.90:0/1351084454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff32c070b00 msgr2=0x7ff32c139800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:36.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.476+0000 7ff33121b640 1 --2- 172.21.15.90:0/1351084454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff32c070b00 0x7ff32c139800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:36.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.476+0000 7ff33121b640 1 -- 172.21.15.90:0/1351084454 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff31c0094d0 con 0x7ff32c1509f0 2024-03-11T20:51:36.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.480+0000 7ff33121b640 1 --2- 172.21.15.90:0/1351084454 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff32c1509f0 0x7ff32c1545f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff31c01b8e0 tx=0x7ff31c01ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:36.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.484+0000 7ff3227fc640 1 -- 172.21.15.90:0/1351084454 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31c0b0d70 con 0x7ff32c1509f0 2024-03-11T20:51:36.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.484+0000 7ff3227fc640 1 -- 172.21.15.90:0/1351084454 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff31c0ba440 con 0x7ff32c1509f0 2024-03-11T20:51:36.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.484+0000 7ff333c80640 1 -- 172.21.15.90:0/1351084454 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff32c136230 con 0x7ff32c1509f0 2024-03-11T20:51:36.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.484+0000 7ff3227fc640 1 -- 172.21.15.90:0/1351084454 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31c0ba7f0 con 0x7ff32c1509f0 2024-03-11T20:51:36.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.484+0000 7ff333c80640 1 -- 172.21.15.90:0/1351084454 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff32c13edd0 con 0x7ff32c1509f0 2024-03-11T20:51:36.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.484+0000 7ff333c80640 1 -- 172.21.15.90:0/1351084454 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff32c139520 con 0x7ff32c1509f0 2024-03-11T20:51:36.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.764+0000 7ff3227fc640 1 -- 172.21.15.90:0/1351084454 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7ff31c0ba950 con 0x7ff32c1509f0 2024-03-11T20:51:36.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.764+0000 7ff3227fc640 1 --2- 172.21.15.90:0/1351084454 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7ff310066e90 0x7ff310069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:36.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.764+0000 7ff331a1c640 1 --2- 172.21.15.90:0/1351084454 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7ff310066e90 0x7ff310069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:36.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.764+0000 7ff331a1c640 1 --2- 172.21.15.90:0/1351084454 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7ff310066e90 0x7ff310069350 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff328007fd0 tx=0x7ff328009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:36.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.804+0000 7ff3227fc640 1 -- 172.21.15.90:0/1351084454 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7ff31c0f3c40 con 0x7ff32c1509f0 2024-03-11T20:51:36.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.808+0000 7ff3227fc640 1 -- 172.21.15.90:0/1351084454 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff31c0efee0 con 0x7ff32c1509f0 2024-03-11T20:51:36.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.968+0000 7ff333c80640 1 -- 172.21.15.90:0/1351084454 --> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff32c0c3da0 con 0x7ff310066e90 2024-03-11T20:51:36.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.976+0000 7ff3227fc640 1 -- 172.21.15.90:0/1351084454 <== mgr.4101 v2:172.21.15.90:6800/1999720161 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+334701 (secure 0 0 0) 0x7ff32c0c3da0 con 0x7ff310066e90 2024-03-11T20:51:36.979 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T20:51:36.981 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-03-11T20:51:36.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.980+0000 7ff333c80640 1 -- 172.21.15.90:0/1351084454 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7ff310066e90 msgr2=0x7ff310069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:36.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.980+0000 7ff333c80640 1 --2- 172.21.15.90:0/1351084454 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7ff310066e90 0x7ff310069350 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff328007fd0 tx=0x7ff328009040 comp rx=0 tx=0).stop 2024-03-11T20:51:36.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.980+0000 7ff333c80640 1 -- 172.21.15.90:0/1351084454 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff32c1509f0 msgr2=0x7ff32c1545f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:36.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.980+0000 7ff333c80640 1 --2- 172.21.15.90:0/1351084454 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff32c1509f0 0x7ff32c1545f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff31c01b8e0 tx=0x7ff31c01ba70 comp rx=0 tx=0).stop 2024-03-11T20:51:36.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.980+0000 7ff333c80640 1 -- 172.21.15.90:0/1351084454 shutdown_connections 2024-03-11T20:51:36.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.980+0000 7ff333c80640 1 --2- 172.21.15.90:0/1351084454 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7ff310066e90 0x7ff310069350 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:36.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.980+0000 7ff333c80640 1 --2- 172.21.15.90:0/1351084454 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ff32c150dd0 0x7ff32c154b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:36.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.980+0000 7ff333c80640 1 --2- 172.21.15.90:0/1351084454 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff32c1509f0 0x7ff32c1545f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:36.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.980+0000 7ff333c80640 1 --2- 172.21.15.90:0/1351084454 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ff32c070b00 0x7ff32c139800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:36.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.980+0000 7ff333c80640 1 -- 172.21.15.90:0/1351084454 >> 172.21.15.90:0/1351084454 conn(0x7ff32c06ac00 msgr2=0x7ff32c06afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:36.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.980+0000 7ff333c80640 1 -- 172.21.15.90:0/1351084454 shutdown_connections 2024-03-11T20:51:36.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:36.980+0000 7ff333c80640 1 -- 172.21.15.90:0/1351084454 wait complete. 2024-03-11T20:51:36.997 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":4115,"stamp":"2024-03-11T20:51:35.451217+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":211048067,"num_objects":150,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":150,"num_whiteouts":0,"num_read":7774,"num_read_kb":1584288,"num_write":72048,"num_write_kb":4420358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76413,"ondisk_log_size":76413,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1577152,"kb_used_data":414912,"kb_used_omap":12,"kb_used_meta":1162163,"kb_avail":753397568,"statfs":{"total":773094113280,"available":771479109632,"internally_reserved":0,"allocated":424869888,"data_stored":423141350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":1190055476},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7707,"num_objects":-1,"num_object_clones":0,"num_object_copies":-4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":25,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005057"},"pg_stats":[{"pgid":"2.1f","version":"30'681","reported_seq":777,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:18.510855+0000","last_change":"2024-03-11T18:35:56.290140+0000","last_active":"2024-03-11T20:51:18.510855+0000","last_peered":"2024-03-11T20:51:18.510855+0000","last_clean":"2024-03-11T20:51:18.510855+0000","last_became_active":"2024-03-11T18:35:56.289178+0000","last_became_peered":"2024-03-11T18:35:56.289178+0000","last_unstale":"2024-03-11T20:51:18.510855+0000","last_undegraded":"2024-03-11T20:51:18.510855+0000","last_fullsized":"2024-03-11T20:51:18.510855+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":681,"log_dups_size":0,"ondisk_log_size":681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:59:46.211033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389280,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":643,"num_write_kb":8491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"30'202","reported_seq":298,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.761828+0000","last_change":"2024-03-11T18:35:57.676047+0000","last_active":"2024-03-11T20:51:29.761828+0000","last_peered":"2024-03-11T20:51:29.761828+0000","last_clean":"2024-03-11T20:51:29.761828+0000","last_became_active":"2024-03-11T18:35:57.675105+0000","last_became_peered":"2024-03-11T18:35:57.675105+0000","last_unstale":"2024-03-11T20:51:29.761828+0000","last_undegraded":"2024-03-11T20:51:29.761828+0000","last_fullsized":"2024-03-11T20:51:29.761828+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:45:39.057074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"30'811","reported_seq":1003,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:10.788625+0000","last_change":"2024-03-11T18:36:04.607351+0000","last_active":"2024-03-11T20:51:10.788625+0000","last_peered":"2024-03-11T20:51:10.788625+0000","last_clean":"2024-03-11T20:51:10.788625+0000","last_became_active":"2024-03-11T18:36:04.599562+0000","last_became_peered":"2024-03-11T18:36:04.599562+0000","last_unstale":"2024-03-11T20:51:10.788625+0000","last_undegraded":"2024-03-11T20:51:10.788625+0000","last_fullsized":"2024-03-11T20:51:10.788625+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":811,"log_dups_size":0,"ondisk_log_size":811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T18:57:51.134410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":119,"num_read_kb":64172,"num_write":795,"num_write_kb":180282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"30'654","reported_seq":752,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:49:28.711104+0000","last_change":"2024-03-11T18:35:56.291253+0000","last_active":"2024-03-11T20:49:28.711104+0000","last_peered":"2024-03-11T20:49:28.711104+0000","last_clean":"2024-03-11T20:49:28.711104+0000","last_became_active":"2024-03-11T18:35:56.290862+0000","last_became_peered":"2024-03-11T18:35:56.290862+0000","last_unstale":"2024-03-11T20:49:28.711104+0000","last_undegraded":"2024-03-11T20:49:28.711104+0000","last_fullsized":"2024-03-11T20:49:28.711104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":654,"log_dups_size":0,"ondisk_log_size":654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:36:17.809744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":603,"num_write_kb":8489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"30'187","reported_seq":286,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.887083+0000","last_change":"2024-03-11T18:37:04.143178+0000","last_active":"2024-03-11T20:51:29.887083+0000","last_peered":"2024-03-11T20:51:29.887083+0000","last_clean":"2024-03-11T20:51:29.887083+0000","last_became_active":"2024-03-11T18:37:04.142880+0000","last_became_peered":"2024-03-11T18:37:04.142880+0000","last_unstale":"2024-03-11T20:51:29.887083+0000","last_undegraded":"2024-03-11T20:51:29.887083+0000","last_fullsized":"2024-03-11T20:51:29.887083+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:58:36.259380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"30'785","reported_seq":972,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:17.717779+0000","last_change":"2024-03-11T18:37:04.274577+0000","last_active":"2024-03-11T20:51:17.717779+0000","last_peered":"2024-03-11T20:51:17.717779+0000","last_clean":"2024-03-11T20:51:17.717779+0000","last_became_active":"2024-03-11T18:37:04.274018+0000","last_became_peered":"2024-03-11T18:37:04.274018+0000","last_unstale":"2024-03-11T20:51:17.717779+0000","last_undegraded":"2024-03-11T20:51:17.717779+0000","last_fullsized":"2024-03-11T20:51:17.717779+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":785,"log_dups_size":0,"ondisk_log_size":785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:51:28.017349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":106,"num_read_kb":55696,"num_write":774,"num_write_kb":160695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2,3],"acting":[4,6,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"30'1734","reported_seq":1819,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:50:57.274928+0000","last_change":"2024-03-11T18:35:56.355147+0000","last_active":"2024-03-11T20:50:57.274928+0000","last_peered":"2024-03-11T20:50:57.274928+0000","last_clean":"2024-03-11T20:50:57.274928+0000","last_became_active":"2024-03-11T18:35:56.354388+0000","last_became_peered":"2024-03-11T18:35:56.354388+0000","last_unstale":"2024-03-11T20:50:57.274928+0000","last_undegraded":"2024-03-11T20:50:57.274928+0000","last_fullsized":"2024-03-11T20:50:57.274928+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1734,"log_dups_size":0,"ondisk_log_size":1734,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:03:55.046045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1669,"num_write_kb":17224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"30'190","reported_seq":273,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.629989+0000","last_change":"2024-03-11T18:35:57.641486+0000","last_active":"2024-03-11T20:51:29.629989+0000","last_peered":"2024-03-11T20:51:29.629989+0000","last_clean":"2024-03-11T20:51:29.629989+0000","last_became_active":"2024-03-11T18:35:57.641052+0000","last_became_peered":"2024-03-11T18:35:57.641052+0000","last_unstale":"2024-03-11T20:51:29.629989+0000","last_undegraded":"2024-03-11T20:51:29.629989+0000","last_fullsized":"2024-03-11T20:51:29.629989+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:29:45.662029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"30'808","reported_seq":973,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:08.067327+0000","last_change":"2024-03-11T18:36:04.629291+0000","last_active":"2024-03-11T20:51:08.067327+0000","last_peered":"2024-03-11T20:51:08.067327+0000","last_clean":"2024-03-11T20:51:08.067327+0000","last_became_active":"2024-03-11T18:36:04.624110+0000","last_became_peered":"2024-03-11T18:36:04.624110+0000","last_unstale":"2024-03-11T20:51:08.067327+0000","last_undegraded":"2024-03-11T20:51:08.067327+0000","last_fullsized":"2024-03-11T20:51:08.067327+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":808,"log_dups_size":0,"ondisk_log_size":808,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:04:03.368322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":93,"num_read_kb":49009,"num_write":797,"num_write_kb":196902,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"30'832","reported_seq":917,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:50:57.274905+0000","last_change":"2024-03-11T18:35:56.298903+0000","last_active":"2024-03-11T20:50:57.274905+0000","last_peered":"2024-03-11T20:50:57.274905+0000","last_clean":"2024-03-11T20:50:57.274905+0000","last_became_active":"2024-03-11T18:35:56.297582+0000","last_became_peered":"2024-03-11T18:35:56.297582+0000","last_unstale":"2024-03-11T20:50:57.274905+0000","last_undegraded":"2024-03-11T20:50:57.274905+0000","last_fullsized":"2024-03-11T20:50:57.274905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":832,"log_dups_size":0,"ondisk_log_size":832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:53:25.278916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":777,"num_write_kb":4502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"30'203","reported_seq":286,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.623592+0000","last_change":"2024-03-11T18:35:57.640122+0000","last_active":"2024-03-11T20:51:29.623592+0000","last_peered":"2024-03-11T20:51:29.623592+0000","last_clean":"2024-03-11T20:51:29.623592+0000","last_became_active":"2024-03-11T18:35:57.639770+0000","last_became_peered":"2024-03-11T18:35:57.639770+0000","last_unstale":"2024-03-11T20:51:29.623592+0000","last_undegraded":"2024-03-11T20:51:29.623592+0000","last_fullsized":"2024-03-11T20:51:29.623592+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:39:25.767184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"30'729","reported_seq":825,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:08.238310+0000","last_change":"2024-03-11T18:36:04.627149+0000","last_active":"2024-03-11T20:51:08.238310+0000","last_peered":"2024-03-11T20:51:08.238310+0000","last_clean":"2024-03-11T20:51:08.238310+0000","last_became_active":"2024-03-11T18:36:04.626108+0000","last_became_peered":"2024-03-11T18:36:04.626108+0000","last_unstale":"2024-03-11T20:51:08.238310+0000","last_undegraded":"2024-03-11T20:51:08.238310+0000","last_fullsized":"2024-03-11T20:51:08.238310+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":729,"log_dups_size":0,"ondisk_log_size":729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:03:40.619618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":20804,"num_write":712,"num_write_kb":76413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"30'216","reported_seq":307,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.960165+0000","last_change":"2024-03-11T18:35:57.616739+0000","last_active":"2024-03-11T20:51:29.960165+0000","last_peered":"2024-03-11T20:51:29.960165+0000","last_clean":"2024-03-11T20:51:29.960165+0000","last_became_active":"2024-03-11T18:35:57.616398+0000","last_became_peered":"2024-03-11T18:35:57.616398+0000","last_unstale":"2024-03-11T20:51:29.960165+0000","last_undegraded":"2024-03-11T20:51:29.960165+0000","last_fullsized":"2024-03-11T20:51:29.960165+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:53:30.324898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"30'1448","reported_seq":1539,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:49:55.997509+0000","last_change":"2024-03-11T18:35:56.304762+0000","last_active":"2024-03-11T20:49:55.997509+0000","last_peered":"2024-03-11T20:49:55.997509+0000","last_clean":"2024-03-11T20:49:55.997509+0000","last_became_active":"2024-03-11T18:35:56.302803+0000","last_became_peered":"2024-03-11T18:35:56.302803+0000","last_unstale":"2024-03-11T20:49:55.997509+0000","last_undegraded":"2024-03-11T20:49:55.997509+0000","last_fullsized":"2024-03-11T20:49:55.997509+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1448,"log_dups_size":0,"ondisk_log_size":1448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:31:18.726996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1389,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"30'793","reported_seq":965,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:50:56.185068+0000","last_change":"2024-03-11T18:36:04.626968+0000","last_active":"2024-03-11T20:50:56.185068+0000","last_peered":"2024-03-11T20:50:56.185068+0000","last_clean":"2024-03-11T20:50:56.185068+0000","last_became_active":"2024-03-11T18:36:04.626337+0000","last_became_peered":"2024-03-11T18:36:04.626337+0000","last_unstale":"2024-03-11T20:50:56.185068+0000","last_undegraded":"2024-03-11T20:50:56.185068+0000","last_fullsized":"2024-03-11T20:50:56.185068+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":793,"log_dups_size":0,"ondisk_log_size":793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:53:37.432826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":94,"num_read_kb":46134,"num_write":781,"num_write_kb":151379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"30'211","reported_seq":300,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.763712+0000","last_change":"2024-03-11T18:35:57.689835+0000","last_active":"2024-03-11T20:51:29.763712+0000","last_peered":"2024-03-11T20:51:29.763712+0000","last_clean":"2024-03-11T20:51:29.763712+0000","last_became_active":"2024-03-11T18:35:57.689039+0000","last_became_peered":"2024-03-11T18:35:57.689039+0000","last_unstale":"2024-03-11T20:51:29.763712+0000","last_undegraded":"2024-03-11T20:51:29.763712+0000","last_fullsized":"2024-03-11T20:51:29.763712+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:38:27.733669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"30'1235","reported_seq":1328,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:49:27.679855+0000","last_change":"2024-03-11T18:35:56.291070+0000","last_active":"2024-03-11T20:49:27.679855+0000","last_peered":"2024-03-11T20:49:27.679855+0000","last_clean":"2024-03-11T20:49:27.679855+0000","last_became_active":"2024-03-11T18:35:56.290504+0000","last_became_peered":"2024-03-11T18:35:56.290504+0000","last_unstale":"2024-03-11T20:49:27.679855+0000","last_undegraded":"2024-03-11T20:49:27.679855+0000","last_fullsized":"2024-03-11T20:49:27.679855+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1235,"log_dups_size":0,"ondisk_log_size":1235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:29:31.660243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1168,"num_write_kb":8796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"30'848","reported_seq":1062,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:12.395257+0000","last_change":"2024-03-11T18:36:04.626150+0000","last_active":"2024-03-11T20:51:12.395257+0000","last_peered":"2024-03-11T20:51:12.395257+0000","last_clean":"2024-03-11T20:51:12.395257+0000","last_became_active":"2024-03-11T18:36:04.625084+0000","last_became_peered":"2024-03-11T18:36:04.625084+0000","last_unstale":"2024-03-11T20:51:12.395257+0000","last_undegraded":"2024-03-11T20:51:12.395257+0000","last_fullsized":"2024-03-11T20:51:12.395257+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":848,"log_dups_size":0,"ondisk_log_size":848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:28:58.373219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":136,"num_read_kb":82099,"num_write":832,"num_write_kb":151378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"30'209","reported_seq":300,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.771259+0000","last_change":"2024-03-11T18:35:57.684535+0000","last_active":"2024-03-11T20:51:29.771259+0000","last_peered":"2024-03-11T20:51:29.771259+0000","last_clean":"2024-03-11T20:51:29.771259+0000","last_became_active":"2024-03-11T18:35:57.681098+0000","last_became_peered":"2024-03-11T18:35:57.681098+0000","last_unstale":"2024-03-11T20:51:29.771259+0000","last_undegraded":"2024-03-11T20:51:29.771259+0000","last_fullsized":"2024-03-11T20:51:29.771259+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:25:49.707568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"30'483","reported_seq":576,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:49:29.907364+0000","last_change":"2024-03-11T18:35:56.290745+0000","last_active":"2024-03-11T20:49:29.907364+0000","last_peered":"2024-03-11T20:49:29.907364+0000","last_clean":"2024-03-11T20:49:29.907364+0000","last_became_active":"2024-03-11T18:35:56.290006+0000","last_became_peered":"2024-03-11T18:35:56.290006+0000","last_unstale":"2024-03-11T20:49:29.907364+0000","last_undegraded":"2024-03-11T20:49:29.907364+0000","last_fullsized":"2024-03-11T20:49:29.907364+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:06:02.844765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194394,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":433,"num_write_kb":4314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"30'749","reported_seq":894,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:08.589998+0000","last_change":"2024-03-11T18:36:04.620689+0000","last_active":"2024-03-11T20:51:08.589998+0000","last_peered":"2024-03-11T20:51:08.589998+0000","last_clean":"2024-03-11T20:51:08.589998+0000","last_became_active":"2024-03-11T18:36:04.620282+0000","last_became_peered":"2024-03-11T18:36:04.620282+0000","last_unstale":"2024-03-11T20:51:08.589998+0000","last_undegraded":"2024-03-11T20:51:08.589998+0000","last_fullsized":"2024-03-11T20:51:08.589998+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":749,"log_dups_size":0,"ondisk_log_size":749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:54:16.807299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":68,"num_read_kb":34102,"num_write":731,"num_write_kb":109760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"30'224","reported_seq":315,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.532383+0000","last_change":"2024-03-11T18:35:57.615234+0000","last_active":"2024-03-11T20:51:29.532383+0000","last_peered":"2024-03-11T20:51:29.532383+0000","last_clean":"2024-03-11T20:51:29.532383+0000","last_became_active":"2024-03-11T18:35:57.614885+0000","last_became_peered":"2024-03-11T18:35:57.614885+0000","last_unstale":"2024-03-11T20:51:29.532383+0000","last_undegraded":"2024-03-11T20:51:29.532383+0000","last_fullsized":"2024-03-11T20:51:29.532383+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":224,"log_dups_size":0,"ondisk_log_size":224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:22:13.730630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"30'348","reported_seq":441,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:50:17.268278+0000","last_change":"2024-03-11T18:35:56.304617+0000","last_active":"2024-03-11T20:50:17.268278+0000","last_peered":"2024-03-11T20:50:17.268278+0000","last_clean":"2024-03-11T20:50:17.268278+0000","last_became_active":"2024-03-11T18:35:56.301698+0000","last_became_peered":"2024-03-11T18:35:56.301698+0000","last_unstale":"2024-03-11T20:50:17.268278+0000","last_undegraded":"2024-03-11T20:50:17.268278+0000","last_fullsized":"2024-03-11T20:50:17.268278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:29:31.660243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":289,"num_write_kb":4232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"30'714","reported_seq":907,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:15.480325+0000","last_change":"2024-03-11T18:37:04.272234+0000","last_active":"2024-03-11T20:51:15.480325+0000","last_peered":"2024-03-11T20:51:15.480325+0000","last_clean":"2024-03-11T20:51:15.480325+0000","last_became_active":"2024-03-11T18:37:04.271646+0000","last_became_peered":"2024-03-11T18:37:04.271646+0000","last_unstale":"2024-03-11T20:51:15.480325+0000","last_undegraded":"2024-03-11T20:51:15.480325+0000","last_fullsized":"2024-03-11T20:51:15.480325+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:51:13.205384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":108,"num_read_kb":62192,"num_write":690,"num_write_kb":100980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,2],"acting":[4,3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"30'216","reported_seq":307,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.862313+0000","last_change":"2024-03-11T18:35:57.506963+0000","last_active":"2024-03-11T20:51:29.862313+0000","last_peered":"2024-03-11T20:51:29.862313+0000","last_clean":"2024-03-11T20:51:29.862313+0000","last_became_active":"2024-03-11T18:35:57.506038+0000","last_became_peered":"2024-03-11T18:35:57.506038+0000","last_unstale":"2024-03-11T20:51:29.862313+0000","last_undegraded":"2024-03-11T20:51:29.862313+0000","last_fullsized":"2024-03-11T20:51:29.862313+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:52:06.024245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"30'413","reported_seq":506,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:50:17.304480+0000","last_change":"2024-03-11T18:35:56.364512+0000","last_active":"2024-03-11T20:50:17.304480+0000","last_peered":"2024-03-11T20:50:17.304480+0000","last_clean":"2024-03-11T20:50:17.304480+0000","last_became_active":"2024-03-11T18:35:56.363731+0000","last_became_peered":"2024-03-11T18:35:56.363731+0000","last_unstale":"2024-03-11T20:50:17.304480+0000","last_undegraded":"2024-03-11T20:50:17.304480+0000","last_fullsized":"2024-03-11T20:50:17.304480+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:36:17.809744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":342,"num_write_kb":4261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"30'829","reported_seq":1056,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:14.415675+0000","last_change":"2024-03-11T18:36:04.643782+0000","last_active":"2024-03-11T20:51:14.415675+0000","last_peered":"2024-03-11T20:51:14.415675+0000","last_clean":"2024-03-11T20:51:14.415675+0000","last_became_active":"2024-03-11T18:36:04.642133+0000","last_became_peered":"2024-03-11T18:36:04.642133+0000","last_unstale":"2024-03-11T20:51:14.415675+0000","last_undegraded":"2024-03-11T20:51:14.415675+0000","last_fullsized":"2024-03-11T20:51:14.415675+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":829,"log_dups_size":0,"ondisk_log_size":829,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:53:33.601193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":144,"num_read_kb":78357,"num_write":819,"num_write_kb":238245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"30'168","reported_seq":259,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.764491+0000","last_change":"2024-03-11T18:35:57.675207+0000","last_active":"2024-03-11T20:51:29.764491+0000","last_peered":"2024-03-11T20:51:29.764491+0000","last_clean":"2024-03-11T20:51:29.764491+0000","last_became_active":"2024-03-11T18:35:57.674472+0000","last_became_peered":"2024-03-11T18:35:57.674472+0000","last_unstale":"2024-03-11T20:51:29.764491+0000","last_undegraded":"2024-03-11T20:51:29.764491+0000","last_fullsized":"2024-03-11T20:51:29.764491+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:29:45.662029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"30'6434","reported_seq":6527,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:22.287372+0000","last_change":"2024-03-11T18:35:56.348479+0000","last_active":"2024-03-11T20:51:22.287372+0000","last_peered":"2024-03-11T20:51:22.287372+0000","last_clean":"2024-03-11T20:51:22.287372+0000","last_became_active":"2024-03-11T18:35:56.348040+0000","last_became_peered":"2024-03-11T18:35:56.348040+0000","last_unstale":"2024-03-11T20:51:22.287372+0000","last_undegraded":"2024-03-11T20:51:22.287372+0000","last_fullsized":"2024-03-11T20:51:22.287372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":6434,"log_dups_size":0,"ondisk_log_size":6434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:27:26.471550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17421,"num_write_kb":6249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"30'799","reported_seq":995,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:50:55.170434+0000","last_change":"2024-03-11T18:36:04.599805+0000","last_active":"2024-03-11T20:50:55.170434+0000","last_peered":"2024-03-11T20:50:55.170434+0000","last_clean":"2024-03-11T20:50:55.170434+0000","last_became_active":"2024-03-11T18:36:04.599065+0000","last_became_peered":"2024-03-11T18:36:04.599065+0000","last_unstale":"2024-03-11T20:50:55.170434+0000","last_undegraded":"2024-03-11T20:50:55.170434+0000","last_fullsized":"2024-03-11T20:50:55.170434+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":799,"log_dups_size":0,"ondisk_log_size":799,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:36:07.187003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":111,"num_read_kb":58643,"num_write":786,"num_write_kb":122786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"30'199","reported_seq":293,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.664765+0000","last_change":"2024-03-11T18:35:57.538551+0000","last_active":"2024-03-11T20:51:29.664765+0000","last_peered":"2024-03-11T20:51:29.664765+0000","last_clean":"2024-03-11T20:51:29.664765+0000","last_became_active":"2024-03-11T18:35:57.537706+0000","last_became_peered":"2024-03-11T18:35:57.537706+0000","last_unstale":"2024-03-11T20:51:29.664765+0000","last_undegraded":"2024-03-11T20:51:29.664765+0000","last_fullsized":"2024-03-11T20:51:29.664765+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:03:56.260394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"30'985","reported_seq":1080,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:49:47.414091+0000","last_change":"2024-03-11T18:35:56.302575+0000","last_active":"2024-03-11T20:49:47.414091+0000","last_peered":"2024-03-11T20:49:47.414091+0000","last_clean":"2024-03-11T20:49:47.414091+0000","last_became_active":"2024-03-11T18:35:56.301819+0000","last_became_peered":"2024-03-11T18:35:56.301819+0000","last_unstale":"2024-03-11T20:49:47.414091+0000","last_undegraded":"2024-03-11T20:49:47.414091+0000","last_fullsized":"2024-03-11T20:49:47.414091+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":985,"log_dups_size":0,"ondisk_log_size":985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:38:26.519320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":935,"num_write_kb":8676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"30'751","reported_seq":913,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:14.544182+0000","last_change":"2024-03-11T18:36:04.609282+0000","last_active":"2024-03-11T20:51:14.544182+0000","last_peered":"2024-03-11T20:51:14.544182+0000","last_clean":"2024-03-11T20:51:14.544182+0000","last_became_active":"2024-03-11T18:36:04.608260+0000","last_became_peered":"2024-03-11T18:36:04.608260+0000","last_unstale":"2024-03-11T20:51:14.544182+0000","last_undegraded":"2024-03-11T20:51:14.544182+0000","last_fullsized":"2024-03-11T20:51:14.544182+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":751,"log_dups_size":0,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:49:28.143755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":79,"num_read_kb":54998,"num_write":738,"num_write_kb":142673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"30'218","reported_seq":311,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.614589+0000","last_change":"2024-03-11T18:35:57.688727+0000","last_active":"2024-03-11T20:51:29.614589+0000","last_peered":"2024-03-11T20:51:29.614589+0000","last_clean":"2024-03-11T20:51:29.614589+0000","last_became_active":"2024-03-11T18:35:57.688209+0000","last_became_peered":"2024-03-11T18:35:57.688209+0000","last_unstale":"2024-03-11T20:51:29.614589+0000","last_undegraded":"2024-03-11T20:51:29.614589+0000","last_fullsized":"2024-03-11T20:51:29.614589+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:58:13.389498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"30'373","reported_seq":469,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:49:53.793880+0000","last_change":"2024-03-11T18:35:56.293842+0000","last_active":"2024-03-11T20:49:53.793880+0000","last_peered":"2024-03-11T20:49:53.793880+0000","last_clean":"2024-03-11T20:49:53.793880+0000","last_became_active":"2024-03-11T18:35:56.293276+0000","last_became_peered":"2024-03-11T18:35:56.293276+0000","last_unstale":"2024-03-11T20:49:53.793880+0000","last_undegraded":"2024-03-11T20:49:53.793880+0000","last_fullsized":"2024-03-11T20:49:53.793880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:04:13.101160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":309,"num_write_kb":4248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"30'742","reported_seq":881,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:15.435885+0000","last_change":"2024-03-11T18:36:04.626320+0000","last_active":"2024-03-11T20:51:15.435885+0000","last_peered":"2024-03-11T20:51:15.435885+0000","last_clean":"2024-03-11T20:51:15.435885+0000","last_became_active":"2024-03-11T18:36:04.625663+0000","last_became_peered":"2024-03-11T18:36:04.625663+0000","last_unstale":"2024-03-11T20:51:15.435885+0000","last_undegraded":"2024-03-11T20:51:15.435885+0000","last_fullsized":"2024-03-11T20:51:15.435885+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":742,"log_dups_size":0,"ondisk_log_size":742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:39:32.875112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":56,"num_read_kb":20684,"num_write":725,"num_write_kb":55759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"30'199","reported_seq":293,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.548835+0000","last_change":"2024-03-11T18:35:57.612242+0000","last_active":"2024-03-11T20:51:29.548835+0000","last_peered":"2024-03-11T20:51:29.548835+0000","last_clean":"2024-03-11T20:51:29.548835+0000","last_became_active":"2024-03-11T18:35:57.611509+0000","last_became_peered":"2024-03-11T18:35:57.611509+0000","last_unstale":"2024-03-11T20:51:29.548835+0000","last_undegraded":"2024-03-11T20:51:29.548835+0000","last_fullsized":"2024-03-11T20:51:29.548835+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:45:28.543623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"30'1190","reported_seq":1286,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:49:17.484772+0000","last_change":"2024-03-11T18:35:56.298766+0000","last_active":"2024-03-11T20:49:17.484772+0000","last_peered":"2024-03-11T20:49:17.484772+0000","last_clean":"2024-03-11T20:49:17.484772+0000","last_became_active":"2024-03-11T18:35:56.297747+0000","last_became_peered":"2024-03-11T18:35:56.297747+0000","last_unstale":"2024-03-11T20:49:17.484772+0000","last_undegraded":"2024-03-11T20:49:17.484772+0000","last_fullsized":"2024-03-11T20:49:17.484772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1190,"log_dups_size":0,"ondisk_log_size":1190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:13:39.177689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389196,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1123,"num_write_kb":8732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"30'749","reported_seq":963,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:17.573144+0000","last_change":"2024-03-11T18:36:04.635773+0000","last_active":"2024-03-11T20:51:17.573144+0000","last_peered":"2024-03-11T20:51:17.573144+0000","last_clean":"2024-03-11T20:51:17.573144+0000","last_became_active":"2024-03-11T18:36:04.635261+0000","last_became_peered":"2024-03-11T18:36:04.635261+0000","last_unstale":"2024-03-11T20:51:17.573144+0000","last_undegraded":"2024-03-11T20:51:17.573144+0000","last_fullsized":"2024-03-11T20:51:17.573144+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":749,"log_dups_size":0,"ondisk_log_size":749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:44:18.985469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":131,"num_read_kb":65646,"num_write":733,"num_write_kb":179820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"30'211","reported_seq":307,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.668044+0000","last_change":"2024-03-11T18:35:57.538552+0000","last_active":"2024-03-11T20:51:29.668044+0000","last_peered":"2024-03-11T20:51:29.668044+0000","last_clean":"2024-03-11T20:51:29.668044+0000","last_became_active":"2024-03-11T18:35:57.537717+0000","last_became_peered":"2024-03-11T18:35:57.537717+0000","last_unstale":"2024-03-11T20:51:29.668044+0000","last_undegraded":"2024-03-11T20:51:29.668044+0000","last_fullsized":"2024-03-11T20:51:29.668044+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:53:26.493265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"30'4762","reported_seq":9555,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:24.467774+0000","last_change":"2024-03-11T18:35:56.294572+0000","last_active":"2024-03-11T20:51:24.467774+0000","last_peered":"2024-03-11T20:51:24.467774+0000","last_clean":"2024-03-11T20:51:24.467774+0000","last_became_active":"2024-03-11T18:35:56.294096+0000","last_became_peered":"2024-03-11T18:35:56.294096+0000","last_unstale":"2024-03-11T20:51:24.467774+0000","last_undegraded":"2024-03-11T20:51:24.467774+0000","last_fullsized":"2024-03-11T20:51:24.467774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":4762,"log_dups_size":0,"ondisk_log_size":4762,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:22:12.516281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1555811,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4697,"num_read_kb":5145,"num_write":4704,"num_write_kb":5149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"30'682","reported_seq":802,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:08.234604+0000","last_change":"2024-03-11T18:36:04.620762+0000","last_active":"2024-03-11T20:51:08.234604+0000","last_peered":"2024-03-11T20:51:08.234604+0000","last_clean":"2024-03-11T20:51:08.234604+0000","last_became_active":"2024-03-11T18:36:04.620356+0000","last_became_peered":"2024-03-11T18:36:04.620356+0000","last_unstale":"2024-03-11T20:51:08.234604+0000","last_undegraded":"2024-03-11T20:51:08.234604+0000","last_fullsized":"2024-03-11T20:51:08.234604+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":682,"log_dups_size":0,"ondisk_log_size":682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:58:20.497426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":17597,"num_write":672,"num_write_kb":47486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"30'198","reported_seq":292,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.887154+0000","last_change":"2024-03-11T18:35:57.602773+0000","last_active":"2024-03-11T20:51:29.887154+0000","last_peered":"2024-03-11T20:51:29.887154+0000","last_clean":"2024-03-11T20:51:29.887154+0000","last_became_active":"2024-03-11T18:35:57.602060+0000","last_became_peered":"2024-03-11T18:35:57.602060+0000","last_unstale":"2024-03-11T20:51:29.887154+0000","last_undegraded":"2024-03-11T20:51:29.887154+0000","last_fullsized":"2024-03-11T20:51:29.887154+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:36:22.201707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"30'1021","reported_seq":1119,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:50:32.270816+0000","last_change":"2024-03-11T18:35:56.305283+0000","last_active":"2024-03-11T20:50:32.270816+0000","last_peered":"2024-03-11T20:50:32.270816+0000","last_clean":"2024-03-11T20:50:32.270816+0000","last_became_active":"2024-03-11T18:35:56.303063+0000","last_became_peered":"2024-03-11T18:35:56.303063+0000","last_unstale":"2024-03-11T20:50:32.270816+0000","last_undegraded":"2024-03-11T20:50:32.270816+0000","last_fullsized":"2024-03-11T20:50:32.270816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1021,"log_dups_size":0,"ondisk_log_size":1021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T18:51:00.795706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":963,"num_write_kb":12732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"30'852","reported_seq":1017,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:15.381877+0000","last_change":"2024-03-11T18:36:04.617247+0000","last_active":"2024-03-11T20:51:15.381877+0000","last_peered":"2024-03-11T20:51:15.381877+0000","last_clean":"2024-03-11T20:51:15.381877+0000","last_became_active":"2024-03-11T18:36:04.610765+0000","last_became_peered":"2024-03-11T18:36:04.610765+0000","last_unstale":"2024-03-11T20:51:15.381877+0000","last_undegraded":"2024-03-11T20:51:15.381877+0000","last_fullsized":"2024-03-11T20:51:15.381877+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:22:19.187685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":85,"num_read_kb":37203,"num_write":843,"num_write_kb":103063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"30'185","reported_seq":279,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.764151+0000","last_change":"2024-03-11T18:35:57.684207+0000","last_active":"2024-03-11T20:51:29.764151+0000","last_peered":"2024-03-11T20:51:29.764151+0000","last_clean":"2024-03-11T20:51:29.764151+0000","last_became_active":"2024-03-11T18:35:57.682653+0000","last_became_peered":"2024-03-11T18:35:57.682653+0000","last_unstale":"2024-03-11T20:51:29.764151+0000","last_undegraded":"2024-03-11T20:51:29.764151+0000","last_fullsized":"2024-03-11T20:51:29.764151+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":185,"log_dups_size":0,"ondisk_log_size":185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:28:51.265291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"30'371","reported_seq":467,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:08.599939+0000","last_change":"2024-03-11T18:35:56.297078+0000","last_active":"2024-03-11T20:51:08.599939+0000","last_peered":"2024-03-11T20:51:08.599939+0000","last_clean":"2024-03-11T20:51:08.599939+0000","last_became_active":"2024-03-11T18:35:56.296705+0000","last_became_peered":"2024-03-11T18:35:56.296705+0000","last_unstale":"2024-03-11T20:51:08.599939+0000","last_undegraded":"2024-03-11T20:51:08.599939+0000","last_fullsized":"2024-03-11T20:51:08.599939+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:53:25.278916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194372,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":305,"num_write_kb":4243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":253,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:47:47.393591+0000","last_change":"2024-03-11T18:35:18.199528+0000","last_active":"2024-03-11T20:47:47.393591+0000","last_peered":"2024-03-11T20:47:47.393591+0000","last_clean":"2024-03-11T20:47:47.393591+0000","last_became_active":"2024-03-11T18:35:18.197797+0000","last_became_peered":"2024-03-11T18:35:18.197797+0000","last_unstale":"2024-03-11T20:47:47.393591+0000","last_undegraded":"2024-03-11T20:47:47.393591+0000","last_fullsized":"2024-03-11T20:47:47.393591+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:16.207500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:16.207500+0000","last_clean_scrub_stamp":"2024-03-11T18:35:16.207500+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:29:00.357496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"30'785","reported_seq":953,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:10.792501+0000","last_change":"2024-03-11T18:36:04.599815+0000","last_active":"2024-03-11T20:51:10.792501+0000","last_peered":"2024-03-11T20:51:10.792501+0000","last_clean":"2024-03-11T20:51:10.792501+0000","last_became_active":"2024-03-11T18:36:04.598777+0000","last_became_peered":"2024-03-11T18:36:04.598777+0000","last_unstale":"2024-03-11T20:51:10.792501+0000","last_undegraded":"2024-03-11T20:51:10.792501+0000","last_fullsized":"2024-03-11T20:51:10.792501+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":785,"log_dups_size":0,"ondisk_log_size":785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:39:32.875112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":86,"num_read_kb":36797,"num_write":762,"num_write_kb":114420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"30'207","reported_seq":276,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.794305+0000","last_change":"2024-03-11T18:37:04.141957+0000","last_active":"2024-03-11T20:51:29.794305+0000","last_peered":"2024-03-11T20:51:29.794305+0000","last_clean":"2024-03-11T20:51:29.794305+0000","last_became_active":"2024-03-11T18:37:04.141663+0000","last_became_peered":"2024-03-11T18:37:04.141663+0000","last_unstale":"2024-03-11T20:51:29.794305+0000","last_undegraded":"2024-03-11T20:51:29.794305+0000","last_fullsized":"2024-03-11T20:51:29.794305+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:25:52.716744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"30'286","reported_seq":377,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:03.112535+0000","last_change":"2024-03-11T18:35:56.294626+0000","last_active":"2024-03-11T20:51:03.112535+0000","last_peered":"2024-03-11T20:51:03.112535+0000","last_clean":"2024-03-11T20:51:03.112535+0000","last_became_active":"2024-03-11T18:35:56.293961+0000","last_became_peered":"2024-03-11T18:35:56.293961+0000","last_unstale":"2024-03-11T20:51:03.112535+0000","last_undegraded":"2024-03-11T20:51:03.112535+0000","last_fullsized":"2024-03-11T20:51:03.112535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:50:38.709532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4196056,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":223,"num_write_kb":4226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"30'822","reported_seq":987,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:14.297709+0000","last_change":"2024-03-11T18:36:04.619862+0000","last_active":"2024-03-11T20:51:14.297709+0000","last_peered":"2024-03-11T20:51:14.297709+0000","last_clean":"2024-03-11T20:51:14.297709+0000","last_became_active":"2024-03-11T18:36:04.618908+0000","last_became_peered":"2024-03-11T18:36:04.618908+0000","last_unstale":"2024-03-11T20:51:14.297709+0000","last_undegraded":"2024-03-11T20:51:14.297709+0000","last_fullsized":"2024-03-11T20:51:14.297709+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":822,"log_dups_size":0,"ondisk_log_size":822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:34:46.433500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":85,"num_read_kb":42087,"num_write":807,"num_write_kb":152305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"30'221","reported_seq":310,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.603226+0000","last_change":"2024-03-11T18:35:57.686416+0000","last_active":"2024-03-11T20:51:29.603226+0000","last_peered":"2024-03-11T20:51:29.603226+0000","last_clean":"2024-03-11T20:51:29.603226+0000","last_became_active":"2024-03-11T18:35:57.685828+0000","last_became_peered":"2024-03-11T18:35:57.685828+0000","last_unstale":"2024-03-11T20:51:29.603226+0000","last_undegraded":"2024-03-11T20:51:29.603226+0000","last_fullsized":"2024-03-11T20:51:29.603226+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:41:42.263652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"30'406","reported_seq":497,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:49:30.772686+0000","last_change":"2024-03-11T18:35:56.295539+0000","last_active":"2024-03-11T20:49:30.772686+0000","last_peered":"2024-03-11T20:49:30.772686+0000","last_clean":"2024-03-11T20:49:30.772686+0000","last_became_active":"2024-03-11T18:35:56.295123+0000","last_became_peered":"2024-03-11T18:35:56.295123+0000","last_unstale":"2024-03-11T20:49:30.772686+0000","last_undegraded":"2024-03-11T20:49:30.772686+0000","last_fullsized":"2024-03-11T20:49:30.772686+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:03:06.585728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":350,"num_write_kb":4273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"30'821","reported_seq":1021,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:04.302500+0000","last_change":"2024-03-11T18:36:04.616750+0000","last_active":"2024-03-11T20:51:04.302500+0000","last_peered":"2024-03-11T20:51:04.302500+0000","last_clean":"2024-03-11T20:51:04.302500+0000","last_became_active":"2024-03-11T18:36:04.616338+0000","last_became_peered":"2024-03-11T18:36:04.616338+0000","last_unstale":"2024-03-11T20:51:04.302500+0000","last_undegraded":"2024-03-11T20:51:04.302500+0000","last_fullsized":"2024-03-11T20:51:04.302500+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":821,"log_dups_size":0,"ondisk_log_size":821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:14:42.558491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":122,"num_read_kb":70346,"num_write":827,"num_write_kb":188969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"30'199","reported_seq":288,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.638812+0000","last_change":"2024-03-11T18:35:57.689888+0000","last_active":"2024-03-11T20:51:29.638812+0000","last_peered":"2024-03-11T20:51:29.638812+0000","last_clean":"2024-03-11T20:51:29.638812+0000","last_became_active":"2024-03-11T18:35:57.689424+0000","last_became_peered":"2024-03-11T18:35:57.689424+0000","last_unstale":"2024-03-11T20:51:29.638812+0000","last_undegraded":"2024-03-11T20:51:29.638812+0000","last_fullsized":"2024-03-11T20:51:29.638812+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:25:16.564261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"30'861","reported_seq":952,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:49:04.742266+0000","last_change":"2024-03-11T18:35:56.293862+0000","last_active":"2024-03-11T20:49:04.742266+0000","last_peered":"2024-03-11T20:49:04.742266+0000","last_clean":"2024-03-11T20:49:04.742266+0000","last_became_active":"2024-03-11T18:35:56.293585+0000","last_became_peered":"2024-03-11T18:35:56.293585+0000","last_unstale":"2024-03-11T20:49:04.742266+0000","last_undegraded":"2024-03-11T20:49:04.742266+0000","last_fullsized":"2024-03-11T20:49:04.742266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":861,"log_dups_size":0,"ondisk_log_size":861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:45:27.329274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":804,"num_write_kb":4496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"30'804","reported_seq":982,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:17.590386+0000","last_change":"2024-03-11T18:36:04.627729+0000","last_active":"2024-03-11T20:51:17.590386+0000","last_peered":"2024-03-11T20:51:17.590386+0000","last_clean":"2024-03-11T20:51:17.590386+0000","last_became_active":"2024-03-11T18:36:04.626596+0000","last_became_peered":"2024-03-11T18:36:04.626596+0000","last_unstale":"2024-03-11T20:51:17.590386+0000","last_undegraded":"2024-03-11T20:51:17.590386+0000","last_fullsized":"2024-03-11T20:51:17.590386+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:51:28.017349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":98,"num_read_kb":46229,"num_write":782,"num_write_kb":114631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"30'184","reported_seq":273,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.619621+0000","last_change":"2024-03-11T18:35:57.690506+0000","last_active":"2024-03-11T20:51:29.619621+0000","last_peered":"2024-03-11T20:51:29.619621+0000","last_clean":"2024-03-11T20:51:29.619621+0000","last_became_active":"2024-03-11T18:35:57.690223+0000","last_became_peered":"2024-03-11T18:35:57.690223+0000","last_unstale":"2024-03-11T20:51:29.619621+0000","last_undegraded":"2024-03-11T20:51:29.619621+0000","last_fullsized":"2024-03-11T20:51:29.619621+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:58:13.389498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"30'815","reported_seq":905,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:50:41.080749+0000","last_change":"2024-03-11T18:35:56.257209+0000","last_active":"2024-03-11T20:50:41.080749+0000","last_peered":"2024-03-11T20:50:41.080749+0000","last_clean":"2024-03-11T20:50:41.080749+0000","last_became_active":"2024-03-11T18:35:56.256454+0000","last_became_peered":"2024-03-11T18:35:56.256454+0000","last_unstale":"2024-03-11T20:50:41.080749+0000","last_undegraded":"2024-03-11T20:50:41.080749+0000","last_fullsized":"2024-03-11T20:50:41.080749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":815,"log_dups_size":0,"ondisk_log_size":815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:32:06.369248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":757,"num_write_kb":4472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"30'833","reported_seq":1020,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:08.316459+0000","last_change":"2024-03-11T18:36:04.644184+0000","last_active":"2024-03-11T20:51:08.316459+0000","last_peered":"2024-03-11T20:51:08.316459+0000","last_clean":"2024-03-11T20:51:08.316459+0000","last_became_active":"2024-03-11T18:36:04.642079+0000","last_became_peered":"2024-03-11T18:36:04.642079+0000","last_unstale":"2024-03-11T20:51:08.316459+0000","last_undegraded":"2024-03-11T20:51:08.316459+0000","last_fullsized":"2024-03-11T20:51:08.316459+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":833,"log_dups_size":0,"ondisk_log_size":833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:36:26.132020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":107,"num_read_kb":63565,"num_write":823,"num_write_kb":172061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"30'239","reported_seq":326,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.606890+0000","last_change":"2024-03-11T18:35:57.687311+0000","last_active":"2024-03-11T20:51:29.606890+0000","last_peered":"2024-03-11T20:51:29.606890+0000","last_clean":"2024-03-11T20:51:29.606890+0000","last_became_active":"2024-03-11T18:35:57.686850+0000","last_became_peered":"2024-03-11T18:35:57.686850+0000","last_unstale":"2024-03-11T20:51:29.606890+0000","last_undegraded":"2024-03-11T20:51:29.606890+0000","last_fullsized":"2024-03-11T20:51:29.606890+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:29:45.662029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"30'899","reported_seq":988,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:16.314784+0000","last_change":"2024-03-11T18:35:56.291232+0000","last_active":"2024-03-11T20:51:16.314784+0000","last_peered":"2024-03-11T20:51:16.314784+0000","last_clean":"2024-03-11T20:51:16.314784+0000","last_became_active":"2024-03-11T18:35:56.290944+0000","last_became_peered":"2024-03-11T18:35:56.290944+0000","last_unstale":"2024-03-11T20:51:16.314784+0000","last_undegraded":"2024-03-11T20:51:16.314784+0000","last_fullsized":"2024-03-11T20:51:16.314784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":899,"log_dups_size":0,"ondisk_log_size":899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:49:27.827765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":846,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"30'791","reported_seq":968,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:18.444965+0000","last_change":"2024-03-11T18:36:04.602142+0000","last_active":"2024-03-11T20:51:18.444965+0000","last_peered":"2024-03-11T20:51:18.444965+0000","last_clean":"2024-03-11T20:51:18.444965+0000","last_became_active":"2024-03-11T18:36:04.601224+0000","last_became_peered":"2024-03-11T18:36:04.601224+0000","last_unstale":"2024-03-11T20:51:18.444965+0000","last_undegraded":"2024-03-11T20:51:18.444965+0000","last_fullsized":"2024-03-11T20:51:18.444965+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:17:42.075355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":59304,"num_write":781,"num_write_kb":152027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"30'219","reported_seq":306,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.941879+0000","last_change":"2024-03-11T18:35:57.611305+0000","last_active":"2024-03-11T20:51:29.941879+0000","last_peered":"2024-03-11T20:51:29.941879+0000","last_clean":"2024-03-11T20:51:29.941879+0000","last_became_active":"2024-03-11T18:35:57.610955+0000","last_became_peered":"2024-03-11T18:35:57.610955+0000","last_unstale":"2024-03-11T20:51:29.941879+0000","last_undegraded":"2024-03-11T20:51:29.941879+0000","last_fullsized":"2024-03-11T20:51:29.941879+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:31:19.941345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"30'60","reported_seq":149,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:17.314474+0000","last_change":"2024-03-11T18:35:56.361952+0000","last_active":"2024-03-11T20:51:17.314474+0000","last_peered":"2024-03-11T20:51:17.314474+0000","last_clean":"2024-03-11T20:51:17.314474+0000","last_became_active":"2024-03-11T18:35:56.361141+0000","last_became_peered":"2024-03-11T18:35:56.361141+0000","last_unstale":"2024-03-11T20:51:17.314474+0000","last_undegraded":"2024-03-11T20:51:17.314474+0000","last_fullsized":"2024-03-11T20:51:17.314474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:04:13.101160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"30'743","reported_seq":894,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:15.483428+0000","last_change":"2024-03-11T18:36:04.593604+0000","last_active":"2024-03-11T20:51:15.483428+0000","last_peered":"2024-03-11T20:51:15.483428+0000","last_clean":"2024-03-11T20:51:15.483428+0000","last_became_active":"2024-03-11T18:36:04.592304+0000","last_became_peered":"2024-03-11T18:36:04.592304+0000","last_unstale":"2024-03-11T20:51:15.483428+0000","last_undegraded":"2024-03-11T20:51:15.483428+0000","last_fullsized":"2024-03-11T20:51:15.483428+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":743,"log_dups_size":0,"ondisk_log_size":743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:31:04.738940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":77,"num_read_kb":52017,"num_write":733,"num_write_kb":82372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"30'206","reported_seq":293,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.637397+0000","last_change":"2024-03-11T18:35:57.689404+0000","last_active":"2024-03-11T20:51:29.637397+0000","last_peered":"2024-03-11T20:51:29.637397+0000","last_clean":"2024-03-11T20:51:29.637397+0000","last_became_active":"2024-03-11T18:35:57.689139+0000","last_became_peered":"2024-03-11T18:35:57.689139+0000","last_unstale":"2024-03-11T20:51:29.637397+0000","last_undegraded":"2024-03-11T20:51:29.637397+0000","last_fullsized":"2024-03-11T20:51:29.637397+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:54:09.699371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"30'60","reported_seq":149,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:19.415920+0000","last_change":"2024-03-11T18:35:56.297682+0000","last_active":"2024-03-11T20:51:19.415920+0000","last_peered":"2024-03-11T20:51:19.415920+0000","last_clean":"2024-03-11T20:51:19.415920+0000","last_became_active":"2024-03-11T18:35:56.296426+0000","last_became_peered":"2024-03-11T18:35:56.296426+0000","last_unstale":"2024-03-11T20:51:19.415920+0000","last_undegraded":"2024-03-11T20:51:19.415920+0000","last_fullsized":"2024-03-11T20:51:19.415920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:28:50.050942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"30'724","reported_seq":923,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:03.154615+0000","last_change":"2024-03-11T18:36:04.590780+0000","last_active":"2024-03-11T20:51:03.154615+0000","last_peered":"2024-03-11T20:51:03.154615+0000","last_clean":"2024-03-11T20:51:03.154615+0000","last_became_active":"2024-03-11T18:36:04.590014+0000","last_became_peered":"2024-03-11T18:36:04.590014+0000","last_unstale":"2024-03-11T20:51:03.154615+0000","last_undegraded":"2024-03-11T20:51:03.154615+0000","last_fullsized":"2024-03-11T20:51:03.154615+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":724,"log_dups_size":0,"ondisk_log_size":724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:51:13.205384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":124,"num_read_kb":69166,"num_write":714,"num_write_kb":127024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"30'208","reported_seq":295,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.875165+0000","last_change":"2024-03-11T18:35:57.671638+0000","last_active":"2024-03-11T20:51:29.875165+0000","last_peered":"2024-03-11T20:51:29.875165+0000","last_clean":"2024-03-11T20:51:29.875165+0000","last_became_active":"2024-03-11T18:35:57.670893+0000","last_became_peered":"2024-03-11T18:35:57.670893+0000","last_unstale":"2024-03-11T20:51:29.875165+0000","last_undegraded":"2024-03-11T20:51:29.875165+0000","last_fullsized":"2024-03-11T20:51:29.875165+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:31:21.395133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"30'592","reported_seq":681,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:49:43.697234+0000","last_change":"2024-03-11T18:35:56.251580+0000","last_active":"2024-03-11T20:49:43.697234+0000","last_peered":"2024-03-11T20:49:43.697234+0000","last_clean":"2024-03-11T20:49:43.697234+0000","last_became_active":"2024-03-11T18:35:56.251287+0000","last_became_peered":"2024-03-11T18:35:56.251287+0000","last_unstale":"2024-03-11T20:49:43.697234+0000","last_undegraded":"2024-03-11T20:49:43.697234+0000","last_fullsized":"2024-03-11T20:49:43.697234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":592,"log_dups_size":0,"ondisk_log_size":592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:30:39.907360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388676,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":533,"num_write_kb":8445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"30'738","reported_seq":846,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:12.396881+0000","last_change":"2024-03-11T18:36:04.625563+0000","last_active":"2024-03-11T20:51:12.396881+0000","last_peered":"2024-03-11T20:51:12.396881+0000","last_clean":"2024-03-11T20:51:12.396881+0000","last_became_active":"2024-03-11T18:36:04.618911+0000","last_became_peered":"2024-03-11T18:36:04.618911+0000","last_unstale":"2024-03-11T20:51:12.396881+0000","last_undegraded":"2024-03-11T20:51:12.396881+0000","last_fullsized":"2024-03-11T20:51:12.396881+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":738,"log_dups_size":0,"ondisk_log_size":738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:31:04.738940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":10131,"num_write":722,"num_write_kb":109498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"30'204","reported_seq":289,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.606383+0000","last_change":"2024-03-11T18:35:57.697641+0000","last_active":"2024-03-11T20:51:29.606383+0000","last_peered":"2024-03-11T20:51:29.606383+0000","last_clean":"2024-03-11T20:51:29.606383+0000","last_became_active":"2024-03-11T18:35:57.697263+0000","last_became_peered":"2024-03-11T18:35:57.697263+0000","last_unstale":"2024-03-11T20:51:29.606383+0000","last_undegraded":"2024-03-11T20:51:29.606383+0000","last_fullsized":"2024-03-11T20:51:29.606383+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:34:39.325572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"30'2202","reported_seq":2290,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:01.825523+0000","last_change":"2024-03-11T18:35:56.355612+0000","last_active":"2024-03-11T20:51:01.825523+0000","last_peered":"2024-03-11T20:51:01.825523+0000","last_clean":"2024-03-11T20:51:01.825523+0000","last_became_active":"2024-03-11T18:35:56.354827+0000","last_became_peered":"2024-03-11T18:35:56.354827+0000","last_unstale":"2024-03-11T20:51:01.825523+0000","last_undegraded":"2024-03-11T20:51:01.825523+0000","last_fullsized":"2024-03-11T20:51:01.825523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":2202,"log_dups_size":0,"ondisk_log_size":2202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:06:02.844765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583584,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2157,"num_write_kb":13630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"30'771","reported_seq":924,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:14.409334+0000","last_change":"2024-03-11T18:37:04.259892+0000","last_active":"2024-03-11T20:51:14.409334+0000","last_peered":"2024-03-11T20:51:14.409334+0000","last_clean":"2024-03-11T20:51:14.409334+0000","last_became_active":"2024-03-11T18:37:04.259051+0000","last_became_peered":"2024-03-11T18:37:04.259051+0000","last_unstale":"2024-03-11T20:51:14.409334+0000","last_undegraded":"2024-03-11T20:51:14.409334+0000","last_fullsized":"2024-03-11T20:51:14.409334+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":771,"log_dups_size":0,"ondisk_log_size":771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:36:51.665733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":84,"num_read_kb":37526,"num_write":755,"num_write_kb":85949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2,0],"acting":[5,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"30'201","reported_seq":287,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.590814+0000","last_change":"2024-03-11T18:35:57.679035+0000","last_active":"2024-03-11T20:51:29.590814+0000","last_peered":"2024-03-11T20:51:29.590814+0000","last_clean":"2024-03-11T20:51:29.590814+0000","last_became_active":"2024-03-11T18:35:57.678565+0000","last_became_peered":"2024-03-11T18:35:57.678565+0000","last_unstale":"2024-03-11T20:51:29.590814+0000","last_undegraded":"2024-03-11T20:51:29.590814+0000","last_fullsized":"2024-03-11T20:51:29.590814+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:18:05.980207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"30'3437","reported_seq":3524,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.294958+0000","last_change":"2024-03-11T18:35:56.288132+0000","last_active":"2024-03-11T20:51:29.294958+0000","last_peered":"2024-03-11T20:51:29.294958+0000","last_clean":"2024-03-11T20:51:29.294958+0000","last_became_active":"2024-03-11T18:35:56.287846+0000","last_became_peered":"2024-03-11T18:35:56.287846+0000","last_unstale":"2024-03-11T20:51:29.294958+0000","last_undegraded":"2024-03-11T20:51:29.294958+0000","last_fullsized":"2024-03-11T20:51:29.294958+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":3437,"log_dups_size":0,"ondisk_log_size":3437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T18:51:00.795706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7567240,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3391,"num_write_kb":10463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"30'736","reported_seq":902,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:10.794831+0000","last_change":"2024-03-11T18:36:04.643164+0000","last_active":"2024-03-11T20:51:10.794831+0000","last_peered":"2024-03-11T20:51:10.794831+0000","last_clean":"2024-03-11T20:51:10.794831+0000","last_became_active":"2024-03-11T18:36:04.642374+0000","last_became_peered":"2024-03-11T18:36:04.642374+0000","last_unstale":"2024-03-11T20:51:10.794831+0000","last_undegraded":"2024-03-11T20:51:10.794831+0000","last_fullsized":"2024-03-11T20:51:10.794831+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":736,"log_dups_size":0,"ondisk_log_size":736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:25:56.815496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":40504,"num_write":721,"num_write_kb":107102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"30'197","reported_seq":282,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.866466+0000","last_change":"2024-03-11T18:35:57.636550+0000","last_active":"2024-03-11T20:51:29.866466+0000","last_peered":"2024-03-11T20:51:29.866466+0000","last_clean":"2024-03-11T20:51:29.866466+0000","last_became_active":"2024-03-11T18:35:57.636093+0000","last_became_peered":"2024-03-11T18:35:57.636093+0000","last_unstale":"2024-03-11T20:51:29.866466+0000","last_undegraded":"2024-03-11T20:51:29.866466+0000","last_fullsized":"2024-03-11T20:51:29.866466+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:33:31.344465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"30'1238","reported_seq":1325,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:50:53.074114+0000","last_change":"2024-03-11T18:35:56.355480+0000","last_active":"2024-03-11T20:50:53.074114+0000","last_peered":"2024-03-11T20:50:53.074114+0000","last_clean":"2024-03-11T20:50:53.074114+0000","last_became_active":"2024-03-11T18:35:56.354759+0000","last_became_peered":"2024-03-11T18:35:56.354759+0000","last_unstale":"2024-03-11T20:50:53.074114+0000","last_undegraded":"2024-03-11T20:50:53.074114+0000","last_fullsized":"2024-03-11T20:50:53.074114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1238,"log_dups_size":0,"ondisk_log_size":1238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:45:37.842725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1180,"num_write_kb":8784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"30'803","reported_seq":939,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:12.607404+0000","last_change":"2024-03-11T18:36:04.617364+0000","last_active":"2024-03-11T20:51:12.607404+0000","last_peered":"2024-03-11T20:51:12.607404+0000","last_clean":"2024-03-11T20:51:12.607404+0000","last_became_active":"2024-03-11T18:36:04.613159+0000","last_became_peered":"2024-03-11T18:36:04.613159+0000","last_unstale":"2024-03-11T20:51:12.607404+0000","last_undegraded":"2024-03-11T20:51:12.607404+0000","last_fullsized":"2024-03-11T20:51:12.607404+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":803,"log_dups_size":0,"ondisk_log_size":803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:25:23.672189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":38200,"num_write":794,"num_write_kb":117163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"30'218","reported_seq":303,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.626148+0000","last_change":"2024-03-11T18:35:57.683426+0000","last_active":"2024-03-11T20:51:29.626148+0000","last_peered":"2024-03-11T20:51:29.626148+0000","last_clean":"2024-03-11T20:51:29.626148+0000","last_became_active":"2024-03-11T18:35:57.683150+0000","last_became_peered":"2024-03-11T18:35:57.683150+0000","last_unstale":"2024-03-11T20:51:29.626148+0000","last_undegraded":"2024-03-11T20:51:29.626148+0000","last_fullsized":"2024-03-11T20:51:29.626148+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:36:00.079075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"30'63","reported_seq":150,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.294390+0000","last_change":"2024-03-11T18:35:56.361613+0000","last_active":"2024-03-11T20:51:29.294390+0000","last_peered":"2024-03-11T20:51:29.294390+0000","last_clean":"2024-03-11T20:51:29.294390+0000","last_became_active":"2024-03-11T18:35:56.359629+0000","last_became_peered":"2024-03-11T18:35:56.359629+0000","last_unstale":"2024-03-11T20:51:29.294390+0000","last_undegraded":"2024-03-11T20:51:29.294390+0000","last_fullsized":"2024-03-11T20:51:29.294390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:03:06.585728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"30'836","reported_seq":1036,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:12.888024+0000","last_change":"2024-03-11T18:36:04.608168+0000","last_active":"2024-03-11T20:51:12.888024+0000","last_peered":"2024-03-11T20:51:12.888024+0000","last_clean":"2024-03-11T20:51:12.888024+0000","last_became_active":"2024-03-11T18:36:04.607063+0000","last_became_peered":"2024-03-11T18:36:04.607063+0000","last_unstale":"2024-03-11T20:51:12.888024+0000","last_undegraded":"2024-03-11T20:51:12.888024+0000","last_fullsized":"2024-03-11T20:51:12.888024+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":836,"log_dups_size":0,"ondisk_log_size":836,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:53:41.996024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":124,"num_read_kb":69905,"num_write":818,"num_write_kb":155514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"30'815","reported_seq":1016,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:08.312913+0000","last_change":"2024-03-11T18:36:04.607992+0000","last_active":"2024-03-11T20:51:08.312913+0000","last_peered":"2024-03-11T20:51:08.312913+0000","last_clean":"2024-03-11T20:51:08.312913+0000","last_became_active":"2024-03-11T18:36:04.607109+0000","last_became_peered":"2024-03-11T18:36:04.607109+0000","last_unstale":"2024-03-11T20:51:08.312913+0000","last_undegraded":"2024-03-11T20:51:08.312913+0000","last_fullsized":"2024-03-11T20:51:08.312913+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":815,"log_dups_size":0,"ondisk_log_size":815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:14:37.678854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":116,"num_read_kb":66679,"num_write":801,"num_write_kb":181203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"30'218","reported_seq":302,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.672110+0000","last_change":"2024-03-11T18:35:57.680534+0000","last_active":"2024-03-11T20:51:29.672110+0000","last_peered":"2024-03-11T20:51:29.672110+0000","last_clean":"2024-03-11T20:51:29.672110+0000","last_became_active":"2024-03-11T18:35:57.679642+0000","last_became_peered":"2024-03-11T18:35:57.679642+0000","last_unstale":"2024-03-11T20:51:29.672110+0000","last_undegraded":"2024-03-11T20:51:29.672110+0000","last_fullsized":"2024-03-11T20:51:29.672110+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:25:49.707568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"30'1317","reported_seq":1404,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:17.349647+0000","last_change":"2024-03-11T18:35:56.303510+0000","last_active":"2024-03-11T20:51:17.349647+0000","last_peered":"2024-03-11T20:51:17.349647+0000","last_clean":"2024-03-11T20:51:17.349647+0000","last_became_active":"2024-03-11T18:35:56.302758+0000","last_became_peered":"2024-03-11T18:35:56.302758+0000","last_unstale":"2024-03-11T20:51:17.349647+0000","last_undegraded":"2024-03-11T20:51:17.349647+0000","last_fullsized":"2024-03-11T20:51:17.349647+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1317,"log_dups_size":0,"ondisk_log_size":1317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:03:32.297342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583494,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1254,"num_write_kb":12926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"30'766","reported_seq":981,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:50:55.156176+0000","last_change":"2024-03-11T18:36:04.618818+0000","last_active":"2024-03-11T20:50:55.156176+0000","last_peered":"2024-03-11T20:50:55.156176+0000","last_clean":"2024-03-11T20:50:55.156176+0000","last_became_active":"2024-03-11T18:36:04.617868+0000","last_became_peered":"2024-03-11T18:36:04.617868+0000","last_unstale":"2024-03-11T20:50:55.156176+0000","last_undegraded":"2024-03-11T20:50:55.156176+0000","last_fullsized":"2024-03-11T20:50:55.156176+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":766,"log_dups_size":0,"ondisk_log_size":766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:16:10.508913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":132,"num_read_kb":68723,"num_write":745,"num_write_kb":131559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"30'225","reported_seq":310,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.771467+0000","last_change":"2024-03-11T18:35:57.690347+0000","last_active":"2024-03-11T20:51:29.771467+0000","last_peered":"2024-03-11T20:51:29.771467+0000","last_clean":"2024-03-11T20:51:29.771467+0000","last_became_active":"2024-03-11T18:35:57.689146+0000","last_became_peered":"2024-03-11T18:35:57.689146+0000","last_unstale":"2024-03-11T20:51:29.771467+0000","last_undegraded":"2024-03-11T20:51:29.771467+0000","last_fullsized":"2024-03-11T20:51:29.771467+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:03:56.260394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"30'834","reported_seq":920,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:49:31.476898+0000","last_change":"2024-03-11T18:35:56.348355+0000","last_active":"2024-03-11T20:49:31.476898+0000","last_peered":"2024-03-11T20:49:31.476898+0000","last_clean":"2024-03-11T20:49:31.476898+0000","last_became_active":"2024-03-11T18:35:56.347631+0000","last_became_peered":"2024-03-11T18:35:56.347631+0000","last_unstale":"2024-03-11T20:49:31.476898+0000","last_undegraded":"2024-03-11T20:49:31.476898+0000","last_fullsized":"2024-03-11T20:49:31.476898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":834,"log_dups_size":0,"ondisk_log_size":834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:06:02.844765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":793,"num_write_kb":4496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"30'1909","reported_seq":1994,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:49:42.823141+0000","last_change":"2024-03-11T18:35:56.361254+0000","last_active":"2024-03-11T20:49:42.823141+0000","last_peered":"2024-03-11T20:49:42.823141+0000","last_clean":"2024-03-11T20:49:42.823141+0000","last_became_active":"2024-03-11T18:35:56.360580+0000","last_became_peered":"2024-03-11T18:35:56.360580+0000","last_unstale":"2024-03-11T20:49:42.823141+0000","last_undegraded":"2024-03-11T20:49:42.823141+0000","last_fullsized":"2024-03-11T20:49:42.823141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1909,"log_dups_size":0,"ondisk_log_size":1909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:27:26.471550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1850,"num_write_kb":17317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"30'183","reported_seq":266,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.874784+0000","last_change":"2024-03-11T18:35:57.602778+0000","last_active":"2024-03-11T20:51:29.874784+0000","last_peered":"2024-03-11T20:51:29.874784+0000","last_clean":"2024-03-11T20:51:29.874784+0000","last_became_active":"2024-03-11T18:35:57.601990+0000","last_became_peered":"2024-03-11T18:35:57.601990+0000","last_unstale":"2024-03-11T20:51:29.874784+0000","last_undegraded":"2024-03-11T20:51:29.874784+0000","last_fullsized":"2024-03-11T20:51:29.874784+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:56:59.014871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"30'692","reported_seq":830,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:08.239134+0000","last_change":"2024-03-11T18:36:04.606098+0000","last_active":"2024-03-11T20:51:08.239134+0000","last_peered":"2024-03-11T20:51:08.239134+0000","last_clean":"2024-03-11T20:51:08.239134+0000","last_became_active":"2024-03-11T18:36:04.604823+0000","last_became_peered":"2024-03-11T18:36:04.604823+0000","last_unstale":"2024-03-11T20:51:08.239134+0000","last_undegraded":"2024-03-11T20:51:08.239134+0000","last_fullsized":"2024-03-11T20:51:08.239134+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:52:04.862452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":66,"num_read_kb":37267,"num_write":684,"num_write_kb":109032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"30'332","reported_seq":417,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:50:56.328124+0000","last_change":"2024-03-11T18:35:56.296312+0000","last_active":"2024-03-11T20:50:56.328124+0000","last_peered":"2024-03-11T20:50:56.328124+0000","last_clean":"2024-03-11T20:50:56.328124+0000","last_became_active":"2024-03-11T18:35:56.296090+0000","last_became_peered":"2024-03-11T18:35:56.296090+0000","last_unstale":"2024-03-11T20:50:56.328124+0000","last_undegraded":"2024-03-11T20:50:56.328124+0000","last_fullsized":"2024-03-11T20:50:56.328124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:03:06.585728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4195508,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":278,"num_write_kb":4222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"30'199","reported_seq":282,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.579273+0000","last_change":"2024-03-11T18:35:57.611809+0000","last_active":"2024-03-11T20:51:29.579273+0000","last_peered":"2024-03-11T20:51:29.579273+0000","last_clean":"2024-03-11T20:51:29.579273+0000","last_became_active":"2024-03-11T18:35:57.611097+0000","last_became_peered":"2024-03-11T18:35:57.611097+0000","last_unstale":"2024-03-11T20:51:29.579273+0000","last_undegraded":"2024-03-11T20:51:29.579273+0000","last_fullsized":"2024-03-11T20:51:29.579273+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:49:29.042114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"30'703","reported_seq":834,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:06.329767+0000","last_change":"2024-03-11T18:36:04.590064+0000","last_active":"2024-03-11T20:51:06.329767+0000","last_peered":"2024-03-11T20:51:06.329767+0000","last_clean":"2024-03-11T20:51:06.329767+0000","last_became_active":"2024-03-11T18:36:04.589037+0000","last_became_peered":"2024-03-11T18:36:04.589037+0000","last_unstale":"2024-03-11T20:51:06.329767+0000","last_undegraded":"2024-03-11T20:51:06.329767+0000","last_fullsized":"2024-03-11T20:51:06.329767+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:51:28.017349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":59,"num_read_kb":23261,"num_write":682,"num_write_kb":69804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"30'217","reported_seq":311,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.960628+0000","last_change":"2024-03-11T18:35:57.614944+0000","last_active":"2024-03-11T20:51:29.960628+0000","last_peered":"2024-03-11T20:51:29.960628+0000","last_clean":"2024-03-11T20:51:29.960628+0000","last_became_active":"2024-03-11T18:35:57.614144+0000","last_became_peered":"2024-03-11T18:35:57.614144+0000","last_unstale":"2024-03-11T20:51:29.960628+0000","last_undegraded":"2024-03-11T20:51:29.960628+0000","last_fullsized":"2024-03-11T20:51:29.960628+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:53:26.493265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"30'222","reported_seq":316,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.797673+0000","last_change":"2024-03-11T18:35:57.630211+0000","last_active":"2024-03-11T20:51:29.797673+0000","last_peered":"2024-03-11T20:51:29.797673+0000","last_clean":"2024-03-11T20:51:29.797673+0000","last_became_active":"2024-03-11T18:35:57.629599+0000","last_became_peered":"2024-03-11T18:35:57.629599+0000","last_unstale":"2024-03-11T20:51:29.797673+0000","last_undegraded":"2024-03-11T20:51:29.797673+0000","last_fullsized":"2024-03-11T20:51:29.797673+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":222,"log_dups_size":0,"ondisk_log_size":222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:18:05.980207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"30'236","reported_seq":330,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.602616+0000","last_change":"2024-03-11T18:35:57.688700+0000","last_active":"2024-03-11T20:51:29.602616+0000","last_peered":"2024-03-11T20:51:29.602616+0000","last_clean":"2024-03-11T20:51:29.602616+0000","last_became_active":"2024-03-11T18:35:57.687968+0000","last_became_peered":"2024-03-11T18:35:57.687968+0000","last_unstale":"2024-03-11T20:51:29.602616+0000","last_undegraded":"2024-03-11T20:51:29.602616+0000","last_fullsized":"2024-03-11T20:51:29.602616+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":236,"log_dups_size":0,"ondisk_log_size":236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:25:16.564261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"30'203","reported_seq":296,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.872107+0000","last_change":"2024-03-11T18:35:57.680730+0000","last_active":"2024-03-11T20:51:29.872107+0000","last_peered":"2024-03-11T20:51:29.872107+0000","last_clean":"2024-03-11T20:51:29.872107+0000","last_became_active":"2024-03-11T18:35:57.679935+0000","last_became_peered":"2024-03-11T18:35:57.679935+0000","last_unstale":"2024-03-11T20:51:29.872107+0000","last_undegraded":"2024-03-11T20:51:29.872107+0000","last_fullsized":"2024-03-11T20:51:29.872107+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:21:21.894805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"30'184","reported_seq":275,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.764136+0000","last_change":"2024-03-11T18:35:57.690799+0000","last_active":"2024-03-11T20:51:29.764136+0000","last_peered":"2024-03-11T20:51:29.764136+0000","last_clean":"2024-03-11T20:51:29.764136+0000","last_became_active":"2024-03-11T18:35:57.686623+0000","last_became_peered":"2024-03-11T18:35:57.686623+0000","last_unstale":"2024-03-11T20:51:29.764136+0000","last_undegraded":"2024-03-11T20:51:29.764136+0000","last_fullsized":"2024-03-11T20:51:29.764136+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:53:30.324898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"30'208","reported_seq":299,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.659529+0000","last_change":"2024-03-11T18:35:57.691955+0000","last_active":"2024-03-11T20:51:29.659529+0000","last_peered":"2024-03-11T20:51:29.659529+0000","last_clean":"2024-03-11T20:51:29.659529+0000","last_became_active":"2024-03-11T18:35:57.691413+0000","last_became_peered":"2024-03-11T18:35:57.691413+0000","last_unstale":"2024-03-11T20:51:29.659529+0000","last_undegraded":"2024-03-11T20:51:29.659529+0000","last_fullsized":"2024-03-11T20:51:29.659529+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:31:19.941345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"30'244","reported_seq":335,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.963800+0000","last_change":"2024-03-11T18:35:57.684695+0000","last_active":"2024-03-11T20:51:29.963800+0000","last_peered":"2024-03-11T20:51:29.963800+0000","last_clean":"2024-03-11T20:51:29.963800+0000","last_became_active":"2024-03-11T18:35:57.684227+0000","last_became_peered":"2024-03-11T18:35:57.684227+0000","last_unstale":"2024-03-11T20:51:29.963800+0000","last_undegraded":"2024-03-11T20:51:29.963800+0000","last_fullsized":"2024-03-11T20:51:29.963800+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":244,"log_dups_size":0,"ondisk_log_size":244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:28:51.265291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"30'193","reported_seq":284,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.760629+0000","last_change":"2024-03-11T18:35:57.683549+0000","last_active":"2024-03-11T20:51:29.760629+0000","last_peered":"2024-03-11T20:51:29.760629+0000","last_clean":"2024-03-11T20:51:29.760629+0000","last_became_active":"2024-03-11T18:35:57.680610+0000","last_became_peered":"2024-03-11T18:35:57.680610+0000","last_unstale":"2024-03-11T20:51:29.760629+0000","last_undegraded":"2024-03-11T20:51:29.760629+0000","last_fullsized":"2024-03-11T20:51:29.760629+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:25:52.716744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"30'218","reported_seq":289,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.791145+0000","last_change":"2024-03-11T18:37:04.264086+0000","last_active":"2024-03-11T20:51:29.791145+0000","last_peered":"2024-03-11T20:51:29.791145+0000","last_clean":"2024-03-11T20:51:29.791145+0000","last_became_active":"2024-03-11T18:37:04.263764+0000","last_became_peered":"2024-03-11T18:37:04.263764+0000","last_unstale":"2024-03-11T20:51:29.791145+0000","last_undegraded":"2024-03-11T20:51:29.791145+0000","last_fullsized":"2024-03-11T20:51:29.791145+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:34:39.325572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"30'205","reported_seq":294,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.764435+0000","last_change":"2024-03-11T18:35:57.681630+0000","last_active":"2024-03-11T20:51:29.764435+0000","last_peered":"2024-03-11T20:51:29.764435+0000","last_clean":"2024-03-11T20:51:29.764435+0000","last_became_active":"2024-03-11T18:35:57.678876+0000","last_became_peered":"2024-03-11T18:35:57.678876+0000","last_unstale":"2024-03-11T20:51:29.764435+0000","last_undegraded":"2024-03-11T20:51:29.764435+0000","last_fullsized":"2024-03-11T20:51:29.764435+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:03:33.511691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"30'208","reported_seq":297,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.623113+0000","last_change":"2024-03-11T18:35:57.679589+0000","last_active":"2024-03-11T20:51:29.623113+0000","last_peered":"2024-03-11T20:51:29.623113+0000","last_clean":"2024-03-11T20:51:29.623113+0000","last_became_active":"2024-03-11T18:35:57.678838+0000","last_became_peered":"2024-03-11T18:35:57.678838+0000","last_unstale":"2024-03-11T20:51:29.623113+0000","last_undegraded":"2024-03-11T20:51:29.623113+0000","last_fullsized":"2024-03-11T20:51:29.623113+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:54:09.699371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"30'213","reported_seq":302,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.872622+0000","last_change":"2024-03-11T18:35:57.674571+0000","last_active":"2024-03-11T20:51:29.872622+0000","last_peered":"2024-03-11T20:51:29.872622+0000","last_clean":"2024-03-11T20:51:29.872622+0000","last_became_active":"2024-03-11T18:35:57.671573+0000","last_became_peered":"2024-03-11T18:35:57.671573+0000","last_unstale":"2024-03-11T20:51:29.872622+0000","last_undegraded":"2024-03-11T20:51:29.872622+0000","last_fullsized":"2024-03-11T20:51:29.872622+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:35:10.403666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"30'221","reported_seq":310,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.959781+0000","last_change":"2024-03-11T18:35:57.615939+0000","last_active":"2024-03-11T20:51:29.959781+0000","last_peered":"2024-03-11T20:51:29.959781+0000","last_clean":"2024-03-11T20:51:29.959781+0000","last_became_active":"2024-03-11T18:35:57.615579+0000","last_became_peered":"2024-03-11T18:35:57.615579+0000","last_unstale":"2024-03-11T20:51:29.959781+0000","last_undegraded":"2024-03-11T20:51:29.959781+0000","last_fullsized":"2024-03-11T20:51:29.959781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:03:33.511691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"30'206","reported_seq":295,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.632447+0000","last_change":"2024-03-11T18:35:57.688955+0000","last_active":"2024-03-11T20:51:29.632447+0000","last_peered":"2024-03-11T20:51:29.632447+0000","last_clean":"2024-03-11T20:51:29.632447+0000","last_became_active":"2024-03-11T18:35:57.688365+0000","last_became_peered":"2024-03-11T18:35:57.688365+0000","last_unstale":"2024-03-11T20:51:29.632447+0000","last_undegraded":"2024-03-11T20:51:29.632447+0000","last_fullsized":"2024-03-11T20:51:29.632447+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:44:11.877541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"30'205","reported_seq":292,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.795195+0000","last_change":"2024-03-11T18:35:57.602044+0000","last_active":"2024-03-11T20:51:29.795195+0000","last_peered":"2024-03-11T20:51:29.795195+0000","last_clean":"2024-03-11T20:51:29.795195+0000","last_became_active":"2024-03-11T18:35:57.601334+0000","last_became_peered":"2024-03-11T18:35:57.601334+0000","last_unstale":"2024-03-11T20:51:29.795195+0000","last_undegraded":"2024-03-11T20:51:29.795195+0000","last_fullsized":"2024-03-11T20:51:29.795195+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:39:12.693619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"30'206","reported_seq":293,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.612090+0000","last_change":"2024-03-11T18:35:57.677114+0000","last_active":"2024-03-11T20:51:29.612090+0000","last_peered":"2024-03-11T20:51:29.612090+0000","last_clean":"2024-03-11T20:51:29.612090+0000","last_became_active":"2024-03-11T18:35:57.676075+0000","last_became_peered":"2024-03-11T18:35:57.676075+0000","last_unstale":"2024-03-11T20:51:29.612090+0000","last_undegraded":"2024-03-11T20:51:29.612090+0000","last_fullsized":"2024-03-11T20:51:29.612090+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:44:11.877541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"30'219","reported_seq":306,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.614892+0000","last_change":"2024-03-11T18:35:57.641392+0000","last_active":"2024-03-11T20:51:29.614892+0000","last_peered":"2024-03-11T20:51:29.614892+0000","last_clean":"2024-03-11T20:51:29.614892+0000","last_became_active":"2024-03-11T18:35:57.640934+0000","last_became_peered":"2024-03-11T18:35:57.640934+0000","last_unstale":"2024-03-11T20:51:29.614892+0000","last_undegraded":"2024-03-11T20:51:29.614892+0000","last_fullsized":"2024-03-11T20:51:29.614892+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:25:52.716744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"30'213","reported_seq":300,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.664702+0000","last_change":"2024-03-11T18:35:57.605291+0000","last_active":"2024-03-11T20:51:29.664702+0000","last_peered":"2024-03-11T20:51:29.664702+0000","last_clean":"2024-03-11T20:51:29.664702+0000","last_became_active":"2024-03-11T18:35:57.604645+0000","last_became_peered":"2024-03-11T18:35:57.604645+0000","last_unstale":"2024-03-11T20:51:29.664702+0000","last_undegraded":"2024-03-11T20:51:29.664702+0000","last_fullsized":"2024-03-11T20:51:29.664702+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:03:33.511691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"30'210","reported_seq":296,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.888659+0000","last_change":"2024-03-11T18:35:57.503994+0000","last_active":"2024-03-11T20:51:29.888659+0000","last_peered":"2024-03-11T20:51:29.888659+0000","last_clean":"2024-03-11T20:51:29.888659+0000","last_became_active":"2024-03-11T18:35:57.503362+0000","last_became_peered":"2024-03-11T18:35:57.503362+0000","last_unstale":"2024-03-11T20:51:29.888659+0000","last_undegraded":"2024-03-11T20:51:29.888659+0000","last_fullsized":"2024-03-11T20:51:29.888659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:03:58.875164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"30'195","reported_seq":260,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.789716+0000","last_change":"2024-03-11T18:37:04.252064+0000","last_active":"2024-03-11T20:51:29.789716+0000","last_peered":"2024-03-11T20:51:29.789716+0000","last_clean":"2024-03-11T20:51:29.789716+0000","last_became_active":"2024-03-11T18:37:04.251092+0000","last_became_peered":"2024-03-11T18:37:04.251092+0000","last_unstale":"2024-03-11T20:51:29.789716+0000","last_undegraded":"2024-03-11T20:51:29.789716+0000","last_fullsized":"2024-03-11T20:51:29.789716+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:29:45.662029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"30'181","reported_seq":266,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.575251+0000","last_change":"2024-03-11T18:35:57.613374+0000","last_active":"2024-03-11T20:51:29.575251+0000","last_peered":"2024-03-11T20:51:29.575251+0000","last_clean":"2024-03-11T20:51:29.575251+0000","last_became_active":"2024-03-11T18:35:57.613050+0000","last_became_peered":"2024-03-11T18:35:57.613050+0000","last_unstale":"2024-03-11T20:51:29.575251+0000","last_undegraded":"2024-03-11T20:51:29.575251+0000","last_fullsized":"2024-03-11T20:51:29.575251+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:50:39.923881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"30'186","reported_seq":271,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.959316+0000","last_change":"2024-03-11T18:35:57.636565+0000","last_active":"2024-03-11T20:51:29.959316+0000","last_peered":"2024-03-11T20:51:29.959316+0000","last_clean":"2024-03-11T20:51:29.959316+0000","last_became_active":"2024-03-11T18:35:57.635978+0000","last_became_peered":"2024-03-11T18:35:57.635978+0000","last_unstale":"2024-03-11T20:51:29.959316+0000","last_undegraded":"2024-03-11T20:51:29.959316+0000","last_fullsized":"2024-03-11T20:51:29.959316+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:03:56.260394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"30'206","reported_seq":291,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.964837+0000","last_change":"2024-03-11T18:35:57.641714+0000","last_active":"2024-03-11T20:51:29.964837+0000","last_peered":"2024-03-11T20:51:29.964837+0000","last_clean":"2024-03-11T20:51:29.964837+0000","last_became_active":"2024-03-11T18:35:57.641417+0000","last_became_peered":"2024-03-11T18:35:57.641417+0000","last_unstale":"2024-03-11T20:51:29.964837+0000","last_undegraded":"2024-03-11T20:51:29.964837+0000","last_fullsized":"2024-03-11T20:51:29.964837+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:25:49.707568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"30'210","reported_seq":294,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.626373+0000","last_change":"2024-03-11T18:35:57.689703+0000","last_active":"2024-03-11T20:51:29.626373+0000","last_peered":"2024-03-11T20:51:29.626373+0000","last_clean":"2024-03-11T20:51:29.626373+0000","last_became_active":"2024-03-11T18:35:57.689056+0000","last_became_peered":"2024-03-11T18:35:57.689056+0000","last_unstale":"2024-03-11T20:51:29.626373+0000","last_undegraded":"2024-03-11T20:51:29.626373+0000","last_fullsized":"2024-03-11T20:51:29.626373+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:41:42.263652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"30'194","reported_seq":277,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.772230+0000","last_change":"2024-03-11T18:35:57.684305+0000","last_active":"2024-03-11T20:51:29.772230+0000","last_peered":"2024-03-11T20:51:29.772230+0000","last_clean":"2024-03-11T20:51:29.772230+0000","last_became_active":"2024-03-11T18:35:57.682947+0000","last_became_peered":"2024-03-11T18:35:57.682947+0000","last_unstale":"2024-03-11T20:51:29.772230+0000","last_undegraded":"2024-03-11T20:51:29.772230+0000","last_fullsized":"2024-03-11T20:51:29.772230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:53:26.493265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"30'210","reported_seq":293,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.585163+0000","last_change":"2024-03-11T18:35:57.692717+0000","last_active":"2024-03-11T20:51:29.585163+0000","last_peered":"2024-03-11T20:51:29.585163+0000","last_clean":"2024-03-11T20:51:29.585163+0000","last_became_active":"2024-03-11T18:35:57.692504+0000","last_became_peered":"2024-03-11T18:35:57.692504+0000","last_unstale":"2024-03-11T20:51:29.585163+0000","last_undegraded":"2024-03-11T20:51:29.585163+0000","last_fullsized":"2024-03-11T20:51:29.585163+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:25:52.716744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"30'194","reported_seq":277,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.612590+0000","last_change":"2024-03-11T18:35:57.640483+0000","last_active":"2024-03-11T20:51:29.612590+0000","last_peered":"2024-03-11T20:51:29.612590+0000","last_clean":"2024-03-11T20:51:29.612590+0000","last_became_active":"2024-03-11T18:35:57.639886+0000","last_became_peered":"2024-03-11T18:35:57.639886+0000","last_unstale":"2024-03-11T20:51:29.612590+0000","last_undegraded":"2024-03-11T20:51:29.612590+0000","last_fullsized":"2024-03-11T20:51:29.612590+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:34:39.325572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"30'212","reported_seq":295,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.603982+0000","last_change":"2024-03-11T18:35:57.687096+0000","last_active":"2024-03-11T20:51:29.603982+0000","last_peered":"2024-03-11T20:51:29.603982+0000","last_clean":"2024-03-11T20:51:29.603982+0000","last_became_active":"2024-03-11T18:35:57.686771+0000","last_became_peered":"2024-03-11T18:35:57.686771+0000","last_unstale":"2024-03-11T20:51:29.603982+0000","last_undegraded":"2024-03-11T20:51:29.603982+0000","last_fullsized":"2024-03-11T20:51:29.603982+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:16:03.400985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"30'196","reported_seq":292,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.628723+0000","last_change":"2024-03-11T18:35:57.694707+0000","last_active":"2024-03-11T20:51:29.628723+0000","last_peered":"2024-03-11T20:51:29.628723+0000","last_clean":"2024-03-11T20:51:29.628723+0000","last_became_active":"2024-03-11T18:35:57.694104+0000","last_became_peered":"2024-03-11T18:35:57.694104+0000","last_unstale":"2024-03-11T20:51:29.628723+0000","last_undegraded":"2024-03-11T20:51:29.628723+0000","last_fullsized":"2024-03-11T20:51:29.628723+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":196,"log_dups_size":0,"ondisk_log_size":196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:47:28.579467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"30'191","reported_seq":285,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.874739+0000","last_change":"2024-03-11T18:35:57.538548+0000","last_active":"2024-03-11T20:51:29.874739+0000","last_peered":"2024-03-11T20:51:29.874739+0000","last_clean":"2024-03-11T20:51:29.874739+0000","last_became_active":"2024-03-11T18:35:57.537704+0000","last_became_peered":"2024-03-11T18:35:57.537704+0000","last_unstale":"2024-03-11T20:51:29.874739+0000","last_undegraded":"2024-03-11T20:51:29.874739+0000","last_fullsized":"2024-03-11T20:51:29.874739+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:40:52.251981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"30'214","reported_seq":308,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.659184+0000","last_change":"2024-03-11T18:35:57.507126+0000","last_active":"2024-03-11T20:51:29.659184+0000","last_peered":"2024-03-11T20:51:29.659184+0000","last_clean":"2024-03-11T20:51:29.659184+0000","last_became_active":"2024-03-11T18:35:57.506429+0000","last_became_peered":"2024-03-11T18:35:57.506429+0000","last_unstale":"2024-03-11T20:51:29.659184+0000","last_undegraded":"2024-03-11T20:51:29.659184+0000","last_fullsized":"2024-03-11T20:51:29.659184+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:28:51.265291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"30'213","reported_seq":307,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.608987+0000","last_change":"2024-03-11T18:35:57.687817+0000","last_active":"2024-03-11T20:51:29.608987+0000","last_peered":"2024-03-11T20:51:29.608987+0000","last_clean":"2024-03-11T20:51:29.608987+0000","last_became_active":"2024-03-11T18:35:57.687532+0000","last_became_peered":"2024-03-11T18:35:57.687532+0000","last_unstale":"2024-03-11T20:51:29.608987+0000","last_undegraded":"2024-03-11T20:51:29.608987+0000","last_fullsized":"2024-03-11T20:51:29.608987+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:47:28.579467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2961,"num_read_kb":1579043,"num_write":24379,"num_write_kb":4188254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24824,"ondisk_log_size":24824,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13197,"ondisk_log_size":13197,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":210457699,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":4697,"num_read_kb":5145,"num_write":47510,"num_write_kb":229688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":421158912,"data_stored":420915398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38275,"ondisk_log_size":38275,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719478378,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":200100,"kb_used_data":78748,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94171740,"statfs":{"total":96636764160,"available":96431861760,"internally_reserved":0,"allocated":80637952,"data_stored":80415478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":124254666},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":16,"seq":68719478378,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":245524,"kb_used_data":61772,"kb_used_omap":1,"kb_used_meta":183742,"kb_avail":94126316,"statfs":{"total":96636764160,"available":96385347584,"internally_reserved":0,"allocated":63254528,"data_stored":63047256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":188152262},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":16,"seq":68719478376,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":200588,"kb_used_data":48708,"kb_used_omap":1,"kb_used_meta":151870,"kb_avail":94171252,"statfs":{"total":96636764160,"available":96431362048,"internally_reserved":0,"allocated":49876992,"data_stored":49644774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":155515334},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":16,"seq":68719478377,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":183188,"kb_used_data":57676,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94188652,"statfs":{"total":96636764160,"available":96449179648,"internally_reserved":0,"allocated":59060224,"data_stored":58852820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424511080,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":218248,"kb_used_data":42816,"kb_used_omap":1,"kb_used_meta":175422,"kb_avail":94153592,"statfs":{"total":96636764160,"available":96413278208,"internally_reserved":0,"allocated":43843584,"data_stored":43631469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":179632583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424511080,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":152012,"kb_used_data":22340,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94219828,"statfs":{"total":96636764160,"available":96481103872,"internally_reserved":0,"allocated":22876160,"data_stored":22659915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424511081,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":228216,"kb_used_data":56880,"kb_used_omap":1,"kb_used_meta":171326,"kb_avail":94143624,"statfs":{"total":96636764160,"available":96403070976,"internally_reserved":0,"allocated":58245120,"data_stored":58030400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":175438280},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129543783,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149276,"kb_used_data":45972,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94222564,"statfs":{"total":96636764160,"available":96483905536,"internally_reserved":0,"allocated":47075328,"data_stored":46859238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":46166016,"data_stored":46138218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57929728,"data_stored":57899748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":22560768,"data_stored":22529263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":43528192,"data_stored":43500817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":58744832,"data_stored":58722168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49561600,"data_stored":49514122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":62939136,"data_stored":62916604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":79728640,"data_stored":79694458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-11T20:51:37.002 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-11T20:51:37.002 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-11T20:51:37.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.160+0000 7f921ea41640 1 Processor -- start 2024-03-11T20:51:37.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.160+0000 7f921ea41640 1 -- start start 2024-03-11T20:51:37.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.160+0000 7f921ea41640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f921806e210 0x7f9218157c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:37.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.160+0000 7f921ea41640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f921806f430 con 0x7f9218151b30 2024-03-11T20:51:37.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.160+0000 7f921ea41640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f921806e820 con 0x7f9218070b00 2024-03-11T20:51:37.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.160+0000 7f921ea41640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92180c65b0 con 0x7f921806e210 2024-03-11T20:51:37.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.160+0000 7f9217fff640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f9218151b30 0x7f92180c5de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:57698/0 (socket says 172.21.15.90:57698) 2024-03-11T20:51:37.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.160+0000 7f9217fff640 1 -- 172.21.15.90:0/379424453 learned_addr learned my addr 172.21.15.90:0/379424453 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T20:51:37.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.160+0000 7f921cfde640 1 --2- 172.21.15.90:0/379424453 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f921806e210 0x7f9218157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:37.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.164+0000 7f9216ffd640 1 -- 172.21.15.90:0/379424453 <== mon.0 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 563078041 0 0) 0x7f921806f430 con 0x7f9218151b30 2024-03-11T20:51:37.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.164+0000 7f9216ffd640 1 -- 172.21.15.90:0/379424453 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9200003530 con 0x7f9218151b30 2024-03-11T20:51:37.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.168+0000 7f9216ffd640 1 -- 172.21.15.90:0/379424453 <== mon.1 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 519811298 0 0) 0x7f921806e820 con 0x7f9218070b00 2024-03-11T20:51:37.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.168+0000 7f9216ffd640 1 -- 172.21.15.90:0/379424453 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f921806f430 con 0x7f9218070b00 2024-03-11T20:51:37.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.168+0000 7f9216ffd640 1 -- 172.21.15.90:0/379424453 <== mon.0 v1:172.21.15.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1065175532 0 0) 0x7f9200003530 con 0x7f9218151b30 2024-03-11T20:51:37.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.168+0000 7f9216ffd640 1 -- 172.21.15.90:0/379424453 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f921806e210 msgr2=0x7f9218157c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:37.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.168+0000 7f9216ffd640 1 --2- 172.21.15.90:0/379424453 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f921806e210 0x7f9218157c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:37.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.168+0000 7f9216ffd640 1 -- 172.21.15.90:0/379424453 >> v1:172.21.15.92:6789/0 conn(0x7f9218070b00 legacy=0x7f92180c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:37.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.168+0000 7f9216ffd640 1 -- 172.21.15.90:0/379424453 --> v1:172.21.15.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f921806eb00 con 0x7f9218151b30 2024-03-11T20:51:37.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.168+0000 7f9216ffd640 1 -- 172.21.15.90:0/379424453 <== mon.0 v1:172.21.15.90:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 691735435 0 0) 0x7f920c001680 con 0x7f9218151b30 2024-03-11T20:51:37.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.168+0000 7f9216ffd640 1 -- 172.21.15.90:0/379424453 >> v1:172.21.15.90:6789/0 conn(0x7f9218151b30 legacy=0x7f92180c5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:37.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.168+0000 7f9216ffd640 1 --2- 172.21.15.90:0/379424453 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9200003b40 0x7f9200003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:37.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.168+0000 7f9216ffd640 1 --2- 172.21.15.90:0/379424453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f92000047d0 0x7f9200004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:37.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.168+0000 7f9216ffd640 1 --2- 172.21.15.90:0/379424453 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f921806e210 0x7f9200005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:37.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.168+0000 7f9216ffd640 1 -- 172.21.15.90:0/379424453 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920000ba40 con 0x7f9200003b40 2024-03-11T20:51:37.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.168+0000 7f9216ffd640 1 -- 172.21.15.90:0/379424453 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920000bb80 con 0x7f92000047d0 2024-03-11T20:51:37.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.168+0000 7f9216ffd640 1 -- 172.21.15.90:0/379424453 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f920000bd40 con 0x7f921806e210 2024-03-11T20:51:37.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.172+0000 7f9217fff640 1 --2- 172.21.15.90:0/379424453 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9200003b40 0x7f9200003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:37.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.172+0000 7f92177fe640 1 --2- 172.21.15.90:0/379424453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f92000047d0 0x7f9200004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:37.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.180+0000 7f9217fff640 1 -- 172.21.15.90:0/379424453 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f921806e210 msgr2=0x7f9200005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:37.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.180+0000 7f9217fff640 1 --2- 172.21.15.90:0/379424453 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f921806e210 0x7f9200005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:37.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.180+0000 7f9217fff640 1 -- 172.21.15.90:0/379424453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f92000047d0 msgr2=0x7f9200004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:37.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.180+0000 7f9217fff640 1 --2- 172.21.15.90:0/379424453 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f92000047d0 0x7f9200004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:37.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.180+0000 7f9217fff640 1 -- 172.21.15.90:0/379424453 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9200003530 con 0x7f9200003b40 2024-03-11T20:51:37.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.180+0000 7f9217fff640 1 --2- 172.21.15.90:0/379424453 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9200003b40 0x7f9200003f40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f920c007fb0 tx=0x7f920c01cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abd4c8a20e2fa246 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:37.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.188+0000 7f9216ffd640 1 -- 172.21.15.90:0/379424453 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920c005ae0 con 0x7f9200003b40 2024-03-11T20:51:37.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.188+0000 7f9216ffd640 1 -- 172.21.15.90:0/379424453 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f920c005c40 con 0x7f9200003b40 2024-03-11T20:51:37.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.188+0000 7f9216ffd640 1 -- 172.21.15.90:0/379424453 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920c0072b0 con 0x7f9200003b40 2024-03-11T20:51:37.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.188+0000 7f921ea41640 1 -- 172.21.15.90:0/379424453 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9200003b40 msgr2=0x7f9200003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:37.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.188+0000 7f921ea41640 1 --2- 172.21.15.90:0/379424453 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9200003b40 0x7f9200003f40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f920c007fb0 tx=0x7f920c01cb10 comp rx=0 tx=0).stop 2024-03-11T20:51:37.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.188+0000 7f921cfde640 1 -- 172.21.15.90:0/379424453 reap_dead start 2024-03-11T20:51:37.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.188+0000 7f921ea41640 1 -- 172.21.15.90:0/379424453 shutdown_connections 2024-03-11T20:51:37.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.188+0000 7f921ea41640 1 -- 172.21.15.90:0/379424453 >> 172.21.15.90:0/379424453 conn(0x7f921806ac00 msgr2=0x7f921806b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:37.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.188+0000 7f921ea41640 1 -- 172.21.15.90:0/379424453 shutdown_connections 2024-03-11T20:51:37.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.188+0000 7f921ea41640 1 -- 172.21.15.90:0/379424453 wait complete. 2024-03-11T20:51:37.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.188+0000 7f921ea41640 1 Processor -- start 2024-03-11T20:51:37.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.188+0000 7f921ea41640 1 -- start start 2024-03-11T20:51:37.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.188+0000 7f921ea41640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9200003b40 0x7f921813b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:37.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.188+0000 7f921ea41640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f921814b710 0x7f9218140020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:37.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.188+0000 7f921ea41640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f921814da00 0x7f9218140560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:37.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.188+0000 7f921ea41640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9218141960 con 0x7f9200003b40 2024-03-11T20:51:37.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.188+0000 7f921ea41640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f921814b280 con 0x7f921814da00 2024-03-11T20:51:37.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.188+0000 7f921ea41640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f921814a3d0 con 0x7f921814b710 2024-03-11T20:51:37.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.192+0000 7f9217fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9200003b40 0x7f921813b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:37.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.192+0000 7f9217fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9200003b40 0x7f921813b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58518/0 (socket says 172.21.15.90:58518) 2024-03-11T20:51:37.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.192+0000 7f9217fff640 1 -- 172.21.15.90:0/1374228232 learned_addr learned my addr 172.21.15.90:0/1374228232 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:51:37.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.196+0000 7f9217fff640 1 -- 172.21.15.90:0/1374228232 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f921814b710 msgr2=0x7f9218140020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:37.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.196+0000 7f92177fe640 1 --2- 172.21.15.90:0/1374228232 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f921814b710 0x7f9218140020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:37.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.196+0000 7f9217fff640 1 --2- 172.21.15.90:0/1374228232 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f921814b710 0x7f9218140020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:37.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.196+0000 7f9217fff640 1 -- 172.21.15.90:0/1374228232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f921814da00 msgr2=0x7f9218140560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:37.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.196+0000 7f9217fff640 1 --2- 172.21.15.90:0/1374228232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f921814da00 0x7f9218140560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:37.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.196+0000 7f9217fff640 1 -- 172.21.15.90:0/1374228232 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f921814a670 con 0x7f9200003b40 2024-03-11T20:51:37.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.200+0000 7f9217fff640 1 --2- 172.21.15.90:0/1374228232 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9200003b40 0x7f921813b6d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f920c007fb0 tx=0x7f9200005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:37.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.204+0000 7f9214ff9640 1 -- 172.21.15.90:0/1374228232 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920c0b4070 con 0x7f9200003b40 2024-03-11T20:51:37.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.204+0000 7f9214ff9640 1 -- 172.21.15.90:0/1374228232 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f920c005db0 con 0x7f9200003b40 2024-03-11T20:51:37.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.204+0000 7f9214ff9640 1 -- 172.21.15.90:0/1374228232 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920c001680 con 0x7f9200003b40 2024-03-11T20:51:37.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.204+0000 7f921ea41640 1 -- 172.21.15.90:0/1374228232 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f921814a950 con 0x7f9200003b40 2024-03-11T20:51:37.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.204+0000 7f921ea41640 1 -- 172.21.15.90:0/1374228232 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f921814af10 con 0x7f9200003b40 2024-03-11T20:51:37.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.204+0000 7f921ea41640 1 -- 172.21.15.90:0/1374228232 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f921804eef0 con 0x7f9200003b40 2024-03-11T20:51:37.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.492+0000 7f9214ff9640 1 -- 172.21.15.90:0/1374228232 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f920c007290 con 0x7f9200003b40 2024-03-11T20:51:37.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.492+0000 7f9214ff9640 1 --2- 172.21.15.90:0/1374228232 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f9204066ec0 0x7f9204069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:37.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.496+0000 7f92177fe640 1 --2- 172.21.15.90:0/1374228232 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f9204066ec0 0x7f9204069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:37.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.496+0000 7f92177fe640 1 --2- 172.21.15.90:0/1374228232 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f9204066ec0 0x7f9204069380 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f92180cc450 tx=0x7f9208009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:37.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.528+0000 7f9214ff9640 1 -- 172.21.15.90:0/1374228232 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f920c0ac370 con 0x7f9200003b40 2024-03-11T20:51:37.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.528+0000 7f9214ff9640 1 -- 172.21.15.90:0/1374228232 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f920c0eb170 con 0x7f9200003b40 2024-03-11T20:51:37.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.704+0000 7f921ea41640 1 -- 172.21.15.90:0/1374228232 --> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9218163300 con 0x7f9204066ec0 2024-03-11T20:51:37.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.724+0000 7f9214ff9640 1 -- 172.21.15.90:0/1374228232 <== mgr.4101 v2:172.21.15.90:6800/1999720161 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+334611 (secure 0 0 0) 0x7f920800d050 con 0x7f9204066ec0 2024-03-11T20:51:37.726 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T20:51:37.728 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-03-11T20:51:37.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.728+0000 7f921ea41640 1 -- 172.21.15.90:0/1374228232 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f9204066ec0 msgr2=0x7f9204069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:37.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.728+0000 7f921ea41640 1 --2- 172.21.15.90:0/1374228232 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f9204066ec0 0x7f9204069380 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f92180cc450 tx=0x7f9208009040 comp rx=0 tx=0).stop 2024-03-11T20:51:37.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.728+0000 7f921ea41640 1 -- 172.21.15.90:0/1374228232 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9200003b40 msgr2=0x7f921813b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:37.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.728+0000 7f921ea41640 1 --2- 172.21.15.90:0/1374228232 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9200003b40 0x7f921813b6d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f920c007fb0 tx=0x7f9200005800 comp rx=0 tx=0).stop 2024-03-11T20:51:37.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.728+0000 7f921ea41640 1 -- 172.21.15.90:0/1374228232 shutdown_connections 2024-03-11T20:51:37.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.728+0000 7f921ea41640 1 --2- 172.21.15.90:0/1374228232 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f9204066ec0 0x7f9204069380 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:37.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.728+0000 7f921ea41640 1 --2- 172.21.15.90:0/1374228232 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f921814da00 0x7f9218140560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:37.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.728+0000 7f921ea41640 1 --2- 172.21.15.90:0/1374228232 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f921814b710 0x7f9218140020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:37.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.728+0000 7f921ea41640 1 --2- 172.21.15.90:0/1374228232 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9200003b40 0x7f921813b6d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:37.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.728+0000 7f921ea41640 1 -- 172.21.15.90:0/1374228232 >> 172.21.15.90:0/1374228232 conn(0x7f921806ac00 msgr2=0x7f9218071370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:37.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.728+0000 7f921ea41640 1 -- 172.21.15.90:0/1374228232 shutdown_connections 2024-03-11T20:51:37.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.728+0000 7f921ea41640 1 -- 172.21.15.90:0/1374228232 wait complete. 2024-03-11T20:51:37.745 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":4116,"stamp":"2024-03-11T20:51:37.451854+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":211048067,"num_objects":150,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":150,"num_whiteouts":0,"num_read":7774,"num_read_kb":1584288,"num_write":72048,"num_write_kb":4420358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76413,"ondisk_log_size":76413,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1577152,"kb_used_data":414912,"kb_used_omap":12,"kb_used_meta":1162163,"kb_avail":753397568,"statfs":{"total":773094113280,"available":771479109632,"internally_reserved":0,"allocated":424869888,"data_stored":423141350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":1190055476},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":10473,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":24,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004873"},"pg_stats":[{"pgid":"2.1f","version":"30'681","reported_seq":777,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:18.510855+0000","last_change":"2024-03-11T18:35:56.290140+0000","last_active":"2024-03-11T20:51:18.510855+0000","last_peered":"2024-03-11T20:51:18.510855+0000","last_clean":"2024-03-11T20:51:18.510855+0000","last_became_active":"2024-03-11T18:35:56.289178+0000","last_became_peered":"2024-03-11T18:35:56.289178+0000","last_unstale":"2024-03-11T20:51:18.510855+0000","last_undegraded":"2024-03-11T20:51:18.510855+0000","last_fullsized":"2024-03-11T20:51:18.510855+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":681,"log_dups_size":0,"ondisk_log_size":681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:59:46.211033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389280,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":643,"num_write_kb":8491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"30'202","reported_seq":298,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.761828+0000","last_change":"2024-03-11T18:35:57.676047+0000","last_active":"2024-03-11T20:51:29.761828+0000","last_peered":"2024-03-11T20:51:29.761828+0000","last_clean":"2024-03-11T20:51:29.761828+0000","last_became_active":"2024-03-11T18:35:57.675105+0000","last_became_peered":"2024-03-11T18:35:57.675105+0000","last_unstale":"2024-03-11T20:51:29.761828+0000","last_undegraded":"2024-03-11T20:51:29.761828+0000","last_fullsized":"2024-03-11T20:51:29.761828+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:45:39.057074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"30'811","reported_seq":1003,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:10.788625+0000","last_change":"2024-03-11T18:36:04.607351+0000","last_active":"2024-03-11T20:51:10.788625+0000","last_peered":"2024-03-11T20:51:10.788625+0000","last_clean":"2024-03-11T20:51:10.788625+0000","last_became_active":"2024-03-11T18:36:04.599562+0000","last_became_peered":"2024-03-11T18:36:04.599562+0000","last_unstale":"2024-03-11T20:51:10.788625+0000","last_undegraded":"2024-03-11T20:51:10.788625+0000","last_fullsized":"2024-03-11T20:51:10.788625+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":811,"log_dups_size":0,"ondisk_log_size":811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T18:57:51.134410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":119,"num_read_kb":64172,"num_write":795,"num_write_kb":180282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"30'654","reported_seq":752,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:49:28.711104+0000","last_change":"2024-03-11T18:35:56.291253+0000","last_active":"2024-03-11T20:49:28.711104+0000","last_peered":"2024-03-11T20:49:28.711104+0000","last_clean":"2024-03-11T20:49:28.711104+0000","last_became_active":"2024-03-11T18:35:56.290862+0000","last_became_peered":"2024-03-11T18:35:56.290862+0000","last_unstale":"2024-03-11T20:49:28.711104+0000","last_undegraded":"2024-03-11T20:49:28.711104+0000","last_fullsized":"2024-03-11T20:49:28.711104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":654,"log_dups_size":0,"ondisk_log_size":654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:36:17.809744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":603,"num_write_kb":8489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"30'187","reported_seq":286,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.887083+0000","last_change":"2024-03-11T18:37:04.143178+0000","last_active":"2024-03-11T20:51:29.887083+0000","last_peered":"2024-03-11T20:51:29.887083+0000","last_clean":"2024-03-11T20:51:29.887083+0000","last_became_active":"2024-03-11T18:37:04.142880+0000","last_became_peered":"2024-03-11T18:37:04.142880+0000","last_unstale":"2024-03-11T20:51:29.887083+0000","last_undegraded":"2024-03-11T20:51:29.887083+0000","last_fullsized":"2024-03-11T20:51:29.887083+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:58:36.259380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"30'785","reported_seq":972,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:17.717779+0000","last_change":"2024-03-11T18:37:04.274577+0000","last_active":"2024-03-11T20:51:17.717779+0000","last_peered":"2024-03-11T20:51:17.717779+0000","last_clean":"2024-03-11T20:51:17.717779+0000","last_became_active":"2024-03-11T18:37:04.274018+0000","last_became_peered":"2024-03-11T18:37:04.274018+0000","last_unstale":"2024-03-11T20:51:17.717779+0000","last_undegraded":"2024-03-11T20:51:17.717779+0000","last_fullsized":"2024-03-11T20:51:17.717779+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":785,"log_dups_size":0,"ondisk_log_size":785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:51:28.017349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":106,"num_read_kb":55696,"num_write":774,"num_write_kb":160695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2,3],"acting":[4,6,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"30'1734","reported_seq":1819,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:50:57.274928+0000","last_change":"2024-03-11T18:35:56.355147+0000","last_active":"2024-03-11T20:50:57.274928+0000","last_peered":"2024-03-11T20:50:57.274928+0000","last_clean":"2024-03-11T20:50:57.274928+0000","last_became_active":"2024-03-11T18:35:56.354388+0000","last_became_peered":"2024-03-11T18:35:56.354388+0000","last_unstale":"2024-03-11T20:50:57.274928+0000","last_undegraded":"2024-03-11T20:50:57.274928+0000","last_fullsized":"2024-03-11T20:50:57.274928+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1734,"log_dups_size":0,"ondisk_log_size":1734,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:03:55.046045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1669,"num_write_kb":17224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"30'190","reported_seq":273,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.629989+0000","last_change":"2024-03-11T18:35:57.641486+0000","last_active":"2024-03-11T20:51:29.629989+0000","last_peered":"2024-03-11T20:51:29.629989+0000","last_clean":"2024-03-11T20:51:29.629989+0000","last_became_active":"2024-03-11T18:35:57.641052+0000","last_became_peered":"2024-03-11T18:35:57.641052+0000","last_unstale":"2024-03-11T20:51:29.629989+0000","last_undegraded":"2024-03-11T20:51:29.629989+0000","last_fullsized":"2024-03-11T20:51:29.629989+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:29:45.662029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"30'808","reported_seq":973,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:08.067327+0000","last_change":"2024-03-11T18:36:04.629291+0000","last_active":"2024-03-11T20:51:08.067327+0000","last_peered":"2024-03-11T20:51:08.067327+0000","last_clean":"2024-03-11T20:51:08.067327+0000","last_became_active":"2024-03-11T18:36:04.624110+0000","last_became_peered":"2024-03-11T18:36:04.624110+0000","last_unstale":"2024-03-11T20:51:08.067327+0000","last_undegraded":"2024-03-11T20:51:08.067327+0000","last_fullsized":"2024-03-11T20:51:08.067327+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":808,"log_dups_size":0,"ondisk_log_size":808,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:04:03.368322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":93,"num_read_kb":49009,"num_write":797,"num_write_kb":196902,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"30'832","reported_seq":917,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:50:57.274905+0000","last_change":"2024-03-11T18:35:56.298903+0000","last_active":"2024-03-11T20:50:57.274905+0000","last_peered":"2024-03-11T20:50:57.274905+0000","last_clean":"2024-03-11T20:50:57.274905+0000","last_became_active":"2024-03-11T18:35:56.297582+0000","last_became_peered":"2024-03-11T18:35:56.297582+0000","last_unstale":"2024-03-11T20:50:57.274905+0000","last_undegraded":"2024-03-11T20:50:57.274905+0000","last_fullsized":"2024-03-11T20:50:57.274905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":832,"log_dups_size":0,"ondisk_log_size":832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:53:25.278916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":777,"num_write_kb":4502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"30'203","reported_seq":286,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.623592+0000","last_change":"2024-03-11T18:35:57.640122+0000","last_active":"2024-03-11T20:51:29.623592+0000","last_peered":"2024-03-11T20:51:29.623592+0000","last_clean":"2024-03-11T20:51:29.623592+0000","last_became_active":"2024-03-11T18:35:57.639770+0000","last_became_peered":"2024-03-11T18:35:57.639770+0000","last_unstale":"2024-03-11T20:51:29.623592+0000","last_undegraded":"2024-03-11T20:51:29.623592+0000","last_fullsized":"2024-03-11T20:51:29.623592+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:39:25.767184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"30'729","reported_seq":825,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:08.238310+0000","last_change":"2024-03-11T18:36:04.627149+0000","last_active":"2024-03-11T20:51:08.238310+0000","last_peered":"2024-03-11T20:51:08.238310+0000","last_clean":"2024-03-11T20:51:08.238310+0000","last_became_active":"2024-03-11T18:36:04.626108+0000","last_became_peered":"2024-03-11T18:36:04.626108+0000","last_unstale":"2024-03-11T20:51:08.238310+0000","last_undegraded":"2024-03-11T20:51:08.238310+0000","last_fullsized":"2024-03-11T20:51:08.238310+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":729,"log_dups_size":0,"ondisk_log_size":729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:03:40.619618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":20804,"num_write":712,"num_write_kb":76413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"30'216","reported_seq":307,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.960165+0000","last_change":"2024-03-11T18:35:57.616739+0000","last_active":"2024-03-11T20:51:29.960165+0000","last_peered":"2024-03-11T20:51:29.960165+0000","last_clean":"2024-03-11T20:51:29.960165+0000","last_became_active":"2024-03-11T18:35:57.616398+0000","last_became_peered":"2024-03-11T18:35:57.616398+0000","last_unstale":"2024-03-11T20:51:29.960165+0000","last_undegraded":"2024-03-11T20:51:29.960165+0000","last_fullsized":"2024-03-11T20:51:29.960165+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:53:30.324898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"30'1448","reported_seq":1539,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:49:55.997509+0000","last_change":"2024-03-11T18:35:56.304762+0000","last_active":"2024-03-11T20:49:55.997509+0000","last_peered":"2024-03-11T20:49:55.997509+0000","last_clean":"2024-03-11T20:49:55.997509+0000","last_became_active":"2024-03-11T18:35:56.302803+0000","last_became_peered":"2024-03-11T18:35:56.302803+0000","last_unstale":"2024-03-11T20:49:55.997509+0000","last_undegraded":"2024-03-11T20:49:55.997509+0000","last_fullsized":"2024-03-11T20:49:55.997509+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1448,"log_dups_size":0,"ondisk_log_size":1448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:31:18.726996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1389,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"30'793","reported_seq":965,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:50:56.185068+0000","last_change":"2024-03-11T18:36:04.626968+0000","last_active":"2024-03-11T20:50:56.185068+0000","last_peered":"2024-03-11T20:50:56.185068+0000","last_clean":"2024-03-11T20:50:56.185068+0000","last_became_active":"2024-03-11T18:36:04.626337+0000","last_became_peered":"2024-03-11T18:36:04.626337+0000","last_unstale":"2024-03-11T20:50:56.185068+0000","last_undegraded":"2024-03-11T20:50:56.185068+0000","last_fullsized":"2024-03-11T20:50:56.185068+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":793,"log_dups_size":0,"ondisk_log_size":793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:53:37.432826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":94,"num_read_kb":46134,"num_write":781,"num_write_kb":151379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"30'211","reported_seq":300,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.763712+0000","last_change":"2024-03-11T18:35:57.689835+0000","last_active":"2024-03-11T20:51:29.763712+0000","last_peered":"2024-03-11T20:51:29.763712+0000","last_clean":"2024-03-11T20:51:29.763712+0000","last_became_active":"2024-03-11T18:35:57.689039+0000","last_became_peered":"2024-03-11T18:35:57.689039+0000","last_unstale":"2024-03-11T20:51:29.763712+0000","last_undegraded":"2024-03-11T20:51:29.763712+0000","last_fullsized":"2024-03-11T20:51:29.763712+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:38:27.733669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"30'1235","reported_seq":1328,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:49:27.679855+0000","last_change":"2024-03-11T18:35:56.291070+0000","last_active":"2024-03-11T20:49:27.679855+0000","last_peered":"2024-03-11T20:49:27.679855+0000","last_clean":"2024-03-11T20:49:27.679855+0000","last_became_active":"2024-03-11T18:35:56.290504+0000","last_became_peered":"2024-03-11T18:35:56.290504+0000","last_unstale":"2024-03-11T20:49:27.679855+0000","last_undegraded":"2024-03-11T20:49:27.679855+0000","last_fullsized":"2024-03-11T20:49:27.679855+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1235,"log_dups_size":0,"ondisk_log_size":1235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:29:31.660243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1168,"num_write_kb":8796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"30'848","reported_seq":1062,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:12.395257+0000","last_change":"2024-03-11T18:36:04.626150+0000","last_active":"2024-03-11T20:51:12.395257+0000","last_peered":"2024-03-11T20:51:12.395257+0000","last_clean":"2024-03-11T20:51:12.395257+0000","last_became_active":"2024-03-11T18:36:04.625084+0000","last_became_peered":"2024-03-11T18:36:04.625084+0000","last_unstale":"2024-03-11T20:51:12.395257+0000","last_undegraded":"2024-03-11T20:51:12.395257+0000","last_fullsized":"2024-03-11T20:51:12.395257+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":848,"log_dups_size":0,"ondisk_log_size":848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:28:58.373219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":136,"num_read_kb":82099,"num_write":832,"num_write_kb":151378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"30'209","reported_seq":300,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.771259+0000","last_change":"2024-03-11T18:35:57.684535+0000","last_active":"2024-03-11T20:51:29.771259+0000","last_peered":"2024-03-11T20:51:29.771259+0000","last_clean":"2024-03-11T20:51:29.771259+0000","last_became_active":"2024-03-11T18:35:57.681098+0000","last_became_peered":"2024-03-11T18:35:57.681098+0000","last_unstale":"2024-03-11T20:51:29.771259+0000","last_undegraded":"2024-03-11T20:51:29.771259+0000","last_fullsized":"2024-03-11T20:51:29.771259+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:25:49.707568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"30'483","reported_seq":576,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:49:29.907364+0000","last_change":"2024-03-11T18:35:56.290745+0000","last_active":"2024-03-11T20:49:29.907364+0000","last_peered":"2024-03-11T20:49:29.907364+0000","last_clean":"2024-03-11T20:49:29.907364+0000","last_became_active":"2024-03-11T18:35:56.290006+0000","last_became_peered":"2024-03-11T18:35:56.290006+0000","last_unstale":"2024-03-11T20:49:29.907364+0000","last_undegraded":"2024-03-11T20:49:29.907364+0000","last_fullsized":"2024-03-11T20:49:29.907364+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:06:02.844765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194394,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":433,"num_write_kb":4314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"30'749","reported_seq":894,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:08.589998+0000","last_change":"2024-03-11T18:36:04.620689+0000","last_active":"2024-03-11T20:51:08.589998+0000","last_peered":"2024-03-11T20:51:08.589998+0000","last_clean":"2024-03-11T20:51:08.589998+0000","last_became_active":"2024-03-11T18:36:04.620282+0000","last_became_peered":"2024-03-11T18:36:04.620282+0000","last_unstale":"2024-03-11T20:51:08.589998+0000","last_undegraded":"2024-03-11T20:51:08.589998+0000","last_fullsized":"2024-03-11T20:51:08.589998+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":749,"log_dups_size":0,"ondisk_log_size":749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:54:16.807299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":68,"num_read_kb":34102,"num_write":731,"num_write_kb":109760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"30'224","reported_seq":315,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.532383+0000","last_change":"2024-03-11T18:35:57.615234+0000","last_active":"2024-03-11T20:51:29.532383+0000","last_peered":"2024-03-11T20:51:29.532383+0000","last_clean":"2024-03-11T20:51:29.532383+0000","last_became_active":"2024-03-11T18:35:57.614885+0000","last_became_peered":"2024-03-11T18:35:57.614885+0000","last_unstale":"2024-03-11T20:51:29.532383+0000","last_undegraded":"2024-03-11T20:51:29.532383+0000","last_fullsized":"2024-03-11T20:51:29.532383+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":224,"log_dups_size":0,"ondisk_log_size":224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:22:13.730630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"30'348","reported_seq":441,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:50:17.268278+0000","last_change":"2024-03-11T18:35:56.304617+0000","last_active":"2024-03-11T20:50:17.268278+0000","last_peered":"2024-03-11T20:50:17.268278+0000","last_clean":"2024-03-11T20:50:17.268278+0000","last_became_active":"2024-03-11T18:35:56.301698+0000","last_became_peered":"2024-03-11T18:35:56.301698+0000","last_unstale":"2024-03-11T20:50:17.268278+0000","last_undegraded":"2024-03-11T20:50:17.268278+0000","last_fullsized":"2024-03-11T20:50:17.268278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:29:31.660243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":289,"num_write_kb":4232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"30'714","reported_seq":907,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:15.480325+0000","last_change":"2024-03-11T18:37:04.272234+0000","last_active":"2024-03-11T20:51:15.480325+0000","last_peered":"2024-03-11T20:51:15.480325+0000","last_clean":"2024-03-11T20:51:15.480325+0000","last_became_active":"2024-03-11T18:37:04.271646+0000","last_became_peered":"2024-03-11T18:37:04.271646+0000","last_unstale":"2024-03-11T20:51:15.480325+0000","last_undegraded":"2024-03-11T20:51:15.480325+0000","last_fullsized":"2024-03-11T20:51:15.480325+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:51:13.205384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":108,"num_read_kb":62192,"num_write":690,"num_write_kb":100980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,2],"acting":[4,3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"30'216","reported_seq":307,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.862313+0000","last_change":"2024-03-11T18:35:57.506963+0000","last_active":"2024-03-11T20:51:29.862313+0000","last_peered":"2024-03-11T20:51:29.862313+0000","last_clean":"2024-03-11T20:51:29.862313+0000","last_became_active":"2024-03-11T18:35:57.506038+0000","last_became_peered":"2024-03-11T18:35:57.506038+0000","last_unstale":"2024-03-11T20:51:29.862313+0000","last_undegraded":"2024-03-11T20:51:29.862313+0000","last_fullsized":"2024-03-11T20:51:29.862313+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:52:06.024245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"30'413","reported_seq":506,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:50:17.304480+0000","last_change":"2024-03-11T18:35:56.364512+0000","last_active":"2024-03-11T20:50:17.304480+0000","last_peered":"2024-03-11T20:50:17.304480+0000","last_clean":"2024-03-11T20:50:17.304480+0000","last_became_active":"2024-03-11T18:35:56.363731+0000","last_became_peered":"2024-03-11T18:35:56.363731+0000","last_unstale":"2024-03-11T20:50:17.304480+0000","last_undegraded":"2024-03-11T20:50:17.304480+0000","last_fullsized":"2024-03-11T20:50:17.304480+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:36:17.809744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":342,"num_write_kb":4261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"30'829","reported_seq":1056,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:14.415675+0000","last_change":"2024-03-11T18:36:04.643782+0000","last_active":"2024-03-11T20:51:14.415675+0000","last_peered":"2024-03-11T20:51:14.415675+0000","last_clean":"2024-03-11T20:51:14.415675+0000","last_became_active":"2024-03-11T18:36:04.642133+0000","last_became_peered":"2024-03-11T18:36:04.642133+0000","last_unstale":"2024-03-11T20:51:14.415675+0000","last_undegraded":"2024-03-11T20:51:14.415675+0000","last_fullsized":"2024-03-11T20:51:14.415675+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":829,"log_dups_size":0,"ondisk_log_size":829,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:53:33.601193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":144,"num_read_kb":78357,"num_write":819,"num_write_kb":238245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"30'168","reported_seq":259,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.764491+0000","last_change":"2024-03-11T18:35:57.675207+0000","last_active":"2024-03-11T20:51:29.764491+0000","last_peered":"2024-03-11T20:51:29.764491+0000","last_clean":"2024-03-11T20:51:29.764491+0000","last_became_active":"2024-03-11T18:35:57.674472+0000","last_became_peered":"2024-03-11T18:35:57.674472+0000","last_unstale":"2024-03-11T20:51:29.764491+0000","last_undegraded":"2024-03-11T20:51:29.764491+0000","last_fullsized":"2024-03-11T20:51:29.764491+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:29:45.662029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"30'6434","reported_seq":6527,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:22.287372+0000","last_change":"2024-03-11T18:35:56.348479+0000","last_active":"2024-03-11T20:51:22.287372+0000","last_peered":"2024-03-11T20:51:22.287372+0000","last_clean":"2024-03-11T20:51:22.287372+0000","last_became_active":"2024-03-11T18:35:56.348040+0000","last_became_peered":"2024-03-11T18:35:56.348040+0000","last_unstale":"2024-03-11T20:51:22.287372+0000","last_undegraded":"2024-03-11T20:51:22.287372+0000","last_fullsized":"2024-03-11T20:51:22.287372+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":6434,"log_dups_size":0,"ondisk_log_size":6434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:27:26.471550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17421,"num_write_kb":6249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"30'799","reported_seq":995,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:50:55.170434+0000","last_change":"2024-03-11T18:36:04.599805+0000","last_active":"2024-03-11T20:50:55.170434+0000","last_peered":"2024-03-11T20:50:55.170434+0000","last_clean":"2024-03-11T20:50:55.170434+0000","last_became_active":"2024-03-11T18:36:04.599065+0000","last_became_peered":"2024-03-11T18:36:04.599065+0000","last_unstale":"2024-03-11T20:50:55.170434+0000","last_undegraded":"2024-03-11T20:50:55.170434+0000","last_fullsized":"2024-03-11T20:50:55.170434+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":799,"log_dups_size":0,"ondisk_log_size":799,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:36:07.187003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":111,"num_read_kb":58643,"num_write":786,"num_write_kb":122786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"30'199","reported_seq":293,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.664765+0000","last_change":"2024-03-11T18:35:57.538551+0000","last_active":"2024-03-11T20:51:29.664765+0000","last_peered":"2024-03-11T20:51:29.664765+0000","last_clean":"2024-03-11T20:51:29.664765+0000","last_became_active":"2024-03-11T18:35:57.537706+0000","last_became_peered":"2024-03-11T18:35:57.537706+0000","last_unstale":"2024-03-11T20:51:29.664765+0000","last_undegraded":"2024-03-11T20:51:29.664765+0000","last_fullsized":"2024-03-11T20:51:29.664765+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:03:56.260394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"30'985","reported_seq":1080,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:49:47.414091+0000","last_change":"2024-03-11T18:35:56.302575+0000","last_active":"2024-03-11T20:49:47.414091+0000","last_peered":"2024-03-11T20:49:47.414091+0000","last_clean":"2024-03-11T20:49:47.414091+0000","last_became_active":"2024-03-11T18:35:56.301819+0000","last_became_peered":"2024-03-11T18:35:56.301819+0000","last_unstale":"2024-03-11T20:49:47.414091+0000","last_undegraded":"2024-03-11T20:49:47.414091+0000","last_fullsized":"2024-03-11T20:49:47.414091+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":985,"log_dups_size":0,"ondisk_log_size":985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:38:26.519320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":935,"num_write_kb":8676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"30'751","reported_seq":913,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:14.544182+0000","last_change":"2024-03-11T18:36:04.609282+0000","last_active":"2024-03-11T20:51:14.544182+0000","last_peered":"2024-03-11T20:51:14.544182+0000","last_clean":"2024-03-11T20:51:14.544182+0000","last_became_active":"2024-03-11T18:36:04.608260+0000","last_became_peered":"2024-03-11T18:36:04.608260+0000","last_unstale":"2024-03-11T20:51:14.544182+0000","last_undegraded":"2024-03-11T20:51:14.544182+0000","last_fullsized":"2024-03-11T20:51:14.544182+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":751,"log_dups_size":0,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:49:28.143755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":79,"num_read_kb":54998,"num_write":738,"num_write_kb":142673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"30'218","reported_seq":311,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.614589+0000","last_change":"2024-03-11T18:35:57.688727+0000","last_active":"2024-03-11T20:51:29.614589+0000","last_peered":"2024-03-11T20:51:29.614589+0000","last_clean":"2024-03-11T20:51:29.614589+0000","last_became_active":"2024-03-11T18:35:57.688209+0000","last_became_peered":"2024-03-11T18:35:57.688209+0000","last_unstale":"2024-03-11T20:51:29.614589+0000","last_undegraded":"2024-03-11T20:51:29.614589+0000","last_fullsized":"2024-03-11T20:51:29.614589+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:58:13.389498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"30'373","reported_seq":469,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:49:53.793880+0000","last_change":"2024-03-11T18:35:56.293842+0000","last_active":"2024-03-11T20:49:53.793880+0000","last_peered":"2024-03-11T20:49:53.793880+0000","last_clean":"2024-03-11T20:49:53.793880+0000","last_became_active":"2024-03-11T18:35:56.293276+0000","last_became_peered":"2024-03-11T18:35:56.293276+0000","last_unstale":"2024-03-11T20:49:53.793880+0000","last_undegraded":"2024-03-11T20:49:53.793880+0000","last_fullsized":"2024-03-11T20:49:53.793880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:04:13.101160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":309,"num_write_kb":4248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"30'742","reported_seq":881,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:15.435885+0000","last_change":"2024-03-11T18:36:04.626320+0000","last_active":"2024-03-11T20:51:15.435885+0000","last_peered":"2024-03-11T20:51:15.435885+0000","last_clean":"2024-03-11T20:51:15.435885+0000","last_became_active":"2024-03-11T18:36:04.625663+0000","last_became_peered":"2024-03-11T18:36:04.625663+0000","last_unstale":"2024-03-11T20:51:15.435885+0000","last_undegraded":"2024-03-11T20:51:15.435885+0000","last_fullsized":"2024-03-11T20:51:15.435885+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":742,"log_dups_size":0,"ondisk_log_size":742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:39:32.875112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":56,"num_read_kb":20684,"num_write":725,"num_write_kb":55759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"30'199","reported_seq":293,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.548835+0000","last_change":"2024-03-11T18:35:57.612242+0000","last_active":"2024-03-11T20:51:29.548835+0000","last_peered":"2024-03-11T20:51:29.548835+0000","last_clean":"2024-03-11T20:51:29.548835+0000","last_became_active":"2024-03-11T18:35:57.611509+0000","last_became_peered":"2024-03-11T18:35:57.611509+0000","last_unstale":"2024-03-11T20:51:29.548835+0000","last_undegraded":"2024-03-11T20:51:29.548835+0000","last_fullsized":"2024-03-11T20:51:29.548835+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:45:28.543623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"30'1190","reported_seq":1286,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:49:17.484772+0000","last_change":"2024-03-11T18:35:56.298766+0000","last_active":"2024-03-11T20:49:17.484772+0000","last_peered":"2024-03-11T20:49:17.484772+0000","last_clean":"2024-03-11T20:49:17.484772+0000","last_became_active":"2024-03-11T18:35:56.297747+0000","last_became_peered":"2024-03-11T18:35:56.297747+0000","last_unstale":"2024-03-11T20:49:17.484772+0000","last_undegraded":"2024-03-11T20:49:17.484772+0000","last_fullsized":"2024-03-11T20:49:17.484772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1190,"log_dups_size":0,"ondisk_log_size":1190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:13:39.177689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389196,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1123,"num_write_kb":8732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"30'749","reported_seq":963,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:17.573144+0000","last_change":"2024-03-11T18:36:04.635773+0000","last_active":"2024-03-11T20:51:17.573144+0000","last_peered":"2024-03-11T20:51:17.573144+0000","last_clean":"2024-03-11T20:51:17.573144+0000","last_became_active":"2024-03-11T18:36:04.635261+0000","last_became_peered":"2024-03-11T18:36:04.635261+0000","last_unstale":"2024-03-11T20:51:17.573144+0000","last_undegraded":"2024-03-11T20:51:17.573144+0000","last_fullsized":"2024-03-11T20:51:17.573144+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":749,"log_dups_size":0,"ondisk_log_size":749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:44:18.985469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":131,"num_read_kb":65646,"num_write":733,"num_write_kb":179820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"30'211","reported_seq":307,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.668044+0000","last_change":"2024-03-11T18:35:57.538552+0000","last_active":"2024-03-11T20:51:29.668044+0000","last_peered":"2024-03-11T20:51:29.668044+0000","last_clean":"2024-03-11T20:51:29.668044+0000","last_became_active":"2024-03-11T18:35:57.537717+0000","last_became_peered":"2024-03-11T18:35:57.537717+0000","last_unstale":"2024-03-11T20:51:29.668044+0000","last_undegraded":"2024-03-11T20:51:29.668044+0000","last_fullsized":"2024-03-11T20:51:29.668044+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:53:26.493265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"30'4762","reported_seq":9555,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:24.467774+0000","last_change":"2024-03-11T18:35:56.294572+0000","last_active":"2024-03-11T20:51:24.467774+0000","last_peered":"2024-03-11T20:51:24.467774+0000","last_clean":"2024-03-11T20:51:24.467774+0000","last_became_active":"2024-03-11T18:35:56.294096+0000","last_became_peered":"2024-03-11T18:35:56.294096+0000","last_unstale":"2024-03-11T20:51:24.467774+0000","last_undegraded":"2024-03-11T20:51:24.467774+0000","last_fullsized":"2024-03-11T20:51:24.467774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":4762,"log_dups_size":0,"ondisk_log_size":4762,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:22:12.516281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1555811,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4697,"num_read_kb":5145,"num_write":4704,"num_write_kb":5149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"30'682","reported_seq":802,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:08.234604+0000","last_change":"2024-03-11T18:36:04.620762+0000","last_active":"2024-03-11T20:51:08.234604+0000","last_peered":"2024-03-11T20:51:08.234604+0000","last_clean":"2024-03-11T20:51:08.234604+0000","last_became_active":"2024-03-11T18:36:04.620356+0000","last_became_peered":"2024-03-11T18:36:04.620356+0000","last_unstale":"2024-03-11T20:51:08.234604+0000","last_undegraded":"2024-03-11T20:51:08.234604+0000","last_fullsized":"2024-03-11T20:51:08.234604+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":682,"log_dups_size":0,"ondisk_log_size":682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:58:20.497426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":17597,"num_write":672,"num_write_kb":47486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"30'198","reported_seq":292,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.887154+0000","last_change":"2024-03-11T18:35:57.602773+0000","last_active":"2024-03-11T20:51:29.887154+0000","last_peered":"2024-03-11T20:51:29.887154+0000","last_clean":"2024-03-11T20:51:29.887154+0000","last_became_active":"2024-03-11T18:35:57.602060+0000","last_became_peered":"2024-03-11T18:35:57.602060+0000","last_unstale":"2024-03-11T20:51:29.887154+0000","last_undegraded":"2024-03-11T20:51:29.887154+0000","last_fullsized":"2024-03-11T20:51:29.887154+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:36:22.201707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"30'1021","reported_seq":1119,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:50:32.270816+0000","last_change":"2024-03-11T18:35:56.305283+0000","last_active":"2024-03-11T20:50:32.270816+0000","last_peered":"2024-03-11T20:50:32.270816+0000","last_clean":"2024-03-11T20:50:32.270816+0000","last_became_active":"2024-03-11T18:35:56.303063+0000","last_became_peered":"2024-03-11T18:35:56.303063+0000","last_unstale":"2024-03-11T20:50:32.270816+0000","last_undegraded":"2024-03-11T20:50:32.270816+0000","last_fullsized":"2024-03-11T20:50:32.270816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1021,"log_dups_size":0,"ondisk_log_size":1021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T18:51:00.795706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":963,"num_write_kb":12732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"30'852","reported_seq":1017,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:15.381877+0000","last_change":"2024-03-11T18:36:04.617247+0000","last_active":"2024-03-11T20:51:15.381877+0000","last_peered":"2024-03-11T20:51:15.381877+0000","last_clean":"2024-03-11T20:51:15.381877+0000","last_became_active":"2024-03-11T18:36:04.610765+0000","last_became_peered":"2024-03-11T18:36:04.610765+0000","last_unstale":"2024-03-11T20:51:15.381877+0000","last_undegraded":"2024-03-11T20:51:15.381877+0000","last_fullsized":"2024-03-11T20:51:15.381877+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:22:19.187685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":85,"num_read_kb":37203,"num_write":843,"num_write_kb":103063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"30'185","reported_seq":279,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.764151+0000","last_change":"2024-03-11T18:35:57.684207+0000","last_active":"2024-03-11T20:51:29.764151+0000","last_peered":"2024-03-11T20:51:29.764151+0000","last_clean":"2024-03-11T20:51:29.764151+0000","last_became_active":"2024-03-11T18:35:57.682653+0000","last_became_peered":"2024-03-11T18:35:57.682653+0000","last_unstale":"2024-03-11T20:51:29.764151+0000","last_undegraded":"2024-03-11T20:51:29.764151+0000","last_fullsized":"2024-03-11T20:51:29.764151+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":185,"log_dups_size":0,"ondisk_log_size":185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:28:51.265291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"30'371","reported_seq":467,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:08.599939+0000","last_change":"2024-03-11T18:35:56.297078+0000","last_active":"2024-03-11T20:51:08.599939+0000","last_peered":"2024-03-11T20:51:08.599939+0000","last_clean":"2024-03-11T20:51:08.599939+0000","last_became_active":"2024-03-11T18:35:56.296705+0000","last_became_peered":"2024-03-11T18:35:56.296705+0000","last_unstale":"2024-03-11T20:51:08.599939+0000","last_undegraded":"2024-03-11T20:51:08.599939+0000","last_fullsized":"2024-03-11T20:51:08.599939+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:53:25.278916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194372,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":305,"num_write_kb":4243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":253,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:47:47.393591+0000","last_change":"2024-03-11T18:35:18.199528+0000","last_active":"2024-03-11T20:47:47.393591+0000","last_peered":"2024-03-11T20:47:47.393591+0000","last_clean":"2024-03-11T20:47:47.393591+0000","last_became_active":"2024-03-11T18:35:18.197797+0000","last_became_peered":"2024-03-11T18:35:18.197797+0000","last_unstale":"2024-03-11T20:47:47.393591+0000","last_undegraded":"2024-03-11T20:47:47.393591+0000","last_fullsized":"2024-03-11T20:47:47.393591+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:16.207500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:16.207500+0000","last_clean_scrub_stamp":"2024-03-11T18:35:16.207500+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:29:00.357496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"30'785","reported_seq":953,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:10.792501+0000","last_change":"2024-03-11T18:36:04.599815+0000","last_active":"2024-03-11T20:51:10.792501+0000","last_peered":"2024-03-11T20:51:10.792501+0000","last_clean":"2024-03-11T20:51:10.792501+0000","last_became_active":"2024-03-11T18:36:04.598777+0000","last_became_peered":"2024-03-11T18:36:04.598777+0000","last_unstale":"2024-03-11T20:51:10.792501+0000","last_undegraded":"2024-03-11T20:51:10.792501+0000","last_fullsized":"2024-03-11T20:51:10.792501+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":785,"log_dups_size":0,"ondisk_log_size":785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:39:32.875112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":86,"num_read_kb":36797,"num_write":762,"num_write_kb":114420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"30'207","reported_seq":276,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.794305+0000","last_change":"2024-03-11T18:37:04.141957+0000","last_active":"2024-03-11T20:51:29.794305+0000","last_peered":"2024-03-11T20:51:29.794305+0000","last_clean":"2024-03-11T20:51:29.794305+0000","last_became_active":"2024-03-11T18:37:04.141663+0000","last_became_peered":"2024-03-11T18:37:04.141663+0000","last_unstale":"2024-03-11T20:51:29.794305+0000","last_undegraded":"2024-03-11T20:51:29.794305+0000","last_fullsized":"2024-03-11T20:51:29.794305+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:25:52.716744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"30'286","reported_seq":377,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:03.112535+0000","last_change":"2024-03-11T18:35:56.294626+0000","last_active":"2024-03-11T20:51:03.112535+0000","last_peered":"2024-03-11T20:51:03.112535+0000","last_clean":"2024-03-11T20:51:03.112535+0000","last_became_active":"2024-03-11T18:35:56.293961+0000","last_became_peered":"2024-03-11T18:35:56.293961+0000","last_unstale":"2024-03-11T20:51:03.112535+0000","last_undegraded":"2024-03-11T20:51:03.112535+0000","last_fullsized":"2024-03-11T20:51:03.112535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:50:38.709532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4196056,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":223,"num_write_kb":4226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"30'822","reported_seq":987,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:14.297709+0000","last_change":"2024-03-11T18:36:04.619862+0000","last_active":"2024-03-11T20:51:14.297709+0000","last_peered":"2024-03-11T20:51:14.297709+0000","last_clean":"2024-03-11T20:51:14.297709+0000","last_became_active":"2024-03-11T18:36:04.618908+0000","last_became_peered":"2024-03-11T18:36:04.618908+0000","last_unstale":"2024-03-11T20:51:14.297709+0000","last_undegraded":"2024-03-11T20:51:14.297709+0000","last_fullsized":"2024-03-11T20:51:14.297709+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":822,"log_dups_size":0,"ondisk_log_size":822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:34:46.433500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":85,"num_read_kb":42087,"num_write":807,"num_write_kb":152305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"30'221","reported_seq":310,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.603226+0000","last_change":"2024-03-11T18:35:57.686416+0000","last_active":"2024-03-11T20:51:29.603226+0000","last_peered":"2024-03-11T20:51:29.603226+0000","last_clean":"2024-03-11T20:51:29.603226+0000","last_became_active":"2024-03-11T18:35:57.685828+0000","last_became_peered":"2024-03-11T18:35:57.685828+0000","last_unstale":"2024-03-11T20:51:29.603226+0000","last_undegraded":"2024-03-11T20:51:29.603226+0000","last_fullsized":"2024-03-11T20:51:29.603226+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:41:42.263652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"30'406","reported_seq":497,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:49:30.772686+0000","last_change":"2024-03-11T18:35:56.295539+0000","last_active":"2024-03-11T20:49:30.772686+0000","last_peered":"2024-03-11T20:49:30.772686+0000","last_clean":"2024-03-11T20:49:30.772686+0000","last_became_active":"2024-03-11T18:35:56.295123+0000","last_became_peered":"2024-03-11T18:35:56.295123+0000","last_unstale":"2024-03-11T20:49:30.772686+0000","last_undegraded":"2024-03-11T20:49:30.772686+0000","last_fullsized":"2024-03-11T20:49:30.772686+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:03:06.585728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":350,"num_write_kb":4273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"30'821","reported_seq":1021,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:04.302500+0000","last_change":"2024-03-11T18:36:04.616750+0000","last_active":"2024-03-11T20:51:04.302500+0000","last_peered":"2024-03-11T20:51:04.302500+0000","last_clean":"2024-03-11T20:51:04.302500+0000","last_became_active":"2024-03-11T18:36:04.616338+0000","last_became_peered":"2024-03-11T18:36:04.616338+0000","last_unstale":"2024-03-11T20:51:04.302500+0000","last_undegraded":"2024-03-11T20:51:04.302500+0000","last_fullsized":"2024-03-11T20:51:04.302500+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":821,"log_dups_size":0,"ondisk_log_size":821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:14:42.558491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":122,"num_read_kb":70346,"num_write":827,"num_write_kb":188969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"30'199","reported_seq":288,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.638812+0000","last_change":"2024-03-11T18:35:57.689888+0000","last_active":"2024-03-11T20:51:29.638812+0000","last_peered":"2024-03-11T20:51:29.638812+0000","last_clean":"2024-03-11T20:51:29.638812+0000","last_became_active":"2024-03-11T18:35:57.689424+0000","last_became_peered":"2024-03-11T18:35:57.689424+0000","last_unstale":"2024-03-11T20:51:29.638812+0000","last_undegraded":"2024-03-11T20:51:29.638812+0000","last_fullsized":"2024-03-11T20:51:29.638812+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:25:16.564261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"30'861","reported_seq":952,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:49:04.742266+0000","last_change":"2024-03-11T18:35:56.293862+0000","last_active":"2024-03-11T20:49:04.742266+0000","last_peered":"2024-03-11T20:49:04.742266+0000","last_clean":"2024-03-11T20:49:04.742266+0000","last_became_active":"2024-03-11T18:35:56.293585+0000","last_became_peered":"2024-03-11T18:35:56.293585+0000","last_unstale":"2024-03-11T20:49:04.742266+0000","last_undegraded":"2024-03-11T20:49:04.742266+0000","last_fullsized":"2024-03-11T20:49:04.742266+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":861,"log_dups_size":0,"ondisk_log_size":861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:45:27.329274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":804,"num_write_kb":4496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"30'804","reported_seq":982,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:17.590386+0000","last_change":"2024-03-11T18:36:04.627729+0000","last_active":"2024-03-11T20:51:17.590386+0000","last_peered":"2024-03-11T20:51:17.590386+0000","last_clean":"2024-03-11T20:51:17.590386+0000","last_became_active":"2024-03-11T18:36:04.626596+0000","last_became_peered":"2024-03-11T18:36:04.626596+0000","last_unstale":"2024-03-11T20:51:17.590386+0000","last_undegraded":"2024-03-11T20:51:17.590386+0000","last_fullsized":"2024-03-11T20:51:17.590386+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:51:28.017349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":98,"num_read_kb":46229,"num_write":782,"num_write_kb":114631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"30'184","reported_seq":273,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.619621+0000","last_change":"2024-03-11T18:35:57.690506+0000","last_active":"2024-03-11T20:51:29.619621+0000","last_peered":"2024-03-11T20:51:29.619621+0000","last_clean":"2024-03-11T20:51:29.619621+0000","last_became_active":"2024-03-11T18:35:57.690223+0000","last_became_peered":"2024-03-11T18:35:57.690223+0000","last_unstale":"2024-03-11T20:51:29.619621+0000","last_undegraded":"2024-03-11T20:51:29.619621+0000","last_fullsized":"2024-03-11T20:51:29.619621+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:58:13.389498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"30'815","reported_seq":906,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:14.415105+0000","last_change":"2024-03-11T18:35:56.257209+0000","last_active":"2024-03-11T20:51:14.415105+0000","last_peered":"2024-03-11T20:51:14.415105+0000","last_clean":"2024-03-11T20:51:14.415105+0000","last_became_active":"2024-03-11T18:35:56.256454+0000","last_became_peered":"2024-03-11T18:35:56.256454+0000","last_unstale":"2024-03-11T20:51:14.415105+0000","last_undegraded":"2024-03-11T20:51:14.415105+0000","last_fullsized":"2024-03-11T20:51:14.415105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":815,"log_dups_size":0,"ondisk_log_size":815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:32:06.369248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":757,"num_write_kb":4472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"30'833","reported_seq":1020,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:08.316459+0000","last_change":"2024-03-11T18:36:04.644184+0000","last_active":"2024-03-11T20:51:08.316459+0000","last_peered":"2024-03-11T20:51:08.316459+0000","last_clean":"2024-03-11T20:51:08.316459+0000","last_became_active":"2024-03-11T18:36:04.642079+0000","last_became_peered":"2024-03-11T18:36:04.642079+0000","last_unstale":"2024-03-11T20:51:08.316459+0000","last_undegraded":"2024-03-11T20:51:08.316459+0000","last_fullsized":"2024-03-11T20:51:08.316459+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":833,"log_dups_size":0,"ondisk_log_size":833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:36:26.132020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":107,"num_read_kb":63565,"num_write":823,"num_write_kb":172061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"30'239","reported_seq":326,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.606890+0000","last_change":"2024-03-11T18:35:57.687311+0000","last_active":"2024-03-11T20:51:29.606890+0000","last_peered":"2024-03-11T20:51:29.606890+0000","last_clean":"2024-03-11T20:51:29.606890+0000","last_became_active":"2024-03-11T18:35:57.686850+0000","last_became_peered":"2024-03-11T18:35:57.686850+0000","last_unstale":"2024-03-11T20:51:29.606890+0000","last_undegraded":"2024-03-11T20:51:29.606890+0000","last_fullsized":"2024-03-11T20:51:29.606890+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:29:45.662029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"30'899","reported_seq":988,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:16.314784+0000","last_change":"2024-03-11T18:35:56.291232+0000","last_active":"2024-03-11T20:51:16.314784+0000","last_peered":"2024-03-11T20:51:16.314784+0000","last_clean":"2024-03-11T20:51:16.314784+0000","last_became_active":"2024-03-11T18:35:56.290944+0000","last_became_peered":"2024-03-11T18:35:56.290944+0000","last_unstale":"2024-03-11T20:51:16.314784+0000","last_undegraded":"2024-03-11T20:51:16.314784+0000","last_fullsized":"2024-03-11T20:51:16.314784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":899,"log_dups_size":0,"ondisk_log_size":899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:49:27.827765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":846,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"30'791","reported_seq":968,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:18.444965+0000","last_change":"2024-03-11T18:36:04.602142+0000","last_active":"2024-03-11T20:51:18.444965+0000","last_peered":"2024-03-11T20:51:18.444965+0000","last_clean":"2024-03-11T20:51:18.444965+0000","last_became_active":"2024-03-11T18:36:04.601224+0000","last_became_peered":"2024-03-11T18:36:04.601224+0000","last_unstale":"2024-03-11T20:51:18.444965+0000","last_undegraded":"2024-03-11T20:51:18.444965+0000","last_fullsized":"2024-03-11T20:51:18.444965+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:17:42.075355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":59304,"num_write":781,"num_write_kb":152027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"30'219","reported_seq":306,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.941879+0000","last_change":"2024-03-11T18:35:57.611305+0000","last_active":"2024-03-11T20:51:29.941879+0000","last_peered":"2024-03-11T20:51:29.941879+0000","last_clean":"2024-03-11T20:51:29.941879+0000","last_became_active":"2024-03-11T18:35:57.610955+0000","last_became_peered":"2024-03-11T18:35:57.610955+0000","last_unstale":"2024-03-11T20:51:29.941879+0000","last_undegraded":"2024-03-11T20:51:29.941879+0000","last_fullsized":"2024-03-11T20:51:29.941879+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:31:19.941345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"30'60","reported_seq":149,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:17.314474+0000","last_change":"2024-03-11T18:35:56.361952+0000","last_active":"2024-03-11T20:51:17.314474+0000","last_peered":"2024-03-11T20:51:17.314474+0000","last_clean":"2024-03-11T20:51:17.314474+0000","last_became_active":"2024-03-11T18:35:56.361141+0000","last_became_peered":"2024-03-11T18:35:56.361141+0000","last_unstale":"2024-03-11T20:51:17.314474+0000","last_undegraded":"2024-03-11T20:51:17.314474+0000","last_fullsized":"2024-03-11T20:51:17.314474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:04:13.101160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"30'743","reported_seq":894,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:15.483428+0000","last_change":"2024-03-11T18:36:04.593604+0000","last_active":"2024-03-11T20:51:15.483428+0000","last_peered":"2024-03-11T20:51:15.483428+0000","last_clean":"2024-03-11T20:51:15.483428+0000","last_became_active":"2024-03-11T18:36:04.592304+0000","last_became_peered":"2024-03-11T18:36:04.592304+0000","last_unstale":"2024-03-11T20:51:15.483428+0000","last_undegraded":"2024-03-11T20:51:15.483428+0000","last_fullsized":"2024-03-11T20:51:15.483428+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":743,"log_dups_size":0,"ondisk_log_size":743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:31:04.738940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":77,"num_read_kb":52017,"num_write":733,"num_write_kb":82372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"30'206","reported_seq":293,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.637397+0000","last_change":"2024-03-11T18:35:57.689404+0000","last_active":"2024-03-11T20:51:29.637397+0000","last_peered":"2024-03-11T20:51:29.637397+0000","last_clean":"2024-03-11T20:51:29.637397+0000","last_became_active":"2024-03-11T18:35:57.689139+0000","last_became_peered":"2024-03-11T18:35:57.689139+0000","last_unstale":"2024-03-11T20:51:29.637397+0000","last_undegraded":"2024-03-11T20:51:29.637397+0000","last_fullsized":"2024-03-11T20:51:29.637397+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:54:09.699371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"30'60","reported_seq":149,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:19.415920+0000","last_change":"2024-03-11T18:35:56.297682+0000","last_active":"2024-03-11T20:51:19.415920+0000","last_peered":"2024-03-11T20:51:19.415920+0000","last_clean":"2024-03-11T20:51:19.415920+0000","last_became_active":"2024-03-11T18:35:56.296426+0000","last_became_peered":"2024-03-11T18:35:56.296426+0000","last_unstale":"2024-03-11T20:51:19.415920+0000","last_undegraded":"2024-03-11T20:51:19.415920+0000","last_fullsized":"2024-03-11T20:51:19.415920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:28:50.050942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"30'724","reported_seq":923,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:03.154615+0000","last_change":"2024-03-11T18:36:04.590780+0000","last_active":"2024-03-11T20:51:03.154615+0000","last_peered":"2024-03-11T20:51:03.154615+0000","last_clean":"2024-03-11T20:51:03.154615+0000","last_became_active":"2024-03-11T18:36:04.590014+0000","last_became_peered":"2024-03-11T18:36:04.590014+0000","last_unstale":"2024-03-11T20:51:03.154615+0000","last_undegraded":"2024-03-11T20:51:03.154615+0000","last_fullsized":"2024-03-11T20:51:03.154615+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":724,"log_dups_size":0,"ondisk_log_size":724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:51:13.205384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":124,"num_read_kb":69166,"num_write":714,"num_write_kb":127024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"30'208","reported_seq":295,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.875165+0000","last_change":"2024-03-11T18:35:57.671638+0000","last_active":"2024-03-11T20:51:29.875165+0000","last_peered":"2024-03-11T20:51:29.875165+0000","last_clean":"2024-03-11T20:51:29.875165+0000","last_became_active":"2024-03-11T18:35:57.670893+0000","last_became_peered":"2024-03-11T18:35:57.670893+0000","last_unstale":"2024-03-11T20:51:29.875165+0000","last_undegraded":"2024-03-11T20:51:29.875165+0000","last_fullsized":"2024-03-11T20:51:29.875165+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:31:21.395133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"30'592","reported_seq":681,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:49:43.697234+0000","last_change":"2024-03-11T18:35:56.251580+0000","last_active":"2024-03-11T20:49:43.697234+0000","last_peered":"2024-03-11T20:49:43.697234+0000","last_clean":"2024-03-11T20:49:43.697234+0000","last_became_active":"2024-03-11T18:35:56.251287+0000","last_became_peered":"2024-03-11T18:35:56.251287+0000","last_unstale":"2024-03-11T20:49:43.697234+0000","last_undegraded":"2024-03-11T20:49:43.697234+0000","last_fullsized":"2024-03-11T20:49:43.697234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":592,"log_dups_size":0,"ondisk_log_size":592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:30:39.907360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388676,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":533,"num_write_kb":8445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"30'738","reported_seq":846,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:12.396881+0000","last_change":"2024-03-11T18:36:04.625563+0000","last_active":"2024-03-11T20:51:12.396881+0000","last_peered":"2024-03-11T20:51:12.396881+0000","last_clean":"2024-03-11T20:51:12.396881+0000","last_became_active":"2024-03-11T18:36:04.618911+0000","last_became_peered":"2024-03-11T18:36:04.618911+0000","last_unstale":"2024-03-11T20:51:12.396881+0000","last_undegraded":"2024-03-11T20:51:12.396881+0000","last_fullsized":"2024-03-11T20:51:12.396881+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":738,"log_dups_size":0,"ondisk_log_size":738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:31:04.738940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":10131,"num_write":722,"num_write_kb":109498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"30'204","reported_seq":289,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.606383+0000","last_change":"2024-03-11T18:35:57.697641+0000","last_active":"2024-03-11T20:51:29.606383+0000","last_peered":"2024-03-11T20:51:29.606383+0000","last_clean":"2024-03-11T20:51:29.606383+0000","last_became_active":"2024-03-11T18:35:57.697263+0000","last_became_peered":"2024-03-11T18:35:57.697263+0000","last_unstale":"2024-03-11T20:51:29.606383+0000","last_undegraded":"2024-03-11T20:51:29.606383+0000","last_fullsized":"2024-03-11T20:51:29.606383+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:34:39.325572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"30'2202","reported_seq":2290,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:01.825523+0000","last_change":"2024-03-11T18:35:56.355612+0000","last_active":"2024-03-11T20:51:01.825523+0000","last_peered":"2024-03-11T20:51:01.825523+0000","last_clean":"2024-03-11T20:51:01.825523+0000","last_became_active":"2024-03-11T18:35:56.354827+0000","last_became_peered":"2024-03-11T18:35:56.354827+0000","last_unstale":"2024-03-11T20:51:01.825523+0000","last_undegraded":"2024-03-11T20:51:01.825523+0000","last_fullsized":"2024-03-11T20:51:01.825523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":2202,"log_dups_size":0,"ondisk_log_size":2202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:06:02.844765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583584,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2157,"num_write_kb":13630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"30'771","reported_seq":924,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:14.409334+0000","last_change":"2024-03-11T18:37:04.259892+0000","last_active":"2024-03-11T20:51:14.409334+0000","last_peered":"2024-03-11T20:51:14.409334+0000","last_clean":"2024-03-11T20:51:14.409334+0000","last_became_active":"2024-03-11T18:37:04.259051+0000","last_became_peered":"2024-03-11T18:37:04.259051+0000","last_unstale":"2024-03-11T20:51:14.409334+0000","last_undegraded":"2024-03-11T20:51:14.409334+0000","last_fullsized":"2024-03-11T20:51:14.409334+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":771,"log_dups_size":0,"ondisk_log_size":771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:36:51.665733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":84,"num_read_kb":37526,"num_write":755,"num_write_kb":85949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2,0],"acting":[5,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"30'201","reported_seq":287,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.590814+0000","last_change":"2024-03-11T18:35:57.679035+0000","last_active":"2024-03-11T20:51:29.590814+0000","last_peered":"2024-03-11T20:51:29.590814+0000","last_clean":"2024-03-11T20:51:29.590814+0000","last_became_active":"2024-03-11T18:35:57.678565+0000","last_became_peered":"2024-03-11T18:35:57.678565+0000","last_unstale":"2024-03-11T20:51:29.590814+0000","last_undegraded":"2024-03-11T20:51:29.590814+0000","last_fullsized":"2024-03-11T20:51:29.590814+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:18:05.980207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"30'3437","reported_seq":3524,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.294958+0000","last_change":"2024-03-11T18:35:56.288132+0000","last_active":"2024-03-11T20:51:29.294958+0000","last_peered":"2024-03-11T20:51:29.294958+0000","last_clean":"2024-03-11T20:51:29.294958+0000","last_became_active":"2024-03-11T18:35:56.287846+0000","last_became_peered":"2024-03-11T18:35:56.287846+0000","last_unstale":"2024-03-11T20:51:29.294958+0000","last_undegraded":"2024-03-11T20:51:29.294958+0000","last_fullsized":"2024-03-11T20:51:29.294958+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":3437,"log_dups_size":0,"ondisk_log_size":3437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T18:51:00.795706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7567240,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3391,"num_write_kb":10463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"30'736","reported_seq":902,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:10.794831+0000","last_change":"2024-03-11T18:36:04.643164+0000","last_active":"2024-03-11T20:51:10.794831+0000","last_peered":"2024-03-11T20:51:10.794831+0000","last_clean":"2024-03-11T20:51:10.794831+0000","last_became_active":"2024-03-11T18:36:04.642374+0000","last_became_peered":"2024-03-11T18:36:04.642374+0000","last_unstale":"2024-03-11T20:51:10.794831+0000","last_undegraded":"2024-03-11T20:51:10.794831+0000","last_fullsized":"2024-03-11T20:51:10.794831+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":736,"log_dups_size":0,"ondisk_log_size":736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:25:56.815496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":40504,"num_write":721,"num_write_kb":107102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"30'197","reported_seq":282,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.866466+0000","last_change":"2024-03-11T18:35:57.636550+0000","last_active":"2024-03-11T20:51:29.866466+0000","last_peered":"2024-03-11T20:51:29.866466+0000","last_clean":"2024-03-11T20:51:29.866466+0000","last_became_active":"2024-03-11T18:35:57.636093+0000","last_became_peered":"2024-03-11T18:35:57.636093+0000","last_unstale":"2024-03-11T20:51:29.866466+0000","last_undegraded":"2024-03-11T20:51:29.866466+0000","last_fullsized":"2024-03-11T20:51:29.866466+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:33:31.344465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"30'1238","reported_seq":1325,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:50:53.074114+0000","last_change":"2024-03-11T18:35:56.355480+0000","last_active":"2024-03-11T20:50:53.074114+0000","last_peered":"2024-03-11T20:50:53.074114+0000","last_clean":"2024-03-11T20:50:53.074114+0000","last_became_active":"2024-03-11T18:35:56.354759+0000","last_became_peered":"2024-03-11T18:35:56.354759+0000","last_unstale":"2024-03-11T20:50:53.074114+0000","last_undegraded":"2024-03-11T20:50:53.074114+0000","last_fullsized":"2024-03-11T20:50:53.074114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1238,"log_dups_size":0,"ondisk_log_size":1238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:45:37.842725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1180,"num_write_kb":8784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"30'803","reported_seq":939,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:12.607404+0000","last_change":"2024-03-11T18:36:04.617364+0000","last_active":"2024-03-11T20:51:12.607404+0000","last_peered":"2024-03-11T20:51:12.607404+0000","last_clean":"2024-03-11T20:51:12.607404+0000","last_became_active":"2024-03-11T18:36:04.613159+0000","last_became_peered":"2024-03-11T18:36:04.613159+0000","last_unstale":"2024-03-11T20:51:12.607404+0000","last_undegraded":"2024-03-11T20:51:12.607404+0000","last_fullsized":"2024-03-11T20:51:12.607404+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":803,"log_dups_size":0,"ondisk_log_size":803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:25:23.672189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":38200,"num_write":794,"num_write_kb":117163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"30'218","reported_seq":303,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.626148+0000","last_change":"2024-03-11T18:35:57.683426+0000","last_active":"2024-03-11T20:51:29.626148+0000","last_peered":"2024-03-11T20:51:29.626148+0000","last_clean":"2024-03-11T20:51:29.626148+0000","last_became_active":"2024-03-11T18:35:57.683150+0000","last_became_peered":"2024-03-11T18:35:57.683150+0000","last_unstale":"2024-03-11T20:51:29.626148+0000","last_undegraded":"2024-03-11T20:51:29.626148+0000","last_fullsized":"2024-03-11T20:51:29.626148+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:36:00.079075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"30'63","reported_seq":150,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.294390+0000","last_change":"2024-03-11T18:35:56.361613+0000","last_active":"2024-03-11T20:51:29.294390+0000","last_peered":"2024-03-11T20:51:29.294390+0000","last_clean":"2024-03-11T20:51:29.294390+0000","last_became_active":"2024-03-11T18:35:56.359629+0000","last_became_peered":"2024-03-11T18:35:56.359629+0000","last_unstale":"2024-03-11T20:51:29.294390+0000","last_undegraded":"2024-03-11T20:51:29.294390+0000","last_fullsized":"2024-03-11T20:51:29.294390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:03:06.585728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"30'836","reported_seq":1036,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:12.888024+0000","last_change":"2024-03-11T18:36:04.608168+0000","last_active":"2024-03-11T20:51:12.888024+0000","last_peered":"2024-03-11T20:51:12.888024+0000","last_clean":"2024-03-11T20:51:12.888024+0000","last_became_active":"2024-03-11T18:36:04.607063+0000","last_became_peered":"2024-03-11T18:36:04.607063+0000","last_unstale":"2024-03-11T20:51:12.888024+0000","last_undegraded":"2024-03-11T20:51:12.888024+0000","last_fullsized":"2024-03-11T20:51:12.888024+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":836,"log_dups_size":0,"ondisk_log_size":836,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:53:41.996024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":124,"num_read_kb":69905,"num_write":818,"num_write_kb":155514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"30'815","reported_seq":1016,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:08.312913+0000","last_change":"2024-03-11T18:36:04.607992+0000","last_active":"2024-03-11T20:51:08.312913+0000","last_peered":"2024-03-11T20:51:08.312913+0000","last_clean":"2024-03-11T20:51:08.312913+0000","last_became_active":"2024-03-11T18:36:04.607109+0000","last_became_peered":"2024-03-11T18:36:04.607109+0000","last_unstale":"2024-03-11T20:51:08.312913+0000","last_undegraded":"2024-03-11T20:51:08.312913+0000","last_fullsized":"2024-03-11T20:51:08.312913+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":815,"log_dups_size":0,"ondisk_log_size":815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:14:37.678854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":116,"num_read_kb":66679,"num_write":801,"num_write_kb":181203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"30'218","reported_seq":302,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.672110+0000","last_change":"2024-03-11T18:35:57.680534+0000","last_active":"2024-03-11T20:51:29.672110+0000","last_peered":"2024-03-11T20:51:29.672110+0000","last_clean":"2024-03-11T20:51:29.672110+0000","last_became_active":"2024-03-11T18:35:57.679642+0000","last_became_peered":"2024-03-11T18:35:57.679642+0000","last_unstale":"2024-03-11T20:51:29.672110+0000","last_undegraded":"2024-03-11T20:51:29.672110+0000","last_fullsized":"2024-03-11T20:51:29.672110+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:25:49.707568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"30'1317","reported_seq":1404,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:17.349647+0000","last_change":"2024-03-11T18:35:56.303510+0000","last_active":"2024-03-11T20:51:17.349647+0000","last_peered":"2024-03-11T20:51:17.349647+0000","last_clean":"2024-03-11T20:51:17.349647+0000","last_became_active":"2024-03-11T18:35:56.302758+0000","last_became_peered":"2024-03-11T18:35:56.302758+0000","last_unstale":"2024-03-11T20:51:17.349647+0000","last_undegraded":"2024-03-11T20:51:17.349647+0000","last_fullsized":"2024-03-11T20:51:17.349647+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1317,"log_dups_size":0,"ondisk_log_size":1317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:03:32.297342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583494,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1254,"num_write_kb":12926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"30'766","reported_seq":981,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:50:55.156176+0000","last_change":"2024-03-11T18:36:04.618818+0000","last_active":"2024-03-11T20:50:55.156176+0000","last_peered":"2024-03-11T20:50:55.156176+0000","last_clean":"2024-03-11T20:50:55.156176+0000","last_became_active":"2024-03-11T18:36:04.617868+0000","last_became_peered":"2024-03-11T18:36:04.617868+0000","last_unstale":"2024-03-11T20:50:55.156176+0000","last_undegraded":"2024-03-11T20:50:55.156176+0000","last_fullsized":"2024-03-11T20:50:55.156176+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":766,"log_dups_size":0,"ondisk_log_size":766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:16:10.508913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":132,"num_read_kb":68723,"num_write":745,"num_write_kb":131559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"30'225","reported_seq":310,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.771467+0000","last_change":"2024-03-11T18:35:57.690347+0000","last_active":"2024-03-11T20:51:29.771467+0000","last_peered":"2024-03-11T20:51:29.771467+0000","last_clean":"2024-03-11T20:51:29.771467+0000","last_became_active":"2024-03-11T18:35:57.689146+0000","last_became_peered":"2024-03-11T18:35:57.689146+0000","last_unstale":"2024-03-11T20:51:29.771467+0000","last_undegraded":"2024-03-11T20:51:29.771467+0000","last_fullsized":"2024-03-11T20:51:29.771467+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:03:56.260394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"30'834","reported_seq":920,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:49:31.476898+0000","last_change":"2024-03-11T18:35:56.348355+0000","last_active":"2024-03-11T20:49:31.476898+0000","last_peered":"2024-03-11T20:49:31.476898+0000","last_clean":"2024-03-11T20:49:31.476898+0000","last_became_active":"2024-03-11T18:35:56.347631+0000","last_became_peered":"2024-03-11T18:35:56.347631+0000","last_unstale":"2024-03-11T20:49:31.476898+0000","last_undegraded":"2024-03-11T20:49:31.476898+0000","last_fullsized":"2024-03-11T20:49:31.476898+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":834,"log_dups_size":0,"ondisk_log_size":834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:06:02.844765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":793,"num_write_kb":4496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"30'1909","reported_seq":1994,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:49:42.823141+0000","last_change":"2024-03-11T18:35:56.361254+0000","last_active":"2024-03-11T20:49:42.823141+0000","last_peered":"2024-03-11T20:49:42.823141+0000","last_clean":"2024-03-11T20:49:42.823141+0000","last_became_active":"2024-03-11T18:35:56.360580+0000","last_became_peered":"2024-03-11T18:35:56.360580+0000","last_unstale":"2024-03-11T20:49:42.823141+0000","last_undegraded":"2024-03-11T20:49:42.823141+0000","last_fullsized":"2024-03-11T20:49:42.823141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1909,"log_dups_size":0,"ondisk_log_size":1909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:27:26.471550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1850,"num_write_kb":17317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"30'183","reported_seq":266,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.874784+0000","last_change":"2024-03-11T18:35:57.602778+0000","last_active":"2024-03-11T20:51:29.874784+0000","last_peered":"2024-03-11T20:51:29.874784+0000","last_clean":"2024-03-11T20:51:29.874784+0000","last_became_active":"2024-03-11T18:35:57.601990+0000","last_became_peered":"2024-03-11T18:35:57.601990+0000","last_unstale":"2024-03-11T20:51:29.874784+0000","last_undegraded":"2024-03-11T20:51:29.874784+0000","last_fullsized":"2024-03-11T20:51:29.874784+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:56:59.014871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"30'692","reported_seq":830,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:08.239134+0000","last_change":"2024-03-11T18:36:04.606098+0000","last_active":"2024-03-11T20:51:08.239134+0000","last_peered":"2024-03-11T20:51:08.239134+0000","last_clean":"2024-03-11T20:51:08.239134+0000","last_became_active":"2024-03-11T18:36:04.604823+0000","last_became_peered":"2024-03-11T18:36:04.604823+0000","last_unstale":"2024-03-11T20:51:08.239134+0000","last_undegraded":"2024-03-11T20:51:08.239134+0000","last_fullsized":"2024-03-11T20:51:08.239134+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:52:04.862452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":66,"num_read_kb":37267,"num_write":684,"num_write_kb":109032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"30'332","reported_seq":417,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:50:56.328124+0000","last_change":"2024-03-11T18:35:56.296312+0000","last_active":"2024-03-11T20:50:56.328124+0000","last_peered":"2024-03-11T20:50:56.328124+0000","last_clean":"2024-03-11T20:50:56.328124+0000","last_became_active":"2024-03-11T18:35:56.296090+0000","last_became_peered":"2024-03-11T18:35:56.296090+0000","last_unstale":"2024-03-11T20:50:56.328124+0000","last_undegraded":"2024-03-11T20:50:56.328124+0000","last_fullsized":"2024-03-11T20:50:56.328124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:03:06.585728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4195508,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":278,"num_write_kb":4222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"30'199","reported_seq":282,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.579273+0000","last_change":"2024-03-11T18:35:57.611809+0000","last_active":"2024-03-11T20:51:29.579273+0000","last_peered":"2024-03-11T20:51:29.579273+0000","last_clean":"2024-03-11T20:51:29.579273+0000","last_became_active":"2024-03-11T18:35:57.611097+0000","last_became_peered":"2024-03-11T18:35:57.611097+0000","last_unstale":"2024-03-11T20:51:29.579273+0000","last_undegraded":"2024-03-11T20:51:29.579273+0000","last_fullsized":"2024-03-11T20:51:29.579273+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:49:29.042114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"30'703","reported_seq":834,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:06.329767+0000","last_change":"2024-03-11T18:36:04.590064+0000","last_active":"2024-03-11T20:51:06.329767+0000","last_peered":"2024-03-11T20:51:06.329767+0000","last_clean":"2024-03-11T20:51:06.329767+0000","last_became_active":"2024-03-11T18:36:04.589037+0000","last_became_peered":"2024-03-11T18:36:04.589037+0000","last_unstale":"2024-03-11T20:51:06.329767+0000","last_undegraded":"2024-03-11T20:51:06.329767+0000","last_fullsized":"2024-03-11T20:51:06.329767+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:51:28.017349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":59,"num_read_kb":23261,"num_write":682,"num_write_kb":69804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"30'217","reported_seq":311,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.960628+0000","last_change":"2024-03-11T18:35:57.614944+0000","last_active":"2024-03-11T20:51:29.960628+0000","last_peered":"2024-03-11T20:51:29.960628+0000","last_clean":"2024-03-11T20:51:29.960628+0000","last_became_active":"2024-03-11T18:35:57.614144+0000","last_became_peered":"2024-03-11T18:35:57.614144+0000","last_unstale":"2024-03-11T20:51:29.960628+0000","last_undegraded":"2024-03-11T20:51:29.960628+0000","last_fullsized":"2024-03-11T20:51:29.960628+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:53:26.493265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"30'222","reported_seq":316,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.797673+0000","last_change":"2024-03-11T18:35:57.630211+0000","last_active":"2024-03-11T20:51:29.797673+0000","last_peered":"2024-03-11T20:51:29.797673+0000","last_clean":"2024-03-11T20:51:29.797673+0000","last_became_active":"2024-03-11T18:35:57.629599+0000","last_became_peered":"2024-03-11T18:35:57.629599+0000","last_unstale":"2024-03-11T20:51:29.797673+0000","last_undegraded":"2024-03-11T20:51:29.797673+0000","last_fullsized":"2024-03-11T20:51:29.797673+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":222,"log_dups_size":0,"ondisk_log_size":222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:18:05.980207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"30'236","reported_seq":330,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.602616+0000","last_change":"2024-03-11T18:35:57.688700+0000","last_active":"2024-03-11T20:51:29.602616+0000","last_peered":"2024-03-11T20:51:29.602616+0000","last_clean":"2024-03-11T20:51:29.602616+0000","last_became_active":"2024-03-11T18:35:57.687968+0000","last_became_peered":"2024-03-11T18:35:57.687968+0000","last_unstale":"2024-03-11T20:51:29.602616+0000","last_undegraded":"2024-03-11T20:51:29.602616+0000","last_fullsized":"2024-03-11T20:51:29.602616+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":236,"log_dups_size":0,"ondisk_log_size":236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:25:16.564261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"30'203","reported_seq":296,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.872107+0000","last_change":"2024-03-11T18:35:57.680730+0000","last_active":"2024-03-11T20:51:29.872107+0000","last_peered":"2024-03-11T20:51:29.872107+0000","last_clean":"2024-03-11T20:51:29.872107+0000","last_became_active":"2024-03-11T18:35:57.679935+0000","last_became_peered":"2024-03-11T18:35:57.679935+0000","last_unstale":"2024-03-11T20:51:29.872107+0000","last_undegraded":"2024-03-11T20:51:29.872107+0000","last_fullsized":"2024-03-11T20:51:29.872107+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:21:21.894805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"30'184","reported_seq":275,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.764136+0000","last_change":"2024-03-11T18:35:57.690799+0000","last_active":"2024-03-11T20:51:29.764136+0000","last_peered":"2024-03-11T20:51:29.764136+0000","last_clean":"2024-03-11T20:51:29.764136+0000","last_became_active":"2024-03-11T18:35:57.686623+0000","last_became_peered":"2024-03-11T18:35:57.686623+0000","last_unstale":"2024-03-11T20:51:29.764136+0000","last_undegraded":"2024-03-11T20:51:29.764136+0000","last_fullsized":"2024-03-11T20:51:29.764136+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:53:30.324898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"30'208","reported_seq":299,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.659529+0000","last_change":"2024-03-11T18:35:57.691955+0000","last_active":"2024-03-11T20:51:29.659529+0000","last_peered":"2024-03-11T20:51:29.659529+0000","last_clean":"2024-03-11T20:51:29.659529+0000","last_became_active":"2024-03-11T18:35:57.691413+0000","last_became_peered":"2024-03-11T18:35:57.691413+0000","last_unstale":"2024-03-11T20:51:29.659529+0000","last_undegraded":"2024-03-11T20:51:29.659529+0000","last_fullsized":"2024-03-11T20:51:29.659529+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:31:19.941345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"30'244","reported_seq":335,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.963800+0000","last_change":"2024-03-11T18:35:57.684695+0000","last_active":"2024-03-11T20:51:29.963800+0000","last_peered":"2024-03-11T20:51:29.963800+0000","last_clean":"2024-03-11T20:51:29.963800+0000","last_became_active":"2024-03-11T18:35:57.684227+0000","last_became_peered":"2024-03-11T18:35:57.684227+0000","last_unstale":"2024-03-11T20:51:29.963800+0000","last_undegraded":"2024-03-11T20:51:29.963800+0000","last_fullsized":"2024-03-11T20:51:29.963800+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":244,"log_dups_size":0,"ondisk_log_size":244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:28:51.265291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"30'193","reported_seq":284,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.760629+0000","last_change":"2024-03-11T18:35:57.683549+0000","last_active":"2024-03-11T20:51:29.760629+0000","last_peered":"2024-03-11T20:51:29.760629+0000","last_clean":"2024-03-11T20:51:29.760629+0000","last_became_active":"2024-03-11T18:35:57.680610+0000","last_became_peered":"2024-03-11T18:35:57.680610+0000","last_unstale":"2024-03-11T20:51:29.760629+0000","last_undegraded":"2024-03-11T20:51:29.760629+0000","last_fullsized":"2024-03-11T20:51:29.760629+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:25:52.716744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"30'218","reported_seq":289,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.791145+0000","last_change":"2024-03-11T18:37:04.264086+0000","last_active":"2024-03-11T20:51:29.791145+0000","last_peered":"2024-03-11T20:51:29.791145+0000","last_clean":"2024-03-11T20:51:29.791145+0000","last_became_active":"2024-03-11T18:37:04.263764+0000","last_became_peered":"2024-03-11T18:37:04.263764+0000","last_unstale":"2024-03-11T20:51:29.791145+0000","last_undegraded":"2024-03-11T20:51:29.791145+0000","last_fullsized":"2024-03-11T20:51:29.791145+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:34:39.325572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"30'205","reported_seq":294,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.764435+0000","last_change":"2024-03-11T18:35:57.681630+0000","last_active":"2024-03-11T20:51:29.764435+0000","last_peered":"2024-03-11T20:51:29.764435+0000","last_clean":"2024-03-11T20:51:29.764435+0000","last_became_active":"2024-03-11T18:35:57.678876+0000","last_became_peered":"2024-03-11T18:35:57.678876+0000","last_unstale":"2024-03-11T20:51:29.764435+0000","last_undegraded":"2024-03-11T20:51:29.764435+0000","last_fullsized":"2024-03-11T20:51:29.764435+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:03:33.511691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"30'208","reported_seq":297,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.623113+0000","last_change":"2024-03-11T18:35:57.679589+0000","last_active":"2024-03-11T20:51:29.623113+0000","last_peered":"2024-03-11T20:51:29.623113+0000","last_clean":"2024-03-11T20:51:29.623113+0000","last_became_active":"2024-03-11T18:35:57.678838+0000","last_became_peered":"2024-03-11T18:35:57.678838+0000","last_unstale":"2024-03-11T20:51:29.623113+0000","last_undegraded":"2024-03-11T20:51:29.623113+0000","last_fullsized":"2024-03-11T20:51:29.623113+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:54:09.699371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"30'213","reported_seq":302,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.872622+0000","last_change":"2024-03-11T18:35:57.674571+0000","last_active":"2024-03-11T20:51:29.872622+0000","last_peered":"2024-03-11T20:51:29.872622+0000","last_clean":"2024-03-11T20:51:29.872622+0000","last_became_active":"2024-03-11T18:35:57.671573+0000","last_became_peered":"2024-03-11T18:35:57.671573+0000","last_unstale":"2024-03-11T20:51:29.872622+0000","last_undegraded":"2024-03-11T20:51:29.872622+0000","last_fullsized":"2024-03-11T20:51:29.872622+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T20:35:10.403666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"30'221","reported_seq":310,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.959781+0000","last_change":"2024-03-11T18:35:57.615939+0000","last_active":"2024-03-11T20:51:29.959781+0000","last_peered":"2024-03-11T20:51:29.959781+0000","last_clean":"2024-03-11T20:51:29.959781+0000","last_became_active":"2024-03-11T18:35:57.615579+0000","last_became_peered":"2024-03-11T18:35:57.615579+0000","last_unstale":"2024-03-11T20:51:29.959781+0000","last_undegraded":"2024-03-11T20:51:29.959781+0000","last_fullsized":"2024-03-11T20:51:29.959781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:03:33.511691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"30'206","reported_seq":295,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.632447+0000","last_change":"2024-03-11T18:35:57.688955+0000","last_active":"2024-03-11T20:51:29.632447+0000","last_peered":"2024-03-11T20:51:29.632447+0000","last_clean":"2024-03-11T20:51:29.632447+0000","last_became_active":"2024-03-11T18:35:57.688365+0000","last_became_peered":"2024-03-11T18:35:57.688365+0000","last_unstale":"2024-03-11T20:51:29.632447+0000","last_undegraded":"2024-03-11T20:51:29.632447+0000","last_fullsized":"2024-03-11T20:51:29.632447+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:44:11.877541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"30'205","reported_seq":292,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.795195+0000","last_change":"2024-03-11T18:35:57.602044+0000","last_active":"2024-03-11T20:51:29.795195+0000","last_peered":"2024-03-11T20:51:29.795195+0000","last_clean":"2024-03-11T20:51:29.795195+0000","last_became_active":"2024-03-11T18:35:57.601334+0000","last_became_peered":"2024-03-11T18:35:57.601334+0000","last_unstale":"2024-03-11T20:51:29.795195+0000","last_undegraded":"2024-03-11T20:51:29.795195+0000","last_fullsized":"2024-03-11T20:51:29.795195+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:39:12.693619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"30'206","reported_seq":293,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.612090+0000","last_change":"2024-03-11T18:35:57.677114+0000","last_active":"2024-03-11T20:51:29.612090+0000","last_peered":"2024-03-11T20:51:29.612090+0000","last_clean":"2024-03-11T20:51:29.612090+0000","last_became_active":"2024-03-11T18:35:57.676075+0000","last_became_peered":"2024-03-11T18:35:57.676075+0000","last_unstale":"2024-03-11T20:51:29.612090+0000","last_undegraded":"2024-03-11T20:51:29.612090+0000","last_fullsized":"2024-03-11T20:51:29.612090+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:44:11.877541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"30'219","reported_seq":306,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.614892+0000","last_change":"2024-03-11T18:35:57.641392+0000","last_active":"2024-03-11T20:51:29.614892+0000","last_peered":"2024-03-11T20:51:29.614892+0000","last_clean":"2024-03-11T20:51:29.614892+0000","last_became_active":"2024-03-11T18:35:57.640934+0000","last_became_peered":"2024-03-11T18:35:57.640934+0000","last_unstale":"2024-03-11T20:51:29.614892+0000","last_undegraded":"2024-03-11T20:51:29.614892+0000","last_fullsized":"2024-03-11T20:51:29.614892+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:25:52.716744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"30'213","reported_seq":300,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.664702+0000","last_change":"2024-03-11T18:35:57.605291+0000","last_active":"2024-03-11T20:51:29.664702+0000","last_peered":"2024-03-11T20:51:29.664702+0000","last_clean":"2024-03-11T20:51:29.664702+0000","last_became_active":"2024-03-11T18:35:57.604645+0000","last_became_peered":"2024-03-11T18:35:57.604645+0000","last_unstale":"2024-03-11T20:51:29.664702+0000","last_undegraded":"2024-03-11T20:51:29.664702+0000","last_fullsized":"2024-03-11T20:51:29.664702+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:03:33.511691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"30'210","reported_seq":296,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.888659+0000","last_change":"2024-03-11T18:35:57.503994+0000","last_active":"2024-03-11T20:51:29.888659+0000","last_peered":"2024-03-11T20:51:29.888659+0000","last_clean":"2024-03-11T20:51:29.888659+0000","last_became_active":"2024-03-11T18:35:57.503362+0000","last_became_peered":"2024-03-11T18:35:57.503362+0000","last_unstale":"2024-03-11T20:51:29.888659+0000","last_undegraded":"2024-03-11T20:51:29.888659+0000","last_fullsized":"2024-03-11T20:51:29.888659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:03:58.875164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"30'195","reported_seq":260,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.789716+0000","last_change":"2024-03-11T18:37:04.252064+0000","last_active":"2024-03-11T20:51:29.789716+0000","last_peered":"2024-03-11T20:51:29.789716+0000","last_clean":"2024-03-11T20:51:29.789716+0000","last_became_active":"2024-03-11T18:37:04.251092+0000","last_became_peered":"2024-03-11T18:37:04.251092+0000","last_unstale":"2024-03-11T20:51:29.789716+0000","last_undegraded":"2024-03-11T20:51:29.789716+0000","last_fullsized":"2024-03-11T20:51:29.789716+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:29:45.662029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"30'181","reported_seq":266,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.575251+0000","last_change":"2024-03-11T18:35:57.613374+0000","last_active":"2024-03-11T20:51:29.575251+0000","last_peered":"2024-03-11T20:51:29.575251+0000","last_clean":"2024-03-11T20:51:29.575251+0000","last_became_active":"2024-03-11T18:35:57.613050+0000","last_became_peered":"2024-03-11T18:35:57.613050+0000","last_unstale":"2024-03-11T20:51:29.575251+0000","last_undegraded":"2024-03-11T20:51:29.575251+0000","last_fullsized":"2024-03-11T20:51:29.575251+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:50:39.923881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"30'186","reported_seq":271,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.959316+0000","last_change":"2024-03-11T18:35:57.636565+0000","last_active":"2024-03-11T20:51:29.959316+0000","last_peered":"2024-03-11T20:51:29.959316+0000","last_clean":"2024-03-11T20:51:29.959316+0000","last_became_active":"2024-03-11T18:35:57.635978+0000","last_became_peered":"2024-03-11T18:35:57.635978+0000","last_unstale":"2024-03-11T20:51:29.959316+0000","last_undegraded":"2024-03-11T20:51:29.959316+0000","last_fullsized":"2024-03-11T20:51:29.959316+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:03:56.260394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"30'206","reported_seq":291,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.964837+0000","last_change":"2024-03-11T18:35:57.641714+0000","last_active":"2024-03-11T20:51:29.964837+0000","last_peered":"2024-03-11T20:51:29.964837+0000","last_clean":"2024-03-11T20:51:29.964837+0000","last_became_active":"2024-03-11T18:35:57.641417+0000","last_became_peered":"2024-03-11T18:35:57.641417+0000","last_unstale":"2024-03-11T20:51:29.964837+0000","last_undegraded":"2024-03-11T20:51:29.964837+0000","last_fullsized":"2024-03-11T20:51:29.964837+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:25:49.707568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"30'210","reported_seq":294,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.626373+0000","last_change":"2024-03-11T18:35:57.689703+0000","last_active":"2024-03-11T20:51:29.626373+0000","last_peered":"2024-03-11T20:51:29.626373+0000","last_clean":"2024-03-11T20:51:29.626373+0000","last_became_active":"2024-03-11T18:35:57.689056+0000","last_became_peered":"2024-03-11T18:35:57.689056+0000","last_unstale":"2024-03-11T20:51:29.626373+0000","last_undegraded":"2024-03-11T20:51:29.626373+0000","last_fullsized":"2024-03-11T20:51:29.626373+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:41:42.263652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"30'194","reported_seq":277,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.772230+0000","last_change":"2024-03-11T18:35:57.684305+0000","last_active":"2024-03-11T20:51:29.772230+0000","last_peered":"2024-03-11T20:51:29.772230+0000","last_clean":"2024-03-11T20:51:29.772230+0000","last_became_active":"2024-03-11T18:35:57.682947+0000","last_became_peered":"2024-03-11T18:35:57.682947+0000","last_unstale":"2024-03-11T20:51:29.772230+0000","last_undegraded":"2024-03-11T20:51:29.772230+0000","last_fullsized":"2024-03-11T20:51:29.772230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:53:26.493265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"30'210","reported_seq":293,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.585163+0000","last_change":"2024-03-11T18:35:57.692717+0000","last_active":"2024-03-11T20:51:29.585163+0000","last_peered":"2024-03-11T20:51:29.585163+0000","last_clean":"2024-03-11T20:51:29.585163+0000","last_became_active":"2024-03-11T18:35:57.692504+0000","last_became_peered":"2024-03-11T18:35:57.692504+0000","last_unstale":"2024-03-11T20:51:29.585163+0000","last_undegraded":"2024-03-11T20:51:29.585163+0000","last_fullsized":"2024-03-11T20:51:29.585163+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:25:52.716744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"30'194","reported_seq":277,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.612590+0000","last_change":"2024-03-11T18:35:57.640483+0000","last_active":"2024-03-11T20:51:29.612590+0000","last_peered":"2024-03-11T20:51:29.612590+0000","last_clean":"2024-03-11T20:51:29.612590+0000","last_became_active":"2024-03-11T18:35:57.639886+0000","last_became_peered":"2024-03-11T18:35:57.639886+0000","last_unstale":"2024-03-11T20:51:29.612590+0000","last_undegraded":"2024-03-11T20:51:29.612590+0000","last_fullsized":"2024-03-11T20:51:29.612590+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T19:34:39.325572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"30'212","reported_seq":295,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.603982+0000","last_change":"2024-03-11T18:35:57.687096+0000","last_active":"2024-03-11T20:51:29.603982+0000","last_peered":"2024-03-11T20:51:29.603982+0000","last_clean":"2024-03-11T20:51:29.603982+0000","last_became_active":"2024-03-11T18:35:57.686771+0000","last_became_peered":"2024-03-11T18:35:57.686771+0000","last_unstale":"2024-03-11T20:51:29.603982+0000","last_undegraded":"2024-03-11T20:51:29.603982+0000","last_fullsized":"2024-03-11T20:51:29.603982+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:16:03.400985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"30'196","reported_seq":292,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.628723+0000","last_change":"2024-03-11T18:35:57.694707+0000","last_active":"2024-03-11T20:51:29.628723+0000","last_peered":"2024-03-11T20:51:29.628723+0000","last_clean":"2024-03-11T20:51:29.628723+0000","last_became_active":"2024-03-11T18:35:57.694104+0000","last_became_peered":"2024-03-11T18:35:57.694104+0000","last_unstale":"2024-03-11T20:51:29.628723+0000","last_undegraded":"2024-03-11T20:51:29.628723+0000","last_fullsized":"2024-03-11T20:51:29.628723+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":196,"log_dups_size":0,"ondisk_log_size":196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:47:28.579467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"30'191","reported_seq":285,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.874739+0000","last_change":"2024-03-11T18:35:57.538548+0000","last_active":"2024-03-11T20:51:29.874739+0000","last_peered":"2024-03-11T20:51:29.874739+0000","last_clean":"2024-03-11T20:51:29.874739+0000","last_became_active":"2024-03-11T18:35:57.537704+0000","last_became_peered":"2024-03-11T18:35:57.537704+0000","last_unstale":"2024-03-11T20:51:29.874739+0000","last_undegraded":"2024-03-11T20:51:29.874739+0000","last_fullsized":"2024-03-11T20:51:29.874739+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:40:52.251981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"30'214","reported_seq":308,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.659184+0000","last_change":"2024-03-11T18:35:57.507126+0000","last_active":"2024-03-11T20:51:29.659184+0000","last_peered":"2024-03-11T20:51:29.659184+0000","last_clean":"2024-03-11T20:51:29.659184+0000","last_became_active":"2024-03-11T18:35:57.506429+0000","last_became_peered":"2024-03-11T18:35:57.506429+0000","last_unstale":"2024-03-11T20:51:29.659184+0000","last_undegraded":"2024-03-11T20:51:29.659184+0000","last_fullsized":"2024-03-11T20:51:29.659184+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:28:51.265291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"30'213","reported_seq":307,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.608987+0000","last_change":"2024-03-11T18:35:57.687817+0000","last_active":"2024-03-11T20:51:29.608987+0000","last_peered":"2024-03-11T20:51:29.608987+0000","last_clean":"2024-03-11T20:51:29.608987+0000","last_became_active":"2024-03-11T18:35:57.687532+0000","last_became_peered":"2024-03-11T18:35:57.687532+0000","last_unstale":"2024-03-11T20:51:29.608987+0000","last_undegraded":"2024-03-11T20:51:29.608987+0000","last_fullsized":"2024-03-11T20:51:29.608987+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:47:28.579467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2961,"num_read_kb":1579043,"num_write":24379,"num_write_kb":4188254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24824,"ondisk_log_size":24824,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13197,"ondisk_log_size":13197,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":210457699,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":4697,"num_read_kb":5145,"num_write":47510,"num_write_kb":229688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":421158912,"data_stored":420915398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38275,"ondisk_log_size":38275,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719478379,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":200100,"kb_used_data":78748,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94171740,"statfs":{"total":96636764160,"available":96431861760,"internally_reserved":0,"allocated":80637952,"data_stored":80415478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":124254666},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":16,"seq":68719478379,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":245524,"kb_used_data":61772,"kb_used_omap":1,"kb_used_meta":183742,"kb_avail":94126316,"statfs":{"total":96636764160,"available":96385347584,"internally_reserved":0,"allocated":63254528,"data_stored":63047256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":188152262},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":16,"seq":68719478377,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":200588,"kb_used_data":48708,"kb_used_omap":1,"kb_used_meta":151870,"kb_avail":94171252,"statfs":{"total":96636764160,"available":96431362048,"internally_reserved":0,"allocated":49876992,"data_stored":49644774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":155515334},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":16,"seq":68719478378,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":183188,"kb_used_data":57676,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94188652,"statfs":{"total":96636764160,"available":96449179648,"internally_reserved":0,"allocated":59060224,"data_stored":58852820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424511081,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":218248,"kb_used_data":42816,"kb_used_omap":1,"kb_used_meta":175422,"kb_avail":94153592,"statfs":{"total":96636764160,"available":96413278208,"internally_reserved":0,"allocated":43843584,"data_stored":43631469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":179632583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424511081,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":152012,"kb_used_data":22340,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94219828,"statfs":{"total":96636764160,"available":96481103872,"internally_reserved":0,"allocated":22876160,"data_stored":22659915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424511082,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":228216,"kb_used_data":56880,"kb_used_omap":1,"kb_used_meta":171326,"kb_avail":94143624,"statfs":{"total":96636764160,"available":96403070976,"internally_reserved":0,"allocated":58245120,"data_stored":58030400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":175438280},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129543783,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149276,"kb_used_data":45972,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94222564,"statfs":{"total":96636764160,"available":96483905536,"internally_reserved":0,"allocated":47075328,"data_stored":46859238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":46166016,"data_stored":46138218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57929728,"data_stored":57899748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":22560768,"data_stored":22529263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":43528192,"data_stored":43500817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":58744832,"data_stored":58722168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49561600,"data_stored":49514122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":62939136,"data_stored":62916604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":79728640,"data_stored":79694458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-11T20:51:37.750 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T20:51:37.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.920+0000 7f12e86e9640 1 Processor -- start 2024-03-11T20:51:37.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.920+0000 7f12e86e9640 1 -- start start 2024-03-11T20:51:37.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.920+0000 7f12e86e9640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f12e0151c70 0x7f12e006bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:37.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.920+0000 7f12e86e9640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f12e0066d60 0x7f12e006c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:37.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.920+0000 7f12e86e9640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12e00c6720 con 0x7f12e00c7f20 2024-03-11T20:51:37.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.920+0000 7f12e86e9640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12e00cc7b0 con 0x7f12e0151c70 2024-03-11T20:51:37.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.920+0000 7f12e86e9640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12e00c5d20 con 0x7f12e0066d60 2024-03-11T20:51:37.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.940+0000 7f12e5c84640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f12e00c7f20 0x7f12e0157ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:57706/0 (socket says 172.21.15.90:57706) 2024-03-11T20:51:37.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.940+0000 7f12e5c84640 1 -- 172.21.15.90:0/3021453535 learned_addr learned my addr 172.21.15.90:0/3021453535 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T20:51:37.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.944+0000 7f12e5483640 1 -- 172.21.15.90:0/3021453535 <== mon.0 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 501115278 0 0) 0x7f12e00c6720 con 0x7f12e00c7f20 2024-03-11T20:51:37.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.944+0000 7f12e5483640 1 -- 172.21.15.90:0/3021453535 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12cc003530 con 0x7f12e00c7f20 2024-03-11T20:51:37.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.948+0000 7f12e5483640 1 -- 172.21.15.90:0/3021453535 <== mon.0 v1:172.21.15.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4134417764 0 0) 0x7f12cc003530 con 0x7f12e00c7f20 2024-03-11T20:51:37.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.948+0000 7f12e5483640 1 -- 172.21.15.90:0/3021453535 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f12e0066d60 msgr2=0x7f12e006c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:37.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.948+0000 7f12e5483640 1 --2- 172.21.15.90:0/3021453535 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f12e0066d60 0x7f12e006c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:37.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.948+0000 7f12e5483640 1 -- 172.21.15.90:0/3021453535 >> v2:172.21.15.92:3300/0 conn(0x7f12e0151c70 msgr2=0x7f12e006bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:37.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.948+0000 7f12e5483640 1 --2- 172.21.15.90:0/3021453535 >> v2:172.21.15.92:3300/0 conn(0x7f12e0151c70 0x7f12e006bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:37.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.948+0000 7f12e5483640 1 -- 172.21.15.90:0/3021453535 --> v1:172.21.15.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e00c5a40 con 0x7f12e00c7f20 2024-03-11T20:51:37.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.948+0000 7f12e5483640 1 -- 172.21.15.90:0/3021453535 <== mon.0 v1:172.21.15.90:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 691735435 0 0) 0x7f12d0001680 con 0x7f12e00c7f20 2024-03-11T20:51:37.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.948+0000 7f12e5483640 1 -- 172.21.15.90:0/3021453535 >> v1:172.21.15.90:6789/0 conn(0x7f12e00c7f20 legacy=0x7f12e0157ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:37.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.948+0000 7f12e5483640 1 --2- 172.21.15.90:0/3021453535 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f12e0066d60 0x7f12cc003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:37.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.948+0000 7f12e5483640 1 --2- 172.21.15.90:0/3021453535 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12cc0040e0 0x7f12cc0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:37.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.948+0000 7f12e5483640 1 --2- 172.21.15.90:0/3021453535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12cc004f10 0x7f12cc00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:37.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.948+0000 7f12e5483640 1 -- 172.21.15.90:0/3021453535 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e00cc7b0 con 0x7f12cc0040e0 2024-03-11T20:51:37.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.948+0000 7f12e5483640 1 -- 172.21.15.90:0/3021453535 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e00c5d20 con 0x7f12cc004f10 2024-03-11T20:51:37.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.948+0000 7f12e5483640 1 -- 172.21.15.90:0/3021453535 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc00bed0 con 0x7f12e0066d60 2024-03-11T20:51:37.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.952+0000 7f12e5c84640 1 --2- 172.21.15.90:0/3021453535 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12cc0040e0 0x7f12cc0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:37.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.956+0000 7f12e5c84640 1 -- 172.21.15.90:0/3021453535 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f12e0066d60 msgr2=0x7f12cc003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:37.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.956+0000 7f12e5c84640 1 --2- 172.21.15.90:0/3021453535 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f12e0066d60 0x7f12cc003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:37.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.956+0000 7f12e5c84640 1 -- 172.21.15.90:0/3021453535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12cc004f10 msgr2=0x7f12cc00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:37.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.956+0000 7f12e5c84640 1 --2- 172.21.15.90:0/3021453535 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12cc004f10 0x7f12cc00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:37.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.956+0000 7f12e5c84640 1 -- 172.21.15.90:0/3021453535 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12cc003530 con 0x7f12cc0040e0 2024-03-11T20:51:37.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.960+0000 7f12e5c84640 1 --2- 172.21.15.90:0/3021453535 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12cc0040e0 0x7f12cc0047d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f12e00710b0 tx=0x7f12d001caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef60a913a879a1d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:37.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.964+0000 7f12e5483640 1 -- 172.21.15.90:0/3021453535 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d00058e0 con 0x7f12cc0040e0 2024-03-11T20:51:37.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.964+0000 7f12e5483640 1 -- 172.21.15.90:0/3021453535 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12d0005a40 con 0x7f12cc0040e0 2024-03-11T20:51:37.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.964+0000 7f12e5483640 1 -- 172.21.15.90:0/3021453535 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d0005e30 con 0x7f12cc0040e0 2024-03-11T20:51:37.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.964+0000 7f12e86e9640 1 -- 172.21.15.90:0/3021453535 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12cc0040e0 msgr2=0x7f12cc0047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:37.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.964+0000 7f12e86e9640 1 --2- 172.21.15.90:0/3021453535 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12cc0040e0 0x7f12cc0047d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f12e00710b0 tx=0x7f12d001caf0 comp rx=0 tx=0).stop 2024-03-11T20:51:37.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.964+0000 7f12e6c86640 1 -- 172.21.15.90:0/3021453535 reap_dead start 2024-03-11T20:51:37.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.964+0000 7f12e86e9640 1 -- 172.21.15.90:0/3021453535 shutdown_connections 2024-03-11T20:51:37.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.964+0000 7f12e86e9640 1 -- 172.21.15.90:0/3021453535 >> 172.21.15.90:0/3021453535 conn(0x7f12e00cb880 msgr2=0x7f12e00cbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:37.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.964+0000 7f12e86e9640 1 -- 172.21.15.90:0/3021453535 shutdown_connections 2024-03-11T20:51:37.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.964+0000 7f12e86e9640 1 -- 172.21.15.90:0/3021453535 wait complete. 2024-03-11T20:51:37.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.968+0000 7f12e86e9640 1 Processor -- start 2024-03-11T20:51:37.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.968+0000 7f12e86e9640 1 -- start start 2024-03-11T20:51:37.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.968+0000 7f12e86e9640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12cc0040e0 0x7f12e0156f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:37.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.968+0000 7f12e86e9640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f12e00c2d80 0x7f12e0157440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:37.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.968+0000 7f12e86e9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12e0147de0 0x7f12e01f64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:37.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.968+0000 7f12e86e9640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e0156a70 con 0x7f12cc0040e0 2024-03-11T20:51:37.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.968+0000 7f12e86e9640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e0147020 con 0x7f12e0147de0 2024-03-11T20:51:37.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.968+0000 7f12e86e9640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12e01472e0 con 0x7f12e00c2d80 2024-03-11T20:51:37.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.968+0000 7f12e6485640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12cc0040e0 0x7f12e0156f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:37.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.968+0000 7f12e6485640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12cc0040e0 0x7f12e0156f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58540/0 (socket says 172.21.15.90:58540) 2024-03-11T20:51:37.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.968+0000 7f12e6485640 1 -- 172.21.15.90:0/3840937330 learned_addr learned my addr 172.21.15.90:0/3840937330 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:51:37.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.972+0000 7f12e6485640 1 -- 172.21.15.90:0/3840937330 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f12e00c2d80 msgr2=0x7f12e0157440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:37.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.972+0000 7f12e6485640 1 --2- 172.21.15.90:0/3840937330 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f12e00c2d80 0x7f12e0157440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:37.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.972+0000 7f12e6485640 1 -- 172.21.15.90:0/3840937330 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12e0147de0 msgr2=0x7f12e01f64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:37.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.972+0000 7f12e6485640 1 --2- 172.21.15.90:0/3840937330 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12e0147de0 0x7f12e01f64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:37.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.972+0000 7f12e6485640 1 -- 172.21.15.90:0/3840937330 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e01475c0 con 0x7f12cc0040e0 2024-03-11T20:51:37.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.976+0000 7f12e6485640 1 --2- 172.21.15.90:0/3840937330 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12cc0040e0 0x7f12e0156f00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f12dc00b520 tx=0x7f12dc00b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:37.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.980+0000 7f12d6ffd640 1 -- 172.21.15.90:0/3840937330 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12dc010890 con 0x7f12cc0040e0 2024-03-11T20:51:37.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.980+0000 7f12d6ffd640 1 -- 172.21.15.90:0/3840937330 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12dc0109f0 con 0x7f12cc0040e0 2024-03-11T20:51:37.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.980+0000 7f12d6ffd640 1 -- 172.21.15.90:0/3840937330 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12dc010da0 con 0x7f12cc0040e0 2024-03-11T20:51:37.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.980+0000 7f12e86e9640 1 -- 172.21.15.90:0/3840937330 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12e0156700 con 0x7f12cc0040e0 2024-03-11T20:51:37.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.980+0000 7f12e86e9640 1 -- 172.21.15.90:0/3840937330 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12e01f6a00 con 0x7f12cc0040e0 2024-03-11T20:51:37.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:37.980+0000 7f12e86e9640 1 -- 172.21.15.90:0/3840937330 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12e0156560 con 0x7f12cc0040e0 2024-03-11T20:51:38.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:38.000+0000 7f12d6ffd640 1 -- 172.21.15.90:0/3840937330 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f12dc023480 con 0x7f12cc0040e0 2024-03-11T20:51:38.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:38.004+0000 7f12d6ffd640 1 --2- 172.21.15.90:0/3840937330 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f12c4066ec0 0x7f12c4069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:38.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:38.004+0000 7f12e5c84640 1 --2- 172.21.15.90:0/3840937330 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f12c4066ec0 0x7f12c4069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:38.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:38.004+0000 7f12e5c84640 1 --2- 172.21.15.90:0/3840937330 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f12c4066ec0 0x7f12c4069380 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f12d00aa000 tx=0x7f12e006e110 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:38.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:38.308+0000 7f12d6ffd640 1 -- 172.21.15.90:0/3840937330 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f12dc055fb0 con 0x7f12cc0040e0 2024-03-11T20:51:38.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:38.308+0000 7f12d6ffd640 1 -- 172.21.15.90:0/3840937330 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f12dc051b50 con 0x7f12cc0040e0 2024-03-11T20:51:38.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:38.480+0000 7f12e86e9640 1 -- 172.21.15.90:0/3840937330 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f12e0156560 con 0x7f12cc0040e0 2024-03-11T20:51:38.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:38.572+0000 7f12d6ffd640 1 -- 172.21.15.90:0/3840937330 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+18819 (secure 0 0 0) 0x7f12e0156560 con 0x7f12cc0040e0 2024-03-11T20:51:38.571 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T20:51:38.571 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":30,"fsid":"76d56b10-6c29-476a-804b-8482ea90f9e2","created":"2024-03-11T18:34:54.521741+0000","modified":"2024-03-11T18:37:03.877940+0000","last_up_change":"2024-03-11T18:35:16.207500+0000","last_in_change":"2024-03-11T18:35:05.213634+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T18:35:16.193842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T18:35:54.001469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T18:35:55.917321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T18:36:02.210943+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"34decff9-7280-4d84-9665-d6af8ea6de84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6819","nonce":1755667921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6821","nonce":1755667921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6825","nonce":1755667921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6823","nonce":1755667921}]},"public_addr":"172.21.15.90:6819/1755667921","cluster_addr":"172.21.15.90:6821/1755667921","heartbeat_back_addr":"172.21.15.90:6825/1755667921","heartbeat_front_addr":"172.21.15.90:6823/1755667921","state":["exists","up"]},{"osd":1,"uuid":"8ecdce9e-5486-415a-a609-dd16e945440e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6807","nonce":438682081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6809","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6811","nonce":438682081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6817","nonce":438682081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6815","nonce":438682081}]},"public_addr":"172.21.15.90:6807/438682081","cluster_addr":"172.21.15.90:6811/438682081","heartbeat_back_addr":"172.21.15.90:6817/438682081","heartbeat_front_addr":"172.21.15.90:6815/438682081","state":["exists","up"]},{"osd":2,"uuid":"737486d4-0e90-4c82-9e79-bf37c812e5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6827","nonce":2837704738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6829","nonce":2837704738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6833","nonce":2837704738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6831","nonce":2837704738}]},"public_addr":"172.21.15.90:6827/2837704738","cluster_addr":"172.21.15.90:6829/2837704738","heartbeat_back_addr":"172.21.15.90:6833/2837704738","heartbeat_front_addr":"172.21.15.90:6831/2837704738","state":["exists","up"]},{"osd":3,"uuid":"698bd661-bbef-4699-a619-f7bfe23ce12c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6803","nonce":240987971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6805","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6806","nonce":240987971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6813","nonce":240987971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6810","nonce":240987971}]},"public_addr":"172.21.15.90:6803/240987971","cluster_addr":"172.21.15.90:6806/240987971","heartbeat_back_addr":"172.21.15.90:6813/240987971","heartbeat_front_addr":"172.21.15.90:6810/240987971","state":["exists","up"]},{"osd":4,"uuid":"aba9e31f-73d9-43fe-a10b-2dc5832968b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6801","nonce":502685275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6803","nonce":502685275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6807","nonce":502685275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6805","nonce":502685275}]},"public_addr":"172.21.15.92:6801/502685275","cluster_addr":"172.21.15.92:6803/502685275","heartbeat_back_addr":"172.21.15.92:6807/502685275","heartbeat_front_addr":"172.21.15.92:6805/502685275","state":["exists","up"]},{"osd":5,"uuid":"bdd56766-4b28-4007-8de3-b5b08cf335d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6809","nonce":200142070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6811","nonce":200142070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6815","nonce":200142070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6813","nonce":200142070}]},"public_addr":"172.21.15.92:6809/200142070","cluster_addr":"172.21.15.92:6811/200142070","heartbeat_back_addr":"172.21.15.92:6815/200142070","heartbeat_front_addr":"172.21.15.92:6813/200142070","state":["exists","up"]},{"osd":6,"uuid":"6c580a59-610f-455e-a96b-033299dcc118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6817","nonce":1919010204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6819","nonce":1919010204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6823","nonce":1919010204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6821","nonce":1919010204}]},"public_addr":"172.21.15.92:6817/1919010204","cluster_addr":"172.21.15.92:6819/1919010204","heartbeat_back_addr":"172.21.15.92:6823/1919010204","heartbeat_front_addr":"172.21.15.92:6821/1919010204","state":["exists","up"]},{"osd":7,"uuid":"958184cc-aebf-4c5b-91a0-2a3e38a6b42b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6825","nonce":3475925719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6827","nonce":3475925719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6831","nonce":3475925719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6829","nonce":3475925719}]},"public_addr":"172.21.15.92:6825/3475925719","cluster_addr":"172.21.15.92:6827/3475925719","heartbeat_back_addr":"172.21.15.92:6831/3475925719","heartbeat_front_addr":"172.21.15.92:6829/3475925719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.816294+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.048084+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.137975+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.968246+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.016829+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.931676+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.041673+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.079404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.33","mappings":[{"from":3,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T20:51:38.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:38.572+0000 7f12e86e9640 1 -- 172.21.15.90:0/3840937330 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f12c4066ec0 msgr2=0x7f12c4069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:38.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:38.572+0000 7f12e86e9640 1 --2- 172.21.15.90:0/3840937330 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f12c4066ec0 0x7f12c4069380 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f12d00aa000 tx=0x7f12e006e110 comp rx=0 tx=0).stop 2024-03-11T20:51:38.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:38.572+0000 7f12e86e9640 1 -- 172.21.15.90:0/3840937330 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12cc0040e0 msgr2=0x7f12e0156f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:38.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:38.572+0000 7f12e86e9640 1 --2- 172.21.15.90:0/3840937330 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12cc0040e0 0x7f12e0156f00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f12dc00b520 tx=0x7f12dc00b9e0 comp rx=0 tx=0).stop 2024-03-11T20:51:38.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:38.572+0000 7f12e86e9640 1 -- 172.21.15.90:0/3840937330 shutdown_connections 2024-03-11T20:51:38.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:38.572+0000 7f12e86e9640 1 --2- 172.21.15.90:0/3840937330 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f12c4066ec0 0x7f12c4069380 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:38.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:38.572+0000 7f12e86e9640 1 --2- 172.21.15.90:0/3840937330 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f12e0147de0 0x7f12e01f64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:38.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:38.572+0000 7f12e86e9640 1 --2- 172.21.15.90:0/3840937330 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f12e00c2d80 0x7f12e0157440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:38.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:38.572+0000 7f12e86e9640 1 --2- 172.21.15.90:0/3840937330 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12cc0040e0 0x7f12e0156f00 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:38.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:38.572+0000 7f12e86e9640 1 -- 172.21.15.90:0/3840937330 >> 172.21.15.90:0/3840937330 conn(0x7f12e00cb880 msgr2=0x7f12e014c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:38.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:38.572+0000 7f12e86e9640 1 -- 172.21.15.90:0/3840937330 shutdown_connections 2024-03-11T20:51:38.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:38.572+0000 7f12e86e9640 1 -- 172.21.15.90:0/3840937330 wait complete. 2024-03-11T20:51:39.598 INFO:tasks.ceph:Scrubbing osd.0 2024-03-11T20:51:39.598 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-03-11T20:51:39.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.736+0000 7f09606fa640 1 Processor -- start 2024-03-11T20:51:39.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.736+0000 7f09606fa640 1 -- start start 2024-03-11T20:51:39.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.736+0000 7f09606fa640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f095806ae60 0x7f095815dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:39.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.736+0000 7f09606fa640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f095815e150 con 0x7f095806b240 2024-03-11T20:51:39.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.736+0000 7f09606fa640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09580c6e50 con 0x7f09580711f0 2024-03-11T20:51:39.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.736+0000 7f09606fa640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09580c78e0 con 0x7f095806ae60 2024-03-11T20:51:39.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.736+0000 7f095e496640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f095806ae60 0x7f095815dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:39.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.736+0000 7f095e496640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f095806ae60 0x7f095815dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58542/0 (socket says 172.21.15.90:58542) 2024-03-11T20:51:39.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.736+0000 7f095e496640 1 -- 172.21.15.90:0/41044741 learned_addr learned my addr 172.21.15.90:0/41044741 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:51:39.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.740+0000 7f095e496640 1 -- 172.21.15.90:0/41044741 >> v1:172.21.15.92:6789/0 conn(0x7f09580711f0 legacy=0x7f0958153a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:39.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.740+0000 7f095e496640 1 -- 172.21.15.90:0/41044741 >> v1:172.21.15.90:6789/0 conn(0x7f095806b240 legacy=0x7f0958070ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:39.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.740+0000 7f095e496640 1 -- 172.21.15.90:0/41044741 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09580c7bc0 con 0x7f095806ae60 2024-03-11T20:51:39.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.744+0000 7f095e496640 1 --2- 172.21.15.90:0/41044741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f095806ae60 0x7f095815dc10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0954009e50 tx=0x7f095401f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80ebd305f1289ca server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:39.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.748+0000 7f095d494640 1 -- 172.21.15.90:0/41044741 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09540af400 con 0x7f095806ae60 2024-03-11T20:51:39.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.748+0000 7f095d494640 1 -- 172.21.15.90:0/41044741 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09540ab050 con 0x7f095806ae60 2024-03-11T20:51:39.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.748+0000 7f095d494640 1 -- 172.21.15.90:0/41044741 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09540af780 con 0x7f095806ae60 2024-03-11T20:51:39.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.752+0000 7f09606fa640 1 -- 172.21.15.90:0/41044741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f095806ae60 msgr2=0x7f095815dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:39.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.752+0000 7f09606fa640 1 --2- 172.21.15.90:0/41044741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f095806ae60 0x7f095815dc10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0954009e50 tx=0x7f095401f1e0 comp rx=0 tx=0).stop 2024-03-11T20:51:39.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.752+0000 7f09606fa640 1 -- 172.21.15.90:0/41044741 shutdown_connections 2024-03-11T20:51:39.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.752+0000 7f09606fa640 1 --2- 172.21.15.90:0/41044741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f095806ae60 0x7f095815dc10 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:39.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.752+0000 7f09606fa640 1 -- 172.21.15.90:0/41044741 >> 172.21.15.90:0/41044741 conn(0x7f095806d230 msgr2=0x7f095806e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:39.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.752+0000 7f09606fa640 1 -- 172.21.15.90:0/41044741 shutdown_connections 2024-03-11T20:51:39.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.752+0000 7f09606fa640 1 -- 172.21.15.90:0/41044741 wait complete. 2024-03-11T20:51:39.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.752+0000 7f09606fa640 1 Processor -- start 2024-03-11T20:51:39.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.752+0000 7f09606fa640 1 -- start start 2024-03-11T20:51:39.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.752+0000 7f09606fa640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f095806ae60 0x7f09581ef320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:39.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.752+0000 7f09606fa640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f095806b240 0x7f09581f0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:39.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.752+0000 7f09606fa640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f095815a810 0x7f0958159c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:39.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.752+0000 7f09606fa640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095815a150 con 0x7f095806b240 2024-03-11T20:51:39.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.752+0000 7f09606fa640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09581ee310 con 0x7f095806ae60 2024-03-11T20:51:39.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.752+0000 7f09606fa640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09581f3be0 con 0x7f095815a810 2024-03-11T20:51:39.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.752+0000 7f095dc95640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f095806b240 0x7f09581f0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:39.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.752+0000 7f095dc95640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f095806b240 0x7f09581f0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58546/0 (socket says 172.21.15.90:58546) 2024-03-11T20:51:39.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.752+0000 7f095dc95640 1 -- 172.21.15.90:0/2311260578 learned_addr learned my addr 172.21.15.90:0/2311260578 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:51:39.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.756+0000 7f095e496640 1 --2- 172.21.15.90:0/2311260578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f095806ae60 0x7f09581ef320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:39.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.756+0000 7f095ec97640 1 --2- 172.21.15.90:0/2311260578 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f095815a810 0x7f0958159c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:39.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.756+0000 7f095dc95640 1 -- 172.21.15.90:0/2311260578 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f095815a810 msgr2=0x7f0958159c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:39.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.756+0000 7f095dc95640 1 --2- 172.21.15.90:0/2311260578 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f095815a810 0x7f0958159c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:39.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.756+0000 7f095dc95640 1 -- 172.21.15.90:0/2311260578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f095806ae60 msgr2=0x7f09581ef320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:39.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.756+0000 7f095dc95640 1 --2- 172.21.15.90:0/2311260578 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f095806ae60 0x7f09581ef320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:39.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.756+0000 7f095dc95640 1 -- 172.21.15.90:0/2311260578 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0954009b30 con 0x7f095806b240 2024-03-11T20:51:39.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.760+0000 7f095dc95640 1 --2- 172.21.15.90:0/2311260578 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f095806b240 0x7f09581f0d90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f094800c450 tx=0x7f094800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:39.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.764+0000 7f094effd640 1 -- 172.21.15.90:0/2311260578 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0948011790 con 0x7f095806b240 2024-03-11T20:51:39.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.764+0000 7f094effd640 1 -- 172.21.15.90:0/2311260578 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0948011910 con 0x7f095806b240 2024-03-11T20:51:39.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.764+0000 7f094effd640 1 -- 172.21.15.90:0/2311260578 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0948011cc0 con 0x7f095806b240 2024-03-11T20:51:39.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.764+0000 7f09606fa640 1 -- 172.21.15.90:0/2311260578 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09581ee560 con 0x7f095806b240 2024-03-11T20:51:39.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.764+0000 7f09606fa640 1 -- 172.21.15.90:0/2311260578 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09581eeb20 con 0x7f095806b240 2024-03-11T20:51:39.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.764+0000 7f09606fa640 1 -- 172.21.15.90:0/2311260578 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f09580c8980 con 0x7f095806b240 2024-03-11T20:51:39.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.784+0000 7f094effd640 1 -- 172.21.15.90:0/2311260578 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f094801c460 con 0x7f095806b240 2024-03-11T20:51:39.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.788+0000 7f094effd640 1 --2- 172.21.15.90:0/2311260578 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f093c066e70 0x7f093c069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:39.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.788+0000 7f095e496640 1 --2- 172.21.15.90:0/2311260578 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f093c066e70 0x7f093c069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:39.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.788+0000 7f095e496640 1 --2- 172.21.15.90:0/2311260578 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f093c066e70 0x7f093c069330 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f09540b1000 tx=0x7f09540aa000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:39.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.788+0000 7f094effd640 1 -- 172.21.15.90:0/2311260578 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f0948056620 con 0x7f095806b240 2024-03-11T20:51:39.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.788+0000 7f094effd640 1 --2- 172.21.15.90:0/2311260578 >> [v2:172.21.15.90:6818/1755667921,v1:172.21.15.90:6819/1755667921] conn(0x7f093c070810 0x7f093c072cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:39.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.788+0000 7f094effd640 1 -- 172.21.15.90:0/2311260578 --> [v2:172.21.15.90:6818/1755667921,v1:172.21.15.90:6819/1755667921] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f093c073380 con 0x7f093c070810 2024-03-11T20:51:39.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.788+0000 7f094effd640 1 -- 172.21.15.90:0/2311260578 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09480521c0 con 0x7f095806b240 2024-03-11T20:51:39.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.788+0000 7f095ec97640 1 --2- 172.21.15.90:0/2311260578 >> [v2:172.21.15.90:6818/1755667921,v1:172.21.15.90:6819/1755667921] conn(0x7f093c070810 0x7f093c072cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:39.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.788+0000 7f095ec97640 1 --2- 172.21.15.90:0/2311260578 >> [v2:172.21.15.90:6818/1755667921,v1:172.21.15.90:6819/1755667921] conn(0x7f093c070810 0x7f093c072cb0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:39.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.792+0000 7f094effd640 1 -- 172.21.15.90:0/2311260578 <== osd.0 v2:172.21.15.90:6818/1755667921 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f093c073380 con 0x7f093c070810 2024-03-11T20:51:39.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.804+0000 7f09606fa640 1 -- 172.21.15.90:0/2311260578 --> [v2:172.21.15.90:6818/1755667921,v1:172.21.15.90:6819/1755667921] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f09580c8460 con 0x7f093c070810 2024-03-11T20:51:39.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.804+0000 7f094effd640 1 -- 172.21.15.90:0/2311260578 <== osd.0 v2:172.21.15.90:6818/1755667921 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f09500050e0 con 0x7f093c070810 2024-03-11T20:51:39.804 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-03-11T20:51:39.805 INFO:teuthology.orchestra.run.smithi090.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-11T20:51:39.805 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-03-11T20:51:39.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.804+0000 7f09606fa640 1 -- 172.21.15.90:0/2311260578 >> [v2:172.21.15.90:6818/1755667921,v1:172.21.15.90:6819/1755667921] conn(0x7f093c070810 msgr2=0x7f093c072cb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:39.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.804+0000 7f09606fa640 1 --2- 172.21.15.90:0/2311260578 >> [v2:172.21.15.90:6818/1755667921,v1:172.21.15.90:6819/1755667921] conn(0x7f093c070810 0x7f093c072cb0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:39.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.804+0000 7f09606fa640 1 -- 172.21.15.90:0/2311260578 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f093c066e70 msgr2=0x7f093c069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:39.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.804+0000 7f09606fa640 1 --2- 172.21.15.90:0/2311260578 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f093c066e70 0x7f093c069330 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f09540b1000 tx=0x7f09540aa000 comp rx=0 tx=0).stop 2024-03-11T20:51:39.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.804+0000 7f09606fa640 1 -- 172.21.15.90:0/2311260578 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f095806b240 msgr2=0x7f09581f0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:39.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.804+0000 7f09606fa640 1 --2- 172.21.15.90:0/2311260578 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f095806b240 0x7f09581f0d90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f094800c450 tx=0x7f094800c910 comp rx=0 tx=0).stop 2024-03-11T20:51:39.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.804+0000 7f095ec97640 1 -- 172.21.15.90:0/2311260578 reap_dead start 2024-03-11T20:51:39.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.804+0000 7f09606fa640 1 -- 172.21.15.90:0/2311260578 shutdown_connections 2024-03-11T20:51:39.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.804+0000 7f09606fa640 1 -- 172.21.15.90:0/2311260578 >> 172.21.15.90:0/2311260578 conn(0x7f095806d230 msgr2=0x7f0958152330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:39.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.804+0000 7f09606fa640 1 -- 172.21.15.90:0/2311260578 shutdown_connections 2024-03-11T20:51:39.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.804+0000 7f09606fa640 1 -- 172.21.15.90:0/2311260578 wait complete. 2024-03-11T20:51:39.821 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-03-11T20:51:39.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.940+0000 7f1c29313640 1 Processor -- start 2024-03-11T20:51:39.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.940+0000 7f1c29313640 1 -- start start 2024-03-11T20:51:39.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.940+0000 7f1c29313640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f1c24151c70 0x7f1c2406bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:39.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.940+0000 7f1c29313640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c240c6720 con 0x7f1c240c7f20 2024-03-11T20:51:39.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.940+0000 7f1c29313640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c240cc7b0 con 0x7f1c24066d60 2024-03-11T20:51:39.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.940+0000 7f1c29313640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c240c5d20 con 0x7f1c24151c70 2024-03-11T20:51:39.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.980+0000 7f1c22ffd640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f1c24151c70 0x7f1c2406bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:39.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.980+0000 7f1c22ffd640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f1c24151c70 0x7f1c2406bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58550/0 (socket says 172.21.15.90:58550) 2024-03-11T20:51:39.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.980+0000 7f1c22ffd640 1 -- 172.21.15.90:0/51510565 learned_addr learned my addr 172.21.15.90:0/51510565 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:51:39.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.984+0000 7f1c22ffd640 1 -- 172.21.15.90:0/51510565 >> v1:172.21.15.92:6789/0 conn(0x7f1c24066d60 legacy=0x7f1c24162300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:39.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.984+0000 7f1c22ffd640 1 -- 172.21.15.90:0/51510565 >> v1:172.21.15.90:6789/0 conn(0x7f1c240c7f20 legacy=0x7f1c24157ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:39.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.984+0000 7f1c22ffd640 1 -- 172.21.15.90:0/51510565 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c240c5a40 con 0x7f1c24151c70 2024-03-11T20:51:39.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:39.992+0000 7f1c22ffd640 1 --2- 172.21.15.90:0/51510565 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c24151c70 0x7f1c2406bc10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c009e50 tx=0x7f1c0c01c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1235843ca661924 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:39.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.000+0000 7f1c21ffb640 1 -- 172.21.15.90:0/51510565 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c0a5720 con 0x7f1c24151c70 2024-03-11T20:51:39.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.000+0000 7f1c21ffb640 1 -- 172.21.15.90:0/51510565 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c0c0a5880 con 0x7f1c24151c70 2024-03-11T20:51:39.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.000+0000 7f1c21ffb640 1 -- 172.21.15.90:0/51510565 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c0a5c10 con 0x7f1c24151c70 2024-03-11T20:51:39.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.000+0000 7f1c29313640 1 -- 172.21.15.90:0/51510565 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c24151c70 msgr2=0x7f1c2406bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:39.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.000+0000 7f1c29313640 1 --2- 172.21.15.90:0/51510565 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c24151c70 0x7f1c2406bc10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c009e50 tx=0x7f1c0c01c860 comp rx=0 tx=0).stop 2024-03-11T20:51:40.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.000+0000 7f1c29313640 1 -- 172.21.15.90:0/51510565 shutdown_connections 2024-03-11T20:51:40.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.000+0000 7f1c29313640 1 --2- 172.21.15.90:0/51510565 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c24151c70 0x7f1c2406bc10 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:40.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.000+0000 7f1c29313640 1 -- 172.21.15.90:0/51510565 >> 172.21.15.90:0/51510565 conn(0x7f1c240cb880 msgr2=0x7f1c240cbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:40.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.000+0000 7f1c29313640 1 -- 172.21.15.90:0/51510565 shutdown_connections 2024-03-11T20:51:40.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.000+0000 7f1c29313640 1 -- 172.21.15.90:0/51510565 wait complete. 2024-03-11T20:51:40.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.004+0000 7f1c29313640 1 Processor -- start 2024-03-11T20:51:40.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.004+0000 7f1c29313640 1 -- start start 2024-03-11T20:51:40.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.004+0000 7f1c29313640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c24066d60 0x7f1c24140340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:40.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.004+0000 7f1c29313640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1c240c7f20 0x7f1c2413e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:40.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.004+0000 7f1c29313640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c24151c70 0x7f1c241523e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:40.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.004+0000 7f1c29313640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c241f4d10 con 0x7f1c24066d60 2024-03-11T20:51:40.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.004+0000 7f1c29313640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2413f5a0 con 0x7f1c24151c70 2024-03-11T20:51:40.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.004+0000 7f1c29313640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c241f4fb0 con 0x7f1c240c7f20 2024-03-11T20:51:40.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.004+0000 7f1c22ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c24066d60 0x7f1c24140340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:40.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.004+0000 7f1c22ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c24066d60 0x7f1c24140340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58560/0 (socket says 172.21.15.90:58560) 2024-03-11T20:51:40.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.004+0000 7f1c22ffd640 1 -- 172.21.15.90:0/624853703 learned_addr learned my addr 172.21.15.90:0/624853703 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:51:40.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.008+0000 7f1c22ffd640 1 -- 172.21.15.90:0/624853703 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1c240c7f20 msgr2=0x7f1c2413e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:40.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.008+0000 7f1c22ffd640 1 --2- 172.21.15.90:0/624853703 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1c240c7f20 0x7f1c2413e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:40.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.008+0000 7f1c22ffd640 1 -- 172.21.15.90:0/624853703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c24151c70 msgr2=0x7f1c241523e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:40.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.008+0000 7f1c22ffd640 1 --2- 172.21.15.90:0/624853703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c24151c70 0x7f1c241523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:40.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.008+0000 7f1c22ffd640 1 -- 172.21.15.90:0/624853703 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c0c009b30 con 0x7f1c24066d60 2024-03-11T20:51:40.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.012+0000 7f1c22ffd640 1 --2- 172.21.15.90:0/624853703 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c24066d60 0x7f1c24140340 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c01af10 tx=0x7f1c0c019600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:40.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.016+0000 7f1c1b7fe640 1 -- 172.21.15.90:0/624853703 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c0b8670 con 0x7f1c24066d60 2024-03-11T20:51:40.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.016+0000 7f1c1b7fe640 1 -- 172.21.15.90:0/624853703 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c0c0b87d0 con 0x7f1c24066d60 2024-03-11T20:51:40.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.016+0000 7f1c1b7fe640 1 -- 172.21.15.90:0/624853703 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c0b8b60 con 0x7f1c24066d60 2024-03-11T20:51:40.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.016+0000 7f1c29313640 1 -- 172.21.15.90:0/624853703 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c2413f860 con 0x7f1c24066d60 2024-03-11T20:51:40.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.016+0000 7f1c29313640 1 -- 172.21.15.90:0/624853703 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c2413fe20 con 0x7f1c24066d60 2024-03-11T20:51:40.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.016+0000 7f1c29313640 1 -- 172.21.15.90:0/624853703 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c24140010 con 0x7f1c24066d60 2024-03-11T20:51:40.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.304+0000 7f1c1b7fe640 1 -- 172.21.15.90:0/624853703 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f1c0c0b8cc0 con 0x7f1c24066d60 2024-03-11T20:51:40.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.304+0000 7f1c1b7fe640 1 --2- 172.21.15.90:0/624853703 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1c0406b280 0x7f1c0406d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:40.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.304+0000 7f1c227fc640 1 --2- 172.21.15.90:0/624853703 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1c0406b280 0x7f1c0406d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:40.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.304+0000 7f1c227fc640 1 --2- 172.21.15.90:0/624853703 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1c0406b280 0x7f1c0406d740 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1c24147d80 tx=0x7f1c140079c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:40.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.320+0000 7f1c1b7fe640 1 -- 172.21.15.90:0/624853703 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f1c0c0f1790 con 0x7f1c24066d60 2024-03-11T20:51:40.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.320+0000 7f1c1b7fe640 1 -- 172.21.15.90:0/624853703 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1c0c0ed330 con 0x7f1c24066d60 2024-03-11T20:51:40.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.484+0000 7f1c29313640 1 -- 172.21.15.90:0/624853703 --> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f1c2406c150 con 0x7f1c0406b280 2024-03-11T20:51:40.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.484+0000 7f1c1b7fe640 1 -- 172.21.15.90:0/624853703 <== mgr.4101 v2:172.21.15.90:6800/1999720161 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f1c14014400 con 0x7f1c0406b280 2024-03-11T20:51:40.485 INFO:teuthology.orchestra.run.smithi090.stderr:instructed osd(s) 0 to deep-scrub 2024-03-11T20:51:40.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.488+0000 7f1c29313640 1 -- 172.21.15.90:0/624853703 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1c0406b280 msgr2=0x7f1c0406d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:40.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.488+0000 7f1c29313640 1 --2- 172.21.15.90:0/624853703 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1c0406b280 0x7f1c0406d740 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1c24147d80 tx=0x7f1c140079c0 comp rx=0 tx=0).stop 2024-03-11T20:51:40.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.488+0000 7f1c29313640 1 -- 172.21.15.90:0/624853703 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c24066d60 msgr2=0x7f1c24140340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:40.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.488+0000 7f1c29313640 1 --2- 172.21.15.90:0/624853703 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c24066d60 0x7f1c24140340 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c01af10 tx=0x7f1c0c019600 comp rx=0 tx=0).stop 2024-03-11T20:51:40.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.488+0000 7f1c29313640 1 -- 172.21.15.90:0/624853703 shutdown_connections 2024-03-11T20:51:40.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.488+0000 7f1c29313640 1 --2- 172.21.15.90:0/624853703 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1c0406b280 0x7f1c0406d740 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:40.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.488+0000 7f1c29313640 1 --2- 172.21.15.90:0/624853703 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1c24151c70 0x7f1c241523e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:40.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.488+0000 7f1c29313640 1 --2- 172.21.15.90:0/624853703 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f1c240c7f20 0x7f1c2413e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:40.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.488+0000 7f1c29313640 1 --2- 172.21.15.90:0/624853703 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c24066d60 0x7f1c24140340 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:40.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.488+0000 7f1c29313640 1 -- 172.21.15.90:0/624853703 >> 172.21.15.90:0/624853703 conn(0x7f1c240cb880 msgr2=0x7f1c24155850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:40.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.488+0000 7f1c29313640 1 -- 172.21.15.90:0/624853703 shutdown_connections 2024-03-11T20:51:40.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.488+0000 7f1c29313640 1 -- 172.21.15.90:0/624853703 wait complete. 2024-03-11T20:51:40.505 INFO:tasks.ceph:Scrubbing osd.1 2024-03-11T20:51:40.505 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-03-11T20:51:40.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.632+0000 7f029861a640 1 Processor -- start 2024-03-11T20:51:40.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.632+0000 7f029861a640 1 -- start start 2024-03-11T20:51:40.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.632+0000 7f029861a640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f02900c60e0 0x7f029015ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:40.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.632+0000 7f029861a640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f029015bf60 con 0x7f0290151ca0 2024-03-11T20:51:40.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.632+0000 7f029861a640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02900c28e0 con 0x7f029006b700 2024-03-11T20:51:40.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.632+0000 7f029861a640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02900c3490 con 0x7f02900c60e0 2024-03-11T20:51:40.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.632+0000 7f02963b6640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f0290151ca0 0x7f02900c3920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:57740/0 (socket says 172.21.15.90:57740) 2024-03-11T20:51:40.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.632+0000 7f02963b6640 1 -- 172.21.15.90:0/2521847778 learned_addr learned my addr 172.21.15.90:0/2521847778 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T20:51:40.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.632+0000 7f0296bb7640 1 --2- 172.21.15.90:0/2521847778 >> v2:172.21.15.92:3300/0 conn(0x7f02900c60e0 0x7f029015ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:40.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.636+0000 7f02953b4640 1 -- 172.21.15.90:0/2521847778 <== mon.0 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3692298911 0 0) 0x7f028c001120 con 0x7f0290151ca0 2024-03-11T20:51:40.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.636+0000 7f02953b4640 1 -- 172.21.15.90:0/2521847778 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f027c003530 con 0x7f0290151ca0 2024-03-11T20:51:40.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.640+0000 7f02953b4640 1 -- 172.21.15.90:0/2521847778 <== mon.0 v1:172.21.15.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2888499487 0 0) 0x7f027c003530 con 0x7f0290151ca0 2024-03-11T20:51:40.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.640+0000 7f02953b4640 1 -- 172.21.15.90:0/2521847778 >> v2:172.21.15.92:3300/0 conn(0x7f02900c60e0 msgr2=0x7f029015ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:40.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.640+0000 7f02953b4640 1 --2- 172.21.15.90:0/2521847778 >> v2:172.21.15.92:3300/0 conn(0x7f02900c60e0 0x7f029015ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:40.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.640+0000 7f02953b4640 1 -- 172.21.15.90:0/2521847778 >> v1:172.21.15.92:6789/0 conn(0x7f029006b700 legacy=0x7f029006ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:40.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.640+0000 7f02953b4640 1 -- 172.21.15.90:0/2521847778 --> v1:172.21.15.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02900c6670 con 0x7f0290151ca0 2024-03-11T20:51:40.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.640+0000 7f02953b4640 1 -- 172.21.15.90:0/2521847778 <== mon.0 v1:172.21.15.90:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 691735435 0 0) 0x7f028c001810 con 0x7f0290151ca0 2024-03-11T20:51:40.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.640+0000 7f02953b4640 1 -- 172.21.15.90:0/2521847778 >> v1:172.21.15.90:6789/0 conn(0x7f0290151ca0 legacy=0x7f02900c3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:40.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.640+0000 7f02953b4640 1 --2- 172.21.15.90:0/2521847778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f027c0039d0 0x7f027c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:40.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.640+0000 7f02953b4640 1 --2- 172.21.15.90:0/2521847778 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f027c0047d0 0x7f027c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:40.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.640+0000 7f02953b4640 1 --2- 172.21.15.90:0/2521847778 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f027c005190 0x7f027c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:40.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.640+0000 7f02953b4640 1 -- 172.21.15.90:0/2521847778 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02900c3490 con 0x7f027c005190 2024-03-11T20:51:40.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.640+0000 7f02953b4640 1 -- 172.21.15.90:0/2521847778 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f027c00c060 con 0x7f027c0039d0 2024-03-11T20:51:40.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.640+0000 7f02953b4640 1 -- 172.21.15.90:0/2521847778 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f027c00c220 con 0x7f027c0047d0 2024-03-11T20:51:40.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.640+0000 7f0295bb5640 1 --2- 172.21.15.90:0/2521847778 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f027c0047d0 0x7f027c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:40.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.640+0000 7f0296bb7640 1 --2- 172.21.15.90:0/2521847778 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f027c005190 0x7f027c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:40.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.648+0000 7f0296bb7640 1 -- 172.21.15.90:0/2521847778 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f027c0047d0 msgr2=0x7f027c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:40.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.648+0000 7f0296bb7640 1 --2- 172.21.15.90:0/2521847778 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f027c0047d0 0x7f027c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:40.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.648+0000 7f0296bb7640 1 -- 172.21.15.90:0/2521847778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f027c0039d0 msgr2=0x7f027c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:40.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.648+0000 7f0296bb7640 1 --2- 172.21.15.90:0/2521847778 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f027c0039d0 0x7f027c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:40.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.648+0000 7f0296bb7640 1 -- 172.21.15.90:0/2521847778 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f027c003530 con 0x7f027c005190 2024-03-11T20:51:40.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.648+0000 7f0296bb7640 1 -- 172.21.15.90:0/2521847778 reap_dead start 2024-03-11T20:51:40.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.652+0000 7f0296bb7640 1 --2- 172.21.15.90:0/2521847778 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f027c005190 0x7f027c00b990 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0288009fc0 tx=0x7f028801ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1abb34ee063373de server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:40.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.656+0000 7f02953b4640 1 -- 172.21.15.90:0/2521847778 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02880a5cd0 con 0x7f027c005190 2024-03-11T20:51:40.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.656+0000 7f02953b4640 1 -- 172.21.15.90:0/2521847778 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f028800f040 con 0x7f027c005190 2024-03-11T20:51:40.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.656+0000 7f02953b4640 1 -- 172.21.15.90:0/2521847778 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0288003be0 con 0x7f027c005190 2024-03-11T20:51:40.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.656+0000 7f029861a640 1 -- 172.21.15.90:0/2521847778 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f027c005190 msgr2=0x7f027c00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:40.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.656+0000 7f029861a640 1 --2- 172.21.15.90:0/2521847778 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f027c005190 0x7f027c00b990 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0288009fc0 tx=0x7f028801ccc0 comp rx=0 tx=0).stop 2024-03-11T20:51:40.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.656+0000 7f029861a640 1 -- 172.21.15.90:0/2521847778 shutdown_connections 2024-03-11T20:51:40.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.656+0000 7f029861a640 1 --2- 172.21.15.90:0/2521847778 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f027c005190 0x7f027c00b990 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:40.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.656+0000 7f029861a640 1 -- 172.21.15.90:0/2521847778 >> 172.21.15.90:0/2521847778 conn(0x7f029006d230 msgr2=0x7f029006e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:40.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.656+0000 7f029861a640 1 -- 172.21.15.90:0/2521847778 shutdown_connections 2024-03-11T20:51:40.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.656+0000 7f029861a640 1 -- 172.21.15.90:0/2521847778 wait complete. 2024-03-11T20:51:40.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.656+0000 7f029861a640 1 Processor -- start 2024-03-11T20:51:40.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.656+0000 7f029861a640 1 -- start start 2024-03-11T20:51:40.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.660+0000 7f029861a640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f027c005190 0x7f0290154890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:40.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.660+0000 7f029861a640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0290155c90 0x7f02901562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:40.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.660+0000 7f029861a640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02900ca230 0x7f02900ca6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:40.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.660+0000 7f029861a640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029014e5a0 con 0x7f02900ca230 2024-03-11T20:51:40.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.660+0000 7f029861a640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02901580f0 con 0x7f027c005190 2024-03-11T20:51:40.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.660+0000 7f029861a640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0290143a90 con 0x7f0290155c90 2024-03-11T20:51:40.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.660+0000 7f0296bb7640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02900ca230 0x7f02900ca6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:40.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.660+0000 7f0296bb7640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02900ca230 0x7f02900ca6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58570/0 (socket says 172.21.15.90:58570) 2024-03-11T20:51:40.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.660+0000 7f0296bb7640 1 -- 172.21.15.90:0/2671564405 learned_addr learned my addr 172.21.15.90:0/2671564405 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:51:40.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.660+0000 7f0295bb5640 1 --2- 172.21.15.90:0/2671564405 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0290155c90 0x7f02901562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:40.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.660+0000 7f02963b6640 1 --2- 172.21.15.90:0/2671564405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f027c005190 0x7f0290154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:40.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.664+0000 7f0296bb7640 1 -- 172.21.15.90:0/2671564405 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0290155c90 msgr2=0x7f02901562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:40.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.664+0000 7f0296bb7640 1 --2- 172.21.15.90:0/2671564405 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f0290155c90 0x7f02901562f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:40.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.664+0000 7f0296bb7640 1 -- 172.21.15.90:0/2671564405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f027c005190 msgr2=0x7f0290154890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:40.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.664+0000 7f0296bb7640 1 --2- 172.21.15.90:0/2671564405 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f027c005190 0x7f0290154890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:40.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.664+0000 7f0296bb7640 1 -- 172.21.15.90:0/2671564405 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f029014e2c0 con 0x7f02900ca230 2024-03-11T20:51:40.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.664+0000 7f0296bb7640 1 --2- 172.21.15.90:0/2671564405 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02900ca230 0x7f02900ca6b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f02880987f0 tx=0x7f0288009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:40.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.668+0000 7f0286ffd640 1 -- 172.21.15.90:0/2671564405 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02880aed40 con 0x7f02900ca230 2024-03-11T20:51:40.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.668+0000 7f0286ffd640 1 -- 172.21.15.90:0/2671564405 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02880aeec0 con 0x7f02900ca230 2024-03-11T20:51:40.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.668+0000 7f0286ffd640 1 -- 172.21.15.90:0/2671564405 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0288003be0 con 0x7f02900ca230 2024-03-11T20:51:40.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.668+0000 7f029861a640 1 -- 172.21.15.90:0/2671564405 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02901edbd0 con 0x7f02900ca230 2024-03-11T20:51:40.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.668+0000 7f029861a640 1 -- 172.21.15.90:0/2671564405 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02901edeb0 con 0x7f02900ca230 2024-03-11T20:51:40.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.672+0000 7f029861a640 1 -- 172.21.15.90:0/2671564405 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f02900c2e10 con 0x7f02900ca230 2024-03-11T20:51:40.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.692+0000 7f0286ffd640 1 -- 172.21.15.90:0/2671564405 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f0288003d40 con 0x7f02900ca230 2024-03-11T20:51:40.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.692+0000 7f0286ffd640 1 --2- 172.21.15.90:0/2671564405 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0274066e90 0x7f0274069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:40.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.692+0000 7f02963b6640 1 --2- 172.21.15.90:0/2671564405 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0274066e90 0x7f0274069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:40.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.692+0000 7f02963b6640 1 --2- 172.21.15.90:0/2671564405 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0274066e90 0x7f0274069350 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f028c004a90 tx=0x7f028c001e20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:40.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.696+0000 7f0286ffd640 1 -- 172.21.15.90:0/2671564405 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f02880f0bc0 con 0x7f02900ca230 2024-03-11T20:51:40.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.696+0000 7f0286ffd640 1 --2- 172.21.15.90:0/2671564405 >> [v2:172.21.15.90:6804/438682081,v1:172.21.15.90:6807/438682081] conn(0x7f0274070820 0x7f0274072cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:40.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.696+0000 7f0286ffd640 1 -- 172.21.15.90:0/2671564405 --> [v2:172.21.15.90:6804/438682081,v1:172.21.15.90:6807/438682081] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0274073390 con 0x7f0274070820 2024-03-11T20:51:40.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.696+0000 7f0286ffd640 1 -- 172.21.15.90:0/2671564405 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02880ece60 con 0x7f02900ca230 2024-03-11T20:51:40.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.696+0000 7f0295bb5640 1 --2- 172.21.15.90:0/2671564405 >> [v2:172.21.15.90:6804/438682081,v1:172.21.15.90:6807/438682081] conn(0x7f0274070820 0x7f0274072cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:40.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.696+0000 7f0295bb5640 1 --2- 172.21.15.90:0/2671564405 >> [v2:172.21.15.90:6804/438682081,v1:172.21.15.90:6807/438682081] conn(0x7f0274070820 0x7f0274072cc0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:40.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.696+0000 7f0286ffd640 1 -- 172.21.15.90:0/2671564405 <== osd.1 v2:172.21.15.90:6804/438682081 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f0274073390 con 0x7f0274070820 2024-03-11T20:51:40.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.708+0000 7f029861a640 1 -- 172.21.15.90:0/2671564405 --> [v2:172.21.15.90:6804/438682081,v1:172.21.15.90:6807/438682081] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f02900c64f0 con 0x7f0274070820 2024-03-11T20:51:40.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.712+0000 7f0286ffd640 1 -- 172.21.15.90:0/2671564405 <== osd.1 v2:172.21.15.90:6804/438682081 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f0280005050 con 0x7f0274070820 2024-03-11T20:51:40.710 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-03-11T20:51:40.710 INFO:teuthology.orchestra.run.smithi090.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-11T20:51:40.710 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-03-11T20:51:40.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.712+0000 7f029861a640 1 -- 172.21.15.90:0/2671564405 >> [v2:172.21.15.90:6804/438682081,v1:172.21.15.90:6807/438682081] conn(0x7f0274070820 msgr2=0x7f0274072cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:40.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.712+0000 7f029861a640 1 --2- 172.21.15.90:0/2671564405 >> [v2:172.21.15.90:6804/438682081,v1:172.21.15.90:6807/438682081] conn(0x7f0274070820 0x7f0274072cc0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:40.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.712+0000 7f029861a640 1 -- 172.21.15.90:0/2671564405 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0274066e90 msgr2=0x7f0274069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:40.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.712+0000 7f029861a640 1 --2- 172.21.15.90:0/2671564405 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0274066e90 0x7f0274069350 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f028c004a90 tx=0x7f028c001e20 comp rx=0 tx=0).stop 2024-03-11T20:51:40.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.712+0000 7f029861a640 1 -- 172.21.15.90:0/2671564405 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02900ca230 msgr2=0x7f02900ca6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:40.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.712+0000 7f029861a640 1 --2- 172.21.15.90:0/2671564405 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02900ca230 0x7f02900ca6b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f02880987f0 tx=0x7f0288009750 comp rx=0 tx=0).stop 2024-03-11T20:51:40.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.712+0000 7f0296bb7640 1 -- 172.21.15.90:0/2671564405 reap_dead start 2024-03-11T20:51:40.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.712+0000 7f029861a640 1 -- 172.21.15.90:0/2671564405 shutdown_connections 2024-03-11T20:51:40.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.712+0000 7f029861a640 1 -- 172.21.15.90:0/2671564405 >> 172.21.15.90:0/2671564405 conn(0x7f029006d230 msgr2=0x7f0290146b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:40.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.712+0000 7f029861a640 1 -- 172.21.15.90:0/2671564405 shutdown_connections 2024-03-11T20:51:40.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.712+0000 7f029861a640 1 -- 172.21.15.90:0/2671564405 wait complete. 2024-03-11T20:51:40.727 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-03-11T20:51:40.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.848+0000 7f9514510640 1 Processor -- start 2024-03-11T20:51:40.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.848+0000 7f9514510640 1 -- start start 2024-03-11T20:51:40.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.848+0000 7f9514510640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f950c06e9b0 0x7f950c14fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:40.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.848+0000 7f9514510640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f950c070b00 con 0x7f950c1598c0 2024-03-11T20:51:40.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.848+0000 7f9514510640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f950c071df0 con 0x7f950c06f220 2024-03-11T20:51:40.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.848+0000 7f9514510640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f950c15a3b0 con 0x7f950c06e9b0 2024-03-11T20:51:40.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.848+0000 7f95122ac640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f950c1598c0 0x7f950c159ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:57744/0 (socket says 172.21.15.90:57744) 2024-03-11T20:51:40.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.848+0000 7f95122ac640 1 -- 172.21.15.90:0/1988924859 learned_addr learned my addr 172.21.15.90:0/1988924859 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T20:51:40.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.852+0000 7f9512aad640 1 --2- 172.21.15.90:0/1988924859 >> v2:172.21.15.92:3300/0 conn(0x7f950c06e9b0 0x7f950c14fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:40.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.852+0000 7f95112aa640 1 -- 172.21.15.90:0/1988924859 <== mon.0 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3776394811 0 0) 0x7f950c070b00 con 0x7f950c1598c0 2024-03-11T20:51:40.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.852+0000 7f95112aa640 1 -- 172.21.15.90:0/1988924859 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94f8003530 con 0x7f950c1598c0 2024-03-11T20:51:40.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.856+0000 7f95112aa640 1 -- 172.21.15.90:0/1988924859 <== mon.0 v1:172.21.15.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3515333361 0 0) 0x7f94f8003530 con 0x7f950c1598c0 2024-03-11T20:51:40.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.856+0000 7f95112aa640 1 -- 172.21.15.90:0/1988924859 >> v2:172.21.15.92:3300/0 conn(0x7f950c06e9b0 msgr2=0x7f950c14fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:40.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.856+0000 7f95112aa640 1 --2- 172.21.15.90:0/1988924859 >> v2:172.21.15.92:3300/0 conn(0x7f950c06e9b0 0x7f950c14fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:40.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.856+0000 7f95112aa640 1 -- 172.21.15.90:0/1988924859 >> v1:172.21.15.92:6789/0 conn(0x7f950c06f220 legacy=0x7f950c0c3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:40.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.856+0000 7f95112aa640 1 -- 172.21.15.90:0/1988924859 --> v1:172.21.15.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f950c070d50 con 0x7f950c1598c0 2024-03-11T20:51:40.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.856+0000 7f95112aa640 1 -- 172.21.15.90:0/1988924859 <== mon.0 v1:172.21.15.90:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 691735435 0 0) 0x7f9508001680 con 0x7f950c1598c0 2024-03-11T20:51:40.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.856+0000 7f95112aa640 1 -- 172.21.15.90:0/1988924859 >> v1:172.21.15.90:6789/0 conn(0x7f950c1598c0 legacy=0x7f950c159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:40.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.856+0000 7f95112aa640 1 --2- 172.21.15.90:0/1988924859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94f8003980 0x7f94f8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:40.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.856+0000 7f95112aa640 1 --2- 172.21.15.90:0/1988924859 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f94f80047d0 0x7f94f8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:40.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.856+0000 7f95112aa640 1 --2- 172.21.15.90:0/1988924859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94f8005190 0x7f94f800caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:40.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.856+0000 7f95112aa640 1 -- 172.21.15.90:0/1988924859 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f950c15a3b0 con 0x7f94f8005190 2024-03-11T20:51:40.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.856+0000 7f95112aa640 1 -- 172.21.15.90:0/1988924859 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f800d150 con 0x7f94f8003980 2024-03-11T20:51:40.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.856+0000 7f95112aa640 1 -- 172.21.15.90:0/1988924859 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94f800d310 con 0x7f94f80047d0 2024-03-11T20:51:40.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.860+0000 7f9511aab640 1 --2- 172.21.15.90:0/1988924859 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f94f80047d0 0x7f94f8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:40.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.860+0000 7f9512aad640 1 --2- 172.21.15.90:0/1988924859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94f8005190 0x7f94f800caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:40.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.868+0000 7f9512aad640 1 -- 172.21.15.90:0/1988924859 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f94f80047d0 msgr2=0x7f94f8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:40.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.868+0000 7f9512aad640 1 --2- 172.21.15.90:0/1988924859 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f94f80047d0 0x7f94f8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:40.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.868+0000 7f9512aad640 1 -- 172.21.15.90:0/1988924859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94f8003980 msgr2=0x7f94f8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:40.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.868+0000 7f9512aad640 1 --2- 172.21.15.90:0/1988924859 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94f8003980 0x7f94f8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:40.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.868+0000 7f9512aad640 1 -- 172.21.15.90:0/1988924859 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f8003530 con 0x7f94f8005190 2024-03-11T20:51:40.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.868+0000 7f9512aad640 1 -- 172.21.15.90:0/1988924859 reap_dead start 2024-03-11T20:51:40.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.872+0000 7f9512aad640 1 --2- 172.21.15.90:0/1988924859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94f8005190 0x7f94f800caa0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9504009fc0 tx=0x7f950401ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2ca7331c77aa70a server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:40.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.876+0000 7f95112aa640 1 -- 172.21.15.90:0/1988924859 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95040a5cd0 con 0x7f94f8005190 2024-03-11T20:51:40.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.876+0000 7f95112aa640 1 -- 172.21.15.90:0/1988924859 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f950400f040 con 0x7f94f8005190 2024-03-11T20:51:40.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.876+0000 7f95112aa640 1 -- 172.21.15.90:0/1988924859 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9504003be0 con 0x7f94f8005190 2024-03-11T20:51:40.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.876+0000 7f9514510640 1 -- 172.21.15.90:0/1988924859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94f8005190 msgr2=0x7f94f800caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:40.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.876+0000 7f9514510640 1 --2- 172.21.15.90:0/1988924859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94f8005190 0x7f94f800caa0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9504009fc0 tx=0x7f950401ccc0 comp rx=0 tx=0).stop 2024-03-11T20:51:40.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.876+0000 7f9514510640 1 -- 172.21.15.90:0/1988924859 shutdown_connections 2024-03-11T20:51:40.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.876+0000 7f9514510640 1 --2- 172.21.15.90:0/1988924859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94f8005190 0x7f94f800caa0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:40.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.876+0000 7f9514510640 1 -- 172.21.15.90:0/1988924859 >> 172.21.15.90:0/1988924859 conn(0x7f950c06ac00 msgr2=0x7f950c06b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:40.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.876+0000 7f9514510640 1 -- 172.21.15.90:0/1988924859 shutdown_connections 2024-03-11T20:51:40.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.876+0000 7f9514510640 1 -- 172.21.15.90:0/1988924859 wait complete. 2024-03-11T20:51:40.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.876+0000 7f9514510640 1 Processor -- start 2024-03-11T20:51:40.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.876+0000 7f9514510640 1 -- start start 2024-03-11T20:51:40.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.876+0000 7f9514510640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94f8005190 0x7f950c1ece80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:40.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.876+0000 7f9514510640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f950c1f03f0 0x7f950c156350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:40.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.876+0000 7f9514510640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f950c1584b0 0x7f950c158960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:40.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.876+0000 7f9514510640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f950c146930 con 0x7f950c1584b0 2024-03-11T20:51:40.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.876+0000 7f9514510640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f950c158ea0 con 0x7f94f8005190 2024-03-11T20:51:40.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.876+0000 7f9514510640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f950c1ed470 con 0x7f950c1f03f0 2024-03-11T20:51:40.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.880+0000 7f9512aad640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f950c1584b0 0x7f950c158960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:40.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.880+0000 7f9512aad640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f950c1584b0 0x7f950c158960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58584/0 (socket says 172.21.15.90:58584) 2024-03-11T20:51:40.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.880+0000 7f9512aad640 1 -- 172.21.15.90:0/704227589 learned_addr learned my addr 172.21.15.90:0/704227589 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:51:40.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.880+0000 7f95122ac640 1 --2- 172.21.15.90:0/704227589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94f8005190 0x7f950c1ece80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:40.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.880+0000 7f9511aab640 1 --2- 172.21.15.90:0/704227589 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f950c1f03f0 0x7f950c156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:40.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.884+0000 7f9512aad640 1 -- 172.21.15.90:0/704227589 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f950c1f03f0 msgr2=0x7f950c156350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:40.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.884+0000 7f9512aad640 1 --2- 172.21.15.90:0/704227589 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f950c1f03f0 0x7f950c156350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:40.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.884+0000 7f9512aad640 1 -- 172.21.15.90:0/704227589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94f8005190 msgr2=0x7f950c1ece80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:40.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.884+0000 7f9512aad640 1 --2- 172.21.15.90:0/704227589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94f8005190 0x7f950c1ece80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:40.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.884+0000 7f9512aad640 1 -- 172.21.15.90:0/704227589 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f950c1ed730 con 0x7f950c1584b0 2024-03-11T20:51:40.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.884+0000 7f95122ac640 1 --2- 172.21.15.90:0/704227589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94f8005190 0x7f950c1ece80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-11T20:51:40.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.884+0000 7f9512aad640 1 --2- 172.21.15.90:0/704227589 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f950c1584b0 0x7f950c158960 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f950401c730 tx=0x7f9504009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:40.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.888+0000 7f9502ffd640 1 -- 172.21.15.90:0/704227589 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95040aed60 con 0x7f950c1584b0 2024-03-11T20:51:40.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.888+0000 7f9502ffd640 1 -- 172.21.15.90:0/704227589 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95040aeec0 con 0x7f950c1584b0 2024-03-11T20:51:40.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.888+0000 7f9502ffd640 1 -- 172.21.15.90:0/704227589 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9504003bc0 con 0x7f950c1584b0 2024-03-11T20:51:40.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.888+0000 7f9514510640 1 -- 172.21.15.90:0/704227589 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f950c1eda10 con 0x7f950c1584b0 2024-03-11T20:51:40.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.888+0000 7f9514510640 1 -- 172.21.15.90:0/704227589 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f950c1579d0 con 0x7f950c1584b0 2024-03-11T20:51:40.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:40.888+0000 7f9514510640 1 -- 172.21.15.90:0/704227589 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f950c04eef0 con 0x7f950c1584b0 2024-03-11T20:51:41.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.172+0000 7f9502ffd640 1 -- 172.21.15.90:0/704227589 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f9504003d20 con 0x7f950c1584b0 2024-03-11T20:51:41.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.172+0000 7f9502ffd640 1 --2- 172.21.15.90:0/704227589 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f94f0066ec0 0x7f94f0069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:41.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.172+0000 7f95122ac640 1 --2- 172.21.15.90:0/704227589 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f94f0066ec0 0x7f94f0069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:41.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.176+0000 7f95122ac640 1 --2- 172.21.15.90:0/704227589 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f94f0066ec0 0x7f94f0069380 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9508000c10 tx=0x7f9508001770 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:41.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.208+0000 7f9502ffd640 1 -- 172.21.15.90:0/704227589 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f95040f2140 con 0x7f950c1584b0 2024-03-11T20:51:41.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.208+0000 7f9502ffd640 1 -- 172.21.15.90:0/704227589 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f95040edce0 con 0x7f950c1584b0 2024-03-11T20:51:41.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.388+0000 7f9514510640 1 -- 172.21.15.90:0/704227589 --> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f950c0715f0 con 0x7f94f0066ec0 2024-03-11T20:51:41.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.388+0000 7f9502ffd640 1 -- 172.21.15.90:0/704227589 <== mgr.4101 v2:172.21.15.90:6800/1999720161 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f95080058c0 con 0x7f94f0066ec0 2024-03-11T20:51:41.388 INFO:teuthology.orchestra.run.smithi090.stderr:instructed osd(s) 1 to deep-scrub 2024-03-11T20:51:41.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.392+0000 7f9514510640 1 -- 172.21.15.90:0/704227589 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f94f0066ec0 msgr2=0x7f94f0069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:41.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.392+0000 7f9514510640 1 --2- 172.21.15.90:0/704227589 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f94f0066ec0 0x7f94f0069380 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9508000c10 tx=0x7f9508001770 comp rx=0 tx=0).stop 2024-03-11T20:51:41.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.392+0000 7f9514510640 1 -- 172.21.15.90:0/704227589 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f950c1584b0 msgr2=0x7f950c158960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:41.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.392+0000 7f9514510640 1 --2- 172.21.15.90:0/704227589 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f950c1584b0 0x7f950c158960 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f950401c730 tx=0x7f9504009750 comp rx=0 tx=0).stop 2024-03-11T20:51:41.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.392+0000 7f9514510640 1 -- 172.21.15.90:0/704227589 shutdown_connections 2024-03-11T20:51:41.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.392+0000 7f9514510640 1 --2- 172.21.15.90:0/704227589 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f94f0066ec0 0x7f94f0069380 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:41.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.392+0000 7f9514510640 1 --2- 172.21.15.90:0/704227589 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f950c1584b0 0x7f950c158960 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:41.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.392+0000 7f9514510640 1 --2- 172.21.15.90:0/704227589 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f950c1f03f0 0x7f950c156350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:41.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.392+0000 7f9514510640 1 --2- 172.21.15.90:0/704227589 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f94f8005190 0x7f950c1ece80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:41.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.392+0000 7f9514510640 1 -- 172.21.15.90:0/704227589 >> 172.21.15.90:0/704227589 conn(0x7f950c06ac00 msgr2=0x7f950c0c5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:41.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.392+0000 7f9514510640 1 -- 172.21.15.90:0/704227589 shutdown_connections 2024-03-11T20:51:41.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.392+0000 7f9514510640 1 -- 172.21.15.90:0/704227589 wait complete. 2024-03-11T20:51:41.410 INFO:tasks.ceph:Scrubbing osd.2 2024-03-11T20:51:41.410 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-03-11T20:51:41.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.540+0000 7f907398e640 1 Processor -- start 2024-03-11T20:51:41.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.540+0000 7f907398e640 1 -- start start 2024-03-11T20:51:41.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.544+0000 7f907398e640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f906c151ca0 0x7f906c15ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:41.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.544+0000 7f907398e640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f906c06b700 0x7f906c0c3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:41.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.544+0000 7f907398e640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f906c15bf60 con 0x7f906c0c60e0 2024-03-11T20:51:41.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.544+0000 7f907398e640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f906c0c28e0 con 0x7f906c151ca0 2024-03-11T20:51:41.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.544+0000 7f907398e640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f906c0c3490 con 0x7f906c06b700 2024-03-11T20:51:41.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.544+0000 7f9070f29640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f906c06b700 0x7f906c0c3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:41.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.544+0000 7f907172a640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f906c151ca0 0x7f906c15ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:41.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.544+0000 7f9070f29640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f906c06b700 0x7f906c0c3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.90:45002/0 (socket says 172.21.15.90:45002) 2024-03-11T20:51:41.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.544+0000 7f907172a640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f906c151ca0 0x7f906c15ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.90:50230/0 (socket says 172.21.15.90:50230) 2024-03-11T20:51:41.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.544+0000 7f9070f29640 1 -- 172.21.15.90:0/3096910884 learned_addr learned my addr 172.21.15.90:0/3096910884 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:51:41.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.552+0000 7f9070f29640 1 -- 172.21.15.90:0/3096910884 >> v2:172.21.15.92:3300/0 conn(0x7f906c151ca0 msgr2=0x7f906c15ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:41.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.552+0000 7f9070f29640 1 --2- 172.21.15.90:0/3096910884 >> v2:172.21.15.92:3300/0 conn(0x7f906c151ca0 0x7f906c15ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:41.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.552+0000 7f9070f29640 1 -- 172.21.15.90:0/3096910884 >> v1:172.21.15.92:6789/0 conn(0x7f906c0c60e0 legacy=0x7f906c06ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:41.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.552+0000 7f9070f29640 1 -- 172.21.15.90:0/3096910884 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f906c0c6670 con 0x7f906c06b700 2024-03-11T20:51:41.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.556+0000 7f9070f29640 1 --2- 172.21.15.90:0/3096910884 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f906c06b700 0x7f906c0c3920 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f905c009870 tx=0x7f905c01c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96d0f68606903afa server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:41.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.564+0000 7f9063fff640 1 -- 172.21.15.90:0/3096910884 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905c0ab640 con 0x7f906c06b700 2024-03-11T20:51:41.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.564+0000 7f9063fff640 1 -- 172.21.15.90:0/3096910884 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f905c0a7070 con 0x7f906c06b700 2024-03-11T20:51:41.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.564+0000 7f9063fff640 1 -- 172.21.15.90:0/3096910884 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905c0ab9c0 con 0x7f906c06b700 2024-03-11T20:51:41.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.568+0000 7f907398e640 1 -- 172.21.15.90:0/3096910884 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f906c06b700 msgr2=0x7f906c0c3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:41.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.568+0000 7f907398e640 1 --2- 172.21.15.90:0/3096910884 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f906c06b700 0x7f906c0c3920 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f905c009870 tx=0x7f905c01c530 comp rx=0 tx=0).stop 2024-03-11T20:51:41.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.568+0000 7f907398e640 1 -- 172.21.15.90:0/3096910884 shutdown_connections 2024-03-11T20:51:41.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.568+0000 7f907398e640 1 --2- 172.21.15.90:0/3096910884 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f906c06b700 0x7f906c0c3920 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:41.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.568+0000 7f907398e640 1 --2- 172.21.15.90:0/3096910884 >> v2:172.21.15.92:3300/0 conn(0x7f906c151ca0 0x7f906c15ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:41.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.568+0000 7f907398e640 1 -- 172.21.15.90:0/3096910884 >> 172.21.15.90:0/3096910884 conn(0x7f906c06d230 msgr2=0x7f906c06e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:41.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.568+0000 7f907398e640 1 -- 172.21.15.90:0/3096910884 shutdown_connections 2024-03-11T20:51:41.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.568+0000 7f907398e640 1 -- 172.21.15.90:0/3096910884 wait complete. 2024-03-11T20:51:41.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.568+0000 7f907398e640 1 Processor -- start 2024-03-11T20:51:41.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.568+0000 7f907398e640 1 -- start start 2024-03-11T20:51:41.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.568+0000 7f907398e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f906c06b700 0x7f906c1529b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:41.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.568+0000 7f907398e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f906c0c60e0 0x7f906c145300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:41.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.568+0000 7f907398e640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f906c151ca0 0x7f906c1598b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:41.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.568+0000 7f907398e640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906c15bda0 con 0x7f906c0c60e0 2024-03-11T20:51:41.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.568+0000 7f907398e640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906c153040 con 0x7f906c06b700 2024-03-11T20:51:41.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.568+0000 7f907398e640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f906c153da0 con 0x7f906c151ca0 2024-03-11T20:51:41.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.568+0000 7f9070f29640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f906c0c60e0 0x7f906c145300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:41.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.568+0000 7f9070f29640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f906c0c60e0 0x7f906c145300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58596/0 (socket says 172.21.15.90:58596) 2024-03-11T20:51:41.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.568+0000 7f9070f29640 1 -- 172.21.15.90:0/101595754 learned_addr learned my addr 172.21.15.90:0/101595754 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:51:41.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.568+0000 7f907172a640 1 --2- 172.21.15.90:0/101595754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f906c06b700 0x7f906c1529b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:41.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.572+0000 7f9071f2b640 1 --2- 172.21.15.90:0/101595754 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f906c151ca0 0x7f906c1598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:41.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.572+0000 7f9070f29640 1 -- 172.21.15.90:0/101595754 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f906c151ca0 msgr2=0x7f906c1598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:41.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.572+0000 7f9070f29640 1 --2- 172.21.15.90:0/101595754 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f906c151ca0 0x7f906c1598b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:41.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.572+0000 7f9070f29640 1 -- 172.21.15.90:0/101595754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f906c06b700 msgr2=0x7f906c1529b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:41.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.572+0000 7f9070f29640 1 --2- 172.21.15.90:0/101595754 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f906c06b700 0x7f906c1529b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:41.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.572+0000 7f9070f29640 1 -- 172.21.15.90:0/101595754 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f905c0094d0 con 0x7f906c0c60e0 2024-03-11T20:51:41.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.576+0000 7f9070f29640 1 --2- 172.21.15.90:0/101595754 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f906c0c60e0 0x7f906c145300 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f905c0ad040 tx=0x7f905c01ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:41.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.580+0000 7f9061ffb640 1 -- 172.21.15.90:0/101595754 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905c0abe40 con 0x7f906c0c60e0 2024-03-11T20:51:41.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.580+0000 7f9061ffb640 1 -- 172.21.15.90:0/101595754 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f905c0be3f0 con 0x7f906c0c60e0 2024-03-11T20:51:41.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.580+0000 7f9061ffb640 1 -- 172.21.15.90:0/101595754 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905c0be7a0 con 0x7f906c0c60e0 2024-03-11T20:51:41.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.580+0000 7f907398e640 1 -- 172.21.15.90:0/101595754 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f906c154de0 con 0x7f906c0c60e0 2024-03-11T20:51:41.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.580+0000 7f907398e640 1 -- 172.21.15.90:0/101595754 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f906c154300 con 0x7f906c0c60e0 2024-03-11T20:51:41.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.580+0000 7f907398e640 1 -- 172.21.15.90:0/101595754 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f906c06b310 con 0x7f906c0c60e0 2024-03-11T20:51:41.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.600+0000 7f9061ffb640 1 -- 172.21.15.90:0/101595754 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f905c0c8420 con 0x7f906c0c60e0 2024-03-11T20:51:41.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.600+0000 7f9061ffb640 1 --2- 172.21.15.90:0/101595754 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f9050066ec0 0x7f9050069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:41.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.600+0000 7f907172a640 1 --2- 172.21.15.90:0/101595754 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f9050066ec0 0x7f9050069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:41.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.600+0000 7f907172a640 1 --2- 172.21.15.90:0/101595754 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f9050066ec0 0x7f9050069380 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9068009ac0 tx=0x7f9068009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:41.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.600+0000 7f9061ffb640 1 -- 172.21.15.90:0/101595754 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f905c0efc40 con 0x7f906c0c60e0 2024-03-11T20:51:41.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.600+0000 7f9061ffb640 1 --2- 172.21.15.90:0/101595754 >> [v2:172.21.15.90:6826/2837704738,v1:172.21.15.90:6827/2837704738] conn(0x7f9050070850 0x7f9050072cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:41.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.600+0000 7f9061ffb640 1 -- 172.21.15.90:0/101595754 --> [v2:172.21.15.90:6826/2837704738,v1:172.21.15.90:6827/2837704738] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f90500733c0 con 0x7f9050070850 2024-03-11T20:51:41.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.600+0000 7f9071f2b640 1 --2- 172.21.15.90:0/101595754 >> [v2:172.21.15.90:6826/2837704738,v1:172.21.15.90:6827/2837704738] conn(0x7f9050070850 0x7f9050072cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:41.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.604+0000 7f9071f2b640 1 --2- 172.21.15.90:0/101595754 >> [v2:172.21.15.90:6826/2837704738,v1:172.21.15.90:6827/2837704738] conn(0x7f9050070850 0x7f9050072cf0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:41.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.604+0000 7f9061ffb640 1 -- 172.21.15.90:0/101595754 <== osd.2 v2:172.21.15.90:6826/2837704738 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f90500733c0 con 0x7f9050070850 2024-03-11T20:51:41.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.604+0000 7f9061ffb640 1 -- 172.21.15.90:0/101595754 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f905c0ebee0 con 0x7f906c0c60e0 2024-03-11T20:51:41.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.616+0000 7f907398e640 1 -- 172.21.15.90:0/101595754 --> [v2:172.21.15.90:6826/2837704738,v1:172.21.15.90:6827/2837704738] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f906c06a0a0 con 0x7f9050070850 2024-03-11T20:51:41.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.616+0000 7f9061ffb640 1 -- 172.21.15.90:0/101595754 <== osd.2 v2:172.21.15.90:6826/2837704738 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f906c06a0a0 con 0x7f9050070850 2024-03-11T20:51:41.616 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-03-11T20:51:41.616 INFO:teuthology.orchestra.run.smithi090.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-11T20:51:41.616 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-03-11T20:51:41.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.616+0000 7f907398e640 1 -- 172.21.15.90:0/101595754 >> [v2:172.21.15.90:6826/2837704738,v1:172.21.15.90:6827/2837704738] conn(0x7f9050070850 msgr2=0x7f9050072cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:41.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.616+0000 7f907398e640 1 --2- 172.21.15.90:0/101595754 >> [v2:172.21.15.90:6826/2837704738,v1:172.21.15.90:6827/2837704738] conn(0x7f9050070850 0x7f9050072cf0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:41.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.616+0000 7f907398e640 1 -- 172.21.15.90:0/101595754 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f9050066ec0 msgr2=0x7f9050069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:41.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.616+0000 7f907398e640 1 --2- 172.21.15.90:0/101595754 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f9050066ec0 0x7f9050069380 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9068009ac0 tx=0x7f9068009190 comp rx=0 tx=0).stop 2024-03-11T20:51:41.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.616+0000 7f907398e640 1 -- 172.21.15.90:0/101595754 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f906c0c60e0 msgr2=0x7f906c145300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:41.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.616+0000 7f907398e640 1 --2- 172.21.15.90:0/101595754 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f906c0c60e0 0x7f906c145300 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f905c0ad040 tx=0x7f905c01ad70 comp rx=0 tx=0).stop 2024-03-11T20:51:41.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.616+0000 7f9071f2b640 1 -- 172.21.15.90:0/101595754 reap_dead start 2024-03-11T20:51:41.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.616+0000 7f907398e640 1 -- 172.21.15.90:0/101595754 shutdown_connections 2024-03-11T20:51:41.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.616+0000 7f907398e640 1 -- 172.21.15.90:0/101595754 >> 172.21.15.90:0/101595754 conn(0x7f906c06d230 msgr2=0x7f906c0cbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:41.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.616+0000 7f907398e640 1 -- 172.21.15.90:0/101595754 shutdown_connections 2024-03-11T20:51:41.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.616+0000 7f907398e640 1 -- 172.21.15.90:0/101595754 wait complete. 2024-03-11T20:51:41.633 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-03-11T20:51:41.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.752+0000 7fd6a6c08640 1 Processor -- start 2024-03-11T20:51:41.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.752+0000 7fd6a6c08640 1 -- start start 2024-03-11T20:51:41.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.752+0000 7fd6a6c08640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fd6a01509f0 0x7fd6a0157390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:41.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.752+0000 7fd6a6c08640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6a00c5de0 con 0x7fd6a0150dd0 2024-03-11T20:51:41.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.752+0000 7fd6a6c08640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6a0156910 con 0x7fd6a0070b00 2024-03-11T20:51:41.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.752+0000 7fd6a6c08640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6a01578d0 con 0x7fd6a01509f0 2024-03-11T20:51:41.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.752+0000 7fd69ffff640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7fd6a0150dd0 0x7fd6a00c28e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:57752/0 (socket says 172.21.15.90:57752) 2024-03-11T20:51:41.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.752+0000 7fd69ffff640 1 -- 172.21.15.90:0/4019977243 learned_addr learned my addr 172.21.15.90:0/4019977243 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T20:51:41.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.752+0000 7fd6a49a4640 1 --2- 172.21.15.90:0/4019977243 >> v2:172.21.15.92:3300/0 conn(0x7fd6a01509f0 0x7fd6a0157390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:41.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.756+0000 7fd69f7fe640 1 -- 172.21.15.90:0/4019977243 <== mon.0 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2329190531 0 0) 0x7fd6a00c5de0 con 0x7fd6a0150dd0 2024-03-11T20:51:41.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.756+0000 7fd69f7fe640 1 -- 172.21.15.90:0/4019977243 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd688003530 con 0x7fd6a0150dd0 2024-03-11T20:51:41.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.760+0000 7fd69f7fe640 1 -- 172.21.15.90:0/4019977243 <== mon.0 v1:172.21.15.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 533568251 0 0) 0x7fd688003530 con 0x7fd6a0150dd0 2024-03-11T20:51:41.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.760+0000 7fd69f7fe640 1 -- 172.21.15.90:0/4019977243 >> v2:172.21.15.92:3300/0 conn(0x7fd6a01509f0 msgr2=0x7fd6a0157390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:41.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.760+0000 7fd69f7fe640 1 --2- 172.21.15.90:0/4019977243 >> v2:172.21.15.92:3300/0 conn(0x7fd6a01509f0 0x7fd6a0157390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:41.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.760+0000 7fd69f7fe640 1 -- 172.21.15.90:0/4019977243 >> v1:172.21.15.92:6789/0 conn(0x7fd6a0070b00 legacy=0x7fd6a00c4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:41.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.760+0000 7fd69f7fe640 1 -- 172.21.15.90:0/4019977243 --> v1:172.21.15.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6a00c6030 con 0x7fd6a0150dd0 2024-03-11T20:51:41.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.760+0000 7fd69f7fe640 1 -- 172.21.15.90:0/4019977243 <== mon.0 v1:172.21.15.90:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 691735435 0 0) 0x7fd690001680 con 0x7fd6a0150dd0 2024-03-11T20:51:41.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.760+0000 7fd69f7fe640 1 -- 172.21.15.90:0/4019977243 >> v1:172.21.15.90:6789/0 conn(0x7fd6a0150dd0 legacy=0x7fd6a00c28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:41.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.760+0000 7fd69f7fe640 1 --2- 172.21.15.90:0/4019977243 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6880039d0 0x7fd688003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:41.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.760+0000 7fd69f7fe640 1 --2- 172.21.15.90:0/4019977243 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6880047d0 0x7fd688004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:41.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.760+0000 7fd69f7fe640 1 --2- 172.21.15.90:0/4019977243 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd688005190 0x7fd68800dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:41.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.760+0000 7fd69f7fe640 1 -- 172.21.15.90:0/4019977243 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a01578d0 con 0x7fd6880047d0 2024-03-11T20:51:41.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.760+0000 7fd69f7fe640 1 -- 172.21.15.90:0/4019977243 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68800e2a0 con 0x7fd6880039d0 2024-03-11T20:51:41.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.760+0000 7fd69f7fe640 1 -- 172.21.15.90:0/4019977243 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd68800e460 con 0x7fd688005190 2024-03-11T20:51:41.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.760+0000 7fd69ffff640 1 --2- 172.21.15.90:0/4019977243 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6880047d0 0x7fd688004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:41.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.760+0000 7fd6a51a5640 1 --2- 172.21.15.90:0/4019977243 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd688005190 0x7fd68800dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:41.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.764+0000 7fd69ffff640 1 -- 172.21.15.90:0/4019977243 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd688005190 msgr2=0x7fd68800dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:41.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.764+0000 7fd69ffff640 1 --2- 172.21.15.90:0/4019977243 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd688005190 0x7fd68800dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:41.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.764+0000 7fd69ffff640 1 -- 172.21.15.90:0/4019977243 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6880039d0 msgr2=0x7fd688003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:41.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.764+0000 7fd69ffff640 1 --2- 172.21.15.90:0/4019977243 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6880039d0 0x7fd688003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:41.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.764+0000 7fd69ffff640 1 -- 172.21.15.90:0/4019977243 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd688003530 con 0x7fd6880047d0 2024-03-11T20:51:41.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.764+0000 7fd6a51a5640 1 -- 172.21.15.90:0/4019977243 reap_dead start 2024-03-11T20:51:41.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.768+0000 7fd69ffff640 1 --2- 172.21.15.90:0/4019977243 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6880047d0 0x7fd688004c50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd690007fc0 tx=0x7fd69001f270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ead3b814388421e server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:41.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.772+0000 7fd69f7fe640 1 -- 172.21.15.90:0/4019977243 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6900a8e90 con 0x7fd6880047d0 2024-03-11T20:51:41.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.772+0000 7fd69f7fe640 1 -- 172.21.15.90:0/4019977243 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6900058e0 con 0x7fd6880047d0 2024-03-11T20:51:41.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.772+0000 7fd69f7fe640 1 -- 172.21.15.90:0/4019977243 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd690005cd0 con 0x7fd6880047d0 2024-03-11T20:51:41.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.772+0000 7fd6a6c08640 1 -- 172.21.15.90:0/4019977243 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6880047d0 msgr2=0x7fd688004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:41.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.772+0000 7fd6a6c08640 1 --2- 172.21.15.90:0/4019977243 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6880047d0 0x7fd688004c50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd690007fc0 tx=0x7fd69001f270 comp rx=0 tx=0).stop 2024-03-11T20:51:41.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.772+0000 7fd6a6c08640 1 -- 172.21.15.90:0/4019977243 shutdown_connections 2024-03-11T20:51:41.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.772+0000 7fd6a6c08640 1 --2- 172.21.15.90:0/4019977243 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6880047d0 0x7fd688004c50 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:41.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.772+0000 7fd6a6c08640 1 -- 172.21.15.90:0/4019977243 >> 172.21.15.90:0/4019977243 conn(0x7fd6a006ac00 msgr2=0x7fd6a006b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:41.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.772+0000 7fd6a6c08640 1 -- 172.21.15.90:0/4019977243 shutdown_connections 2024-03-11T20:51:41.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.772+0000 7fd6a6c08640 1 -- 172.21.15.90:0/4019977243 wait complete. 2024-03-11T20:51:41.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.776+0000 7fd6a6c08640 1 Processor -- start 2024-03-11T20:51:41.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.776+0000 7fd6a6c08640 1 -- start start 2024-03-11T20:51:41.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.776+0000 7fd6a6c08640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6880047d0 0x7fd6a01ebe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:41.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.776+0000 7fd6a6c08640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd6a013fd00 0x7fd6a01583e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:41.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.776+0000 7fd6a6c08640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6a0157eb0 0x7fd6a0144b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:41.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.776+0000 7fd6a6c08640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a0140cd0 con 0x7fd6a0157eb0 2024-03-11T20:51:41.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.776+0000 7fd6a6c08640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a01ec3b0 con 0x7fd6880047d0 2024-03-11T20:51:41.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.776+0000 7fd6a6c08640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a013f870 con 0x7fd6a013fd00 2024-03-11T20:51:41.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.776+0000 7fd6a51a5640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6a0157eb0 0x7fd6a0144b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:41.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.776+0000 7fd6a51a5640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6a0157eb0 0x7fd6a0144b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58620/0 (socket says 172.21.15.90:58620) 2024-03-11T20:51:41.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.776+0000 7fd6a51a5640 1 -- 172.21.15.90:0/4013375935 learned_addr learned my addr 172.21.15.90:0/4013375935 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:51:41.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.776+0000 7fd69ffff640 1 --2- 172.21.15.90:0/4013375935 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd6a013fd00 0x7fd6a01583e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:41.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.780+0000 7fd6a51a5640 1 -- 172.21.15.90:0/4013375935 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd6a013fd00 msgr2=0x7fd6a01583e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:41.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.780+0000 7fd6a51a5640 1 --2- 172.21.15.90:0/4013375935 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd6a013fd00 0x7fd6a01583e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:41.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.780+0000 7fd6a51a5640 1 -- 172.21.15.90:0/4013375935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6880047d0 msgr2=0x7fd6a01ebe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:41.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.780+0000 7fd6a51a5640 1 --2- 172.21.15.90:0/4013375935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6880047d0 0x7fd6a01ebe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:41.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.780+0000 7fd6a51a5640 1 -- 172.21.15.90:0/4013375935 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6a0158920 con 0x7fd6a0157eb0 2024-03-11T20:51:41.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.784+0000 7fd6a51a5640 1 --2- 172.21.15.90:0/4013375935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6a0157eb0 0x7fd6a0144b30 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd6a00721a0 tx=0x7fd69800fcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:41.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.788+0000 7fd69d7fa640 1 -- 172.21.15.90:0/4013375935 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69800fec0 con 0x7fd6a0157eb0 2024-03-11T20:51:41.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.788+0000 7fd69d7fa640 1 -- 172.21.15.90:0/4013375935 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd698003b90 con 0x7fd6a0157eb0 2024-03-11T20:51:41.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.788+0000 7fd69d7fa640 1 -- 172.21.15.90:0/4013375935 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6980050d0 con 0x7fd6a0157eb0 2024-03-11T20:51:41.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.788+0000 7fd6a6c08640 1 -- 172.21.15.90:0/4013375935 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6a0140750 con 0x7fd6a0157eb0 2024-03-11T20:51:41.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.788+0000 7fd6a6c08640 1 -- 172.21.15.90:0/4013375935 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6a013f220 con 0x7fd6a0157eb0 2024-03-11T20:51:41.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.788+0000 7fd6a6c08640 1 -- 172.21.15.90:0/4013375935 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6a013f080 con 0x7fd6a0157eb0 2024-03-11T20:51:41.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.808+0000 7fd69d7fa640 1 -- 172.21.15.90:0/4013375935 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fd698005230 con 0x7fd6a0157eb0 2024-03-11T20:51:41.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.808+0000 7fd69d7fa640 1 --2- 172.21.15.90:0/4013375935 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fd68c066e40 0x7fd68c069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:41.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.812+0000 7fd6a49a4640 1 --2- 172.21.15.90:0/4013375935 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fd68c066e40 0x7fd68c069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:41.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:41.812+0000 7fd6a49a4640 1 --2- 172.21.15.90:0/4013375935 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fd68c066e40 0x7fd68c069300 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd694006aa0 tx=0x7fd694004780 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:42.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.080+0000 7fd69d7fa640 1 -- 172.21.15.90:0/4013375935 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fd698056390 con 0x7fd6a0157eb0 2024-03-11T20:51:42.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.080+0000 7fd69d7fa640 1 -- 172.21.15.90:0/4013375935 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd698051f30 con 0x7fd6a0157eb0 2024-03-11T20:51:42.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.264+0000 7fd6a6c08640 1 -- 172.21.15.90:0/4013375935 --> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fd6a0156e70 con 0x7fd68c066e40 2024-03-11T20:51:42.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.264+0000 7fd69d7fa640 1 -- 172.21.15.90:0/4013375935 <== mgr.4101 v2:172.21.15.90:6800/1999720161 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd69400e750 con 0x7fd68c066e40 2024-03-11T20:51:42.264 INFO:teuthology.orchestra.run.smithi090.stderr:instructed osd(s) 2 to deep-scrub 2024-03-11T20:51:42.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.268+0000 7fd6a6c08640 1 -- 172.21.15.90:0/4013375935 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fd68c066e40 msgr2=0x7fd68c069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:42.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.268+0000 7fd6a6c08640 1 --2- 172.21.15.90:0/4013375935 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fd68c066e40 0x7fd68c069300 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd694006aa0 tx=0x7fd694004780 comp rx=0 tx=0).stop 2024-03-11T20:51:42.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.268+0000 7fd6a6c08640 1 -- 172.21.15.90:0/4013375935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6a0157eb0 msgr2=0x7fd6a0144b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:42.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.268+0000 7fd6a6c08640 1 --2- 172.21.15.90:0/4013375935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6a0157eb0 0x7fd6a0144b30 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd6a00721a0 tx=0x7fd69800fcd0 comp rx=0 tx=0).stop 2024-03-11T20:51:42.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.268+0000 7fd6a6c08640 1 -- 172.21.15.90:0/4013375935 shutdown_connections 2024-03-11T20:51:42.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.268+0000 7fd6a6c08640 1 --2- 172.21.15.90:0/4013375935 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fd68c066e40 0x7fd68c069300 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:42.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.268+0000 7fd6a6c08640 1 --2- 172.21.15.90:0/4013375935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6a0157eb0 0x7fd6a0144b30 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:42.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.268+0000 7fd6a6c08640 1 --2- 172.21.15.90:0/4013375935 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fd6a013fd00 0x7fd6a01583e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:42.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.268+0000 7fd6a6c08640 1 --2- 172.21.15.90:0/4013375935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fd6880047d0 0x7fd6a01ebe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:42.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.268+0000 7fd6a6c08640 1 -- 172.21.15.90:0/4013375935 >> 172.21.15.90:0/4013375935 conn(0x7fd6a006ac00 msgr2=0x7fd6a006e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:42.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.268+0000 7fd6a6c08640 1 -- 172.21.15.90:0/4013375935 shutdown_connections 2024-03-11T20:51:42.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.268+0000 7fd6a6c08640 1 -- 172.21.15.90:0/4013375935 wait complete. 2024-03-11T20:51:42.284 INFO:tasks.ceph:Scrubbing osd.3 2024-03-11T20:51:42.284 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-03-11T20:51:42.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.404+0000 7f139ce90640 1 Processor -- start 2024-03-11T20:51:42.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.408+0000 7f139ce90640 1 -- start start 2024-03-11T20:51:42.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.408+0000 7f139ce90640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f1398070ae0 0x7f13980c3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:42.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.408+0000 7f139ce90640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1398071750 con 0x7f139806b240 2024-03-11T20:51:42.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.408+0000 7f139ce90640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1398071a30 con 0x7f139806ae60 2024-03-11T20:51:42.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.408+0000 7f139ce90640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1398071d10 con 0x7f1398070ae0 2024-03-11T20:51:42.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.408+0000 7f1396d9d640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f1398070ae0 0x7f13980c3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:42.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.408+0000 7f1396d9d640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f1398070ae0 0x7f13980c3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58636/0 (socket says 172.21.15.90:58636) 2024-03-11T20:51:42.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.408+0000 7f1396d9d640 1 -- 172.21.15.90:0/813500173 learned_addr learned my addr 172.21.15.90:0/813500173 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:51:42.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.412+0000 7f1396d9d640 1 -- 172.21.15.90:0/813500173 >> v1:172.21.15.92:6789/0 conn(0x7f139806ae60 legacy=0x7f139815a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:42.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.412+0000 7f1396d9d640 1 -- 172.21.15.90:0/813500173 >> v1:172.21.15.90:6789/0 conn(0x7f139806b240 legacy=0x7f139814ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:42.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.412+0000 7f1396d9d640 1 -- 172.21.15.90:0/813500173 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1398071ff0 con 0x7f1398070ae0 2024-03-11T20:51:42.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.420+0000 7f1396d9d640 1 --2- 172.21.15.90:0/813500173 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1398070ae0 0x7f13980c3e70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1390009e50 tx=0x7f139001f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d91b2335a4fa66f server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:42.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.504+0000 7f139559a640 1 -- 172.21.15.90:0/813500173 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13900af400 con 0x7f1398070ae0 2024-03-11T20:51:42.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.504+0000 7f139559a640 1 -- 172.21.15.90:0/813500173 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13900ab050 con 0x7f1398070ae0 2024-03-11T20:51:42.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.504+0000 7f139559a640 1 -- 172.21.15.90:0/813500173 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13900af780 con 0x7f1398070ae0 2024-03-11T20:51:42.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.504+0000 7f139ce90640 1 -- 172.21.15.90:0/813500173 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1398070ae0 msgr2=0x7f13980c3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:42.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.504+0000 7f139ce90640 1 --2- 172.21.15.90:0/813500173 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1398070ae0 0x7f13980c3e70 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1390009e50 tx=0x7f139001f1e0 comp rx=0 tx=0).stop 2024-03-11T20:51:42.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.504+0000 7f139ce90640 1 -- 172.21.15.90:0/813500173 shutdown_connections 2024-03-11T20:51:42.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.504+0000 7f139ce90640 1 --2- 172.21.15.90:0/813500173 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1398070ae0 0x7f13980c3e70 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:42.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.504+0000 7f139ce90640 1 -- 172.21.15.90:0/813500173 >> 172.21.15.90:0/813500173 conn(0x7f139806d230 msgr2=0x7f139806e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:42.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.504+0000 7f139ce90640 1 -- 172.21.15.90:0/813500173 shutdown_connections 2024-03-11T20:51:42.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.508+0000 7f139ce90640 1 -- 172.21.15.90:0/813500173 wait complete. 2024-03-11T20:51:42.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.508+0000 7f139ce90640 1 Processor -- start 2024-03-11T20:51:42.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.508+0000 7f139ce90640 1 -- start start 2024-03-11T20:51:42.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.508+0000 7f139ce90640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f139806ae60 0x7f13981f0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:42.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.508+0000 7f139ce90640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f139806b240 0x7f13980cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:42.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.508+0000 7f139ce90640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1398070ae0 0x7f1398161a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:42.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.508+0000 7f139ce90640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13980cb210 con 0x7f139806ae60 2024-03-11T20:51:42.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.508+0000 7f139ce90640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13981f0b20 con 0x7f1398070ae0 2024-03-11T20:51:42.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.508+0000 7f139ce90640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13981f14f0 con 0x7f139806b240 2024-03-11T20:51:42.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.508+0000 7f139659c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f139806ae60 0x7f13981f0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:42.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.508+0000 7f139659c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f139806ae60 0x7f13981f0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58640/0 (socket says 172.21.15.90:58640) 2024-03-11T20:51:42.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.508+0000 7f139659c640 1 -- 172.21.15.90:0/3274767343 learned_addr learned my addr 172.21.15.90:0/3274767343 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:51:42.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.512+0000 7f139659c640 1 -- 172.21.15.90:0/3274767343 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f139806b240 msgr2=0x7f13980cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:42.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.512+0000 7f139659c640 1 --2- 172.21.15.90:0/3274767343 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f139806b240 0x7f13980cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:42.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.512+0000 7f139659c640 1 -- 172.21.15.90:0/3274767343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1398070ae0 msgr2=0x7f1398161a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:42.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.512+0000 7f139659c640 1 --2- 172.21.15.90:0/3274767343 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f1398070ae0 0x7f1398161a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:42.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.512+0000 7f139659c640 1 -- 172.21.15.90:0/3274767343 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1390009b30 con 0x7f139806ae60 2024-03-11T20:51:42.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.516+0000 7f139659c640 1 --2- 172.21.15.90:0/3274767343 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f139806ae60 0x7f13981f0fb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f138800b410 tx=0x7f138800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:42.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.520+0000 7f1386ffd640 1 -- 172.21.15.90:0/3274767343 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1388011830 con 0x7f139806ae60 2024-03-11T20:51:42.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.520+0000 7f1386ffd640 1 -- 172.21.15.90:0/3274767343 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1388011990 con 0x7f139806ae60 2024-03-11T20:51:42.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.520+0000 7f1386ffd640 1 -- 172.21.15.90:0/3274767343 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1388011d40 con 0x7f139806ae60 2024-03-11T20:51:42.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.520+0000 7f139ce90640 1 -- 172.21.15.90:0/3274767343 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13980c7eb0 con 0x7f139806ae60 2024-03-11T20:51:42.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.520+0000 7f139ce90640 1 -- 172.21.15.90:0/3274767343 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1398155280 con 0x7f139806ae60 2024-03-11T20:51:42.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.520+0000 7f139ce90640 1 -- 172.21.15.90:0/3274767343 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f13980c27e0 con 0x7f139806ae60 2024-03-11T20:51:42.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.540+0000 7f1386ffd640 1 -- 172.21.15.90:0/3274767343 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f1388023430 con 0x7f139806ae60 2024-03-11T20:51:42.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.540+0000 7f1386ffd640 1 --2- 172.21.15.90:0/3274767343 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1378066ec0 0x7f1378069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:42.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.540+0000 7f1395d9b640 1 --2- 172.21.15.90:0/3274767343 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1378066ec0 0x7f1378069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:42.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.540+0000 7f1395d9b640 1 --2- 172.21.15.90:0/3274767343 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1378066ec0 0x7f1378069380 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f138c000f70 tx=0x7f138c0079e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:42.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.544+0000 7f1386ffd640 1 -- 172.21.15.90:0/3274767343 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f1388055ff0 con 0x7f139806ae60 2024-03-11T20:51:42.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.544+0000 7f1386ffd640 1 --2- 172.21.15.90:0/3274767343 >> [v2:172.21.15.90:6802/240987971,v1:172.21.15.90:6803/240987971] conn(0x7f1378070850 0x7f1378072cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:42.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.544+0000 7f1386ffd640 1 -- 172.21.15.90:0/3274767343 --> [v2:172.21.15.90:6802/240987971,v1:172.21.15.90:6803/240987971] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f13780733c0 con 0x7f1378070850 2024-03-11T20:51:42.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.544+0000 7f1396d9d640 1 --2- 172.21.15.90:0/3274767343 >> [v2:172.21.15.90:6802/240987971,v1:172.21.15.90:6803/240987971] conn(0x7f1378070850 0x7f1378072cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:42.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.544+0000 7f1396d9d640 1 --2- 172.21.15.90:0/3274767343 >> [v2:172.21.15.90:6802/240987971,v1:172.21.15.90:6803/240987971] conn(0x7f1378070850 0x7f1378072cf0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:42.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.548+0000 7f1386ffd640 1 -- 172.21.15.90:0/3274767343 <== osd.3 v2:172.21.15.90:6802/240987971 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f13780733c0 con 0x7f1378070850 2024-03-11T20:51:42.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.548+0000 7f1386ffd640 1 -- 172.21.15.90:0/3274767343 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1388011ea0 con 0x7f139806ae60 2024-03-11T20:51:42.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.560+0000 7f139ce90640 1 -- 172.21.15.90:0/3274767343 --> [v2:172.21.15.90:6802/240987971,v1:172.21.15.90:6803/240987971] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f13980c8d00 con 0x7f1378070850 2024-03-11T20:51:42.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.560+0000 7f1386ffd640 1 -- 172.21.15.90:0/3274767343 <== osd.3 v2:172.21.15.90:6802/240987971 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f13900ba450 con 0x7f1378070850 2024-03-11T20:51:42.559 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-03-11T20:51:42.560 INFO:teuthology.orchestra.run.smithi090.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-11T20:51:42.560 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-03-11T20:51:42.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.560+0000 7f139ce90640 1 -- 172.21.15.90:0/3274767343 >> [v2:172.21.15.90:6802/240987971,v1:172.21.15.90:6803/240987971] conn(0x7f1378070850 msgr2=0x7f1378072cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:42.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.560+0000 7f139ce90640 1 --2- 172.21.15.90:0/3274767343 >> [v2:172.21.15.90:6802/240987971,v1:172.21.15.90:6803/240987971] conn(0x7f1378070850 0x7f1378072cf0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:42.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.560+0000 7f139ce90640 1 -- 172.21.15.90:0/3274767343 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1378066ec0 msgr2=0x7f1378069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:42.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.560+0000 7f139ce90640 1 --2- 172.21.15.90:0/3274767343 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f1378066ec0 0x7f1378069380 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f138c000f70 tx=0x7f138c0079e0 comp rx=0 tx=0).stop 2024-03-11T20:51:42.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.560+0000 7f139ce90640 1 -- 172.21.15.90:0/3274767343 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f139806ae60 msgr2=0x7f13981f0fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:42.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.560+0000 7f139ce90640 1 --2- 172.21.15.90:0/3274767343 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f139806ae60 0x7f13981f0fb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f138800b410 tx=0x7f138800b8d0 comp rx=0 tx=0).stop 2024-03-11T20:51:42.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.560+0000 7f1396d9d640 1 -- 172.21.15.90:0/3274767343 reap_dead start 2024-03-11T20:51:42.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.560+0000 7f139ce90640 1 -- 172.21.15.90:0/3274767343 shutdown_connections 2024-03-11T20:51:42.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.560+0000 7f139ce90640 1 -- 172.21.15.90:0/3274767343 >> 172.21.15.90:0/3274767343 conn(0x7f139806d230 msgr2=0x7f139806e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:42.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.560+0000 7f139ce90640 1 -- 172.21.15.90:0/3274767343 shutdown_connections 2024-03-11T20:51:42.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.560+0000 7f139ce90640 1 -- 172.21.15.90:0/3274767343 wait complete. 2024-03-11T20:51:42.577 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-03-11T20:51:42.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.708+0000 7f8ceea05640 1 Processor -- start 2024-03-11T20:51:42.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.708+0000 7f8ceea05640 1 -- start start 2024-03-11T20:51:42.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.708+0000 7f8ceea05640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f8ce815da50 0x7f8ce80caf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:42.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.708+0000 7f8ceea05640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ce80cb5b0 con 0x7f8ce81541c0 2024-03-11T20:51:42.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.708+0000 7f8ceea05640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ce8159e10 con 0x7f8ce80cfda0 2024-03-11T20:51:42.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.708+0000 7f8ceea05640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ce80b5410 con 0x7f8ce815da50 2024-03-11T20:51:42.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.708+0000 7f8ce7fff640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f8ce81541c0 0x7f8ce80ca180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:57762/0 (socket says 172.21.15.90:57762) 2024-03-11T20:51:42.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.708+0000 7f8ce7fff640 1 -- 172.21.15.90:0/3372104242 learned_addr learned my addr 172.21.15.90:0/3372104242 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T20:51:42.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.708+0000 7f8cecfa2640 1 --2- 172.21.15.90:0/3372104242 >> v2:172.21.15.92:3300/0 conn(0x7f8ce815da50 0x7f8ce80caf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:42.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.712+0000 7f8ce77fe640 1 -- 172.21.15.90:0/3372104242 <== mon.0 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2004249540 0 0) 0x7f8ce80cb5b0 con 0x7f8ce81541c0 2024-03-11T20:51:42.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.712+0000 7f8ce77fe640 1 -- 172.21.15.90:0/3372104242 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8cd4003530 con 0x7f8ce81541c0 2024-03-11T20:51:42.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.716+0000 7f8ce77fe640 1 -- 172.21.15.90:0/3372104242 <== mon.0 v1:172.21.15.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3803185436 0 0) 0x7f8cd4003530 con 0x7f8ce81541c0 2024-03-11T20:51:42.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.716+0000 7f8ce77fe640 1 -- 172.21.15.90:0/3372104242 >> v2:172.21.15.92:3300/0 conn(0x7f8ce815da50 msgr2=0x7f8ce80caf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:42.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.716+0000 7f8ce77fe640 1 --2- 172.21.15.90:0/3372104242 >> v2:172.21.15.92:3300/0 conn(0x7f8ce815da50 0x7f8ce80caf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:42.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.716+0000 7f8ce77fe640 1 -- 172.21.15.90:0/3372104242 >> v1:172.21.15.92:6789/0 conn(0x7f8ce80cfda0 legacy=0x7f8ce815d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:42.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.716+0000 7f8ce77fe640 1 -- 172.21.15.90:0/3372104242 --> v1:172.21.15.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ce815a0f0 con 0x7f8ce81541c0 2024-03-11T20:51:42.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.716+0000 7f8ce77fe640 1 -- 172.21.15.90:0/3372104242 <== mon.0 v1:172.21.15.90:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 691735435 0 0) 0x7f8cd8001680 con 0x7f8ce81541c0 2024-03-11T20:51:42.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.716+0000 7f8ce77fe640 1 -- 172.21.15.90:0/3372104242 >> v1:172.21.15.90:6789/0 conn(0x7f8ce81541c0 legacy=0x7f8ce80ca180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:42.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.716+0000 7f8ce77fe640 1 --2- 172.21.15.90:0/3372104242 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8cd40039d0 0x7f8cd4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:42.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.716+0000 7f8ce77fe640 1 --2- 172.21.15.90:0/3372104242 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8cd40047d0 0x7f8cd4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:42.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.716+0000 7f8ce77fe640 1 --2- 172.21.15.90:0/3372104242 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8cd4005190 0x7f8cd400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:42.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.716+0000 7f8ce77fe640 1 -- 172.21.15.90:0/3372104242 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd400c010 con 0x7f8cd40039d0 2024-03-11T20:51:42.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.716+0000 7f8ce77fe640 1 -- 172.21.15.90:0/3372104242 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd400c180 con 0x7f8cd40047d0 2024-03-11T20:51:42.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.716+0000 7f8ce77fe640 1 -- 172.21.15.90:0/3372104242 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd400c340 con 0x7f8cd4005190 2024-03-11T20:51:42.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.716+0000 7f8ce7fff640 1 --2- 172.21.15.90:0/3372104242 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8cd40039d0 0x7f8cd4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:42.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.720+0000 7f8cecfa2640 1 --2- 172.21.15.90:0/3372104242 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8cd4005190 0x7f8cd400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:42.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.720+0000 7f8ce7fff640 1 -- 172.21.15.90:0/3372104242 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8cd4005190 msgr2=0x7f8cd400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:42.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.720+0000 7f8ce7fff640 1 --2- 172.21.15.90:0/3372104242 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8cd4005190 0x7f8cd400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:42.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.720+0000 7f8ce7fff640 1 -- 172.21.15.90:0/3372104242 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8cd40047d0 msgr2=0x7f8cd4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:42.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.720+0000 7f8ce7fff640 1 --2- 172.21.15.90:0/3372104242 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8cd40047d0 0x7f8cd4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:42.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.720+0000 7f8ce7fff640 1 -- 172.21.15.90:0/3372104242 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd4003530 con 0x7f8cd40039d0 2024-03-11T20:51:42.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.720+0000 7f8cecfa2640 1 -- 172.21.15.90:0/3372104242 reap_dead start 2024-03-11T20:51:42.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.728+0000 7f8ce7fff640 1 --2- 172.21.15.90:0/3372104242 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8cd40039d0 0x7f8cd4003dd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8cd800c190 tx=0x7f8cd801f310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c18ae10e8352d0b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:42.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.732+0000 7f8ce77fe640 1 -- 172.21.15.90:0/3372104242 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd80058e0 con 0x7f8cd40039d0 2024-03-11T20:51:42.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.732+0000 7f8ce77fe640 1 -- 172.21.15.90:0/3372104242 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cd8005a40 con 0x7f8cd40039d0 2024-03-11T20:51:42.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.732+0000 7f8ce77fe640 1 -- 172.21.15.90:0/3372104242 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd8005e30 con 0x7f8cd40039d0 2024-03-11T20:51:42.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.732+0000 7f8ceea05640 1 -- 172.21.15.90:0/3372104242 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8cd40039d0 msgr2=0x7f8cd4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:42.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.732+0000 7f8ceea05640 1 --2- 172.21.15.90:0/3372104242 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8cd40039d0 0x7f8cd4003dd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8cd800c190 tx=0x7f8cd801f310 comp rx=0 tx=0).stop 2024-03-11T20:51:42.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.732+0000 7f8ceea05640 1 -- 172.21.15.90:0/3372104242 shutdown_connections 2024-03-11T20:51:42.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.732+0000 7f8ceea05640 1 --2- 172.21.15.90:0/3372104242 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8cd40039d0 0x7f8cd4003dd0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:42.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.732+0000 7f8ceea05640 1 -- 172.21.15.90:0/3372104242 >> 172.21.15.90:0/3372104242 conn(0x7f8ce806b620 msgr2=0x7f8ce806ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:42.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.732+0000 7f8ceea05640 1 -- 172.21.15.90:0/3372104242 shutdown_connections 2024-03-11T20:51:42.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.732+0000 7f8ceea05640 1 -- 172.21.15.90:0/3372104242 wait complete. 2024-03-11T20:51:42.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.736+0000 7f8ceea05640 1 Processor -- start 2024-03-11T20:51:42.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.736+0000 7f8ceea05640 1 -- start start 2024-03-11T20:51:42.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.736+0000 7f8ceea05640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ce8156fa0 0x7f8ce815b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:42.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.736+0000 7f8ceea05640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8ce81555d0 0x7f8ce81519d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:42.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.736+0000 7f8ceea05640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ce815c7b0 0x7f8ce80bfec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:42.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.736+0000 7f8ceea05640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce8156a60 con 0x7f8ce8156fa0 2024-03-11T20:51:42.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.736+0000 7f8ceea05640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce815be10 con 0x7f8ce815c7b0 2024-03-11T20:51:42.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.736+0000 7f8ceea05640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce815b5f0 con 0x7f8ce81555d0 2024-03-11T20:51:42.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.736+0000 7f8ce7fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ce8156fa0 0x7f8ce815b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:42.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.736+0000 7f8cdffff640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8ce81555d0 0x7f8ce81519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:42.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.736+0000 7f8ce7fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ce8156fa0 0x7f8ce815b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58656/0 (socket says 172.21.15.90:58656) 2024-03-11T20:51:42.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.736+0000 7f8ce7fff640 1 -- 172.21.15.90:0/3593113347 learned_addr learned my addr 172.21.15.90:0/3593113347 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:51:42.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.736+0000 7f8cecfa2640 1 --2- 172.21.15.90:0/3593113347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ce815c7b0 0x7f8ce80bfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:42.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.740+0000 7f8ce7fff640 1 -- 172.21.15.90:0/3593113347 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8ce81555d0 msgr2=0x7f8ce81519d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:42.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.740+0000 7f8ce7fff640 1 --2- 172.21.15.90:0/3593113347 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8ce81555d0 0x7f8ce81519d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:42.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.740+0000 7f8ce7fff640 1 -- 172.21.15.90:0/3593113347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ce815c7b0 msgr2=0x7f8ce80bfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:42.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.740+0000 7f8ce7fff640 1 --2- 172.21.15.90:0/3593113347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ce815c7b0 0x7f8ce80bfec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:42.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.740+0000 7f8ce7fff640 1 -- 172.21.15.90:0/3593113347 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ce8154810 con 0x7f8ce8156fa0 2024-03-11T20:51:42.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.740+0000 7f8ce7fff640 1 --2- 172.21.15.90:0/3593113347 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ce8156fa0 0x7f8ce815b8d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8cd800c190 tx=0x7f8cd4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:42.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.744+0000 7f8ce57fa640 1 -- 172.21.15.90:0/3593113347 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd80b7070 con 0x7f8ce8156fa0 2024-03-11T20:51:42.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.744+0000 7f8ce57fa640 1 -- 172.21.15.90:0/3593113347 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cd8005bb0 con 0x7f8ce8156fa0 2024-03-11T20:51:42.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.744+0000 7f8ce57fa640 1 -- 172.21.15.90:0/3593113347 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd8007030 con 0x7f8ce8156fa0 2024-03-11T20:51:42.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.744+0000 7f8ceea05640 1 -- 172.21.15.90:0/3593113347 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ce8154af0 con 0x7f8ce8156fa0 2024-03-11T20:51:42.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.744+0000 7f8ceea05640 1 -- 172.21.15.90:0/3593113347 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ce81550b0 con 0x7f8ce8156fa0 2024-03-11T20:51:42.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.748+0000 7f8ceea05640 1 -- 172.21.15.90:0/3593113347 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ce804eef0 con 0x7f8ce8156fa0 2024-03-11T20:51:42.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.768+0000 7f8ce57fa640 1 -- 172.21.15.90:0/3593113347 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f8cd8007190 con 0x7f8ce8156fa0 2024-03-11T20:51:42.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.768+0000 7f8ce57fa640 1 --2- 172.21.15.90:0/3593113347 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f8cc8066e90 0x7f8cc8069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:42.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.768+0000 7f8cdffff640 1 --2- 172.21.15.90:0/3593113347 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f8cc8066e90 0x7f8cc8069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:42.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.768+0000 7f8cdffff640 1 --2- 172.21.15.90:0/3593113347 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f8cc8066e90 0x7f8cc8069350 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8ce80c7890 tx=0x7f8cd0007dc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:42.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:42.772+0000 7f8ce57fa640 1 -- 172.21.15.90:0/3593113347 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f8cd80f2780 con 0x7f8ce8156fa0 2024-03-11T20:51:43.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.020+0000 7f8ce57fa640 1 -- 172.21.15.90:0/3593113347 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8cd80eea20 con 0x7f8ce8156fa0 2024-03-11T20:51:43.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.192+0000 7f8ceea05640 1 -- 172.21.15.90:0/3593113347 --> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f8ce80c6eb0 con 0x7f8cc8066e90 2024-03-11T20:51:43.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.192+0000 7f8ce57fa640 1 -- 172.21.15.90:0/3593113347 <== mgr.4101 v2:172.21.15.90:6800/1999720161 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f8ce80c6eb0 con 0x7f8cc8066e90 2024-03-11T20:51:43.191 INFO:teuthology.orchestra.run.smithi090.stderr:instructed osd(s) 3 to deep-scrub 2024-03-11T20:51:43.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.196+0000 7f8ceea05640 1 -- 172.21.15.90:0/3593113347 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f8cc8066e90 msgr2=0x7f8cc8069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:43.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.196+0000 7f8ceea05640 1 --2- 172.21.15.90:0/3593113347 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f8cc8066e90 0x7f8cc8069350 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8ce80c7890 tx=0x7f8cd0007dc0 comp rx=0 tx=0).stop 2024-03-11T20:51:43.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.196+0000 7f8ceea05640 1 -- 172.21.15.90:0/3593113347 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ce8156fa0 msgr2=0x7f8ce815b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:43.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.196+0000 7f8ceea05640 1 --2- 172.21.15.90:0/3593113347 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ce8156fa0 0x7f8ce815b8d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8cd800c190 tx=0x7f8cd4005800 comp rx=0 tx=0).stop 2024-03-11T20:51:43.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.196+0000 7f8ceea05640 1 -- 172.21.15.90:0/3593113347 shutdown_connections 2024-03-11T20:51:43.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.196+0000 7f8ceea05640 1 --2- 172.21.15.90:0/3593113347 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f8cc8066e90 0x7f8cc8069350 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:43.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.196+0000 7f8ceea05640 1 --2- 172.21.15.90:0/3593113347 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f8ce815c7b0 0x7f8ce80bfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:43.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.196+0000 7f8ceea05640 1 --2- 172.21.15.90:0/3593113347 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f8ce81555d0 0x7f8ce81519d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:43.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.196+0000 7f8ceea05640 1 --2- 172.21.15.90:0/3593113347 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ce8156fa0 0x7f8ce815b8d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:43.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.196+0000 7f8ceea05640 1 -- 172.21.15.90:0/3593113347 >> 172.21.15.90:0/3593113347 conn(0x7f8ce806b620 msgr2=0x7f8ce80ca140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:43.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.196+0000 7f8ceea05640 1 -- 172.21.15.90:0/3593113347 shutdown_connections 2024-03-11T20:51:43.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.196+0000 7f8ceea05640 1 -- 172.21.15.90:0/3593113347 wait complete. 2024-03-11T20:51:43.212 INFO:tasks.ceph:Scrubbing osd.4 2024-03-11T20:51:43.212 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-03-11T20:51:43.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.336+0000 7fcb2197c640 1 Processor -- start 2024-03-11T20:51:43.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.336+0000 7fcb2197c640 1 -- start start 2024-03-11T20:51:43.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.336+0000 7fcb2197c640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fcb1c15a440 0x7fcb1c15a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:43.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.336+0000 7fcb2197c640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fcb1c0c7e70 0x7fcb1c0c82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:43.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.336+0000 7fcb2197c640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb1c15adf0 con 0x7fcb1c150c90 2024-03-11T20:51:43.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.336+0000 7fcb2197c640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb1c159f60 con 0x7fcb1c15a440 2024-03-11T20:51:43.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.336+0000 7fcb2197c640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1c0c2be0 con 0x7fcb1c0c7e70 2024-03-11T20:51:43.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.336+0000 7fcb1affd640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7fcb1c150c90 0x7fcb1c0727c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:57770/0 (socket says 172.21.15.90:57770) 2024-03-11T20:51:43.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.336+0000 7fcb1affd640 1 -- 172.21.15.90:0/230698524 learned_addr learned my addr 172.21.15.90:0/230698524 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T20:51:43.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.336+0000 7fcb1b7fe640 1 --2- 172.21.15.90:0/230698524 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fcb1c0c7e70 0x7fcb1c0c82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:43.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.340+0000 7fcb1a7fc640 1 --2- 172.21.15.90:0/230698524 >> v2:172.21.15.92:3300/0 conn(0x7fcb1c15a440 0x7fcb1c15a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:43.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.340+0000 7fcb19ffb640 1 -- 172.21.15.90:0/230698524 <== mon.0 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3697017029 0 0) 0x7fcb1c15adf0 con 0x7fcb1c150c90 2024-03-11T20:51:43.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.340+0000 7fcb19ffb640 1 -- 172.21.15.90:0/230698524 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb00003530 con 0x7fcb1c150c90 2024-03-11T20:51:43.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.344+0000 7fcb19ffb640 1 -- 172.21.15.90:0/230698524 <== mon.0 v1:172.21.15.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 221688223 0 0) 0x7fcb00003530 con 0x7fcb1c150c90 2024-03-11T20:51:43.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.344+0000 7fcb19ffb640 1 -- 172.21.15.90:0/230698524 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fcb1c0c7e70 msgr2=0x7fcb1c0c82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:43.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.344+0000 7fcb19ffb640 1 --2- 172.21.15.90:0/230698524 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fcb1c0c7e70 0x7fcb1c0c82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:43.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.344+0000 7fcb19ffb640 1 -- 172.21.15.90:0/230698524 >> v2:172.21.15.92:3300/0 conn(0x7fcb1c15a440 msgr2=0x7fcb1c15a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:43.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.344+0000 7fcb19ffb640 1 --2- 172.21.15.90:0/230698524 >> v2:172.21.15.92:3300/0 conn(0x7fcb1c15a440 0x7fcb1c15a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:43.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.344+0000 7fcb19ffb640 1 -- 172.21.15.90:0/230698524 --> v1:172.21.15.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb1c1569f0 con 0x7fcb1c150c90 2024-03-11T20:51:43.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.344+0000 7fcb19ffb640 1 -- 172.21.15.90:0/230698524 <== mon.0 v1:172.21.15.90:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 691735435 0 0) 0x7fcb0c001680 con 0x7fcb1c150c90 2024-03-11T20:51:43.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.344+0000 7fcb19ffb640 1 -- 172.21.15.90:0/230698524 >> v1:172.21.15.90:6789/0 conn(0x7fcb1c150c90 legacy=0x7fcb1c0727c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:43.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.344+0000 7fcb19ffb640 1 --2- 172.21.15.90:0/230698524 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb00003980 0x7fcb00003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:43.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.344+0000 7fcb19ffb640 1 --2- 172.21.15.90:0/230698524 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fcb1c0c7e70 0x7fcb000047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:43.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.344+0000 7fcb19ffb640 1 --2- 172.21.15.90:0/230698524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb00004d10 0x7fcb00005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:43.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.344+0000 7fcb19ffb640 1 -- 172.21.15.90:0/230698524 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1c159f60 con 0x7fcb00003980 2024-03-11T20:51:43.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.344+0000 7fcb19ffb640 1 -- 172.21.15.90:0/230698524 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1c0c2be0 con 0x7fcb00004d10 2024-03-11T20:51:43.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.344+0000 7fcb19ffb640 1 -- 172.21.15.90:0/230698524 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0000b9c0 con 0x7fcb1c0c7e70 2024-03-11T20:51:43.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.348+0000 7fcb1affd640 1 --2- 172.21.15.90:0/230698524 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb00003980 0x7fcb00003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:43.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.352+0000 7fcb1affd640 1 -- 172.21.15.90:0/230698524 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fcb1c0c7e70 msgr2=0x7fcb000047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:43.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.352+0000 7fcb1affd640 1 --2- 172.21.15.90:0/230698524 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fcb1c0c7e70 0x7fcb000047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:43.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.352+0000 7fcb1affd640 1 -- 172.21.15.90:0/230698524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb00004d10 msgr2=0x7fcb00005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:43.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.352+0000 7fcb1affd640 1 --2- 172.21.15.90:0/230698524 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb00004d10 0x7fcb00005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:43.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.352+0000 7fcb1affd640 1 -- 172.21.15.90:0/230698524 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb00003530 con 0x7fcb00003980 2024-03-11T20:51:43.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.356+0000 7fcb1affd640 1 --2- 172.21.15.90:0/230698524 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb00003980 0x7fcb00003da0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c00c110 tx=0x7fcb0c021520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bed3a3f5763afee server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:43.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.360+0000 7fcb19ffb640 1 -- 172.21.15.90:0/230698524 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c005ae0 con 0x7fcb00003980 2024-03-11T20:51:43.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.360+0000 7fcb19ffb640 1 -- 172.21.15.90:0/230698524 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb0c0ad070 con 0x7fcb00003980 2024-03-11T20:51:43.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.360+0000 7fcb19ffb640 1 -- 172.21.15.90:0/230698524 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c005ea0 con 0x7fcb00003980 2024-03-11T20:51:43.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.360+0000 7fcb2197c640 1 -- 172.21.15.90:0/230698524 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb00003980 msgr2=0x7fcb00003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:43.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.360+0000 7fcb2197c640 1 --2- 172.21.15.90:0/230698524 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb00003980 0x7fcb00003da0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c00c110 tx=0x7fcb0c021520 comp rx=0 tx=0).stop 2024-03-11T20:51:43.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.360+0000 7fcb1b7fe640 1 -- 172.21.15.90:0/230698524 reap_dead start 2024-03-11T20:51:43.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.360+0000 7fcb2197c640 1 -- 172.21.15.90:0/230698524 shutdown_connections 2024-03-11T20:51:43.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.360+0000 7fcb2197c640 1 -- 172.21.15.90:0/230698524 >> 172.21.15.90:0/230698524 conn(0x7fcb1c06d230 msgr2=0x7fcb1c06e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:43.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.360+0000 7fcb2197c640 1 -- 172.21.15.90:0/230698524 shutdown_connections 2024-03-11T20:51:43.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.360+0000 7fcb2197c640 1 -- 172.21.15.90:0/230698524 wait complete. 2024-03-11T20:51:43.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.364+0000 7fcb2197c640 1 Processor -- start 2024-03-11T20:51:43.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.364+0000 7fcb2197c640 1 -- start start 2024-03-11T20:51:43.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.364+0000 7fcb2197c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb00003980 0x7fcb1c1f0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:43.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.364+0000 7fcb2197c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb1c156610 0x7fcb1c0ca8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:43.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.364+0000 7fcb2197c640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fcb1c154fd0 0x7fcb1c1f2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:43.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.364+0000 7fcb2197c640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1c154bd0 con 0x7fcb1c156610 2024-03-11T20:51:43.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.364+0000 7fcb2197c640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1c14fcb0 con 0x7fcb00003980 2024-03-11T20:51:43.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.364+0000 7fcb2197c640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1c14ff90 con 0x7fcb1c154fd0 2024-03-11T20:51:43.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.364+0000 7fcb1a7fc640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb1c156610 0x7fcb1c0ca8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:43.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.364+0000 7fcb1a7fc640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb1c156610 0x7fcb1c0ca8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58680/0 (socket says 172.21.15.90:58680) 2024-03-11T20:51:43.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.364+0000 7fcb1a7fc640 1 -- 172.21.15.90:0/4248719302 learned_addr learned my addr 172.21.15.90:0/4248719302 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:51:43.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.364+0000 7fcb1b7fe640 1 --2- 172.21.15.90:0/4248719302 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fcb1c154fd0 0x7fcb1c1f2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:43.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.364+0000 7fcb1affd640 1 --2- 172.21.15.90:0/4248719302 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb00003980 0x7fcb1c1f0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:43.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.368+0000 7fcb1a7fc640 1 -- 172.21.15.90:0/4248719302 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fcb1c154fd0 msgr2=0x7fcb1c1f2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:43.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.368+0000 7fcb1a7fc640 1 --2- 172.21.15.90:0/4248719302 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fcb1c154fd0 0x7fcb1c1f2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:43.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.368+0000 7fcb1a7fc640 1 -- 172.21.15.90:0/4248719302 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb00003980 msgr2=0x7fcb1c1f0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:43.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.368+0000 7fcb1a7fc640 1 --2- 172.21.15.90:0/4248719302 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcb00003980 0x7fcb1c1f0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:43.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.368+0000 7fcb1a7fc640 1 -- 172.21.15.90:0/4248719302 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb1c1547f0 con 0x7fcb1c156610 2024-03-11T20:51:43.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.372+0000 7fcb1a7fc640 1 --2- 172.21.15.90:0/4248719302 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb1c156610 0x7fcb1c0ca8b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcb0800e7c0 tx=0x7fcb0800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:43.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.376+0000 7fcb13fff640 1 -- 172.21.15.90:0/4248719302 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0800eed0 con 0x7fcb1c156610 2024-03-11T20:51:43.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.376+0000 7fcb13fff640 1 -- 172.21.15.90:0/4248719302 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb0800c940 con 0x7fcb1c156610 2024-03-11T20:51:43.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.376+0000 7fcb13fff640 1 -- 172.21.15.90:0/4248719302 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0800ccf0 con 0x7fcb1c156610 2024-03-11T20:51:43.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.376+0000 7fcb2197c640 1 -- 172.21.15.90:0/4248719302 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb1c14f9b0 con 0x7fcb1c156610 2024-03-11T20:51:43.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.376+0000 7fcb2197c640 1 -- 172.21.15.90:0/4248719302 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb1c0cadf0 con 0x7fcb1c156610 2024-03-11T20:51:43.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.376+0000 7fcb2197c640 1 -- 172.21.15.90:0/4248719302 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcb1c157270 con 0x7fcb1c156610 2024-03-11T20:51:43.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.400+0000 7fcb13fff640 1 -- 172.21.15.90:0/4248719302 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fcb08013470 con 0x7fcb1c156610 2024-03-11T20:51:43.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.400+0000 7fcb13fff640 1 --2- 172.21.15.90:0/4248719302 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fcb04066e90 0x7fcb04069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:43.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.400+0000 7fcb1affd640 1 --2- 172.21.15.90:0/4248719302 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fcb04066e90 0x7fcb04069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:43.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.400+0000 7fcb1affd640 1 --2- 172.21.15.90:0/4248719302 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fcb04066e90 0x7fcb04069350 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c00c110 tx=0x7fcb0c0b4040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:43.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.404+0000 7fcb13fff640 1 -- 172.21.15.90:0/4248719302 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fcb08055980 con 0x7fcb1c156610 2024-03-11T20:51:43.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.404+0000 7fcb13fff640 1 --2- 172.21.15.90:0/4248719302 >> [v2:172.21.15.92:6800/502685275,v1:172.21.15.92:6801/502685275] conn(0x7fcb040706b0 0x7fcb04072b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:43.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.404+0000 7fcb13fff640 1 -- 172.21.15.90:0/4248719302 --> [v2:172.21.15.92:6800/502685275,v1:172.21.15.92:6801/502685275] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcb04073220 con 0x7fcb040706b0 2024-03-11T20:51:43.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.404+0000 7fcb13fff640 1 -- 172.21.15.90:0/4248719302 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb08051c20 con 0x7fcb1c156610 2024-03-11T20:51:43.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.404+0000 7fcb1b7fe640 1 --2- 172.21.15.90:0/4248719302 >> [v2:172.21.15.92:6800/502685275,v1:172.21.15.92:6801/502685275] conn(0x7fcb040706b0 0x7fcb04072b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:43.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.408+0000 7fcb1b7fe640 1 --2- 172.21.15.90:0/4248719302 >> [v2:172.21.15.92:6800/502685275,v1:172.21.15.92:6801/502685275] conn(0x7fcb040706b0 0x7fcb04072b50 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:43.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.408+0000 7fcb13fff640 1 -- 172.21.15.90:0/4248719302 <== osd.4 v2:172.21.15.92:6800/502685275 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fcb04073220 con 0x7fcb040706b0 2024-03-11T20:51:43.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.420+0000 7fcb2197c640 1 -- 172.21.15.90:0/4248719302 --> [v2:172.21.15.92:6800/502685275,v1:172.21.15.92:6801/502685275] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fcb1c156f90 con 0x7fcb040706b0 2024-03-11T20:51:43.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.424+0000 7fcb13fff640 1 -- 172.21.15.90:0/4248719302 <== osd.4 v2:172.21.15.92:6800/502685275 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fcb14015370 con 0x7fcb040706b0 2024-03-11T20:51:43.422 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-03-11T20:51:43.422 INFO:teuthology.orchestra.run.smithi090.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-11T20:51:43.422 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-03-11T20:51:43.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.424+0000 7fcb2197c640 1 -- 172.21.15.90:0/4248719302 >> [v2:172.21.15.92:6800/502685275,v1:172.21.15.92:6801/502685275] conn(0x7fcb040706b0 msgr2=0x7fcb04072b50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:43.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.424+0000 7fcb2197c640 1 --2- 172.21.15.90:0/4248719302 >> [v2:172.21.15.92:6800/502685275,v1:172.21.15.92:6801/502685275] conn(0x7fcb040706b0 0x7fcb04072b50 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:43.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.424+0000 7fcb2197c640 1 -- 172.21.15.90:0/4248719302 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fcb04066e90 msgr2=0x7fcb04069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:43.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.424+0000 7fcb2197c640 1 --2- 172.21.15.90:0/4248719302 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fcb04066e90 0x7fcb04069350 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c00c110 tx=0x7fcb0c0b4040 comp rx=0 tx=0).stop 2024-03-11T20:51:43.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.424+0000 7fcb2197c640 1 -- 172.21.15.90:0/4248719302 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb1c156610 msgr2=0x7fcb1c0ca8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:43.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.424+0000 7fcb2197c640 1 --2- 172.21.15.90:0/4248719302 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb1c156610 0x7fcb1c0ca8b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcb0800e7c0 tx=0x7fcb0800ec80 comp rx=0 tx=0).stop 2024-03-11T20:51:43.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.424+0000 7fcb1b7fe640 1 -- 172.21.15.90:0/4248719302 reap_dead start 2024-03-11T20:51:43.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.424+0000 7fcb2197c640 1 -- 172.21.15.90:0/4248719302 shutdown_connections 2024-03-11T20:51:43.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.424+0000 7fcb2197c640 1 -- 172.21.15.90:0/4248719302 >> 172.21.15.90:0/4248719302 conn(0x7fcb1c06d230 msgr2=0x7fcb1c15a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:43.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.424+0000 7fcb2197c640 1 -- 172.21.15.90:0/4248719302 shutdown_connections 2024-03-11T20:51:43.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.424+0000 7fcb2197c640 1 -- 172.21.15.90:0/4248719302 wait complete. 2024-03-11T20:51:43.438 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-03-11T20:51:43.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.556+0000 7f4d6a5d9640 1 Processor -- start 2024-03-11T20:51:43.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.556+0000 7f4d6a5d9640 1 -- start start 2024-03-11T20:51:43.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.556+0000 7f4d6a5d9640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f4d64153fc0 0x7f4d6406bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:43.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.556+0000 7f4d6a5d9640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f4d6415a950 0x7f4d6406c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:43.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.556+0000 7f4d6a5d9640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d64159d80 con 0x7f4d640cc3a0 2024-03-11T20:51:43.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.556+0000 7f4d6a5d9640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d640c7990 con 0x7f4d6415a950 2024-03-11T20:51:43.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.556+0000 7f4d6a5d9640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d640c6e10 con 0x7f4d64153fc0 2024-03-11T20:51:43.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.556+0000 7f4d68b76640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f4d6415a950 0x7f4d6406c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:43.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.556+0000 7f4d637fe640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f4d640cc3a0 0x7f4d6415a240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:45046/0 (socket says 172.21.15.90:45046) 2024-03-11T20:51:43.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.556+0000 7f4d637fe640 1 -- 172.21.15.90:0/2905240360 learned_addr learned my addr 172.21.15.90:0/2905240360 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T20:51:43.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.556+0000 7f4d63fff640 1 --2- 172.21.15.90:0/2905240360 >> v2:172.21.15.92:3300/0 conn(0x7f4d64153fc0 0x7f4d6406bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:43.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.560+0000 7f4d68b76640 1 -- 172.21.15.90:0/2905240360 >> v2:172.21.15.92:3300/0 conn(0x7f4d64153fc0 msgr2=0x7f4d6406bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:43.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.560+0000 7f4d68b76640 1 --2- 172.21.15.90:0/2905240360 >> v2:172.21.15.92:3300/0 conn(0x7f4d64153fc0 0x7f4d6406bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:43.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.560+0000 7f4d68b76640 1 -- 172.21.15.90:0/2905240360 >> v1:172.21.15.90:6789/0 conn(0x7f4d640cc3a0 legacy=0x7f4d6415a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:43.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.560+0000 7f4d68b76640 1 -- 172.21.15.90:0/2905240360 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d640c70f0 con 0x7f4d6415a950 2024-03-11T20:51:43.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.564+0000 7f4d68b76640 1 --2- 172.21.15.90:0/2905240360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4d6415a950 0x7f4d6406c150 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4d640c7ad0 tx=0x7f4d5401ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdbb962dd615c31b server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:43.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.580+0000 7f4d62ffd640 1 -- 172.21.15.90:0/2905240360 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d540a8950 con 0x7f4d6415a950 2024-03-11T20:51:43.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.580+0000 7f4d62ffd640 1 -- 172.21.15.90:0/2905240360 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d540a8ab0 con 0x7f4d6415a950 2024-03-11T20:51:43.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.580+0000 7f4d62ffd640 1 -- 172.21.15.90:0/2905240360 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d540a8e40 con 0x7f4d6415a950 2024-03-11T20:51:43.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.580+0000 7f4d6a5d9640 1 -- 172.21.15.90:0/2905240360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4d6415a950 msgr2=0x7f4d6406c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:43.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.580+0000 7f4d6a5d9640 1 --2- 172.21.15.90:0/2905240360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4d6415a950 0x7f4d6406c150 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4d640c7ad0 tx=0x7f4d5401ec50 comp rx=0 tx=0).stop 2024-03-11T20:51:43.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.580+0000 7f4d6a5d9640 1 -- 172.21.15.90:0/2905240360 shutdown_connections 2024-03-11T20:51:43.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.580+0000 7f4d6a5d9640 1 --2- 172.21.15.90:0/2905240360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4d6415a950 0x7f4d6406c150 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:43.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.580+0000 7f4d6a5d9640 1 --2- 172.21.15.90:0/2905240360 >> v2:172.21.15.92:3300/0 conn(0x7f4d64153fc0 0x7f4d6406bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:43.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.580+0000 7f4d6a5d9640 1 -- 172.21.15.90:0/2905240360 >> 172.21.15.90:0/2905240360 conn(0x7f4d640cb880 msgr2=0x7f4d640cbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:43.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.580+0000 7f4d6a5d9640 1 -- 172.21.15.90:0/2905240360 shutdown_connections 2024-03-11T20:51:43.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.580+0000 7f4d6a5d9640 1 -- 172.21.15.90:0/2905240360 wait complete. 2024-03-11T20:51:43.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.580+0000 7f4d6a5d9640 1 Processor -- start 2024-03-11T20:51:43.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.580+0000 7f4d6a5d9640 1 -- start start 2024-03-11T20:51:43.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.584+0000 7f4d6a5d9640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4d640cc3a0 0x7f4d640c59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:43.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.584+0000 7f4d6a5d9640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4d64153fc0 0x7f4d641f01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:43.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.584+0000 7f4d6a5d9640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d6415a950 0x7f4d640c3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:43.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.584+0000 7f4d6a5d9640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d641516a0 con 0x7f4d640cc3a0 2024-03-11T20:51:43.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.584+0000 7f4d6a5d9640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d641efaa0 con 0x7f4d6415a950 2024-03-11T20:51:43.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.584+0000 7f4d6a5d9640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d64145280 con 0x7f4d64153fc0 2024-03-11T20:51:43.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.584+0000 7f4d63fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4d640cc3a0 0x7f4d640c59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:43.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.584+0000 7f4d63fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4d640cc3a0 0x7f4d640c59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43974/0 (socket says 172.21.15.90:43974) 2024-03-11T20:51:43.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.584+0000 7f4d63fff640 1 -- 172.21.15.90:0/3487572329 learned_addr learned my addr 172.21.15.90:0/3487572329 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:51:43.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.584+0000 7f4d68b76640 1 --2- 172.21.15.90:0/3487572329 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d6415a950 0x7f4d640c3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:43.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.584+0000 7f4d637fe640 1 --2- 172.21.15.90:0/3487572329 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4d64153fc0 0x7f4d641f01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:43.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.588+0000 7f4d63fff640 1 -- 172.21.15.90:0/3487572329 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4d64153fc0 msgr2=0x7f4d641f01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:43.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.588+0000 7f4d63fff640 1 --2- 172.21.15.90:0/3487572329 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4d64153fc0 0x7f4d641f01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:43.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.588+0000 7f4d63fff640 1 -- 172.21.15.90:0/3487572329 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d6415a950 msgr2=0x7f4d640c3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:43.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.588+0000 7f4d63fff640 1 --2- 172.21.15.90:0/3487572329 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d6415a950 0x7f4d640c3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:43.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.588+0000 7f4d63fff640 1 -- 172.21.15.90:0/3487572329 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d54009420 con 0x7f4d640cc3a0 2024-03-11T20:51:43.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.592+0000 7f4d63fff640 1 --2- 172.21.15.90:0/3487572329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4d640cc3a0 0x7f4d640c59b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c009db0 tx=0x7f4d5c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:43.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.676+0000 7f4d60ff9640 1 -- 172.21.15.90:0/3487572329 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c014480 con 0x7f4d640cc3a0 2024-03-11T20:51:43.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.676+0000 7f4d60ff9640 1 -- 172.21.15.90:0/3487572329 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d5c00f040 con 0x7f4d640cc3a0 2024-03-11T20:51:43.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.676+0000 7f4d60ff9640 1 -- 172.21.15.90:0/3487572329 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c014800 con 0x7f4d640cc3a0 2024-03-11T20:51:43.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.676+0000 7f4d6a5d9640 1 -- 172.21.15.90:0/3487572329 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d6414a090 con 0x7f4d640cc3a0 2024-03-11T20:51:43.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.680+0000 7f4d6a5d9640 1 -- 172.21.15.90:0/3487572329 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d6414ae60 con 0x7f4d640cc3a0 2024-03-11T20:51:43.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:43.680+0000 7f4d6a5d9640 1 -- 172.21.15.90:0/3487572329 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d6406a1b0 con 0x7f4d640cc3a0 2024-03-11T20:51:44.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.016+0000 7f4d60ff9640 1 -- 172.21.15.90:0/3487572329 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f4d5c014960 con 0x7f4d640cc3a0 2024-03-11T20:51:44.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.020+0000 7f4d60ff9640 1 --2- 172.21.15.90:0/3487572329 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f4d50066ec0 0x7f4d50069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:44.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.020+0000 7f4d637fe640 1 --2- 172.21.15.90:0/3487572329 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f4d50066ec0 0x7f4d50069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:44.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.020+0000 7f4d637fe640 1 --2- 172.21.15.90:0/3487572329 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f4d50066ec0 0x7f4d50069380 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4d640ca300 tx=0x7f4d58009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:44.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.056+0000 7f4d60ff9640 1 -- 172.21.15.90:0/3487572329 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f4d5c056f40 con 0x7f4d640cc3a0 2024-03-11T20:51:44.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.056+0000 7f4d60ff9640 1 -- 172.21.15.90:0/3487572329 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4d5c052ae0 con 0x7f4d640cc3a0 2024-03-11T20:51:44.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.256+0000 7f4d6a5d9640 1 -- 172.21.15.90:0/3487572329 --> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f4d640c2a00 con 0x7f4d50066ec0 2024-03-11T20:51:44.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.256+0000 7f4d60ff9640 1 -- 172.21.15.90:0/3487572329 <== mgr.4101 v2:172.21.15.90:6800/1999720161 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4d58001350 con 0x7f4d50066ec0 2024-03-11T20:51:44.256 INFO:teuthology.orchestra.run.smithi090.stderr:instructed osd(s) 4 to deep-scrub 2024-03-11T20:51:44.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.260+0000 7f4d6a5d9640 1 -- 172.21.15.90:0/3487572329 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f4d50066ec0 msgr2=0x7f4d50069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:44.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.260+0000 7f4d6a5d9640 1 --2- 172.21.15.90:0/3487572329 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f4d50066ec0 0x7f4d50069380 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4d640ca300 tx=0x7f4d58009040 comp rx=0 tx=0).stop 2024-03-11T20:51:44.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.260+0000 7f4d6a5d9640 1 -- 172.21.15.90:0/3487572329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4d640cc3a0 msgr2=0x7f4d640c59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:44.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.260+0000 7f4d6a5d9640 1 --2- 172.21.15.90:0/3487572329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4d640cc3a0 0x7f4d640c59b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c009db0 tx=0x7f4d5c00c490 comp rx=0 tx=0).stop 2024-03-11T20:51:44.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.260+0000 7f4d6a5d9640 1 -- 172.21.15.90:0/3487572329 shutdown_connections 2024-03-11T20:51:44.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.260+0000 7f4d6a5d9640 1 --2- 172.21.15.90:0/3487572329 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f4d50066ec0 0x7f4d50069380 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:44.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.260+0000 7f4d6a5d9640 1 --2- 172.21.15.90:0/3487572329 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f4d6415a950 0x7f4d640c3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:44.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.260+0000 7f4d6a5d9640 1 --2- 172.21.15.90:0/3487572329 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f4d64153fc0 0x7f4d641f01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:44.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.260+0000 7f4d6a5d9640 1 --2- 172.21.15.90:0/3487572329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4d640cc3a0 0x7f4d640c59b0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:44.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.260+0000 7f4d6a5d9640 1 -- 172.21.15.90:0/3487572329 >> 172.21.15.90:0/3487572329 conn(0x7f4d640cb880 msgr2=0x7f4d64149520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:44.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.260+0000 7f4d6a5d9640 1 -- 172.21.15.90:0/3487572329 shutdown_connections 2024-03-11T20:51:44.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.260+0000 7f4d6a5d9640 1 -- 172.21.15.90:0/3487572329 wait complete. 2024-03-11T20:51:44.276 INFO:tasks.ceph:Scrubbing osd.5 2024-03-11T20:51:44.276 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-03-11T20:51:44.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.404+0000 7fcaf3fe3640 1 Processor -- start 2024-03-11T20:51:44.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.404+0000 7fcaf3fe3640 1 -- start start 2024-03-11T20:51:44.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.404+0000 7fcaf3fe3640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fcaec0cf5f0 0x7fcaec15a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:44.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.404+0000 7fcaf3fe3640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fcaec153a30 0x7fcaec153ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:44.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.404+0000 7fcaf3fe3640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaec0cbbe0 con 0x7fcaec0cf9d0 2024-03-11T20:51:44.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.404+0000 7fcaf3fe3640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaec0cadd0 con 0x7fcaec0cf5f0 2024-03-11T20:51:44.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.404+0000 7fcaf3fe3640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaec1543e0 con 0x7fcaec153a30 2024-03-11T20:51:44.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.404+0000 7fcaf1d7f640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fcaec0cf5f0 0x7fcaec15a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:44.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.404+0000 7fcaf1d7f640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fcaec0cf5f0 0x7fcaec15a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43978/0 (socket says 172.21.15.90:43978) 2024-03-11T20:51:44.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.404+0000 7fcaf1d7f640 1 -- 172.21.15.90:0/2785974599 learned_addr learned my addr 172.21.15.90:0/2785974599 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:51:44.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.408+0000 7fcaf2580640 1 --2- 172.21.15.90:0/2785974599 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fcaec153a30 0x7fcaec153ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:44.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.408+0000 7fcaf1d7f640 1 -- 172.21.15.90:0/2785974599 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fcaec153a30 msgr2=0x7fcaec153ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:44.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.408+0000 7fcaf1d7f640 1 --2- 172.21.15.90:0/2785974599 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fcaec153a30 0x7fcaec153ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:44.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.408+0000 7fcaf1d7f640 1 -- 172.21.15.90:0/2785974599 >> v1:172.21.15.92:6789/0 conn(0x7fcaec0cf9d0 legacy=0x7fcaec0ca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:44.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.408+0000 7fcaf1d7f640 1 -- 172.21.15.90:0/2785974599 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaec15abb0 con 0x7fcaec0cf5f0 2024-03-11T20:51:44.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.412+0000 7fcaf1d7f640 1 --2- 172.21.15.90:0/2785974599 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcaec0cf5f0 0x7fcaec15a670 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fcae8009870 tx=0x7fcae801ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e8d525e860181fd server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:44.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.420+0000 7fcaf0d7d640 1 -- 172.21.15.90:0/2785974599 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae80a8a50 con 0x7fcaec0cf5f0 2024-03-11T20:51:44.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.420+0000 7fcaf0d7d640 1 -- 172.21.15.90:0/2785974599 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcae80a8bb0 con 0x7fcaec0cf5f0 2024-03-11T20:51:44.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.420+0000 7fcaf0d7d640 1 -- 172.21.15.90:0/2785974599 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae80b0440 con 0x7fcaec0cf5f0 2024-03-11T20:51:44.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.420+0000 7fcaf3fe3640 1 -- 172.21.15.90:0/2785974599 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcaec0cf5f0 msgr2=0x7fcaec15a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:44.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.420+0000 7fcaf3fe3640 1 --2- 172.21.15.90:0/2785974599 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcaec0cf5f0 0x7fcaec15a670 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fcae8009870 tx=0x7fcae801ed50 comp rx=0 tx=0).stop 2024-03-11T20:51:44.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.420+0000 7fcaf3fe3640 1 -- 172.21.15.90:0/2785974599 shutdown_connections 2024-03-11T20:51:44.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.420+0000 7fcaf3fe3640 1 --2- 172.21.15.90:0/2785974599 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fcaec153a30 0x7fcaec153ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:44.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.420+0000 7fcaf3fe3640 1 --2- 172.21.15.90:0/2785974599 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcaec0cf5f0 0x7fcaec15a670 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:44.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.420+0000 7fcaf3fe3640 1 -- 172.21.15.90:0/2785974599 >> 172.21.15.90:0/2785974599 conn(0x7fcaec06d770 msgr2=0x7fcaec06b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:44.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.420+0000 7fcaf3fe3640 1 -- 172.21.15.90:0/2785974599 shutdown_connections 2024-03-11T20:51:44.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.420+0000 7fcaf3fe3640 1 -- 172.21.15.90:0/2785974599 wait complete. 2024-03-11T20:51:44.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.420+0000 7fcaf3fe3640 1 Processor -- start 2024-03-11T20:51:44.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.420+0000 7fcaf3fe3640 1 -- start start 2024-03-11T20:51:44.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.420+0000 7fcaf3fe3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcaec0cf9d0 0x7fcaec1528a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:44.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.420+0000 7fcaf3fe3640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcaec153a30 0x7fcaec1ef930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:44.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.420+0000 7fcaf3fe3640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fcaec0b2070 0x7fcaec1efe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:44.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.420+0000 7fcaf3fe3640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaec14f0b0 con 0x7fcaec153a30 2024-03-11T20:51:44.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.420+0000 7fcaf3fe3640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaec14ff30 con 0x7fcaec0cf9d0 2024-03-11T20:51:44.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.420+0000 7fcaf3fe3640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaec0c8f10 con 0x7fcaec0b2070 2024-03-11T20:51:44.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.420+0000 7fcaf157e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcaec153a30 0x7fcaec1ef930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:44.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.424+0000 7fcaf157e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcaec153a30 0x7fcaec1ef930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43982/0 (socket says 172.21.15.90:43982) 2024-03-11T20:51:44.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.424+0000 7fcaf157e640 1 -- 172.21.15.90:0/939115282 learned_addr learned my addr 172.21.15.90:0/939115282 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:51:44.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.424+0000 7fcaf2580640 1 --2- 172.21.15.90:0/939115282 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fcaec0b2070 0x7fcaec1efe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:44.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.424+0000 7fcaf157e640 1 -- 172.21.15.90:0/939115282 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fcaec0b2070 msgr2=0x7fcaec1efe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:44.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.424+0000 7fcaf157e640 1 --2- 172.21.15.90:0/939115282 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fcaec0b2070 0x7fcaec1efe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:44.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.424+0000 7fcaf157e640 1 -- 172.21.15.90:0/939115282 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcaec0cf9d0 msgr2=0x7fcaec1528a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:44.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.424+0000 7fcaf157e640 1 --2- 172.21.15.90:0/939115282 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fcaec0cf9d0 0x7fcaec1528a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:44.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.424+0000 7fcaf157e640 1 -- 172.21.15.90:0/939115282 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae80094d0 con 0x7fcaec153a30 2024-03-11T20:51:44.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.428+0000 7fcaf157e640 1 --2- 172.21.15.90:0/939115282 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcaec153a30 0x7fcaec1ef930 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fcadc00c500 tx=0x7fcadc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:44.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.432+0000 7fcae27fc640 1 -- 172.21.15.90:0/939115282 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcadc005220 con 0x7fcaec153a30 2024-03-11T20:51:44.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.432+0000 7fcae27fc640 1 -- 172.21.15.90:0/939115282 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcadc005380 con 0x7fcaec153a30 2024-03-11T20:51:44.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.432+0000 7fcae27fc640 1 -- 172.21.15.90:0/939115282 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcadc007270 con 0x7fcaec153a30 2024-03-11T20:51:44.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.432+0000 7fcaf3fe3640 1 -- 172.21.15.90:0/939115282 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaec0c91c0 con 0x7fcaec153a30 2024-03-11T20:51:44.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.432+0000 7fcaf3fe3640 1 -- 172.21.15.90:0/939115282 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaec150160 con 0x7fcaec153a30 2024-03-11T20:51:44.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.432+0000 7fcaf3fe3640 1 -- 172.21.15.90:0/939115282 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcaec0c4520 con 0x7fcaec153a30 2024-03-11T20:51:44.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.452+0000 7fcae27fc640 1 -- 172.21.15.90:0/939115282 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fcadc0073d0 con 0x7fcaec153a30 2024-03-11T20:51:44.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.452+0000 7fcae27fc640 1 --2- 172.21.15.90:0/939115282 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fcad0066e90 0x7fcad0069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:44.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.456+0000 7fcaf1d7f640 1 --2- 172.21.15.90:0/939115282 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fcad0066e90 0x7fcad0069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:44.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.456+0000 7fcaf1d7f640 1 --2- 172.21.15.90:0/939115282 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fcad0066e90 0x7fcad0069350 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fcae80b2000 tx=0x7fcae8004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:44.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.456+0000 7fcae27fc640 1 -- 172.21.15.90:0/939115282 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fcadc007680 con 0x7fcaec153a30 2024-03-11T20:51:44.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.456+0000 7fcae27fc640 1 --2- 172.21.15.90:0/939115282 >> [v2:172.21.15.92:6808/200142070,v1:172.21.15.92:6809/200142070] conn(0x7fcad0070810 0x7fcad0072cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:44.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.456+0000 7fcae27fc640 1 -- 172.21.15.90:0/939115282 --> [v2:172.21.15.92:6808/200142070,v1:172.21.15.92:6809/200142070] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcad0073380 con 0x7fcad0070810 2024-03-11T20:51:44.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.460+0000 7fcaf2580640 1 --2- 172.21.15.90:0/939115282 >> [v2:172.21.15.92:6808/200142070,v1:172.21.15.92:6809/200142070] conn(0x7fcad0070810 0x7fcad0072cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:44.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.460+0000 7fcaf2580640 1 --2- 172.21.15.90:0/939115282 >> [v2:172.21.15.92:6808/200142070,v1:172.21.15.92:6809/200142070] conn(0x7fcad0070810 0x7fcad0072cb0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:44.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.460+0000 7fcae27fc640 1 -- 172.21.15.90:0/939115282 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcadc0011d0 con 0x7fcaec153a30 2024-03-11T20:51:44.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.460+0000 7fcae27fc640 1 -- 172.21.15.90:0/939115282 <== osd.5 v2:172.21.15.92:6808/200142070 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fcad0073380 con 0x7fcad0070810 2024-03-11T20:51:44.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.472+0000 7fcaf3fe3640 1 -- 172.21.15.90:0/939115282 --> [v2:172.21.15.92:6808/200142070,v1:172.21.15.92:6809/200142070] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fcaec15d920 con 0x7fcad0070810 2024-03-11T20:51:44.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.476+0000 7fcae27fc640 1 -- 172.21.15.90:0/939115282 <== osd.5 v2:172.21.15.92:6808/200142070 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fcae40083a0 con 0x7fcad0070810 2024-03-11T20:51:44.474 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-03-11T20:51:44.474 INFO:teuthology.orchestra.run.smithi090.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-11T20:51:44.474 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-03-11T20:51:44.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.476+0000 7fcaf3fe3640 1 -- 172.21.15.90:0/939115282 >> [v2:172.21.15.92:6808/200142070,v1:172.21.15.92:6809/200142070] conn(0x7fcad0070810 msgr2=0x7fcad0072cb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:44.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.476+0000 7fcaf3fe3640 1 --2- 172.21.15.90:0/939115282 >> [v2:172.21.15.92:6808/200142070,v1:172.21.15.92:6809/200142070] conn(0x7fcad0070810 0x7fcad0072cb0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:44.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.476+0000 7fcaf3fe3640 1 -- 172.21.15.90:0/939115282 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fcad0066e90 msgr2=0x7fcad0069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:44.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.476+0000 7fcaf3fe3640 1 --2- 172.21.15.90:0/939115282 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fcad0066e90 0x7fcad0069350 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fcae80b2000 tx=0x7fcae8004740 comp rx=0 tx=0).stop 2024-03-11T20:51:44.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.476+0000 7fcaf3fe3640 1 -- 172.21.15.90:0/939115282 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcaec153a30 msgr2=0x7fcaec1ef930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:44.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.476+0000 7fcaf3fe3640 1 --2- 172.21.15.90:0/939115282 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcaec153a30 0x7fcaec1ef930 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fcadc00c500 tx=0x7fcadc00c9c0 comp rx=0 tx=0).stop 2024-03-11T20:51:44.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.476+0000 7fcaf2580640 1 -- 172.21.15.90:0/939115282 reap_dead start 2024-03-11T20:51:44.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.476+0000 7fcaf3fe3640 1 -- 172.21.15.90:0/939115282 shutdown_connections 2024-03-11T20:51:44.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.476+0000 7fcaf3fe3640 1 -- 172.21.15.90:0/939115282 >> 172.21.15.90:0/939115282 conn(0x7fcaec06d770 msgr2=0x7fcaec0cbd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:44.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.476+0000 7fcaf3fe3640 1 -- 172.21.15.90:0/939115282 shutdown_connections 2024-03-11T20:51:44.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.476+0000 7fcaf3fe3640 1 -- 172.21.15.90:0/939115282 wait complete. 2024-03-11T20:51:44.491 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-03-11T20:51:44.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.612+0000 7f23e4c79640 1 Processor -- start 2024-03-11T20:51:44.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.612+0000 7f23e4c79640 1 -- start start 2024-03-11T20:51:44.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.612+0000 7f23e4c79640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f23e0150a90 0x7f23e0157220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:44.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.616+0000 7f23e4c79640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23e00c5e60 con 0x7f23e00c65b0 2024-03-11T20:51:44.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.616+0000 7f23e4c79640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23e0156a60 con 0x7f23e00cc280 2024-03-11T20:51:44.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.616+0000 7f23e4c79640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23e00c55c0 con 0x7f23e0150a90 2024-03-11T20:51:44.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.616+0000 7f23de59c640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f23e0150a90 0x7f23e0157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:44.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.616+0000 7f23ded9d640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f23e00cc280 0x7f23e0161240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.90:33954/0 (socket says 172.21.15.90:33954) 2024-03-11T20:51:44.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.616+0000 7f23ded9d640 1 -- 172.21.15.90:0/2288011906 learned_addr learned my addr 172.21.15.90:0/2288011906 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T20:51:44.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.620+0000 7f23de59c640 1 -- 172.21.15.90:0/2288011906 >> v1:172.21.15.92:6789/0 conn(0x7f23e00cc280 legacy=0x7f23e0161240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:44.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.620+0000 7f23de59c640 1 -- 172.21.15.90:0/2288011906 >> v1:172.21.15.90:6789/0 conn(0x7f23e00c65b0 legacy=0x7f23e00c37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:44.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.620+0000 7f23de59c640 1 -- 172.21.15.90:0/2288011906 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e00c58a0 con 0x7f23e0150a90 2024-03-11T20:51:44.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.636+0000 7f23de59c640 1 --2- 172.21.15.90:0/2288011906 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f23e0150a90 0x7f23e0157220 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f23d00097c0 tx=0x7f23d001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=832bde3ddd6a085e server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:44.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.712+0000 7f23dd59a640 1 -- 172.21.15.90:0/2288011906 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d00ab600 con 0x7f23e0150a90 2024-03-11T20:51:44.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.712+0000 7f23dd59a640 1 -- 172.21.15.90:0/2288011906 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23d00a7050 con 0x7f23e0150a90 2024-03-11T20:51:44.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.712+0000 7f23dd59a640 1 -- 172.21.15.90:0/2288011906 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d00ab980 con 0x7f23e0150a90 2024-03-11T20:51:44.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.712+0000 7f23e4c79640 1 -- 172.21.15.90:0/2288011906 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f23e0150a90 msgr2=0x7f23e0157220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:44.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.712+0000 7f23e4c79640 1 --2- 172.21.15.90:0/2288011906 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f23e0150a90 0x7f23e0157220 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f23d00097c0 tx=0x7f23d001c4a0 comp rx=0 tx=0).stop 2024-03-11T20:51:44.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.712+0000 7f23e4c79640 1 -- 172.21.15.90:0/2288011906 shutdown_connections 2024-03-11T20:51:44.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.712+0000 7f23e4c79640 1 --2- 172.21.15.90:0/2288011906 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f23e0150a90 0x7f23e0157220 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:44.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.712+0000 7f23e4c79640 1 -- 172.21.15.90:0/2288011906 >> 172.21.15.90:0/2288011906 conn(0x7f23e0071aa0 msgr2=0x7f23e0071eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:44.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.712+0000 7f23e4c79640 1 -- 172.21.15.90:0/2288011906 shutdown_connections 2024-03-11T20:51:44.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.712+0000 7f23e4c79640 1 -- 172.21.15.90:0/2288011906 wait complete. 2024-03-11T20:51:44.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.716+0000 7f23e4c79640 1 Processor -- start 2024-03-11T20:51:44.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.716+0000 7f23e4c79640 1 -- start start 2024-03-11T20:51:44.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.716+0000 7f23e4c79640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23e00c65b0 0x7f23e0154b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:44.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.716+0000 7f23e4c79640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f23e00cc280 0x7f23e00c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:44.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.716+0000 7f23e4c79640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f23e0150a90 0x7f23e01f1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:44.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.716+0000 7f23e4c79640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e01f5190 con 0x7f23e0150a90 2024-03-11T20:51:44.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.716+0000 7f23e4c79640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e01f48f0 con 0x7f23e00c65b0 2024-03-11T20:51:44.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.716+0000 7f23e4c79640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23e0155050 con 0x7f23e00cc280 2024-03-11T20:51:44.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.716+0000 7f23ded9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f23e0150a90 0x7f23e01f1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:44.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.716+0000 7f23ded9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f23e0150a90 0x7f23e01f1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44002/0 (socket says 172.21.15.90:44002) 2024-03-11T20:51:44.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.716+0000 7f23ded9d640 1 -- 172.21.15.90:0/3952932352 learned_addr learned my addr 172.21.15.90:0/3952932352 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:51:44.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.720+0000 7f23ded9d640 1 -- 172.21.15.90:0/3952932352 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f23e00cc280 msgr2=0x7f23e00c8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:44.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.720+0000 7f23ded9d640 1 --2- 172.21.15.90:0/3952932352 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f23e00cc280 0x7f23e00c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:44.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.720+0000 7f23ded9d640 1 -- 172.21.15.90:0/3952932352 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23e00c65b0 msgr2=0x7f23e0154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:44.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.720+0000 7f23ded9d640 1 --2- 172.21.15.90:0/3952932352 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23e00c65b0 0x7f23e0154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:44.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.720+0000 7f23ded9d640 1 -- 172.21.15.90:0/3952932352 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d0009420 con 0x7f23e0150a90 2024-03-11T20:51:44.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.724+0000 7f23ded9d640 1 --2- 172.21.15.90:0/3952932352 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f23e0150a90 0x7f23e01f1380 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f23d800cbc0 tx=0x7f23d800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:44.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.736+0000 7f23ceffd640 1 -- 172.21.15.90:0/3952932352 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d800ad40 con 0x7f23e0150a90 2024-03-11T20:51:44.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.736+0000 7f23ceffd640 1 -- 172.21.15.90:0/3952932352 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23d8012400 con 0x7f23e0150a90 2024-03-11T20:51:44.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.736+0000 7f23ceffd640 1 -- 172.21.15.90:0/3952932352 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d800aea0 con 0x7f23e0150a90 2024-03-11T20:51:44.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.736+0000 7f23e4c79640 1 -- 172.21.15.90:0/3952932352 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23e01552f0 con 0x7f23e0150a90 2024-03-11T20:51:44.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.736+0000 7f23e4c79640 1 -- 172.21.15.90:0/3952932352 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23e01f4b40 con 0x7f23e0150a90 2024-03-11T20:51:44.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.736+0000 7f23e4c79640 1 -- 172.21.15.90:0/3952932352 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23e004eef0 con 0x7f23e0150a90 2024-03-11T20:51:44.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.768+0000 7f23ceffd640 1 -- 172.21.15.90:0/3952932352 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f23d8007070 con 0x7f23e0150a90 2024-03-11T20:51:44.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.768+0000 7f23ceffd640 1 --2- 172.21.15.90:0/3952932352 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f23c0066e90 0x7f23c0069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:44.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.768+0000 7f23de59c640 1 --2- 172.21.15.90:0/3952932352 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f23c0066e90 0x7f23c0069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:44.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:44.772+0000 7f23de59c640 1 --2- 172.21.15.90:0/3952932352 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f23c0066e90 0x7f23c0069350 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f23d00ad000 tx=0x7f23d0004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:45.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.056+0000 7f23ceffd640 1 -- 172.21.15.90:0/3952932352 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f23d8055990 con 0x7f23e0150a90 2024-03-11T20:51:45.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.056+0000 7f23ceffd640 1 -- 172.21.15.90:0/3952932352 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f23d8051c30 con 0x7f23e0150a90 2024-03-11T20:51:45.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.232+0000 7f23e4c79640 1 -- 172.21.15.90:0/3952932352 --> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f23e00ccaa0 con 0x7f23c0066e90 2024-03-11T20:51:45.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.232+0000 7f23ceffd640 1 -- 172.21.15.90:0/3952932352 <== mgr.4101 v2:172.21.15.90:6800/1999720161 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f23d00ab760 con 0x7f23c0066e90 2024-03-11T20:51:45.233 INFO:teuthology.orchestra.run.smithi090.stderr:instructed osd(s) 5 to deep-scrub 2024-03-11T20:51:45.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.236+0000 7f23e4c79640 1 -- 172.21.15.90:0/3952932352 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f23c0066e90 msgr2=0x7f23c0069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:45.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.236+0000 7f23e4c79640 1 --2- 172.21.15.90:0/3952932352 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f23c0066e90 0x7f23c0069350 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f23d00ad000 tx=0x7f23d0004740 comp rx=0 tx=0).stop 2024-03-11T20:51:45.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.236+0000 7f23e4c79640 1 -- 172.21.15.90:0/3952932352 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f23e0150a90 msgr2=0x7f23e01f1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:45.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.236+0000 7f23e4c79640 1 --2- 172.21.15.90:0/3952932352 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f23e0150a90 0x7f23e01f1380 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f23d800cbc0 tx=0x7f23d800a430 comp rx=0 tx=0).stop 2024-03-11T20:51:45.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.236+0000 7f23e4c79640 1 -- 172.21.15.90:0/3952932352 shutdown_connections 2024-03-11T20:51:45.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.236+0000 7f23e4c79640 1 --2- 172.21.15.90:0/3952932352 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f23c0066e90 0x7f23c0069350 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:45.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.236+0000 7f23e4c79640 1 --2- 172.21.15.90:0/3952932352 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f23e0150a90 0x7f23e01f1380 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:45.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.236+0000 7f23e4c79640 1 --2- 172.21.15.90:0/3952932352 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f23e00cc280 0x7f23e00c8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:45.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.236+0000 7f23e4c79640 1 --2- 172.21.15.90:0/3952932352 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f23e00c65b0 0x7f23e0154b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:45.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.236+0000 7f23e4c79640 1 -- 172.21.15.90:0/3952932352 >> 172.21.15.90:0/3952932352 conn(0x7f23e0071aa0 msgr2=0x7f23e00c3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:45.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.236+0000 7f23e4c79640 1 -- 172.21.15.90:0/3952932352 shutdown_connections 2024-03-11T20:51:45.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.236+0000 7f23e4c79640 1 -- 172.21.15.90:0/3952932352 wait complete. 2024-03-11T20:51:45.253 INFO:tasks.ceph:Scrubbing osd.6 2024-03-11T20:51:45.253 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-03-11T20:51:45.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.384+0000 7f3c798de640 1 Processor -- start 2024-03-11T20:51:45.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.384+0000 7f3c798de640 1 -- start start 2024-03-11T20:51:45.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.384+0000 7f3c798de640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3c74153de0 0x7f3c74071d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:45.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.384+0000 7f3c798de640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f3c7415a7a0 0x7f3c740caae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:45.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.384+0000 7f3c798de640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c7415abb0 con 0x7f3c740c76c0 2024-03-11T20:51:45.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.384+0000 7f3c798de640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c740c7b10 con 0x7f3c7415a7a0 2024-03-11T20:51:45.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.384+0000 7f3c798de640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c740ca580 con 0x7f3c74153de0 2024-03-11T20:51:45.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.388+0000 7f3c737fe640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f3c7415a7a0 0x7f3c740caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:45.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.388+0000 7f3c6bfff640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f3c740c76c0 0x7f3c740c98c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:45066/0 (socket says 172.21.15.90:45066) 2024-03-11T20:51:45.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.388+0000 7f3c6bfff640 1 -- 172.21.15.90:0/3166525719 learned_addr learned my addr 172.21.15.90:0/3166525719 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T20:51:45.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.388+0000 7f3c72ffd640 1 --2- 172.21.15.90:0/3166525719 >> v2:172.21.15.92:3300/0 conn(0x7f3c74153de0 0x7f3c74071d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:45.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.392+0000 7f3c737fe640 1 -- 172.21.15.90:0/3166525719 >> v2:172.21.15.92:3300/0 conn(0x7f3c74153de0 msgr2=0x7f3c74071d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:45.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.392+0000 7f3c737fe640 1 --2- 172.21.15.90:0/3166525719 >> v2:172.21.15.92:3300/0 conn(0x7f3c74153de0 0x7f3c74071d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:45.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.392+0000 7f3c737fe640 1 -- 172.21.15.90:0/3166525719 >> v1:172.21.15.90:6789/0 conn(0x7f3c740c76c0 legacy=0x7f3c740c98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:45.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.392+0000 7f3c737fe640 1 -- 172.21.15.90:0/3166525719 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c740c9fd0 con 0x7f3c7415a7a0 2024-03-11T20:51:45.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.392+0000 7f3c72ffd640 1 --2- 172.21.15.90:0/3166525719 >> v2:172.21.15.92:3300/0 conn(0x7f3c74153de0 0x7f3c74071d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-11T20:51:45.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.396+0000 7f3c737fe640 1 --2- 172.21.15.90:0/3166525719 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c7415a7a0 0x7f3c740caae0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3c740cc8c0 tx=0x7f3c6c01f110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4243bc1150d25a7b server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:45.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.396+0000 7f3c727fc640 1 -- 172.21.15.90:0/3166525719 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c0af600 con 0x7f3c7415a7a0 2024-03-11T20:51:45.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.400+0000 7f3c727fc640 1 -- 172.21.15.90:0/3166525719 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c6c0ab050 con 0x7f3c7415a7a0 2024-03-11T20:51:45.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.400+0000 7f3c727fc640 1 -- 172.21.15.90:0/3166525719 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c0af980 con 0x7f3c7415a7a0 2024-03-11T20:51:45.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.400+0000 7f3c798de640 1 -- 172.21.15.90:0/3166525719 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c7415a7a0 msgr2=0x7f3c740caae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:45.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.400+0000 7f3c798de640 1 --2- 172.21.15.90:0/3166525719 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c7415a7a0 0x7f3c740caae0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3c740cc8c0 tx=0x7f3c6c01f110 comp rx=0 tx=0).stop 2024-03-11T20:51:45.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.400+0000 7f3c798de640 1 -- 172.21.15.90:0/3166525719 shutdown_connections 2024-03-11T20:51:45.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.400+0000 7f3c798de640 1 --2- 172.21.15.90:0/3166525719 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c7415a7a0 0x7f3c740caae0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:45.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.400+0000 7f3c798de640 1 --2- 172.21.15.90:0/3166525719 >> v2:172.21.15.92:3300/0 conn(0x7f3c74153de0 0x7f3c74071d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:45.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.400+0000 7f3c798de640 1 -- 172.21.15.90:0/3166525719 >> 172.21.15.90:0/3166525719 conn(0x7f3c740cbac0 msgr2=0x7f3c740cbed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:45.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.400+0000 7f3c798de640 1 -- 172.21.15.90:0/3166525719 shutdown_connections 2024-03-11T20:51:45.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.400+0000 7f3c798de640 1 -- 172.21.15.90:0/3166525719 wait complete. 2024-03-11T20:51:45.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.400+0000 7f3c798de640 1 Processor -- start 2024-03-11T20:51:45.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.400+0000 7f3c798de640 1 -- start start 2024-03-11T20:51:45.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.404+0000 7f3c798de640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3c740c76c0 0x7f3c74156450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:45.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.404+0000 7f3c798de640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c74153de0 0x7f3c741f0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:45.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.404+0000 7f3c798de640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c7415a7a0 0x7f3c740cb060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:45.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.404+0000 7f3c798de640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c741ef7e0 con 0x7f3c7415a7a0 2024-03-11T20:51:45.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.404+0000 7f3c798de640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c74155fc0 con 0x7f3c74153de0 2024-03-11T20:51:45.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.404+0000 7f3c798de640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c741ef500 con 0x7f3c740c76c0 2024-03-11T20:51:45.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.404+0000 7f3c737fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c7415a7a0 0x7f3c740cb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:45.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.404+0000 7f3c737fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c7415a7a0 0x7f3c740cb060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44016/0 (socket says 172.21.15.90:44016) 2024-03-11T20:51:45.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.404+0000 7f3c737fe640 1 -- 172.21.15.90:0/1262293102 learned_addr learned my addr 172.21.15.90:0/1262293102 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:51:45.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.404+0000 7f3c72ffd640 1 --2- 172.21.15.90:0/1262293102 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3c740c76c0 0x7f3c74156450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:45.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.404+0000 7f3c6bfff640 1 --2- 172.21.15.90:0/1262293102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c74153de0 0x7f3c741f0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:45.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.408+0000 7f3c737fe640 1 -- 172.21.15.90:0/1262293102 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3c740c76c0 msgr2=0x7f3c74156450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:45.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.408+0000 7f3c737fe640 1 --2- 172.21.15.90:0/1262293102 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f3c740c76c0 0x7f3c74156450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:45.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.408+0000 7f3c737fe640 1 -- 172.21.15.90:0/1262293102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c74153de0 msgr2=0x7f3c741f0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:45.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.408+0000 7f3c737fe640 1 --2- 172.21.15.90:0/1262293102 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3c74153de0 0x7f3c741f0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:45.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.408+0000 7f3c737fe640 1 -- 172.21.15.90:0/1262293102 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c6c009a70 con 0x7f3c7415a7a0 2024-03-11T20:51:45.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.408+0000 7f3c737fe640 1 --2- 172.21.15.90:0/1262293102 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c7415a7a0 0x7f3c740cb060 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c01bd20 tx=0x7f3c6c01beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:45.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.412+0000 7f3c70ff9640 1 -- 172.21.15.90:0/1262293102 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c0b9610 con 0x7f3c7415a7a0 2024-03-11T20:51:45.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.412+0000 7f3c70ff9640 1 -- 172.21.15.90:0/1262293102 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c6c0b9770 con 0x7f3c7415a7a0 2024-03-11T20:51:45.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.412+0000 7f3c70ff9640 1 -- 172.21.15.90:0/1262293102 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c0b9b20 con 0x7f3c7415a7a0 2024-03-11T20:51:45.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.412+0000 7f3c798de640 1 -- 172.21.15.90:0/1262293102 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c74155690 con 0x7f3c7415a7a0 2024-03-11T20:51:45.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.412+0000 7f3c798de640 1 -- 172.21.15.90:0/1262293102 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c74155c50 con 0x7f3c7415a7a0 2024-03-11T20:51:45.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.416+0000 7f3c798de640 1 -- 172.21.15.90:0/1262293102 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3c74159fa0 con 0x7f3c7415a7a0 2024-03-11T20:51:45.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.432+0000 7f3c70ff9640 1 -- 172.21.15.90:0/1262293102 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f3c6c0b6050 con 0x7f3c7415a7a0 2024-03-11T20:51:45.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.436+0000 7f3c70ff9640 1 --2- 172.21.15.90:0/1262293102 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f3c54066e40 0x7f3c54069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:45.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.436+0000 7f3c72ffd640 1 --2- 172.21.15.90:0/1262293102 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f3c54066e40 0x7f3c54069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:45.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.436+0000 7f3c72ffd640 1 --2- 172.21.15.90:0/1262293102 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f3c54066e40 0x7f3c54069300 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3c6400ab60 tx=0x7f3c64008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:45.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.436+0000 7f3c70ff9640 1 -- 172.21.15.90:0/1262293102 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f3c6c0f48e0 con 0x7f3c7415a7a0 2024-03-11T20:51:45.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.436+0000 7f3c70ff9640 1 --2- 172.21.15.90:0/1262293102 >> [v2:172.21.15.92:6816/1919010204,v1:172.21.15.92:6817/1919010204] conn(0x7f3c54070750 0x7f3c54072bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:45.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.436+0000 7f3c70ff9640 1 -- 172.21.15.90:0/1262293102 --> [v2:172.21.15.92:6816/1919010204,v1:172.21.15.92:6817/1919010204] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3c540732c0 con 0x7f3c54070750 2024-03-11T20:51:45.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.436+0000 7f3c6bfff640 1 --2- 172.21.15.90:0/1262293102 >> [v2:172.21.15.92:6816/1919010204,v1:172.21.15.92:6817/1919010204] conn(0x7f3c54070750 0x7f3c54072bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:45.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.440+0000 7f3c6bfff640 1 --2- 172.21.15.90:0/1262293102 >> [v2:172.21.15.92:6816/1919010204,v1:172.21.15.92:6817/1919010204] conn(0x7f3c54070750 0x7f3c54072bf0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:45.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.440+0000 7f3c70ff9640 1 -- 172.21.15.90:0/1262293102 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c6c0afe70 con 0x7f3c7415a7a0 2024-03-11T20:51:45.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.440+0000 7f3c70ff9640 1 -- 172.21.15.90:0/1262293102 <== osd.6 v2:172.21.15.92:6816/1919010204 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f3c540732c0 con 0x7f3c54070750 2024-03-11T20:51:45.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.452+0000 7f3c798de640 1 -- 172.21.15.90:0/1262293102 --> [v2:172.21.15.92:6816/1919010204,v1:172.21.15.92:6817/1919010204] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f3c74159cc0 con 0x7f3c54070750 2024-03-11T20:51:45.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.456+0000 7f3c70ff9640 1 -- 172.21.15.90:0/1262293102 <== osd.6 v2:172.21.15.92:6816/1919010204 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f3c5c006b30 con 0x7f3c54070750 2024-03-11T20:51:45.454 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-03-11T20:51:45.454 INFO:teuthology.orchestra.run.smithi090.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-11T20:51:45.454 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-03-11T20:51:45.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.456+0000 7f3c798de640 1 -- 172.21.15.90:0/1262293102 >> [v2:172.21.15.92:6816/1919010204,v1:172.21.15.92:6817/1919010204] conn(0x7f3c54070750 msgr2=0x7f3c54072bf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:45.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.456+0000 7f3c798de640 1 --2- 172.21.15.90:0/1262293102 >> [v2:172.21.15.92:6816/1919010204,v1:172.21.15.92:6817/1919010204] conn(0x7f3c54070750 0x7f3c54072bf0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:45.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.456+0000 7f3c798de640 1 -- 172.21.15.90:0/1262293102 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f3c54066e40 msgr2=0x7f3c54069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:45.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.456+0000 7f3c798de640 1 --2- 172.21.15.90:0/1262293102 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f3c54066e40 0x7f3c54069300 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3c6400ab60 tx=0x7f3c64008040 comp rx=0 tx=0).stop 2024-03-11T20:51:45.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.456+0000 7f3c798de640 1 -- 172.21.15.90:0/1262293102 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c7415a7a0 msgr2=0x7f3c740cb060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:45.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.456+0000 7f3c798de640 1 --2- 172.21.15.90:0/1262293102 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c7415a7a0 0x7f3c740cb060 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c01bd20 tx=0x7f3c6c01beb0 comp rx=0 tx=0).stop 2024-03-11T20:51:45.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.456+0000 7f3c737fe640 1 -- 172.21.15.90:0/1262293102 reap_dead start 2024-03-11T20:51:45.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.456+0000 7f3c798de640 1 -- 172.21.15.90:0/1262293102 shutdown_connections 2024-03-11T20:51:45.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.456+0000 7f3c798de640 1 -- 172.21.15.90:0/1262293102 >> 172.21.15.90:0/1262293102 conn(0x7f3c740cbac0 msgr2=0x7f3c740c98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:45.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.456+0000 7f3c798de640 1 -- 172.21.15.90:0/1262293102 shutdown_connections 2024-03-11T20:51:45.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.456+0000 7f3c798de640 1 -- 172.21.15.90:0/1262293102 wait complete. 2024-03-11T20:51:45.471 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-03-11T20:51:45.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.592+0000 7f49c1197640 1 Processor -- start 2024-03-11T20:51:45.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.592+0000 7f49c1197640 1 -- start start 2024-03-11T20:51:45.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.592+0000 7f49c1197640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f49bc15da50 0x7f49bc0caf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:45.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.592+0000 7f49c1197640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49bc0cb5b0 con 0x7f49bc1541c0 2024-03-11T20:51:45.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.592+0000 7f49c1197640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49bc159e10 con 0x7f49bc0cfda0 2024-03-11T20:51:45.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.592+0000 7f49c1197640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49bc0b5410 con 0x7f49bc15da50 2024-03-11T20:51:45.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.592+0000 7f49bad9d640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f49bc1541c0 0x7f49bc0ca180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:45080/0 (socket says 172.21.15.90:45080) 2024-03-11T20:51:45.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.592+0000 7f49bad9d640 1 -- 172.21.15.90:0/3789028929 learned_addr learned my addr 172.21.15.90:0/3789028929 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T20:51:45.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.592+0000 7f49bb59e640 1 --2- 172.21.15.90:0/3789028929 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f49bc15da50 0x7f49bc0caf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:45.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.596+0000 7f49b9d9b640 1 -- 172.21.15.90:0/3789028929 <== mon.0 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1244081746 0 0) 0x7f49bc0cb5b0 con 0x7f49bc1541c0 2024-03-11T20:51:45.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.596+0000 7f49b9d9b640 1 -- 172.21.15.90:0/3789028929 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49a4003530 con 0x7f49bc1541c0 2024-03-11T20:51:45.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.600+0000 7f49b9d9b640 1 -- 172.21.15.90:0/3789028929 <== mon.1 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3027152839 0 0) 0x7f49bc159e10 con 0x7f49bc0cfda0 2024-03-11T20:51:45.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.600+0000 7f49b9d9b640 1 -- 172.21.15.90:0/3789028929 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49bc0cb5b0 con 0x7f49bc0cfda0 2024-03-11T20:51:45.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.600+0000 7f49b9d9b640 1 -- 172.21.15.90:0/3789028929 <== mon.0 v1:172.21.15.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3481790144 0 0) 0x7f49a4003530 con 0x7f49bc1541c0 2024-03-11T20:51:45.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.600+0000 7f49b9d9b640 1 -- 172.21.15.90:0/3789028929 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f49bc15da50 msgr2=0x7f49bc0caf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:45.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.600+0000 7f49b9d9b640 1 --2- 172.21.15.90:0/3789028929 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f49bc15da50 0x7f49bc0caf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:45.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.600+0000 7f49b9d9b640 1 -- 172.21.15.90:0/3789028929 >> v1:172.21.15.92:6789/0 conn(0x7f49bc0cfda0 legacy=0x7f49bc15d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:45.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.600+0000 7f49b9d9b640 1 -- 172.21.15.90:0/3789028929 --> v1:172.21.15.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49bc15a0f0 con 0x7f49bc1541c0 2024-03-11T20:51:45.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.600+0000 7f49b9d9b640 1 -- 172.21.15.90:0/3789028929 <== mon.0 v1:172.21.15.90:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 691735435 0 0) 0x7f49ac0016a0 con 0x7f49bc1541c0 2024-03-11T20:51:45.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.600+0000 7f49b9d9b640 1 -- 172.21.15.90:0/3789028929 >> v1:172.21.15.90:6789/0 conn(0x7f49bc1541c0 legacy=0x7f49bc0ca180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:45.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.600+0000 7f49b9d9b640 1 --2- 172.21.15.90:0/3789028929 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49a4003b40 0x7f49a4003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:45.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.600+0000 7f49b9d9b640 1 --2- 172.21.15.90:0/3789028929 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f49bc15da50 0x7f49a40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:45.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.600+0000 7f49b9d9b640 1 --2- 172.21.15.90:0/3789028929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49a4004d10 0x7f49a4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:45.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.600+0000 7f49b9d9b640 1 -- 172.21.15.90:0/3789028929 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a400ba40 con 0x7f49a4003b40 2024-03-11T20:51:45.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.600+0000 7f49b9d9b640 1 -- 172.21.15.90:0/3789028929 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a400bb80 con 0x7f49a4004d10 2024-03-11T20:51:45.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.600+0000 7f49b9d9b640 1 -- 172.21.15.90:0/3789028929 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49a400bd40 con 0x7f49bc15da50 2024-03-11T20:51:45.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.604+0000 7f49bad9d640 1 --2- 172.21.15.90:0/3789028929 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49a4003b40 0x7f49a4003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:45.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.604+0000 7f49bb59e640 1 --2- 172.21.15.90:0/3789028929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49a4004d10 0x7f49a4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:45.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.608+0000 7f49bad9d640 1 -- 172.21.15.90:0/3789028929 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f49bc15da50 msgr2=0x7f49a40047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:45.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.608+0000 7f49bad9d640 1 --2- 172.21.15.90:0/3789028929 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f49bc15da50 0x7f49a40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:45.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.608+0000 7f49bad9d640 1 -- 172.21.15.90:0/3789028929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49a4004d10 msgr2=0x7f49a4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:45.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.608+0000 7f49bad9d640 1 --2- 172.21.15.90:0/3789028929 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49a4004d10 0x7f49a4005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:45.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.608+0000 7f49bad9d640 1 -- 172.21.15.90:0/3789028929 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a4003530 con 0x7f49a4003b40 2024-03-11T20:51:45.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.612+0000 7f49bad9d640 1 --2- 172.21.15.90:0/3789028929 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49a4003b40 0x7f49a4003f40 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f49bc15ae50 tx=0x7f49ac01cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd6298fff2cead93 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:45.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.616+0000 7f49b9d9b640 1 -- 172.21.15.90:0/3789028929 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ac005ae0 con 0x7f49a4003b40 2024-03-11T20:51:45.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.616+0000 7f49b9d9b640 1 -- 172.21.15.90:0/3789028929 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49ac005c40 con 0x7f49a4003b40 2024-03-11T20:51:45.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.616+0000 7f49b9d9b640 1 -- 172.21.15.90:0/3789028929 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ac0a5820 con 0x7f49a4003b40 2024-03-11T20:51:45.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.616+0000 7f49c1197640 1 -- 172.21.15.90:0/3789028929 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49a4003b40 msgr2=0x7f49a4003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:45.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.616+0000 7f49c1197640 1 --2- 172.21.15.90:0/3789028929 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49a4003b40 0x7f49a4003f40 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f49bc15ae50 tx=0x7f49ac01cb20 comp rx=0 tx=0).stop 2024-03-11T20:51:45.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.616+0000 7f49bb59e640 1 -- 172.21.15.90:0/3789028929 reap_dead start 2024-03-11T20:51:45.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.616+0000 7f49c1197640 1 -- 172.21.15.90:0/3789028929 shutdown_connections 2024-03-11T20:51:45.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.616+0000 7f49c1197640 1 -- 172.21.15.90:0/3789028929 >> 172.21.15.90:0/3789028929 conn(0x7f49bc06b620 msgr2=0x7f49bc06ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:45.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.616+0000 7f49c1197640 1 -- 172.21.15.90:0/3789028929 shutdown_connections 2024-03-11T20:51:45.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.616+0000 7f49c1197640 1 -- 172.21.15.90:0/3789028929 wait complete. 2024-03-11T20:51:45.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.616+0000 7f49c1197640 1 Processor -- start 2024-03-11T20:51:45.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.616+0000 7f49c1197640 1 -- start start 2024-03-11T20:51:45.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.616+0000 7f49c1197640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49a4003b40 0x7f49bc1f1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:45.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.616+0000 7f49c1197640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49bc154880 0x7f49bc1533a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:45.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.616+0000 7f49c1197640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f49bc1f2610 0x7f49bc1f2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:45.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.616+0000 7f49c1197640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49bc1560f0 con 0x7f49bc154880 2024-03-11T20:51:45.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.616+0000 7f49c1197640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49bc152f10 con 0x7f49a4003b40 2024-03-11T20:51:45.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.616+0000 7f49c1197640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49bc0cca90 con 0x7f49bc1f2610 2024-03-11T20:51:45.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.620+0000 7f49ba59c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49bc154880 0x7f49bc1533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:45.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.620+0000 7f49ba59c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49bc154880 0x7f49bc1533a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44038/0 (socket says 172.21.15.90:44038) 2024-03-11T20:51:45.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.620+0000 7f49ba59c640 1 -- 172.21.15.90:0/1396826230 learned_addr learned my addr 172.21.15.90:0/1396826230 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:51:45.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.620+0000 7f49bb59e640 1 --2- 172.21.15.90:0/1396826230 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f49bc1f2610 0x7f49bc1f2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:45.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.620+0000 7f49bad9d640 1 --2- 172.21.15.90:0/1396826230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49a4003b40 0x7f49bc1f1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:45.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.624+0000 7f49ba59c640 1 -- 172.21.15.90:0/1396826230 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f49bc1f2610 msgr2=0x7f49bc1f2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:45.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.624+0000 7f49ba59c640 1 --2- 172.21.15.90:0/1396826230 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f49bc1f2610 0x7f49bc1f2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:45.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.624+0000 7f49ba59c640 1 -- 172.21.15.90:0/1396826230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49a4003b40 msgr2=0x7f49bc1f1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:45.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.624+0000 7f49ba59c640 1 --2- 172.21.15.90:0/1396826230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49a4003b40 0x7f49bc1f1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:45.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.624+0000 7f49ba59c640 1 -- 172.21.15.90:0/1396826230 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49bc15d7d0 con 0x7f49bc154880 2024-03-11T20:51:45.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.624+0000 7f49ba59c640 1 --2- 172.21.15.90:0/1396826230 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49bc154880 0x7f49bc1533a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f49b000fa70 tx=0x7f49b000ff30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:45.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.628+0000 7f49a37fe640 1 -- 172.21.15.90:0/1396826230 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b0008480 con 0x7f49bc154880 2024-03-11T20:51:45.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.628+0000 7f49a37fe640 1 -- 172.21.15.90:0/1396826230 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b000d6d0 con 0x7f49bc154880 2024-03-11T20:51:45.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.628+0000 7f49c1197640 1 -- 172.21.15.90:0/1396826230 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49bc155b30 con 0x7f49bc154880 2024-03-11T20:51:45.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.628+0000 7f49a37fe640 1 -- 172.21.15.90:0/1396826230 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b000da60 con 0x7f49bc154880 2024-03-11T20:51:45.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.628+0000 7f49c1197640 1 -- 172.21.15.90:0/1396826230 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49bc152ba0 con 0x7f49bc154880 2024-03-11T20:51:45.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.628+0000 7f49c1197640 1 -- 172.21.15.90:0/1396826230 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49bc04eef0 con 0x7f49bc154880 2024-03-11T20:51:45.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.928+0000 7f49a37fe640 1 -- 172.21.15.90:0/1396826230 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f49b000dbc0 con 0x7f49bc154880 2024-03-11T20:51:45.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.928+0000 7f49a37fe640 1 --2- 172.21.15.90:0/1396826230 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f49a8066e90 0x7f49a8069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:45.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.928+0000 7f49bad9d640 1 --2- 172.21.15.90:0/1396826230 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f49a8066e90 0x7f49a8069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:45.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.928+0000 7f49bad9d640 1 --2- 172.21.15.90:0/1396826230 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f49a8066e90 0x7f49a8069350 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f49ac0ab000 tx=0x7f49ac0a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:45.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.964+0000 7f49a37fe640 1 -- 172.21.15.90:0/1396826230 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f49b0058500 con 0x7f49bc154880 2024-03-11T20:51:45.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:45.964+0000 7f49a37fe640 1 -- 172.21.15.90:0/1396826230 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f49b00547a0 con 0x7f49bc154880 2024-03-11T20:51:46.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.144+0000 7f49c1197640 1 -- 172.21.15.90:0/1396826230 --> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f49bc152a20 con 0x7f49a8066e90 2024-03-11T20:51:46.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.144+0000 7f49a37fe640 1 -- 172.21.15.90:0/1396826230 <== mgr.4101 v2:172.21.15.90:6800/1999720161 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f49ac0b5040 con 0x7f49a8066e90 2024-03-11T20:51:46.143 INFO:teuthology.orchestra.run.smithi090.stderr:instructed osd(s) 6 to deep-scrub 2024-03-11T20:51:46.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.148+0000 7f49c1197640 1 -- 172.21.15.90:0/1396826230 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f49a8066e90 msgr2=0x7f49a8069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:46.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.148+0000 7f49c1197640 1 --2- 172.21.15.90:0/1396826230 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f49a8066e90 0x7f49a8069350 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f49ac0ab000 tx=0x7f49ac0a7070 comp rx=0 tx=0).stop 2024-03-11T20:51:46.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.148+0000 7f49c1197640 1 -- 172.21.15.90:0/1396826230 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49bc154880 msgr2=0x7f49bc1533a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:46.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.148+0000 7f49c1197640 1 --2- 172.21.15.90:0/1396826230 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49bc154880 0x7f49bc1533a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f49b000fa70 tx=0x7f49b000ff30 comp rx=0 tx=0).stop 2024-03-11T20:51:46.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.148+0000 7f49c1197640 1 -- 172.21.15.90:0/1396826230 shutdown_connections 2024-03-11T20:51:46.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.148+0000 7f49c1197640 1 --2- 172.21.15.90:0/1396826230 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f49a8066e90 0x7f49a8069350 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:46.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.148+0000 7f49c1197640 1 --2- 172.21.15.90:0/1396826230 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f49bc1f2610 0x7f49bc1f2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:46.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.148+0000 7f49c1197640 1 --2- 172.21.15.90:0/1396826230 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49bc154880 0x7f49bc1533a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:46.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.148+0000 7f49c1197640 1 --2- 172.21.15.90:0/1396826230 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f49a4003b40 0x7f49bc1f1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:46.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.148+0000 7f49c1197640 1 -- 172.21.15.90:0/1396826230 >> 172.21.15.90:0/1396826230 conn(0x7f49bc06b620 msgr2=0x7f49bc0c7990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:46.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.148+0000 7f49c1197640 1 -- 172.21.15.90:0/1396826230 shutdown_connections 2024-03-11T20:51:46.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.148+0000 7f49c1197640 1 -- 172.21.15.90:0/1396826230 wait complete. 2024-03-11T20:51:46.163 INFO:tasks.ceph:Scrubbing osd.7 2024-03-11T20:51:46.164 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-03-11T20:51:46.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.288+0000 7ffb5a0f1640 1 Processor -- start 2024-03-11T20:51:46.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.288+0000 7ffb5a0f1640 1 -- start start 2024-03-11T20:51:46.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.288+0000 7ffb5a0f1640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7ffb5406c220 0x7ffb54066c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:46.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.288+0000 7ffb5a0f1640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ffb54157c80 0x7ffb54158110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:46.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.288+0000 7ffb5a0f1640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb54158650 con 0x7ffb5406c600 2024-03-11T20:51:46.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.288+0000 7ffb5a0f1640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb540b1de0 con 0x7ffb5406c220 2024-03-11T20:51:46.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.288+0000 7ffb5a0f1640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb540c9a60 con 0x7ffb54157c80 2024-03-11T20:51:46.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.296+0000 7ffb52ffd640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7ffb5406c600 0x7ffb540c64c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:45088/0 (socket says 172.21.15.90:45088) 2024-03-11T20:51:46.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.296+0000 7ffb52ffd640 1 -- 172.21.15.90:0/2833208981 learned_addr learned my addr 172.21.15.90:0/2833208981 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T20:51:46.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.300+0000 7ffb527fc640 1 -- 172.21.15.90:0/2833208981 <== mon.0 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 938820006 0 0) 0x7ffb44001120 con 0x7ffb5406c600 2024-03-11T20:51:46.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.300+0000 7ffb527fc640 1 -- 172.21.15.90:0/2833208981 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb40003530 con 0x7ffb5406c600 2024-03-11T20:51:46.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.304+0000 7ffb527fc640 1 -- 172.21.15.90:0/2833208981 <== mon.0 v1:172.21.15.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1483359116 0 0) 0x7ffb40003530 con 0x7ffb5406c600 2024-03-11T20:51:46.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.304+0000 7ffb527fc640 1 -- 172.21.15.90:0/2833208981 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ffb54157c80 msgr2=0x7ffb54158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:46.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.304+0000 7ffb527fc640 1 --2- 172.21.15.90:0/2833208981 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ffb54157c80 0x7ffb54158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:46.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.304+0000 7ffb527fc640 1 -- 172.21.15.90:0/2833208981 >> v2:172.21.15.92:3300/0 conn(0x7ffb5406c220 msgr2=0x7ffb54066c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:46.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.304+0000 7ffb527fc640 1 --2- 172.21.15.90:0/2833208981 >> v2:172.21.15.92:3300/0 conn(0x7ffb5406c220 0x7ffb54066c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:46.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.304+0000 7ffb527fc640 1 -- 172.21.15.90:0/2833208981 --> v1:172.21.15.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb540c9d40 con 0x7ffb5406c600 2024-03-11T20:51:46.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.304+0000 7ffb527fc640 1 -- 172.21.15.90:0/2833208981 <== mon.0 v1:172.21.15.90:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 691735435 0 0) 0x7ffb44001810 con 0x7ffb5406c600 2024-03-11T20:51:46.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.304+0000 7ffb527fc640 1 -- 172.21.15.90:0/2833208981 >> v1:172.21.15.90:6789/0 conn(0x7ffb5406c600 legacy=0x7ffb540c64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:46.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.304+0000 7ffb527fc640 1 --2- 172.21.15.90:0/2833208981 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ffb54157c80 0x7ffb40003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:46.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.304+0000 7ffb527fc640 1 --2- 172.21.15.90:0/2833208981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb400040e0 0x7ffb400047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:46.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.304+0000 7ffb527fc640 1 --2- 172.21.15.90:0/2833208981 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb40004e30 0x7ffb4000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:46.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.304+0000 7ffb527fc640 1 -- 172.21.15.90:0/2833208981 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb540b1de0 con 0x7ffb40004e30 2024-03-11T20:51:46.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.304+0000 7ffb527fc640 1 -- 172.21.15.90:0/2833208981 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb540c9a60 con 0x7ffb400040e0 2024-03-11T20:51:46.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.304+0000 7ffb527fc640 1 -- 172.21.15.90:0/2833208981 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb40005640 con 0x7ffb54157c80 2024-03-11T20:51:46.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.304+0000 7ffb53fff640 1 --2- 172.21.15.90:0/2833208981 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb40004e30 0x7ffb4000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:46.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.308+0000 7ffb53fff640 1 -- 172.21.15.90:0/2833208981 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ffb54157c80 msgr2=0x7ffb40003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:46.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.308+0000 7ffb53fff640 1 --2- 172.21.15.90:0/2833208981 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ffb54157c80 0x7ffb40003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:46.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.308+0000 7ffb53fff640 1 -- 172.21.15.90:0/2833208981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb400040e0 msgr2=0x7ffb400047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:46.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.308+0000 7ffb53fff640 1 --2- 172.21.15.90:0/2833208981 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb400040e0 0x7ffb400047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:46.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.308+0000 7ffb53fff640 1 -- 172.21.15.90:0/2833208981 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb40003530 con 0x7ffb40004e30 2024-03-11T20:51:46.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.312+0000 7ffb53fff640 1 --2- 172.21.15.90:0/2833208981 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb40004e30 0x7ffb4000b990 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c009ca0 tx=0x7ffb4c01cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcf1f1a86c9be335 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:46.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.316+0000 7ffb527fc640 1 -- 172.21.15.90:0/2833208981 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c0a5c80 con 0x7ffb40004e30 2024-03-11T20:51:46.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.316+0000 7ffb527fc640 1 -- 172.21.15.90:0/2833208981 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb4c0a5e00 con 0x7ffb40004e30 2024-03-11T20:51:46.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.316+0000 7ffb527fc640 1 -- 172.21.15.90:0/2833208981 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c0ad680 con 0x7ffb40004e30 2024-03-11T20:51:46.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.320+0000 7ffb5a0f1640 1 -- 172.21.15.90:0/2833208981 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb40004e30 msgr2=0x7ffb4000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:46.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.320+0000 7ffb5a0f1640 1 --2- 172.21.15.90:0/2833208981 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb40004e30 0x7ffb4000b990 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c009ca0 tx=0x7ffb4c01cc10 comp rx=0 tx=0).stop 2024-03-11T20:51:46.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.320+0000 7ffb53fff640 1 -- 172.21.15.90:0/2833208981 reap_dead start 2024-03-11T20:51:46.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.320+0000 7ffb5a0f1640 1 -- 172.21.15.90:0/2833208981 shutdown_connections 2024-03-11T20:51:46.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.320+0000 7ffb5a0f1640 1 -- 172.21.15.90:0/2833208981 >> 172.21.15.90:0/2833208981 conn(0x7ffb540cbac0 msgr2=0x7ffb540cbed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:46.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.320+0000 7ffb5a0f1640 1 -- 172.21.15.90:0/2833208981 shutdown_connections 2024-03-11T20:51:46.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.320+0000 7ffb5a0f1640 1 -- 172.21.15.90:0/2833208981 wait complete. 2024-03-11T20:51:46.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.320+0000 7ffb5a0f1640 1 Processor -- start 2024-03-11T20:51:46.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.320+0000 7ffb5a0f1640 1 -- start start 2024-03-11T20:51:46.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.320+0000 7ffb5a0f1640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb541f26f0 0x7ffb540cc470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:46.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.320+0000 7ffb5a0f1640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ffb541f2ad0 0x7ffb5414fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:46.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.320+0000 7ffb5a0f1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb5414f4c0 0x7ffb54150130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:46.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.320+0000 7ffb5a0f1640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5414caa0 con 0x7ffb541f26f0 2024-03-11T20:51:46.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.320+0000 7ffb5a0f1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb540c4920 con 0x7ffb5414f4c0 2024-03-11T20:51:46.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.320+0000 7ffb5a0f1640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb540c4e10 con 0x7ffb541f2ad0 2024-03-11T20:51:46.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.320+0000 7ffb537fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb541f26f0 0x7ffb540cc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:46.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.320+0000 7ffb537fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb541f26f0 0x7ffb540cc470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44066/0 (socket says 172.21.15.90:44066) 2024-03-11T20:51:46.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.320+0000 7ffb537fe640 1 -- 172.21.15.90:0/3532888748 learned_addr learned my addr 172.21.15.90:0/3532888748 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:51:46.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.324+0000 7ffb537fe640 1 -- 172.21.15.90:0/3532888748 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ffb541f2ad0 msgr2=0x7ffb5414fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:46.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.324+0000 7ffb537fe640 1 --2- 172.21.15.90:0/3532888748 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7ffb541f2ad0 0x7ffb5414fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:46.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.324+0000 7ffb537fe640 1 -- 172.21.15.90:0/3532888748 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb5414f4c0 msgr2=0x7ffb54150130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:46.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.324+0000 7ffb537fe640 1 --2- 172.21.15.90:0/3532888748 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7ffb5414f4c0 0x7ffb54150130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:46.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.324+0000 7ffb537fe640 1 -- 172.21.15.90:0/3532888748 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb5414ccd0 con 0x7ffb541f26f0 2024-03-11T20:51:46.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.328+0000 7ffb537fe640 1 --2- 172.21.15.90:0/3532888748 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb541f26f0 0x7ffb540cc470 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c00b430 tx=0x7ffb3c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:46.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.332+0000 7ffb4bfff640 1 -- 172.21.15.90:0/3532888748 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c011830 con 0x7ffb541f26f0 2024-03-11T20:51:46.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.332+0000 7ffb4bfff640 1 -- 172.21.15.90:0/3532888748 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb3c011990 con 0x7ffb541f26f0 2024-03-11T20:51:46.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.332+0000 7ffb5a0f1640 1 -- 172.21.15.90:0/3532888748 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb540c4b50 con 0x7ffb541f26f0 2024-03-11T20:51:46.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.332+0000 7ffb4bfff640 1 -- 172.21.15.90:0/3532888748 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c011d40 con 0x7ffb541f26f0 2024-03-11T20:51:46.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.332+0000 7ffb5a0f1640 1 -- 172.21.15.90:0/3532888748 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb5414cf40 con 0x7ffb541f26f0 2024-03-11T20:51:46.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.332+0000 7ffb5a0f1640 1 -- 172.21.15.90:0/3532888748 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ffb540ca300 con 0x7ffb541f26f0 2024-03-11T20:51:46.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.352+0000 7ffb4bfff640 1 -- 172.21.15.90:0/3532888748 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7ffb3c01a400 con 0x7ffb541f26f0 2024-03-11T20:51:46.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.352+0000 7ffb4bfff640 1 --2- 172.21.15.90:0/3532888748 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7ffb34066e90 0x7ffb34069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:46.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.352+0000 7ffb52ffd640 1 --2- 172.21.15.90:0/3532888748 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7ffb34066e90 0x7ffb34069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:46.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.352+0000 7ffb52ffd640 1 --2- 172.21.15.90:0/3532888748 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7ffb34066e90 0x7ffb34069350 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ffb44001060 tx=0x7ffb440011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:46.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.360+0000 7ffb4bfff640 1 -- 172.21.15.90:0/3532888748 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7ffb3c056480 con 0x7ffb541f26f0 2024-03-11T20:51:46.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.360+0000 7ffb4bfff640 1 --2- 172.21.15.90:0/3532888748 >> [v2:172.21.15.92:6824/3475925719,v1:172.21.15.92:6825/3475925719] conn(0x7ffb34070820 0x7ffb34072cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:46.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.360+0000 7ffb4bfff640 1 -- 172.21.15.90:0/3532888748 --> [v2:172.21.15.92:6824/3475925719,v1:172.21.15.92:6825/3475925719] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffb34073390 con 0x7ffb34070820 2024-03-11T20:51:46.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.360+0000 7ffb4bfff640 1 -- 172.21.15.90:0/3532888748 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb3c011ea0 con 0x7ffb541f26f0 2024-03-11T20:51:46.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.360+0000 7ffb53fff640 1 --2- 172.21.15.90:0/3532888748 >> [v2:172.21.15.92:6824/3475925719,v1:172.21.15.92:6825/3475925719] conn(0x7ffb34070820 0x7ffb34072cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:46.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.364+0000 7ffb53fff640 1 --2- 172.21.15.90:0/3532888748 >> [v2:172.21.15.92:6824/3475925719,v1:172.21.15.92:6825/3475925719] conn(0x7ffb34070820 0x7ffb34072cc0 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:46.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.364+0000 7ffb4bfff640 1 -- 172.21.15.90:0/3532888748 <== osd.7 v2:172.21.15.92:6824/3475925719 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ffb34073390 con 0x7ffb34070820 2024-03-11T20:51:46.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.376+0000 7ffb5a0f1640 1 -- 172.21.15.90:0/3532888748 --> [v2:172.21.15.92:6824/3475925719,v1:172.21.15.92:6825/3475925719] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ffb5414d130 con 0x7ffb34070820 2024-03-11T20:51:46.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.380+0000 7ffb4bfff640 1 -- 172.21.15.90:0/3532888748 <== osd.7 v2:172.21.15.92:6824/3475925719 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7ffb5414d130 con 0x7ffb34070820 2024-03-11T20:51:46.377 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-03-11T20:51:46.377 INFO:teuthology.orchestra.run.smithi090.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-11T20:51:46.377 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-03-11T20:51:46.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.380+0000 7ffb5a0f1640 1 -- 172.21.15.90:0/3532888748 >> [v2:172.21.15.92:6824/3475925719,v1:172.21.15.92:6825/3475925719] conn(0x7ffb34070820 msgr2=0x7ffb34072cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:46.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.380+0000 7ffb5a0f1640 1 --2- 172.21.15.90:0/3532888748 >> [v2:172.21.15.92:6824/3475925719,v1:172.21.15.92:6825/3475925719] conn(0x7ffb34070820 0x7ffb34072cc0 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:46.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.380+0000 7ffb5a0f1640 1 -- 172.21.15.90:0/3532888748 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7ffb34066e90 msgr2=0x7ffb34069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:46.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.380+0000 7ffb5a0f1640 1 --2- 172.21.15.90:0/3532888748 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7ffb34066e90 0x7ffb34069350 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ffb44001060 tx=0x7ffb440011a0 comp rx=0 tx=0).stop 2024-03-11T20:51:46.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.380+0000 7ffb5a0f1640 1 -- 172.21.15.90:0/3532888748 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb541f26f0 msgr2=0x7ffb540cc470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:46.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.380+0000 7ffb5a0f1640 1 --2- 172.21.15.90:0/3532888748 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb541f26f0 0x7ffb540cc470 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c00b430 tx=0x7ffb3c00b8f0 comp rx=0 tx=0).stop 2024-03-11T20:51:46.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.380+0000 7ffb53fff640 1 -- 172.21.15.90:0/3532888748 reap_dead start 2024-03-11T20:51:46.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.380+0000 7ffb5a0f1640 1 -- 172.21.15.90:0/3532888748 shutdown_connections 2024-03-11T20:51:46.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.380+0000 7ffb5a0f1640 1 -- 172.21.15.90:0/3532888748 >> 172.21.15.90:0/3532888748 conn(0x7ffb540cbac0 msgr2=0x7ffb5406b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:46.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.380+0000 7ffb5a0f1640 1 -- 172.21.15.90:0/3532888748 shutdown_connections 2024-03-11T20:51:46.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.380+0000 7ffb5a0f1640 1 -- 172.21.15.90:0/3532888748 wait complete. 2024-03-11T20:51:46.393 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-03-11T20:51:46.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.508+0000 7f307ab9d640 1 Processor -- start 2024-03-11T20:51:46.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.508+0000 7f307ab9d640 1 -- start start 2024-03-11T20:51:46.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.508+0000 7f307ab9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f3074070b00 0x7f30740c4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:46.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.508+0000 7f307ab9d640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30740c5de0 con 0x7f3074150dd0 2024-03-11T20:51:46.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.508+0000 7f307ab9d640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3074156910 con 0x7f30741509f0 2024-03-11T20:51:46.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.508+0000 7f307ab9d640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30740c6030 con 0x7f3074070b00 2024-03-11T20:51:46.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.508+0000 7f3073fff640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f3074150dd0 0x7f30740c28e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:45102/0 (socket says 172.21.15.90:45102) 2024-03-11T20:51:46.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.508+0000 7f3073fff640 1 -- 172.21.15.90:0/3097543805 learned_addr learned my addr 172.21.15.90:0/3097543805 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T20:51:46.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.508+0000 7f307913a640 1 --2- 172.21.15.90:0/3097543805 >> v2:172.21.15.92:3300/0 conn(0x7f3074070b00 0x7f30740c4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:46.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.512+0000 7f30737fe640 1 -- 172.21.15.90:0/3097543805 <== mon.0 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3447485202 0 0) 0x7f30740c5de0 con 0x7f3074150dd0 2024-03-11T20:51:46.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.512+0000 7f30737fe640 1 -- 172.21.15.90:0/3097543805 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f305c005740 con 0x7f3074150dd0 2024-03-11T20:51:46.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.516+0000 7f30737fe640 1 -- 172.21.15.90:0/3097543805 <== mon.0 v1:172.21.15.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 720305341 0 0) 0x7f305c005740 con 0x7f3074150dd0 2024-03-11T20:51:46.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.516+0000 7f30737fe640 1 -- 172.21.15.90:0/3097543805 >> v2:172.21.15.92:3300/0 conn(0x7f3074070b00 msgr2=0x7f30740c4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:46.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.516+0000 7f30737fe640 1 --2- 172.21.15.90:0/3097543805 >> v2:172.21.15.92:3300/0 conn(0x7f3074070b00 0x7f30740c4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:46.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.516+0000 7f30737fe640 1 -- 172.21.15.90:0/3097543805 >> v1:172.21.15.92:6789/0 conn(0x7f30741509f0 legacy=0x7f3074157390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:46.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.516+0000 7f30737fe640 1 -- 172.21.15.90:0/3097543805 --> v1:172.21.15.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30740c6310 con 0x7f3074150dd0 2024-03-11T20:51:46.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.516+0000 7f30737fe640 1 -- 172.21.15.90:0/3097543805 <== mon.0 v1:172.21.15.90:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 691735435 0 0) 0x7f3068001680 con 0x7f3074150dd0 2024-03-11T20:51:46.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.516+0000 7f30737fe640 1 -- 172.21.15.90:0/3097543805 >> v1:172.21.15.90:6789/0 conn(0x7f3074150dd0 legacy=0x7f30740c28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:46.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.516+0000 7f30737fe640 1 --2- 172.21.15.90:0/3097543805 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f305c005be0 0x7f305c005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:46.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.516+0000 7f30737fe640 1 --2- 172.21.15.90:0/3097543805 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f305c0069e0 0x7f305c006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:46.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.516+0000 7f30737fe640 1 --2- 172.21.15.90:0/3097543805 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f305c0073a0 0x7f305c00dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:46.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.516+0000 7f30737fe640 1 -- 172.21.15.90:0/3097543805 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30740c6030 con 0x7f305c005be0 2024-03-11T20:51:46.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.516+0000 7f30737fe640 1 -- 172.21.15.90:0/3097543805 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305c00e270 con 0x7f305c0069e0 2024-03-11T20:51:46.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.516+0000 7f30737fe640 1 -- 172.21.15.90:0/3097543805 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f305c00e430 con 0x7f305c0073a0 2024-03-11T20:51:46.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.516+0000 7f307913a640 1 --2- 172.21.15.90:0/3097543805 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f305c0073a0 0x7f305c00dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:46.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.520+0000 7f3078939640 1 --2- 172.21.15.90:0/3097543805 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f305c005be0 0x7f305c005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:46.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.524+0000 7f3078939640 1 -- 172.21.15.90:0/3097543805 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f305c0073a0 msgr2=0x7f305c00dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:46.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.524+0000 7f3078939640 1 --2- 172.21.15.90:0/3097543805 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f305c0073a0 0x7f305c00dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:46.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.524+0000 7f3078939640 1 -- 172.21.15.90:0/3097543805 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f305c0069e0 msgr2=0x7f305c006e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:46.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.524+0000 7f3078939640 1 --2- 172.21.15.90:0/3097543805 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f305c0069e0 0x7f305c006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:46.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.524+0000 7f3078939640 1 -- 172.21.15.90:0/3097543805 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f305c005740 con 0x7f305c005be0 2024-03-11T20:51:46.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.524+0000 7f307913a640 1 -- 172.21.15.90:0/3097543805 reap_dead start 2024-03-11T20:51:46.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.524+0000 7f3078939640 1 --2- 172.21.15.90:0/3097543805 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f305c005be0 0x7f305c005fe0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f306c0044b0 tx=0x7f306c01c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b367cd99fa3b086 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:46.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.528+0000 7f30737fe640 1 -- 172.21.15.90:0/3097543805 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306c0a9660 con 0x7f305c005be0 2024-03-11T20:51:46.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.528+0000 7f30737fe640 1 -- 172.21.15.90:0/3097543805 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f306c0a97e0 con 0x7f305c005be0 2024-03-11T20:51:46.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.528+0000 7f30737fe640 1 -- 172.21.15.90:0/3097543805 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306c0a9b90 con 0x7f305c005be0 2024-03-11T20:51:46.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.532+0000 7f307ab9d640 1 -- 172.21.15.90:0/3097543805 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f305c005be0 msgr2=0x7f305c005fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:46.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.532+0000 7f307ab9d640 1 --2- 172.21.15.90:0/3097543805 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f305c005be0 0x7f305c005fe0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f306c0044b0 tx=0x7f306c01c680 comp rx=0 tx=0).stop 2024-03-11T20:51:46.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.532+0000 7f307ab9d640 1 -- 172.21.15.90:0/3097543805 shutdown_connections 2024-03-11T20:51:46.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.532+0000 7f307ab9d640 1 --2- 172.21.15.90:0/3097543805 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f305c005be0 0x7f305c005fe0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:46.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.532+0000 7f307ab9d640 1 -- 172.21.15.90:0/3097543805 >> 172.21.15.90:0/3097543805 conn(0x7f307406ac00 msgr2=0x7f307406b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:46.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.532+0000 7f307ab9d640 1 -- 172.21.15.90:0/3097543805 shutdown_connections 2024-03-11T20:51:46.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.532+0000 7f307ab9d640 1 -- 172.21.15.90:0/3097543805 wait complete. 2024-03-11T20:51:46.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.532+0000 7f307ab9d640 1 Processor -- start 2024-03-11T20:51:46.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.532+0000 7f307ab9d640 1 -- start start 2024-03-11T20:51:46.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.532+0000 7f307ab9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f305c005be0 0x7f3074150010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:46.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.532+0000 7f307ab9d640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f307413f270 0x7f307413d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:46.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.532+0000 7f307ab9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3074143d10 0x7f30740ca360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:46.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.532+0000 7f307ab9d640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30741ef290 con 0x7f305c005be0 2024-03-11T20:51:46.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.532+0000 7f307ab9d640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307413d690 con 0x7f3074143d10 2024-03-11T20:51:46.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.532+0000 7f307ab9d640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30741ee3c0 con 0x7f307413f270 2024-03-11T20:51:46.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.532+0000 7f3078939640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f305c005be0 0x7f3074150010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:46.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.532+0000 7f3078939640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f305c005be0 0x7f3074150010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44086/0 (socket says 172.21.15.90:44086) 2024-03-11T20:51:46.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.532+0000 7f3078939640 1 -- 172.21.15.90:0/3201664270 learned_addr learned my addr 172.21.15.90:0/3201664270 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:51:46.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.532+0000 7f3073fff640 1 --2- 172.21.15.90:0/3201664270 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f307413f270 0x7f307413d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:46.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.536+0000 7f307913a640 1 --2- 172.21.15.90:0/3201664270 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3074143d10 0x7f30740ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:46.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.536+0000 7f3078939640 1 -- 172.21.15.90:0/3201664270 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f307413f270 msgr2=0x7f307413d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:46.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.536+0000 7f3078939640 1 --2- 172.21.15.90:0/3201664270 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f307413f270 0x7f307413d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:46.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.536+0000 7f3078939640 1 -- 172.21.15.90:0/3201664270 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3074143d10 msgr2=0x7f30740ca360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:46.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.536+0000 7f3078939640 1 --2- 172.21.15.90:0/3201664270 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3074143d10 0x7f30740ca360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:46.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.536+0000 7f3078939640 1 -- 172.21.15.90:0/3201664270 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30741ee680 con 0x7f305c005be0 2024-03-11T20:51:46.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.540+0000 7f3078939640 1 --2- 172.21.15.90:0/3201664270 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f305c005be0 0x7f3074150010 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f306c000f90 tx=0x7f306c019420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:46.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.544+0000 7f30717fa640 1 -- 172.21.15.90:0/3201664270 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306c0ba480 con 0x7f305c005be0 2024-03-11T20:51:46.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.544+0000 7f30717fa640 1 -- 172.21.15.90:0/3201664270 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f306c0ba5e0 con 0x7f305c005be0 2024-03-11T20:51:46.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.544+0000 7f307ab9d640 1 -- 172.21.15.90:0/3201664270 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30741ee960 con 0x7f305c005be0 2024-03-11T20:51:46.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.544+0000 7f30717fa640 1 -- 172.21.15.90:0/3201664270 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306c0ba970 con 0x7f305c005be0 2024-03-11T20:51:46.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.544+0000 7f307ab9d640 1 -- 172.21.15.90:0/3201664270 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30741eef20 con 0x7f305c005be0 2024-03-11T20:51:46.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.544+0000 7f307ab9d640 1 -- 172.21.15.90:0/3201664270 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f307404eef0 con 0x7f305c005be0 2024-03-11T20:51:46.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.564+0000 7f30717fa640 1 -- 172.21.15.90:0/3201664270 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f306c0b13f0 con 0x7f305c005be0 2024-03-11T20:51:46.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.564+0000 7f30717fa640 1 --2- 172.21.15.90:0/3201664270 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f3060066e70 0x7f3060069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:46.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.564+0000 7f3073fff640 1 --2- 172.21.15.90:0/3201664270 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f3060066e70 0x7f3060069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:46.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.564+0000 7f3073fff640 1 --2- 172.21.15.90:0/3201664270 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f3060066e70 0x7f3060069330 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3074140c00 tx=0x7f30680019c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:46.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.852+0000 7f30717fa640 1 -- 172.21.15.90:0/3201664270 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f306c0efe70 con 0x7f305c005be0 2024-03-11T20:51:46.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:46.852+0000 7f30717fa640 1 -- 172.21.15.90:0/3201664270 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f306c0ec110 con 0x7f305c005be0 2024-03-11T20:51:47.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.016+0000 7f307ab9d640 1 -- 172.21.15.90:0/3201664270 --> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f30741544f0 con 0x7f3060066e70 2024-03-11T20:51:47.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.016+0000 7f30717fa640 1 -- 172.21.15.90:0/3201664270 <== mgr.4101 v2:172.21.15.90:6800/1999720161 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f30741544f0 con 0x7f3060066e70 2024-03-11T20:51:47.016 INFO:teuthology.orchestra.run.smithi090.stderr:instructed osd(s) 7 to deep-scrub 2024-03-11T20:51:47.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.020+0000 7f307ab9d640 1 -- 172.21.15.90:0/3201664270 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f3060066e70 msgr2=0x7f3060069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:47.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.020+0000 7f307ab9d640 1 --2- 172.21.15.90:0/3201664270 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f3060066e70 0x7f3060069330 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3074140c00 tx=0x7f30680019c0 comp rx=0 tx=0).stop 2024-03-11T20:51:47.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.020+0000 7f307ab9d640 1 -- 172.21.15.90:0/3201664270 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f305c005be0 msgr2=0x7f3074150010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:47.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.020+0000 7f307ab9d640 1 --2- 172.21.15.90:0/3201664270 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f305c005be0 0x7f3074150010 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f306c000f90 tx=0x7f306c019420 comp rx=0 tx=0).stop 2024-03-11T20:51:47.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.020+0000 7f307ab9d640 1 -- 172.21.15.90:0/3201664270 shutdown_connections 2024-03-11T20:51:47.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.020+0000 7f307ab9d640 1 --2- 172.21.15.90:0/3201664270 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f3060066e70 0x7f3060069330 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:47.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.020+0000 7f307ab9d640 1 --2- 172.21.15.90:0/3201664270 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f3074143d10 0x7f30740ca360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:47.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.020+0000 7f307ab9d640 1 --2- 172.21.15.90:0/3201664270 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f307413f270 0x7f307413d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:47.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.020+0000 7f307ab9d640 1 --2- 172.21.15.90:0/3201664270 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f305c005be0 0x7f3074150010 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:47.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.020+0000 7f307ab9d640 1 -- 172.21.15.90:0/3201664270 >> 172.21.15.90:0/3201664270 conn(0x7f307406ac00 msgr2=0x7f30740cb370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:47.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.020+0000 7f307ab9d640 1 -- 172.21.15.90:0/3201664270 shutdown_connections 2024-03-11T20:51:47.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.020+0000 7f307ab9d640 1 -- 172.21.15.90:0/3201664270 wait complete. 2024-03-11T20:51:47.036 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-11T20:51:47.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.152+0000 7f2bd8bb2640 1 Processor -- start 2024-03-11T20:51:47.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.152+0000 7f2bd8bb2640 1 -- start start 2024-03-11T20:51:47.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.152+0000 7f2bd8bb2640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f2bd4151000 0x7f2bd40c2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:47.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.152+0000 7f2bd8bb2640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bd40c7980 con 0x7f2bd40c5b50 2024-03-11T20:51:47.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.152+0000 7f2bd8bb2640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bd40c2ad0 con 0x7f2bd4066d60 2024-03-11T20:51:47.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.152+0000 7f2bd8bb2640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd40cc820 con 0x7f2bd4151000 2024-03-11T20:51:47.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.152+0000 7f2bd1d9b640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f2bd40c5b50 0x7f2bd40c4ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:45108/0 (socket says 172.21.15.90:45108) 2024-03-11T20:51:47.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.152+0000 7f2bd1d9b640 1 -- 172.21.15.90:0/3336553261 learned_addr learned my addr 172.21.15.90:0/3336553261 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T20:51:47.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.160+0000 7f2bd259c640 1 --2- 172.21.15.90:0/3336553261 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f2bd4151000 0x7f2bd40c2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:47.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.160+0000 7f2bd159a640 1 -- 172.21.15.90:0/3336553261 <== mon.0 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3709068676 0 0) 0x7f2bd40c7980 con 0x7f2bd40c5b50 2024-03-11T20:51:47.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.160+0000 7f2bd159a640 1 -- 172.21.15.90:0/3336553261 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bc0005740 con 0x7f2bd40c5b50 2024-03-11T20:51:47.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.164+0000 7f2bd259c640 1 -- 172.21.15.90:0/3336553261 >> v1:172.21.15.92:6789/0 conn(0x7f2bd4066d60 legacy=0x7f2bd41613a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:51:47.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.164+0000 7f2bd259c640 1 -- 172.21.15.90:0/3336553261 >> v1:172.21.15.90:6789/0 conn(0x7f2bd40c5b50 legacy=0x7f2bd40c4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:47.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.164+0000 7f2bd259c640 1 -- 172.21.15.90:0/3336553261 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd40ca9c0 con 0x7f2bd4151000 2024-03-11T20:51:47.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.172+0000 7f2bd259c640 1 --2- 172.21.15.90:0/3336553261 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f2bd4151000 0x7f2bd40c2d80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc0097f0 tx=0x7f2bbc01c380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fde4d1297b308910 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:47.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.200+0000 7f2bd159a640 1 -- 172.21.15.90:0/3336553261 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc0ab400 con 0x7f2bd4151000 2024-03-11T20:51:47.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.200+0000 7f2bd159a640 1 -- 172.21.15.90:0/3336553261 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bbc0a7050 con 0x7f2bd4151000 2024-03-11T20:51:47.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.200+0000 7f2bd159a640 1 -- 172.21.15.90:0/3336553261 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc0ab780 con 0x7f2bd4151000 2024-03-11T20:51:47.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.204+0000 7f2bd8bb2640 1 -- 172.21.15.90:0/3336553261 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f2bd4151000 msgr2=0x7f2bd40c2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:47.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.204+0000 7f2bd8bb2640 1 --2- 172.21.15.90:0/3336553261 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f2bd4151000 0x7f2bd40c2d80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc0097f0 tx=0x7f2bbc01c380 comp rx=0 tx=0).stop 2024-03-11T20:51:47.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.204+0000 7f2bd8bb2640 1 -- 172.21.15.90:0/3336553261 shutdown_connections 2024-03-11T20:51:47.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.204+0000 7f2bd8bb2640 1 --2- 172.21.15.90:0/3336553261 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f2bd4151000 0x7f2bd40c2d80 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:47.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.204+0000 7f2bd8bb2640 1 -- 172.21.15.90:0/3336553261 >> 172.21.15.90:0/3336553261 conn(0x7f2bd40cb880 msgr2=0x7f2bd40cbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:47.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.204+0000 7f2bd8bb2640 1 -- 172.21.15.90:0/3336553261 shutdown_connections 2024-03-11T20:51:47.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.204+0000 7f2bd8bb2640 1 -- 172.21.15.90:0/3336553261 wait complete. 2024-03-11T20:51:47.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.204+0000 7f2bd8bb2640 1 Processor -- start 2024-03-11T20:51:47.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.204+0000 7f2bd8bb2640 1 -- start start 2024-03-11T20:51:47.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.204+0000 7f2bd8bb2640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f2bd4066d60 0x7f2bd41efd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:47.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.204+0000 7f2bd8bb2640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2bd40c5b50 0x7f2bd4156640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:47.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.204+0000 7f2bd8bb2640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2bd4151000 0x7f2bd41ef320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:47.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.204+0000 7f2bd8bb2640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd41ef860 con 0x7f2bd40c5b50 2024-03-11T20:51:47.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.204+0000 7f2bd8bb2640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd41561b0 con 0x7f2bd4151000 2024-03-11T20:51:47.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.204+0000 7f2bd8bb2640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd415a5c0 con 0x7f2bd4066d60 2024-03-11T20:51:47.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.204+0000 7f2bd2d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2bd4151000 0x7f2bd41ef320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:47.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.208+0000 7f2bd2d9d640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2bd4151000 0x7f2bd41ef320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.90:36374/0 (socket says 172.21.15.90:36374) 2024-03-11T20:51:47.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.208+0000 7f2bd2d9d640 1 -- 172.21.15.90:0/311236932 learned_addr learned my addr 172.21.15.90:0/311236932 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:51:47.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.212+0000 7f2bd2d9d640 1 -- 172.21.15.90:0/311236932 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f2bd4066d60 msgr2=0x7f2bd41efd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:47.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.212+0000 7f2bd2d9d640 1 --2- 172.21.15.90:0/311236932 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f2bd4066d60 0x7f2bd41efd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:47.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.212+0000 7f2bd2d9d640 1 -- 172.21.15.90:0/311236932 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2bd40c5b50 msgr2=0x7f2bd4156640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:47.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.212+0000 7f2bd2d9d640 1 --2- 172.21.15.90:0/311236932 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2bd40c5b50 0x7f2bd4156640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:47.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.212+0000 7f2bd2d9d640 1 -- 172.21.15.90:0/311236932 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bbc009420 con 0x7f2bd4151000 2024-03-11T20:51:47.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.228+0000 7f2bd2d9d640 1 --2- 172.21.15.90:0/311236932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2bd4151000 0x7f2bd41ef320 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc00c410 tx=0x7f2bcc00c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:47.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.236+0000 7f2bcaffd640 1 -- 172.21.15.90:0/311236932 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc003600 con 0x7f2bd4151000 2024-03-11T20:51:47.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.236+0000 7f2bcaffd640 1 -- 172.21.15.90:0/311236932 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bcc003760 con 0x7f2bd4151000 2024-03-11T20:51:47.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.240+0000 7f2bd8bb2640 1 -- 172.21.15.90:0/311236932 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bd41574d0 con 0x7f2bd4151000 2024-03-11T20:51:47.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.240+0000 7f2bcaffd640 1 -- 172.21.15.90:0/311236932 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc003b50 con 0x7f2bd4151000 2024-03-11T20:51:47.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.240+0000 7f2bd8bb2640 1 -- 172.21.15.90:0/311236932 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bd4155b60 con 0x7f2bd4151000 2024-03-11T20:51:47.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.240+0000 7f2bd8bb2640 1 -- 172.21.15.90:0/311236932 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bd41559c0 con 0x7f2bd4151000 2024-03-11T20:51:47.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.276+0000 7f2bcaffd640 1 -- 172.21.15.90:0/311236932 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f2bcc003cb0 con 0x7f2bd4151000 2024-03-11T20:51:47.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.276+0000 7f2bcaffd640 1 --2- 172.21.15.90:0/311236932 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f2bb4066e90 0x7f2bb4069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:51:47.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.276+0000 7f2bd259c640 1 --2- 172.21.15.90:0/311236932 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f2bb4066e90 0x7f2bb4069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:51:47.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.276+0000 7f2bd259c640 1 --2- 172.21.15.90:0/311236932 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f2bb4066e90 0x7f2bb4069350 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc0ad000 tx=0x7f2bbc004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:51:47.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.564+0000 7f2bcaffd640 1 -- 172.21.15.90:0/311236932 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f2bcc0554e0 con 0x7f2bd4151000 2024-03-11T20:51:47.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.596+0000 7f2bcaffd640 1 -- 172.21.15.90:0/311236932 <== mon.1 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2bcc051080 con 0x7f2bd4151000 2024-03-11T20:51:47.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.764+0000 7f2bd8bb2640 1 -- 172.21.15.90:0/311236932 --> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2bd406fbe0 con 0x7f2bb4066e90 2024-03-11T20:51:47.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.788+0000 7f2bcaffd640 1 -- 172.21.15.90:0/311236932 <== mgr.4101 v2:172.21.15.90:6800/1999720161 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+330800 (secure 0 0 0) 0x7f2bd406fbe0 con 0x7f2bb4066e90 2024-03-11T20:51:47.792 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T20:51:47.795 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-03-11T20:51:47.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.792+0000 7f2bd8bb2640 1 -- 172.21.15.90:0/311236932 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f2bb4066e90 msgr2=0x7f2bb4069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:47.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.792+0000 7f2bd8bb2640 1 --2- 172.21.15.90:0/311236932 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f2bb4066e90 0x7f2bb4069350 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc0ad000 tx=0x7f2bbc004740 comp rx=0 tx=0).stop 2024-03-11T20:51:47.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.792+0000 7f2bd8bb2640 1 -- 172.21.15.90:0/311236932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2bd4151000 msgr2=0x7f2bd41ef320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:51:47.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.792+0000 7f2bd8bb2640 1 --2- 172.21.15.90:0/311236932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2bd4151000 0x7f2bd41ef320 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc00c410 tx=0x7f2bcc00c8d0 comp rx=0 tx=0).stop 2024-03-11T20:51:47.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.792+0000 7f2bd8bb2640 1 -- 172.21.15.90:0/311236932 shutdown_connections 2024-03-11T20:51:47.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.792+0000 7f2bd8bb2640 1 --2- 172.21.15.90:0/311236932 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f2bb4066e90 0x7f2bb4069350 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:47.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.792+0000 7f2bd8bb2640 1 --2- 172.21.15.90:0/311236932 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f2bd4151000 0x7f2bd41ef320 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:47.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.792+0000 7f2bd8bb2640 1 --2- 172.21.15.90:0/311236932 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2bd40c5b50 0x7f2bd4156640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:47.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.792+0000 7f2bd8bb2640 1 --2- 172.21.15.90:0/311236932 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f2bd4066d60 0x7f2bd41efd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:51:47.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.792+0000 7f2bd8bb2640 1 -- 172.21.15.90:0/311236932 >> 172.21.15.90:0/311236932 conn(0x7f2bd40cb880 msgr2=0x7f2bd414f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:51:47.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.792+0000 7f2bd8bb2640 1 -- 172.21.15.90:0/311236932 shutdown_connections 2024-03-11T20:51:47.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:51:47.792+0000 7f2bd8bb2640 1 -- 172.21.15.90:0/311236932 wait complete. 2024-03-11T20:51:47.814 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":4121,"stamp":"2024-03-11T20:51:47.455097+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":211048067,"num_objects":150,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":150,"num_whiteouts":0,"num_read":7774,"num_read_kb":1584288,"num_write":72048,"num_write_kb":4420358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":10000,"num_omap_keys":20,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76473,"ondisk_log_size":76473,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1577152,"kb_used_data":414912,"kb_used_omap":12,"kb_used_meta":1162163,"kb_avail":753397568,"statfs":{"total":773094113280,"available":771479109632,"internally_reserved":0,"allocated":424869888,"data_stored":423141350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":1190055476},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":10000,"num_omap_keys":20,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003879"},"pg_stats":[{"pgid":"2.1f","version":"30'689","reported_seq":794,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:42.501287+0000","last_change":"2024-03-11T20:51:42.501371+0000","last_active":"2024-03-11T20:51:42.501287+0000","last_peered":"2024-03-11T20:51:42.501287+0000","last_clean":"2024-03-11T20:51:42.501287+0000","last_became_active":"2024-03-11T18:35:56.289178+0000","last_became_peered":"2024-03-11T18:35:56.289178+0000","last_unstale":"2024-03-11T20:51:42.501287+0000","last_undegraded":"2024-03-11T20:51:42.501287+0000","last_fullsized":"2024-03-11T20:51:42.501287+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'689","last_scrub_stamp":"2024-03-11T20:51:42.501128+0000","last_deep_scrub":"30'689","last_deep_scrub_stamp":"2024-03-11T20:51:42.501128+0000","last_clean_scrub_stamp":"2024-03-11T20:51:42.501128+0000","objects_scrubbed":8,"log_size":689,"log_dups_size":0,"ondisk_log_size":689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:44:39.698011+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389280,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":643,"num_write_kb":8491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"30'202","reported_seq":299,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.763464+0000","last_change":"2024-03-11T18:35:57.676047+0000","last_active":"2024-03-11T20:51:29.763464+0000","last_peered":"2024-03-11T20:51:29.763464+0000","last_clean":"2024-03-11T20:51:29.763464+0000","last_became_active":"2024-03-11T18:35:57.675105+0000","last_became_peered":"2024-03-11T18:35:57.675105+0000","last_unstale":"2024-03-11T20:51:29.763464+0000","last_undegraded":"2024-03-11T20:51:29.763464+0000","last_fullsized":"2024-03-11T20:51:29.763464+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"30'811","reported_seq":1004,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:17.711896+0000","last_change":"2024-03-11T18:36:04.607351+0000","last_active":"2024-03-11T20:51:17.711896+0000","last_peered":"2024-03-11T20:51:17.711896+0000","last_clean":"2024-03-11T20:51:17.711896+0000","last_became_active":"2024-03-11T18:36:04.599562+0000","last_became_peered":"2024-03-11T18:36:04.599562+0000","last_unstale":"2024-03-11T20:51:17.711896+0000","last_undegraded":"2024-03-11T20:51:17.711896+0000","last_fullsized":"2024-03-11T20:51:17.711896+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":811,"log_dups_size":0,"ondisk_log_size":811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":119,"num_read_kb":64172,"num_write":795,"num_write_kb":180282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"30'654","reported_seq":753,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:50:41.325710+0000","last_change":"2024-03-11T18:35:56.291253+0000","last_active":"2024-03-11T20:50:41.325710+0000","last_peered":"2024-03-11T20:50:41.325710+0000","last_clean":"2024-03-11T20:50:41.325710+0000","last_became_active":"2024-03-11T18:35:56.290862+0000","last_became_peered":"2024-03-11T18:35:56.290862+0000","last_unstale":"2024-03-11T20:50:41.325710+0000","last_undegraded":"2024-03-11T20:50:41.325710+0000","last_fullsized":"2024-03-11T20:50:41.325710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":654,"log_dups_size":0,"ondisk_log_size":654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":603,"num_write_kb":8489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"30'187","reported_seq":287,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.903261+0000","last_change":"2024-03-11T18:37:04.143178+0000","last_active":"2024-03-11T20:51:29.903261+0000","last_peered":"2024-03-11T20:51:29.903261+0000","last_clean":"2024-03-11T20:51:29.903261+0000","last_became_active":"2024-03-11T18:37:04.142880+0000","last_became_peered":"2024-03-11T18:37:04.142880+0000","last_unstale":"2024-03-11T20:51:29.903261+0000","last_undegraded":"2024-03-11T20:51:29.903261+0000","last_fullsized":"2024-03-11T20:51:29.903261+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"30'785","reported_seq":973,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:27.367790+0000","last_change":"2024-03-11T18:37:04.274577+0000","last_active":"2024-03-11T20:51:27.367790+0000","last_peered":"2024-03-11T20:51:27.367790+0000","last_clean":"2024-03-11T20:51:27.367790+0000","last_became_active":"2024-03-11T18:37:04.274018+0000","last_became_peered":"2024-03-11T18:37:04.274018+0000","last_unstale":"2024-03-11T20:51:27.367790+0000","last_undegraded":"2024-03-11T20:51:27.367790+0000","last_fullsized":"2024-03-11T20:51:27.367790+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":785,"log_dups_size":0,"ondisk_log_size":785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":106,"num_read_kb":55696,"num_write":774,"num_write_kb":160695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2,3],"acting":[4,6,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"30'1734","reported_seq":1820,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:50:58.626621+0000","last_change":"2024-03-11T18:35:56.355147+0000","last_active":"2024-03-11T20:50:58.626621+0000","last_peered":"2024-03-11T20:50:58.626621+0000","last_clean":"2024-03-11T20:50:58.626621+0000","last_became_active":"2024-03-11T18:35:56.354388+0000","last_became_peered":"2024-03-11T18:35:56.354388+0000","last_unstale":"2024-03-11T20:50:58.626621+0000","last_undegraded":"2024-03-11T20:50:58.626621+0000","last_fullsized":"2024-03-11T20:50:58.626621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1734,"log_dups_size":0,"ondisk_log_size":1734,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1669,"num_write_kb":17224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"30'190","reported_seq":274,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.632507+0000","last_change":"2024-03-11T18:35:57.641486+0000","last_active":"2024-03-11T20:51:29.632507+0000","last_peered":"2024-03-11T20:51:29.632507+0000","last_clean":"2024-03-11T20:51:29.632507+0000","last_became_active":"2024-03-11T18:35:57.641052+0000","last_became_peered":"2024-03-11T18:35:57.641052+0000","last_unstale":"2024-03-11T20:51:29.632507+0000","last_undegraded":"2024-03-11T20:51:29.632507+0000","last_fullsized":"2024-03-11T20:51:29.632507+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"30'808","reported_seq":974,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:09.708970+0000","last_change":"2024-03-11T18:36:04.629291+0000","last_active":"2024-03-11T20:51:09.708970+0000","last_peered":"2024-03-11T20:51:09.708970+0000","last_clean":"2024-03-11T20:51:09.708970+0000","last_became_active":"2024-03-11T18:36:04.624110+0000","last_became_peered":"2024-03-11T18:36:04.624110+0000","last_unstale":"2024-03-11T20:51:09.708970+0000","last_undegraded":"2024-03-11T20:51:09.708970+0000","last_fullsized":"2024-03-11T20:51:09.708970+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":808,"log_dups_size":0,"ondisk_log_size":808,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":93,"num_read_kb":49009,"num_write":797,"num_write_kb":196902,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"30'832","reported_seq":918,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:08.619271+0000","last_change":"2024-03-11T18:35:56.298903+0000","last_active":"2024-03-11T20:51:08.619271+0000","last_peered":"2024-03-11T20:51:08.619271+0000","last_clean":"2024-03-11T20:51:08.619271+0000","last_became_active":"2024-03-11T18:35:56.297582+0000","last_became_peered":"2024-03-11T18:35:56.297582+0000","last_unstale":"2024-03-11T20:51:08.619271+0000","last_undegraded":"2024-03-11T20:51:08.619271+0000","last_fullsized":"2024-03-11T20:51:08.619271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":832,"log_dups_size":0,"ondisk_log_size":832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":777,"num_write_kb":4502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"30'203","reported_seq":287,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.628509+0000","last_change":"2024-03-11T18:35:57.640122+0000","last_active":"2024-03-11T20:51:29.628509+0000","last_peered":"2024-03-11T20:51:29.628509+0000","last_clean":"2024-03-11T20:51:29.628509+0000","last_became_active":"2024-03-11T18:35:57.639770+0000","last_became_peered":"2024-03-11T18:35:57.639770+0000","last_unstale":"2024-03-11T20:51:29.628509+0000","last_undegraded":"2024-03-11T20:51:29.628509+0000","last_fullsized":"2024-03-11T20:51:29.628509+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"30'729","reported_seq":826,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:14.552309+0000","last_change":"2024-03-11T18:36:04.627149+0000","last_active":"2024-03-11T20:51:14.552309+0000","last_peered":"2024-03-11T20:51:14.552309+0000","last_clean":"2024-03-11T20:51:14.552309+0000","last_became_active":"2024-03-11T18:36:04.626108+0000","last_became_peered":"2024-03-11T18:36:04.626108+0000","last_unstale":"2024-03-11T20:51:14.552309+0000","last_undegraded":"2024-03-11T20:51:14.552309+0000","last_fullsized":"2024-03-11T20:51:14.552309+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":729,"log_dups_size":0,"ondisk_log_size":729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":20804,"num_write":712,"num_write_kb":76413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"30'216","reported_seq":308,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.960518+0000","last_change":"2024-03-11T18:35:57.616739+0000","last_active":"2024-03-11T20:51:29.960518+0000","last_peered":"2024-03-11T20:51:29.960518+0000","last_clean":"2024-03-11T20:51:29.960518+0000","last_became_active":"2024-03-11T18:35:57.616398+0000","last_became_peered":"2024-03-11T18:35:57.616398+0000","last_unstale":"2024-03-11T20:51:29.960518+0000","last_undegraded":"2024-03-11T20:51:29.960518+0000","last_fullsized":"2024-03-11T20:51:29.960518+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"30'1448","reported_seq":1540,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:50:57.274882+0000","last_change":"2024-03-11T18:35:56.304762+0000","last_active":"2024-03-11T20:50:57.274882+0000","last_peered":"2024-03-11T20:50:57.274882+0000","last_clean":"2024-03-11T20:50:57.274882+0000","last_became_active":"2024-03-11T18:35:56.302803+0000","last_became_peered":"2024-03-11T18:35:56.302803+0000","last_unstale":"2024-03-11T20:50:57.274882+0000","last_undegraded":"2024-03-11T20:50:57.274882+0000","last_fullsized":"2024-03-11T20:50:57.274882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1448,"log_dups_size":0,"ondisk_log_size":1448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1389,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"30'793","reported_seq":966,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:07.364509+0000","last_change":"2024-03-11T18:36:04.626968+0000","last_active":"2024-03-11T20:51:07.364509+0000","last_peered":"2024-03-11T20:51:07.364509+0000","last_clean":"2024-03-11T20:51:07.364509+0000","last_became_active":"2024-03-11T18:36:04.626337+0000","last_became_peered":"2024-03-11T18:36:04.626337+0000","last_unstale":"2024-03-11T20:51:07.364509+0000","last_undegraded":"2024-03-11T20:51:07.364509+0000","last_fullsized":"2024-03-11T20:51:07.364509+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":793,"log_dups_size":0,"ondisk_log_size":793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":94,"num_read_kb":46134,"num_write":781,"num_write_kb":151379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"30'211","reported_seq":309,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:44.001936+0000","last_change":"2024-03-11T20:51:44.002049+0000","last_active":"2024-03-11T20:51:44.001936+0000","last_peered":"2024-03-11T20:51:44.001936+0000","last_clean":"2024-03-11T20:51:44.001936+0000","last_became_active":"2024-03-11T18:35:57.689039+0000","last_became_peered":"2024-03-11T18:35:57.689039+0000","last_unstale":"2024-03-11T20:51:44.001936+0000","last_undegraded":"2024-03-11T20:51:44.001936+0000","last_fullsized":"2024-03-11T20:51:44.001936+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'211","last_scrub_stamp":"2024-03-11T20:51:44.001743+0000","last_deep_scrub":"30'211","last_deep_scrub_stamp":"2024-03-11T20:51:44.001743+0000","last_clean_scrub_stamp":"2024-03-11T20:51:44.001743+0000","objects_scrubbed":0,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:14:43.227957+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"30'1239","reported_seq":1341,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:44.157527+0000","last_change":"2024-03-11T20:51:44.157640+0000","last_active":"2024-03-11T20:51:44.157527+0000","last_peered":"2024-03-11T20:51:44.157527+0000","last_clean":"2024-03-11T20:51:44.157527+0000","last_became_active":"2024-03-11T18:35:56.290504+0000","last_became_peered":"2024-03-11T18:35:56.290504+0000","last_unstale":"2024-03-11T20:51:44.157527+0000","last_undegraded":"2024-03-11T20:51:44.157527+0000","last_fullsized":"2024-03-11T20:51:44.157527+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'1239","last_scrub_stamp":"2024-03-11T20:51:44.157352+0000","last_deep_scrub":"30'1239","last_deep_scrub_stamp":"2024-03-11T20:51:44.157352+0000","last_clean_scrub_stamp":"2024-03-11T20:51:44.157352+0000","objects_scrubbed":4,"log_size":1239,"log_dups_size":0,"ondisk_log_size":1239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:34:55.996207+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1168,"num_write_kb":8796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"30'848","reported_seq":1063,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:15.404608+0000","last_change":"2024-03-11T18:36:04.626150+0000","last_active":"2024-03-11T20:51:15.404608+0000","last_peered":"2024-03-11T20:51:15.404608+0000","last_clean":"2024-03-11T20:51:15.404608+0000","last_became_active":"2024-03-11T18:36:04.625084+0000","last_became_peered":"2024-03-11T18:36:04.625084+0000","last_unstale":"2024-03-11T20:51:15.404608+0000","last_undegraded":"2024-03-11T20:51:15.404608+0000","last_fullsized":"2024-03-11T20:51:15.404608+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":848,"log_dups_size":0,"ondisk_log_size":848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":136,"num_read_kb":82099,"num_write":832,"num_write_kb":151378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"30'209","reported_seq":309,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:45.022737+0000","last_change":"2024-03-11T20:51:45.022797+0000","last_active":"2024-03-11T20:51:45.022737+0000","last_peered":"2024-03-11T20:51:45.022737+0000","last_clean":"2024-03-11T20:51:45.022737+0000","last_became_active":"2024-03-11T18:35:57.681098+0000","last_became_peered":"2024-03-11T18:35:57.681098+0000","last_unstale":"2024-03-11T20:51:45.022737+0000","last_undegraded":"2024-03-11T20:51:45.022737+0000","last_fullsized":"2024-03-11T20:51:45.022737+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'209","last_scrub_stamp":"2024-03-11T20:51:45.022563+0000","last_deep_scrub":"30'209","last_deep_scrub_stamp":"2024-03-11T20:51:45.022563+0000","last_clean_scrub_stamp":"2024-03-11T20:51:45.022563+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:48:10.282950+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"30'488","reported_seq":590,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:43.173760+0000","last_change":"2024-03-11T20:51:43.173812+0000","last_active":"2024-03-11T20:51:43.173760+0000","last_peered":"2024-03-11T20:51:43.173760+0000","last_clean":"2024-03-11T20:51:43.173760+0000","last_became_active":"2024-03-11T18:35:56.290006+0000","last_became_peered":"2024-03-11T18:35:56.290006+0000","last_unstale":"2024-03-11T20:51:43.173760+0000","last_undegraded":"2024-03-11T20:51:43.173760+0000","last_fullsized":"2024-03-11T20:51:43.173760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'488","last_scrub_stamp":"2024-03-11T20:51:43.173661+0000","last_deep_scrub":"30'488","last_deep_scrub_stamp":"2024-03-11T20:51:43.173661+0000","last_clean_scrub_stamp":"2024-03-11T20:51:43.173661+0000","objects_scrubbed":5,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:25:36.710996+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194394,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":433,"num_write_kb":4314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"30'749","reported_seq":895,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:08.591309+0000","last_change":"2024-03-11T18:36:04.620689+0000","last_active":"2024-03-11T20:51:08.591309+0000","last_peered":"2024-03-11T20:51:08.591309+0000","last_clean":"2024-03-11T20:51:08.591309+0000","last_became_active":"2024-03-11T18:36:04.620282+0000","last_became_peered":"2024-03-11T18:36:04.620282+0000","last_unstale":"2024-03-11T20:51:08.591309+0000","last_undegraded":"2024-03-11T20:51:08.591309+0000","last_fullsized":"2024-03-11T20:51:08.591309+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":749,"log_dups_size":0,"ondisk_log_size":749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":68,"num_read_kb":34102,"num_write":731,"num_write_kb":109760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"30'224","reported_seq":316,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.557763+0000","last_change":"2024-03-11T18:35:57.615234+0000","last_active":"2024-03-11T20:51:29.557763+0000","last_peered":"2024-03-11T20:51:29.557763+0000","last_clean":"2024-03-11T20:51:29.557763+0000","last_became_active":"2024-03-11T18:35:57.614885+0000","last_became_peered":"2024-03-11T18:35:57.614885+0000","last_unstale":"2024-03-11T20:51:29.557763+0000","last_undegraded":"2024-03-11T20:51:29.557763+0000","last_fullsized":"2024-03-11T20:51:29.557763+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":224,"log_dups_size":0,"ondisk_log_size":224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"30'348","reported_seq":442,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:50:27.732962+0000","last_change":"2024-03-11T18:35:56.304617+0000","last_active":"2024-03-11T20:50:27.732962+0000","last_peered":"2024-03-11T20:50:27.732962+0000","last_clean":"2024-03-11T20:50:27.732962+0000","last_became_active":"2024-03-11T18:35:56.301698+0000","last_became_peered":"2024-03-11T18:35:56.301698+0000","last_unstale":"2024-03-11T20:50:27.732962+0000","last_undegraded":"2024-03-11T20:50:27.732962+0000","last_fullsized":"2024-03-11T20:50:27.732962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":289,"num_write_kb":4232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"30'714","reported_seq":908,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:18.529402+0000","last_change":"2024-03-11T18:37:04.272234+0000","last_active":"2024-03-11T20:51:18.529402+0000","last_peered":"2024-03-11T20:51:18.529402+0000","last_clean":"2024-03-11T20:51:18.529402+0000","last_became_active":"2024-03-11T18:37:04.271646+0000","last_became_peered":"2024-03-11T18:37:04.271646+0000","last_unstale":"2024-03-11T20:51:18.529402+0000","last_undegraded":"2024-03-11T20:51:18.529402+0000","last_fullsized":"2024-03-11T20:51:18.529402+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":108,"num_read_kb":62192,"num_write":690,"num_write_kb":100980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,2],"acting":[4,3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"30'216","reported_seq":308,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.921148+0000","last_change":"2024-03-11T18:35:57.506963+0000","last_active":"2024-03-11T20:51:29.921148+0000","last_peered":"2024-03-11T20:51:29.921148+0000","last_clean":"2024-03-11T20:51:29.921148+0000","last_became_active":"2024-03-11T18:35:57.506038+0000","last_became_peered":"2024-03-11T18:35:57.506038+0000","last_unstale":"2024-03-11T20:51:29.921148+0000","last_undegraded":"2024-03-11T20:51:29.921148+0000","last_fullsized":"2024-03-11T20:51:29.921148+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"30'413","reported_seq":507,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:22.447987+0000","last_change":"2024-03-11T18:35:56.364512+0000","last_active":"2024-03-11T20:51:22.447987+0000","last_peered":"2024-03-11T20:51:22.447987+0000","last_clean":"2024-03-11T20:51:22.447987+0000","last_became_active":"2024-03-11T18:35:56.363731+0000","last_became_peered":"2024-03-11T18:35:56.363731+0000","last_unstale":"2024-03-11T20:51:22.447987+0000","last_undegraded":"2024-03-11T20:51:22.447987+0000","last_fullsized":"2024-03-11T20:51:22.447987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":342,"num_write_kb":4261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"30'829","reported_seq":1057,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:32.368627+0000","last_change":"2024-03-11T18:36:04.643782+0000","last_active":"2024-03-11T20:51:32.368627+0000","last_peered":"2024-03-11T20:51:32.368627+0000","last_clean":"2024-03-11T20:51:32.368627+0000","last_became_active":"2024-03-11T18:36:04.642133+0000","last_became_peered":"2024-03-11T18:36:04.642133+0000","last_unstale":"2024-03-11T20:51:32.368627+0000","last_undegraded":"2024-03-11T20:51:32.368627+0000","last_fullsized":"2024-03-11T20:51:32.368627+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":829,"log_dups_size":0,"ondisk_log_size":829,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":144,"num_read_kb":78357,"num_write":819,"num_write_kb":238245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"30'168","reported_seq":260,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.764678+0000","last_change":"2024-03-11T18:35:57.675207+0000","last_active":"2024-03-11T20:51:29.764678+0000","last_peered":"2024-03-11T20:51:29.764678+0000","last_clean":"2024-03-11T20:51:29.764678+0000","last_became_active":"2024-03-11T18:35:57.674472+0000","last_became_peered":"2024-03-11T18:35:57.674472+0000","last_unstale":"2024-03-11T20:51:29.764678+0000","last_undegraded":"2024-03-11T20:51:29.764678+0000","last_fullsized":"2024-03-11T20:51:29.764678+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"30'6434","reported_seq":6528,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:24.287424+0000","last_change":"2024-03-11T18:35:56.348479+0000","last_active":"2024-03-11T20:51:24.287424+0000","last_peered":"2024-03-11T20:51:24.287424+0000","last_clean":"2024-03-11T20:51:24.287424+0000","last_became_active":"2024-03-11T18:35:56.348040+0000","last_became_peered":"2024-03-11T18:35:56.348040+0000","last_unstale":"2024-03-11T20:51:24.287424+0000","last_undegraded":"2024-03-11T20:51:24.287424+0000","last_fullsized":"2024-03-11T20:51:24.287424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":6434,"log_dups_size":0,"ondisk_log_size":6434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17421,"num_write_kb":6249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"30'799","reported_seq":996,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:12.846369+0000","last_change":"2024-03-11T18:36:04.599805+0000","last_active":"2024-03-11T20:51:12.846369+0000","last_peered":"2024-03-11T20:51:12.846369+0000","last_clean":"2024-03-11T20:51:12.846369+0000","last_became_active":"2024-03-11T18:36:04.599065+0000","last_became_peered":"2024-03-11T18:36:04.599065+0000","last_unstale":"2024-03-11T20:51:12.846369+0000","last_undegraded":"2024-03-11T20:51:12.846369+0000","last_fullsized":"2024-03-11T20:51:12.846369+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":799,"log_dups_size":0,"ondisk_log_size":799,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":111,"num_read_kb":58643,"num_write":786,"num_write_kb":122786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"30'199","reported_seq":294,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.665030+0000","last_change":"2024-03-11T18:35:57.538551+0000","last_active":"2024-03-11T20:51:29.665030+0000","last_peered":"2024-03-11T20:51:29.665030+0000","last_clean":"2024-03-11T20:51:29.665030+0000","last_became_active":"2024-03-11T18:35:57.537706+0000","last_became_peered":"2024-03-11T18:35:57.537706+0000","last_unstale":"2024-03-11T20:51:29.665030+0000","last_undegraded":"2024-03-11T20:51:29.665030+0000","last_fullsized":"2024-03-11T20:51:29.665030+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"30'985","reported_seq":1081,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:50:56.200949+0000","last_change":"2024-03-11T18:35:56.302575+0000","last_active":"2024-03-11T20:50:56.200949+0000","last_peered":"2024-03-11T20:50:56.200949+0000","last_clean":"2024-03-11T20:50:56.200949+0000","last_became_active":"2024-03-11T18:35:56.301819+0000","last_became_peered":"2024-03-11T18:35:56.301819+0000","last_unstale":"2024-03-11T20:50:56.200949+0000","last_undegraded":"2024-03-11T20:50:56.200949+0000","last_fullsized":"2024-03-11T20:50:56.200949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":985,"log_dups_size":0,"ondisk_log_size":985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":935,"num_write_kb":8676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"30'751","reported_seq":914,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.306515+0000","last_change":"2024-03-11T18:36:04.609282+0000","last_active":"2024-03-11T20:51:29.306515+0000","last_peered":"2024-03-11T20:51:29.306515+0000","last_clean":"2024-03-11T20:51:29.306515+0000","last_became_active":"2024-03-11T18:36:04.608260+0000","last_became_peered":"2024-03-11T18:36:04.608260+0000","last_unstale":"2024-03-11T20:51:29.306515+0000","last_undegraded":"2024-03-11T20:51:29.306515+0000","last_fullsized":"2024-03-11T20:51:29.306515+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":751,"log_dups_size":0,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":79,"num_read_kb":54998,"num_write":738,"num_write_kb":142673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"30'218","reported_seq":312,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.625482+0000","last_change":"2024-03-11T18:35:57.688727+0000","last_active":"2024-03-11T20:51:29.625482+0000","last_peered":"2024-03-11T20:51:29.625482+0000","last_clean":"2024-03-11T20:51:29.625482+0000","last_became_active":"2024-03-11T18:35:57.688209+0000","last_became_peered":"2024-03-11T18:35:57.688209+0000","last_unstale":"2024-03-11T20:51:29.625482+0000","last_undegraded":"2024-03-11T20:51:29.625482+0000","last_fullsized":"2024-03-11T20:51:29.625482+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"30'374","reported_seq":479,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:45.115684+0000","last_change":"2024-03-11T20:51:45.115773+0000","last_active":"2024-03-11T20:51:45.115684+0000","last_peered":"2024-03-11T20:51:45.115684+0000","last_clean":"2024-03-11T20:51:45.115684+0000","last_became_active":"2024-03-11T18:35:56.293276+0000","last_became_peered":"2024-03-11T18:35:56.293276+0000","last_unstale":"2024-03-11T20:51:45.115684+0000","last_undegraded":"2024-03-11T20:51:45.115684+0000","last_fullsized":"2024-03-11T20:51:45.115684+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'374","last_scrub_stamp":"2024-03-11T20:51:45.115571+0000","last_deep_scrub":"30'374","last_deep_scrub_stamp":"2024-03-11T20:51:45.115571+0000","last_clean_scrub_stamp":"2024-03-11T20:51:45.115571+0000","objects_scrubbed":1,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:52:29.747602+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":309,"num_write_kb":4248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"30'742","reported_seq":882,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:31.389564+0000","last_change":"2024-03-11T18:36:04.626320+0000","last_active":"2024-03-11T20:51:31.389564+0000","last_peered":"2024-03-11T20:51:31.389564+0000","last_clean":"2024-03-11T20:51:31.389564+0000","last_became_active":"2024-03-11T18:36:04.625663+0000","last_became_peered":"2024-03-11T18:36:04.625663+0000","last_unstale":"2024-03-11T20:51:31.389564+0000","last_undegraded":"2024-03-11T20:51:31.389564+0000","last_fullsized":"2024-03-11T20:51:31.389564+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":742,"log_dups_size":0,"ondisk_log_size":742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":56,"num_read_kb":20684,"num_write":725,"num_write_kb":55759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"30'199","reported_seq":302,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:46.668384+0000","last_change":"2024-03-11T20:51:46.668442+0000","last_active":"2024-03-11T20:51:46.668384+0000","last_peered":"2024-03-11T20:51:46.668384+0000","last_clean":"2024-03-11T20:51:46.668384+0000","last_became_active":"2024-03-11T18:35:57.611509+0000","last_became_peered":"2024-03-11T18:35:57.611509+0000","last_unstale":"2024-03-11T20:51:46.668384+0000","last_undegraded":"2024-03-11T20:51:46.668384+0000","last_fullsized":"2024-03-11T20:51:46.668384+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'199","last_scrub_stamp":"2024-03-11T20:51:46.668213+0000","last_deep_scrub":"30'199","last_deep_scrub_stamp":"2024-03-11T20:51:46.668213+0000","last_clean_scrub_stamp":"2024-03-11T20:51:46.668213+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:15:59.701206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"30'1190","reported_seq":1287,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:22.350716+0000","last_change":"2024-03-11T18:35:56.298766+0000","last_active":"2024-03-11T20:51:22.350716+0000","last_peered":"2024-03-11T20:51:22.350716+0000","last_clean":"2024-03-11T20:51:22.350716+0000","last_became_active":"2024-03-11T18:35:56.297747+0000","last_became_peered":"2024-03-11T18:35:56.297747+0000","last_unstale":"2024-03-11T20:51:22.350716+0000","last_undegraded":"2024-03-11T20:51:22.350716+0000","last_fullsized":"2024-03-11T20:51:22.350716+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1190,"log_dups_size":0,"ondisk_log_size":1190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389196,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1123,"num_write_kb":8732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"30'749","reported_seq":964,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:31.389642+0000","last_change":"2024-03-11T18:36:04.635773+0000","last_active":"2024-03-11T20:51:31.389642+0000","last_peered":"2024-03-11T20:51:31.389642+0000","last_clean":"2024-03-11T20:51:31.389642+0000","last_became_active":"2024-03-11T18:36:04.635261+0000","last_became_peered":"2024-03-11T18:36:04.635261+0000","last_unstale":"2024-03-11T20:51:31.389642+0000","last_undegraded":"2024-03-11T20:51:31.389642+0000","last_fullsized":"2024-03-11T20:51:31.389642+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":749,"log_dups_size":0,"ondisk_log_size":749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":131,"num_read_kb":65646,"num_write":733,"num_write_kb":179820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"30'211","reported_seq":308,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.668391+0000","last_change":"2024-03-11T18:35:57.538552+0000","last_active":"2024-03-11T20:51:29.668391+0000","last_peered":"2024-03-11T20:51:29.668391+0000","last_clean":"2024-03-11T20:51:29.668391+0000","last_became_active":"2024-03-11T18:35:57.537717+0000","last_became_peered":"2024-03-11T18:35:57.537717+0000","last_unstale":"2024-03-11T20:51:29.668391+0000","last_undegraded":"2024-03-11T20:51:29.668391+0000","last_fullsized":"2024-03-11T20:51:29.668391+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"30'4764","reported_seq":9566,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:44.821888+0000","last_change":"2024-03-11T20:51:44.821936+0000","last_active":"2024-03-11T20:51:44.821888+0000","last_peered":"2024-03-11T20:51:44.821888+0000","last_clean":"2024-03-11T20:51:44.821888+0000","last_became_active":"2024-03-11T18:35:56.294096+0000","last_became_peered":"2024-03-11T18:35:56.294096+0000","last_unstale":"2024-03-11T20:51:44.821888+0000","last_undegraded":"2024-03-11T20:51:44.821888+0000","last_fullsized":"2024-03-11T20:51:44.821888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'4764","last_scrub_stamp":"2024-03-11T20:51:44.821783+0000","last_deep_scrub":"30'4764","last_deep_scrub_stamp":"2024-03-11T20:51:44.821783+0000","last_clean_scrub_stamp":"2024-03-11T20:51:44.821783+0000","objects_scrubbed":4,"log_size":4764,"log_dups_size":0,"ondisk_log_size":4764,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:27:59.083928+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1555811,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4697,"num_read_kb":5145,"num_write":4704,"num_write_kb":5149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"30'682","reported_seq":803,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:08.256738+0000","last_change":"2024-03-11T18:36:04.620762+0000","last_active":"2024-03-11T20:51:08.256738+0000","last_peered":"2024-03-11T20:51:08.256738+0000","last_clean":"2024-03-11T20:51:08.256738+0000","last_became_active":"2024-03-11T18:36:04.620356+0000","last_became_peered":"2024-03-11T18:36:04.620356+0000","last_unstale":"2024-03-11T20:51:08.256738+0000","last_undegraded":"2024-03-11T20:51:08.256738+0000","last_fullsized":"2024-03-11T20:51:08.256738+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":682,"log_dups_size":0,"ondisk_log_size":682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":17597,"num_write":672,"num_write_kb":47486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"30'198","reported_seq":293,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.921114+0000","last_change":"2024-03-11T18:35:57.602773+0000","last_active":"2024-03-11T20:51:29.921114+0000","last_peered":"2024-03-11T20:51:29.921114+0000","last_clean":"2024-03-11T20:51:29.921114+0000","last_became_active":"2024-03-11T18:35:57.602060+0000","last_became_peered":"2024-03-11T18:35:57.602060+0000","last_unstale":"2024-03-11T20:51:29.921114+0000","last_undegraded":"2024-03-11T20:51:29.921114+0000","last_fullsized":"2024-03-11T20:51:29.921114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"30'1021","reported_seq":1120,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:50:40.135559+0000","last_change":"2024-03-11T18:35:56.305283+0000","last_active":"2024-03-11T20:50:40.135559+0000","last_peered":"2024-03-11T20:50:40.135559+0000","last_clean":"2024-03-11T20:50:40.135559+0000","last_became_active":"2024-03-11T18:35:56.303063+0000","last_became_peered":"2024-03-11T18:35:56.303063+0000","last_unstale":"2024-03-11T20:50:40.135559+0000","last_undegraded":"2024-03-11T20:50:40.135559+0000","last_fullsized":"2024-03-11T20:50:40.135559+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1021,"log_dups_size":0,"ondisk_log_size":1021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":963,"num_write_kb":12732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"30'852","reported_seq":1018,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:15.384134+0000","last_change":"2024-03-11T18:36:04.617247+0000","last_active":"2024-03-11T20:51:15.384134+0000","last_peered":"2024-03-11T20:51:15.384134+0000","last_clean":"2024-03-11T20:51:15.384134+0000","last_became_active":"2024-03-11T18:36:04.610765+0000","last_became_peered":"2024-03-11T18:36:04.610765+0000","last_unstale":"2024-03-11T20:51:15.384134+0000","last_undegraded":"2024-03-11T20:51:15.384134+0000","last_fullsized":"2024-03-11T20:51:15.384134+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":85,"num_read_kb":37203,"num_write":843,"num_write_kb":103063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"30'185","reported_seq":280,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.764433+0000","last_change":"2024-03-11T18:35:57.684207+0000","last_active":"2024-03-11T20:51:29.764433+0000","last_peered":"2024-03-11T20:51:29.764433+0000","last_clean":"2024-03-11T20:51:29.764433+0000","last_became_active":"2024-03-11T18:35:57.682653+0000","last_became_peered":"2024-03-11T18:35:57.682653+0000","last_unstale":"2024-03-11T20:51:29.764433+0000","last_undegraded":"2024-03-11T20:51:29.764433+0000","last_fullsized":"2024-03-11T20:51:29.764433+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":185,"log_dups_size":0,"ondisk_log_size":185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"30'371","reported_seq":468,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:22.350743+0000","last_change":"2024-03-11T18:35:56.297078+0000","last_active":"2024-03-11T20:51:22.350743+0000","last_peered":"2024-03-11T20:51:22.350743+0000","last_clean":"2024-03-11T20:51:22.350743+0000","last_became_active":"2024-03-11T18:35:56.296705+0000","last_became_peered":"2024-03-11T18:35:56.296705+0000","last_unstale":"2024-03-11T20:51:22.350743+0000","last_undegraded":"2024-03-11T20:51:22.350743+0000","last_fullsized":"2024-03-11T20:51:22.350743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194372,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":305,"num_write_kb":4243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":254,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:49:47.414021+0000","last_change":"2024-03-11T18:35:18.199528+0000","last_active":"2024-03-11T20:49:47.414021+0000","last_peered":"2024-03-11T20:49:47.414021+0000","last_clean":"2024-03-11T20:49:47.414021+0000","last_became_active":"2024-03-11T18:35:18.197797+0000","last_became_peered":"2024-03-11T18:35:18.197797+0000","last_unstale":"2024-03-11T20:49:47.414021+0000","last_undegraded":"2024-03-11T20:49:47.414021+0000","last_fullsized":"2024-03-11T20:49:47.414021+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:16.207500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:16.207500+0000","last_clean_scrub_stamp":"2024-03-11T18:35:16.207500+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"30'785","reported_seq":954,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:14.552949+0000","last_change":"2024-03-11T18:36:04.599815+0000","last_active":"2024-03-11T20:51:14.552949+0000","last_peered":"2024-03-11T20:51:14.552949+0000","last_clean":"2024-03-11T20:51:14.552949+0000","last_became_active":"2024-03-11T18:36:04.598777+0000","last_became_peered":"2024-03-11T18:36:04.598777+0000","last_unstale":"2024-03-11T20:51:14.552949+0000","last_undegraded":"2024-03-11T20:51:14.552949+0000","last_fullsized":"2024-03-11T20:51:14.552949+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":785,"log_dups_size":0,"ondisk_log_size":785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":86,"num_read_kb":36797,"num_write":762,"num_write_kb":114420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"30'207","reported_seq":277,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.794571+0000","last_change":"2024-03-11T18:37:04.141957+0000","last_active":"2024-03-11T20:51:29.794571+0000","last_peered":"2024-03-11T20:51:29.794571+0000","last_clean":"2024-03-11T20:51:29.794571+0000","last_became_active":"2024-03-11T18:37:04.141663+0000","last_became_peered":"2024-03-11T18:37:04.141663+0000","last_unstale":"2024-03-11T20:51:29.794571+0000","last_undegraded":"2024-03-11T20:51:29.794571+0000","last_fullsized":"2024-03-11T20:51:29.794571+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"30'289","reported_seq":389,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:45.834207+0000","last_change":"2024-03-11T20:51:45.834271+0000","last_active":"2024-03-11T20:51:45.834207+0000","last_peered":"2024-03-11T20:51:45.834207+0000","last_clean":"2024-03-11T20:51:45.834207+0000","last_became_active":"2024-03-11T18:35:56.293961+0000","last_became_peered":"2024-03-11T18:35:56.293961+0000","last_unstale":"2024-03-11T20:51:45.834207+0000","last_undegraded":"2024-03-11T20:51:45.834207+0000","last_fullsized":"2024-03-11T20:51:45.834207+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'289","last_scrub_stamp":"2024-03-11T20:51:45.834055+0000","last_deep_scrub":"30'289","last_deep_scrub_stamp":"2024-03-11T20:51:45.834055+0000","last_clean_scrub_stamp":"2024-03-11T20:51:45.834055+0000","objects_scrubbed":4,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:08:44.795933+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4196056,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":223,"num_write_kb":4226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"30'822","reported_seq":988,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:15.317555+0000","last_change":"2024-03-11T18:36:04.619862+0000","last_active":"2024-03-11T20:51:15.317555+0000","last_peered":"2024-03-11T20:51:15.317555+0000","last_clean":"2024-03-11T20:51:15.317555+0000","last_became_active":"2024-03-11T18:36:04.618908+0000","last_became_peered":"2024-03-11T18:36:04.618908+0000","last_unstale":"2024-03-11T20:51:15.317555+0000","last_undegraded":"2024-03-11T20:51:15.317555+0000","last_fullsized":"2024-03-11T20:51:15.317555+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":822,"log_dups_size":0,"ondisk_log_size":822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":85,"num_read_kb":42087,"num_write":807,"num_write_kb":152305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"30'221","reported_seq":311,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.616287+0000","last_change":"2024-03-11T18:35:57.686416+0000","last_active":"2024-03-11T20:51:29.616287+0000","last_peered":"2024-03-11T20:51:29.616287+0000","last_clean":"2024-03-11T20:51:29.616287+0000","last_became_active":"2024-03-11T18:35:57.685828+0000","last_became_peered":"2024-03-11T18:35:57.685828+0000","last_unstale":"2024-03-11T20:51:29.616287+0000","last_undegraded":"2024-03-11T20:51:29.616287+0000","last_fullsized":"2024-03-11T20:51:29.616287+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"30'408","reported_seq":508,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:46.107531+0000","last_change":"2024-03-11T20:51:46.107582+0000","last_active":"2024-03-11T20:51:46.107531+0000","last_peered":"2024-03-11T20:51:46.107531+0000","last_clean":"2024-03-11T20:51:46.107531+0000","last_became_active":"2024-03-11T18:35:56.295123+0000","last_became_peered":"2024-03-11T18:35:56.295123+0000","last_unstale":"2024-03-11T20:51:46.107531+0000","last_undegraded":"2024-03-11T20:51:46.107531+0000","last_fullsized":"2024-03-11T20:51:46.107531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'408","last_scrub_stamp":"2024-03-11T20:51:46.107427+0000","last_deep_scrub":"30'408","last_deep_scrub_stamp":"2024-03-11T20:51:46.107427+0000","last_clean_scrub_stamp":"2024-03-11T20:51:46.107427+0000","objects_scrubbed":2,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:29:26.738334+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":350,"num_write_kb":4273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"30'821","reported_seq":1022,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:19.406934+0000","last_change":"2024-03-11T18:36:04.616750+0000","last_active":"2024-03-11T20:51:19.406934+0000","last_peered":"2024-03-11T20:51:19.406934+0000","last_clean":"2024-03-11T20:51:19.406934+0000","last_became_active":"2024-03-11T18:36:04.616338+0000","last_became_peered":"2024-03-11T18:36:04.616338+0000","last_unstale":"2024-03-11T20:51:19.406934+0000","last_undegraded":"2024-03-11T20:51:19.406934+0000","last_fullsized":"2024-03-11T20:51:19.406934+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":821,"log_dups_size":0,"ondisk_log_size":821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":122,"num_read_kb":70346,"num_write":827,"num_write_kb":188969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"30'199","reported_seq":289,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:32.280893+0000","last_change":"2024-03-11T18:35:57.689888+0000","last_active":"2024-03-11T20:51:32.280893+0000","last_peered":"2024-03-11T20:51:32.280893+0000","last_clean":"2024-03-11T20:51:32.280893+0000","last_became_active":"2024-03-11T18:35:57.689424+0000","last_became_peered":"2024-03-11T18:35:57.689424+0000","last_unstale":"2024-03-11T20:51:32.280893+0000","last_undegraded":"2024-03-11T20:51:32.280893+0000","last_fullsized":"2024-03-11T20:51:32.280893+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"30'864","reported_seq":964,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:43.780270+0000","last_change":"2024-03-11T20:51:43.780331+0000","last_active":"2024-03-11T20:51:43.780270+0000","last_peered":"2024-03-11T20:51:43.780270+0000","last_clean":"2024-03-11T20:51:43.780270+0000","last_became_active":"2024-03-11T18:35:56.293585+0000","last_became_peered":"2024-03-11T18:35:56.293585+0000","last_unstale":"2024-03-11T20:51:43.780270+0000","last_undegraded":"2024-03-11T20:51:43.780270+0000","last_fullsized":"2024-03-11T20:51:43.780270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'864","last_scrub_stamp":"2024-03-11T20:51:43.780155+0000","last_deep_scrub":"30'864","last_deep_scrub_stamp":"2024-03-11T20:51:43.780155+0000","last_clean_scrub_stamp":"2024-03-11T20:51:43.780155+0000","objects_scrubbed":3,"log_size":864,"log_dups_size":0,"ondisk_log_size":864,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:45:57.220940+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":804,"num_write_kb":4496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"30'804","reported_seq":983,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:17.714578+0000","last_change":"2024-03-11T18:36:04.627729+0000","last_active":"2024-03-11T20:51:17.714578+0000","last_peered":"2024-03-11T20:51:17.714578+0000","last_clean":"2024-03-11T20:51:17.714578+0000","last_became_active":"2024-03-11T18:36:04.626596+0000","last_became_peered":"2024-03-11T18:36:04.626596+0000","last_unstale":"2024-03-11T20:51:17.714578+0000","last_undegraded":"2024-03-11T20:51:17.714578+0000","last_fullsized":"2024-03-11T20:51:17.714578+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":98,"num_read_kb":46229,"num_write":782,"num_write_kb":114631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"30'184","reported_seq":274,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.631590+0000","last_change":"2024-03-11T18:35:57.690506+0000","last_active":"2024-03-11T20:51:29.631590+0000","last_peered":"2024-03-11T20:51:29.631590+0000","last_clean":"2024-03-11T20:51:29.631590+0000","last_became_active":"2024-03-11T18:35:57.690223+0000","last_became_peered":"2024-03-11T18:35:57.690223+0000","last_unstale":"2024-03-11T20:51:29.631590+0000","last_undegraded":"2024-03-11T20:51:29.631590+0000","last_fullsized":"2024-03-11T20:51:29.631590+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"30'818","reported_seq":918,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:44.730724+0000","last_change":"2024-03-11T20:51:44.730811+0000","last_active":"2024-03-11T20:51:44.730724+0000","last_peered":"2024-03-11T20:51:44.730724+0000","last_clean":"2024-03-11T20:51:44.730724+0000","last_became_active":"2024-03-11T18:35:56.256454+0000","last_became_peered":"2024-03-11T18:35:56.256454+0000","last_unstale":"2024-03-11T20:51:44.730724+0000","last_undegraded":"2024-03-11T20:51:44.730724+0000","last_fullsized":"2024-03-11T20:51:44.730724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'818","last_scrub_stamp":"2024-03-11T20:51:44.730537+0000","last_deep_scrub":"30'818","last_deep_scrub_stamp":"2024-03-11T20:51:44.730537+0000","last_clean_scrub_stamp":"2024-03-11T20:51:44.730537+0000","objects_scrubbed":3,"log_size":818,"log_dups_size":0,"ondisk_log_size":818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:30:31.543385+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":757,"num_write_kb":4472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"30'833","reported_seq":1021,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:12.677085+0000","last_change":"2024-03-11T18:36:04.644184+0000","last_active":"2024-03-11T20:51:12.677085+0000","last_peered":"2024-03-11T20:51:12.677085+0000","last_clean":"2024-03-11T20:51:12.677085+0000","last_became_active":"2024-03-11T18:36:04.642079+0000","last_became_peered":"2024-03-11T18:36:04.642079+0000","last_unstale":"2024-03-11T20:51:12.677085+0000","last_undegraded":"2024-03-11T20:51:12.677085+0000","last_fullsized":"2024-03-11T20:51:12.677085+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":833,"log_dups_size":0,"ondisk_log_size":833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":107,"num_read_kb":63565,"num_write":823,"num_write_kb":172061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"30'239","reported_seq":327,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.637681+0000","last_change":"2024-03-11T18:35:57.687311+0000","last_active":"2024-03-11T20:51:29.637681+0000","last_peered":"2024-03-11T20:51:29.637681+0000","last_clean":"2024-03-11T20:51:29.637681+0000","last_became_active":"2024-03-11T18:35:57.686850+0000","last_became_peered":"2024-03-11T18:35:57.686850+0000","last_unstale":"2024-03-11T20:51:29.637681+0000","last_undegraded":"2024-03-11T20:51:29.637681+0000","last_fullsized":"2024-03-11T20:51:29.637681+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"30'909","reported_seq":1007,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:42.786561+0000","last_change":"2024-03-11T20:51:42.786622+0000","last_active":"2024-03-11T20:51:42.786561+0000","last_peered":"2024-03-11T20:51:42.786561+0000","last_clean":"2024-03-11T20:51:42.786561+0000","last_became_active":"2024-03-11T18:35:56.290944+0000","last_became_peered":"2024-03-11T18:35:56.290944+0000","last_unstale":"2024-03-11T20:51:42.786561+0000","last_undegraded":"2024-03-11T20:51:42.786561+0000","last_fullsized":"2024-03-11T20:51:42.786561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'909","last_scrub_stamp":"2024-03-11T20:51:42.786457+0000","last_deep_scrub":"30'909","last_deep_scrub_stamp":"2024-03-11T20:51:42.786457+0000","last_clean_scrub_stamp":"2024-03-11T20:51:42.786457+0000","objects_scrubbed":10,"log_size":909,"log_dups_size":0,"ondisk_log_size":909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:32:37.677873+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":846,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"30'791","reported_seq":969,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:24.305717+0000","last_change":"2024-03-11T18:36:04.602142+0000","last_active":"2024-03-11T20:51:24.305717+0000","last_peered":"2024-03-11T20:51:24.305717+0000","last_clean":"2024-03-11T20:51:24.305717+0000","last_became_active":"2024-03-11T18:36:04.601224+0000","last_became_peered":"2024-03-11T18:36:04.601224+0000","last_unstale":"2024-03-11T20:51:24.305717+0000","last_undegraded":"2024-03-11T20:51:24.305717+0000","last_fullsized":"2024-03-11T20:51:24.305717+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":59304,"num_write":781,"num_write_kb":152027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"30'219","reported_seq":307,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.943407+0000","last_change":"2024-03-11T18:35:57.611305+0000","last_active":"2024-03-11T20:51:29.943407+0000","last_peered":"2024-03-11T20:51:29.943407+0000","last_clean":"2024-03-11T20:51:29.943407+0000","last_became_active":"2024-03-11T18:35:57.610955+0000","last_became_peered":"2024-03-11T18:35:57.610955+0000","last_unstale":"2024-03-11T20:51:29.943407+0000","last_undegraded":"2024-03-11T20:51:29.943407+0000","last_fullsized":"2024-03-11T20:51:29.943407+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"30'60","reported_seq":150,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.290394+0000","last_change":"2024-03-11T18:35:56.361952+0000","last_active":"2024-03-11T20:51:29.290394+0000","last_peered":"2024-03-11T20:51:29.290394+0000","last_clean":"2024-03-11T20:51:29.290394+0000","last_became_active":"2024-03-11T18:35:56.361141+0000","last_became_peered":"2024-03-11T18:35:56.361141+0000","last_unstale":"2024-03-11T20:51:29.290394+0000","last_undegraded":"2024-03-11T20:51:29.290394+0000","last_fullsized":"2024-03-11T20:51:29.290394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"30'743","reported_seq":895,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:26.333213+0000","last_change":"2024-03-11T18:36:04.593604+0000","last_active":"2024-03-11T20:51:26.333213+0000","last_peered":"2024-03-11T20:51:26.333213+0000","last_clean":"2024-03-11T20:51:26.333213+0000","last_became_active":"2024-03-11T18:36:04.592304+0000","last_became_peered":"2024-03-11T18:36:04.592304+0000","last_unstale":"2024-03-11T20:51:26.333213+0000","last_undegraded":"2024-03-11T20:51:26.333213+0000","last_fullsized":"2024-03-11T20:51:26.333213+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":743,"log_dups_size":0,"ondisk_log_size":743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":77,"num_read_kb":52017,"num_write":733,"num_write_kb":82372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"30'206","reported_seq":294,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.638619+0000","last_change":"2024-03-11T18:35:57.689404+0000","last_active":"2024-03-11T20:51:29.638619+0000","last_peered":"2024-03-11T20:51:29.638619+0000","last_clean":"2024-03-11T20:51:29.638619+0000","last_became_active":"2024-03-11T18:35:57.689139+0000","last_became_peered":"2024-03-11T18:35:57.689139+0000","last_unstale":"2024-03-11T20:51:29.638619+0000","last_undegraded":"2024-03-11T20:51:29.638619+0000","last_fullsized":"2024-03-11T20:51:29.638619+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"30'60","reported_seq":150,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:21.430445+0000","last_change":"2024-03-11T18:35:56.297682+0000","last_active":"2024-03-11T20:51:21.430445+0000","last_peered":"2024-03-11T20:51:21.430445+0000","last_clean":"2024-03-11T20:51:21.430445+0000","last_became_active":"2024-03-11T18:35:56.296426+0000","last_became_peered":"2024-03-11T18:35:56.296426+0000","last_unstale":"2024-03-11T20:51:21.430445+0000","last_undegraded":"2024-03-11T20:51:21.430445+0000","last_fullsized":"2024-03-11T20:51:21.430445+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"30'724","reported_seq":924,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:14.528411+0000","last_change":"2024-03-11T18:36:04.590780+0000","last_active":"2024-03-11T20:51:14.528411+0000","last_peered":"2024-03-11T20:51:14.528411+0000","last_clean":"2024-03-11T20:51:14.528411+0000","last_became_active":"2024-03-11T18:36:04.590014+0000","last_became_peered":"2024-03-11T18:36:04.590014+0000","last_unstale":"2024-03-11T20:51:14.528411+0000","last_undegraded":"2024-03-11T20:51:14.528411+0000","last_fullsized":"2024-03-11T20:51:14.528411+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":724,"log_dups_size":0,"ondisk_log_size":724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":124,"num_read_kb":69166,"num_write":714,"num_write_kb":127024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"30'208","reported_seq":304,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:40.519149+0000","last_change":"2024-03-11T20:51:40.519194+0000","last_active":"2024-03-11T20:51:40.519149+0000","last_peered":"2024-03-11T20:51:40.519149+0000","last_clean":"2024-03-11T20:51:40.519149+0000","last_became_active":"2024-03-11T18:35:57.670893+0000","last_became_peered":"2024-03-11T18:35:57.670893+0000","last_unstale":"2024-03-11T20:51:40.519149+0000","last_undegraded":"2024-03-11T20:51:40.519149+0000","last_fullsized":"2024-03-11T20:51:40.519149+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'208","last_scrub_stamp":"2024-03-11T20:51:40.519026+0000","last_deep_scrub":"30'208","last_deep_scrub_stamp":"2024-03-11T20:51:40.519026+0000","last_clean_scrub_stamp":"2024-03-11T20:51:40.519026+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:28:40.054929+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"30'600","reported_seq":698,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:41.519078+0000","last_change":"2024-03-11T20:51:41.519148+0000","last_active":"2024-03-11T20:51:41.519078+0000","last_peered":"2024-03-11T20:51:41.519078+0000","last_clean":"2024-03-11T20:51:41.519078+0000","last_became_active":"2024-03-11T18:35:56.251287+0000","last_became_peered":"2024-03-11T18:35:56.251287+0000","last_unstale":"2024-03-11T20:51:41.519078+0000","last_undegraded":"2024-03-11T20:51:41.519078+0000","last_fullsized":"2024-03-11T20:51:41.519078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'600","last_scrub_stamp":"2024-03-11T20:51:41.518947+0000","last_deep_scrub":"30'600","last_deep_scrub_stamp":"2024-03-11T20:51:41.518947+0000","last_clean_scrub_stamp":"2024-03-11T20:51:41.518947+0000","objects_scrubbed":8,"log_size":600,"log_dups_size":0,"ondisk_log_size":600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:41:08.769039+0000","scrub_duration":36,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388676,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":533,"num_write_kb":8445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"30'738","reported_seq":847,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:27.351634+0000","last_change":"2024-03-11T18:36:04.625563+0000","last_active":"2024-03-11T20:51:27.351634+0000","last_peered":"2024-03-11T20:51:27.351634+0000","last_clean":"2024-03-11T20:51:27.351634+0000","last_became_active":"2024-03-11T18:36:04.618911+0000","last_became_peered":"2024-03-11T18:36:04.618911+0000","last_unstale":"2024-03-11T20:51:27.351634+0000","last_undegraded":"2024-03-11T20:51:27.351634+0000","last_fullsized":"2024-03-11T20:51:27.351634+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":738,"log_dups_size":0,"ondisk_log_size":738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":10131,"num_write":722,"num_write_kb":109498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"30'204","reported_seq":290,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.620206+0000","last_change":"2024-03-11T18:35:57.697641+0000","last_active":"2024-03-11T20:51:29.620206+0000","last_peered":"2024-03-11T20:51:29.620206+0000","last_clean":"2024-03-11T20:51:29.620206+0000","last_became_active":"2024-03-11T18:35:57.697263+0000","last_became_peered":"2024-03-11T18:35:57.697263+0000","last_unstale":"2024-03-11T20:51:29.620206+0000","last_undegraded":"2024-03-11T20:51:29.620206+0000","last_fullsized":"2024-03-11T20:51:29.620206+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"30'2209","reported_seq":2306,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:46.633128+0000","last_change":"2024-03-11T20:51:46.633274+0000","last_active":"2024-03-11T20:51:46.633128+0000","last_peered":"2024-03-11T20:51:46.633128+0000","last_clean":"2024-03-11T20:51:46.633128+0000","last_became_active":"2024-03-11T18:35:56.354827+0000","last_became_peered":"2024-03-11T18:35:56.354827+0000","last_unstale":"2024-03-11T20:51:46.633128+0000","last_undegraded":"2024-03-11T20:51:46.633128+0000","last_fullsized":"2024-03-11T20:51:46.633128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'2209","last_scrub_stamp":"2024-03-11T20:51:46.632881+0000","last_deep_scrub":"30'2209","last_deep_scrub_stamp":"2024-03-11T20:51:46.632881+0000","last_clean_scrub_stamp":"2024-03-11T20:51:46.632881+0000","objects_scrubbed":8,"log_size":2209,"log_dups_size":0,"ondisk_log_size":2209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:28:00.681266+0000","scrub_duration":59,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583584,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2157,"num_write_kb":13630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"30'771","reported_seq":925,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:15.400678+0000","last_change":"2024-03-11T18:37:04.259892+0000","last_active":"2024-03-11T20:51:15.400678+0000","last_peered":"2024-03-11T20:51:15.400678+0000","last_clean":"2024-03-11T20:51:15.400678+0000","last_became_active":"2024-03-11T18:37:04.259051+0000","last_became_peered":"2024-03-11T18:37:04.259051+0000","last_unstale":"2024-03-11T20:51:15.400678+0000","last_undegraded":"2024-03-11T20:51:15.400678+0000","last_fullsized":"2024-03-11T20:51:15.400678+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":771,"log_dups_size":0,"ondisk_log_size":771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":84,"num_read_kb":37526,"num_write":755,"num_write_kb":85949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2,0],"acting":[5,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"30'201","reported_seq":288,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.591071+0000","last_change":"2024-03-11T18:35:57.679035+0000","last_active":"2024-03-11T20:51:29.591071+0000","last_peered":"2024-03-11T20:51:29.591071+0000","last_clean":"2024-03-11T20:51:29.591071+0000","last_became_active":"2024-03-11T18:35:57.678565+0000","last_became_peered":"2024-03-11T18:35:57.678565+0000","last_unstale":"2024-03-11T20:51:29.591071+0000","last_undegraded":"2024-03-11T20:51:29.591071+0000","last_fullsized":"2024-03-11T20:51:29.591071+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"30'3441","reported_seq":3537,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:42.219110+0000","last_change":"2024-03-11T20:51:42.219190+0000","last_active":"2024-03-11T20:51:42.219110+0000","last_peered":"2024-03-11T20:51:42.219110+0000","last_clean":"2024-03-11T20:51:42.219110+0000","last_became_active":"2024-03-11T18:35:56.287846+0000","last_became_peered":"2024-03-11T18:35:56.287846+0000","last_unstale":"2024-03-11T20:51:42.219110+0000","last_undegraded":"2024-03-11T20:51:42.219110+0000","last_fullsized":"2024-03-11T20:51:42.219110+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'3441","last_scrub_stamp":"2024-03-11T20:51:42.218945+0000","last_deep_scrub":"30'3441","last_deep_scrub_stamp":"2024-03-11T20:51:42.218945+0000","last_clean_scrub_stamp":"2024-03-11T20:51:42.218945+0000","objects_scrubbed":6,"log_size":3441,"log_dups_size":0,"ondisk_log_size":3441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:29:44.432977+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7567240,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3391,"num_write_kb":10463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"30'736","reported_seq":903,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:27.367857+0000","last_change":"2024-03-11T18:36:04.643164+0000","last_active":"2024-03-11T20:51:27.367857+0000","last_peered":"2024-03-11T20:51:27.367857+0000","last_clean":"2024-03-11T20:51:27.367857+0000","last_became_active":"2024-03-11T18:36:04.642374+0000","last_became_peered":"2024-03-11T18:36:04.642374+0000","last_unstale":"2024-03-11T20:51:27.367857+0000","last_undegraded":"2024-03-11T20:51:27.367857+0000","last_fullsized":"2024-03-11T20:51:27.367857+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":736,"log_dups_size":0,"ondisk_log_size":736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":40504,"num_write":721,"num_write_kb":107102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"30'197","reported_seq":283,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.866646+0000","last_change":"2024-03-11T18:35:57.636550+0000","last_active":"2024-03-11T20:51:29.866646+0000","last_peered":"2024-03-11T20:51:29.866646+0000","last_clean":"2024-03-11T20:51:29.866646+0000","last_became_active":"2024-03-11T18:35:57.636093+0000","last_became_peered":"2024-03-11T18:35:57.636093+0000","last_unstale":"2024-03-11T20:51:29.866646+0000","last_undegraded":"2024-03-11T20:51:29.866646+0000","last_fullsized":"2024-03-11T20:51:29.866646+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"30'1238","reported_seq":1326,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:17.349818+0000","last_change":"2024-03-11T18:35:56.355480+0000","last_active":"2024-03-11T20:51:17.349818+0000","last_peered":"2024-03-11T20:51:17.349818+0000","last_clean":"2024-03-11T20:51:17.349818+0000","last_became_active":"2024-03-11T18:35:56.354759+0000","last_became_peered":"2024-03-11T18:35:56.354759+0000","last_unstale":"2024-03-11T20:51:17.349818+0000","last_undegraded":"2024-03-11T20:51:17.349818+0000","last_fullsized":"2024-03-11T20:51:17.349818+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1238,"log_dups_size":0,"ondisk_log_size":1238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1180,"num_write_kb":8784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"30'803","reported_seq":940,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:26.333003+0000","last_change":"2024-03-11T18:36:04.617364+0000","last_active":"2024-03-11T20:51:26.333003+0000","last_peered":"2024-03-11T20:51:26.333003+0000","last_clean":"2024-03-11T20:51:26.333003+0000","last_became_active":"2024-03-11T18:36:04.613159+0000","last_became_peered":"2024-03-11T18:36:04.613159+0000","last_unstale":"2024-03-11T20:51:26.333003+0000","last_undegraded":"2024-03-11T20:51:26.333003+0000","last_fullsized":"2024-03-11T20:51:26.333003+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":803,"log_dups_size":0,"ondisk_log_size":803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":38200,"num_write":794,"num_write_kb":117163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"30'218","reported_seq":304,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.626758+0000","last_change":"2024-03-11T18:35:57.683426+0000","last_active":"2024-03-11T20:51:29.626758+0000","last_peered":"2024-03-11T20:51:29.626758+0000","last_clean":"2024-03-11T20:51:29.626758+0000","last_became_active":"2024-03-11T18:35:57.683150+0000","last_became_peered":"2024-03-11T18:35:57.683150+0000","last_unstale":"2024-03-11T20:51:29.626758+0000","last_undegraded":"2024-03-11T20:51:29.626758+0000","last_fullsized":"2024-03-11T20:51:29.626758+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"30'63","reported_seq":151,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.301631+0000","last_change":"2024-03-11T18:35:56.361613+0000","last_active":"2024-03-11T20:51:29.301631+0000","last_peered":"2024-03-11T20:51:29.301631+0000","last_clean":"2024-03-11T20:51:29.301631+0000","last_became_active":"2024-03-11T18:35:56.359629+0000","last_became_peered":"2024-03-11T18:35:56.359629+0000","last_unstale":"2024-03-11T20:51:29.301631+0000","last_undegraded":"2024-03-11T20:51:29.301631+0000","last_fullsized":"2024-03-11T20:51:29.301631+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"30'836","reported_seq":1037,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:26.333080+0000","last_change":"2024-03-11T18:36:04.608168+0000","last_active":"2024-03-11T20:51:26.333080+0000","last_peered":"2024-03-11T20:51:26.333080+0000","last_clean":"2024-03-11T20:51:26.333080+0000","last_became_active":"2024-03-11T18:36:04.607063+0000","last_became_peered":"2024-03-11T18:36:04.607063+0000","last_unstale":"2024-03-11T20:51:26.333080+0000","last_undegraded":"2024-03-11T20:51:26.333080+0000","last_fullsized":"2024-03-11T20:51:26.333080+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":836,"log_dups_size":0,"ondisk_log_size":836,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":124,"num_read_kb":69905,"num_write":818,"num_write_kb":155514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"30'815","reported_seq":1017,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:17.572410+0000","last_change":"2024-03-11T18:36:04.607992+0000","last_active":"2024-03-11T20:51:17.572410+0000","last_peered":"2024-03-11T20:51:17.572410+0000","last_clean":"2024-03-11T20:51:17.572410+0000","last_became_active":"2024-03-11T18:36:04.607109+0000","last_became_peered":"2024-03-11T18:36:04.607109+0000","last_unstale":"2024-03-11T20:51:17.572410+0000","last_undegraded":"2024-03-11T20:51:17.572410+0000","last_fullsized":"2024-03-11T20:51:17.572410+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":815,"log_dups_size":0,"ondisk_log_size":815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":116,"num_read_kb":66679,"num_write":801,"num_write_kb":181203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"30'218","reported_seq":303,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.672325+0000","last_change":"2024-03-11T18:35:57.680534+0000","last_active":"2024-03-11T20:51:29.672325+0000","last_peered":"2024-03-11T20:51:29.672325+0000","last_clean":"2024-03-11T20:51:29.672325+0000","last_became_active":"2024-03-11T18:35:57.679642+0000","last_became_peered":"2024-03-11T18:35:57.679642+0000","last_unstale":"2024-03-11T20:51:29.672325+0000","last_undegraded":"2024-03-11T20:51:29.672325+0000","last_fullsized":"2024-03-11T20:51:29.672325+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"30'1317","reported_seq":1405,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:19.417741+0000","last_change":"2024-03-11T18:35:56.303510+0000","last_active":"2024-03-11T20:51:19.417741+0000","last_peered":"2024-03-11T20:51:19.417741+0000","last_clean":"2024-03-11T20:51:19.417741+0000","last_became_active":"2024-03-11T18:35:56.302758+0000","last_became_peered":"2024-03-11T18:35:56.302758+0000","last_unstale":"2024-03-11T20:51:19.417741+0000","last_undegraded":"2024-03-11T20:51:19.417741+0000","last_fullsized":"2024-03-11T20:51:19.417741+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1317,"log_dups_size":0,"ondisk_log_size":1317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583494,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1254,"num_write_kb":12926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"30'766","reported_seq":982,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:32.317070+0000","last_change":"2024-03-11T18:36:04.618818+0000","last_active":"2024-03-11T20:51:32.317070+0000","last_peered":"2024-03-11T20:51:32.317070+0000","last_clean":"2024-03-11T20:51:32.317070+0000","last_became_active":"2024-03-11T18:36:04.617868+0000","last_became_peered":"2024-03-11T18:36:04.617868+0000","last_unstale":"2024-03-11T20:51:32.317070+0000","last_undegraded":"2024-03-11T20:51:32.317070+0000","last_fullsized":"2024-03-11T20:51:32.317070+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":766,"log_dups_size":0,"ondisk_log_size":766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":132,"num_read_kb":68723,"num_write":745,"num_write_kb":131559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"30'225","reported_seq":319,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:46.057566+0000","last_change":"2024-03-11T20:51:46.057622+0000","last_active":"2024-03-11T20:51:46.057566+0000","last_peered":"2024-03-11T20:51:46.057566+0000","last_clean":"2024-03-11T20:51:46.057566+0000","last_became_active":"2024-03-11T18:35:57.689146+0000","last_became_peered":"2024-03-11T18:35:57.689146+0000","last_unstale":"2024-03-11T20:51:46.057566+0000","last_undegraded":"2024-03-11T20:51:46.057566+0000","last_fullsized":"2024-03-11T20:51:46.057566+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'225","last_scrub_stamp":"2024-03-11T20:51:46.057403+0000","last_deep_scrub":"30'225","last_deep_scrub_stamp":"2024-03-11T20:51:46.057403+0000","last_clean_scrub_stamp":"2024-03-11T20:51:46.057403+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:07:28.376685+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"30'834","reported_seq":921,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:20.396809+0000","last_change":"2024-03-11T18:35:56.348355+0000","last_active":"2024-03-11T20:51:20.396809+0000","last_peered":"2024-03-11T20:51:20.396809+0000","last_clean":"2024-03-11T20:51:20.396809+0000","last_became_active":"2024-03-11T18:35:56.347631+0000","last_became_peered":"2024-03-11T18:35:56.347631+0000","last_unstale":"2024-03-11T20:51:20.396809+0000","last_undegraded":"2024-03-11T20:51:20.396809+0000","last_fullsized":"2024-03-11T20:51:20.396809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":834,"log_dups_size":0,"ondisk_log_size":834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":793,"num_write_kb":4496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"30'1909","reported_seq":1995,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:50:57.310561+0000","last_change":"2024-03-11T18:35:56.361254+0000","last_active":"2024-03-11T20:50:57.310561+0000","last_peered":"2024-03-11T20:50:57.310561+0000","last_clean":"2024-03-11T20:50:57.310561+0000","last_became_active":"2024-03-11T18:35:56.360580+0000","last_became_peered":"2024-03-11T18:35:56.360580+0000","last_unstale":"2024-03-11T20:50:57.310561+0000","last_undegraded":"2024-03-11T20:50:57.310561+0000","last_fullsized":"2024-03-11T20:50:57.310561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":1909,"log_dups_size":0,"ondisk_log_size":1909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1850,"num_write_kb":17317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"30'183","reported_seq":267,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.920800+0000","last_change":"2024-03-11T18:35:57.602778+0000","last_active":"2024-03-11T20:51:29.920800+0000","last_peered":"2024-03-11T20:51:29.920800+0000","last_clean":"2024-03-11T20:51:29.920800+0000","last_became_active":"2024-03-11T18:35:57.601990+0000","last_became_peered":"2024-03-11T18:35:57.601990+0000","last_unstale":"2024-03-11T20:51:29.920800+0000","last_undegraded":"2024-03-11T20:51:29.920800+0000","last_fullsized":"2024-03-11T20:51:29.920800+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"30'692","reported_seq":831,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:17.584388+0000","last_change":"2024-03-11T18:36:04.606098+0000","last_active":"2024-03-11T20:51:17.584388+0000","last_peered":"2024-03-11T20:51:17.584388+0000","last_clean":"2024-03-11T20:51:17.584388+0000","last_became_active":"2024-03-11T18:36:04.604823+0000","last_became_peered":"2024-03-11T18:36:04.604823+0000","last_unstale":"2024-03-11T20:51:17.584388+0000","last_undegraded":"2024-03-11T20:51:17.584388+0000","last_fullsized":"2024-03-11T20:51:17.584388+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":66,"num_read_kb":37267,"num_write":684,"num_write_kb":109032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"30'332","reported_seq":418,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.294932+0000","last_change":"2024-03-11T18:35:56.296312+0000","last_active":"2024-03-11T20:51:29.294932+0000","last_peered":"2024-03-11T20:51:29.294932+0000","last_clean":"2024-03-11T20:51:29.294932+0000","last_became_active":"2024-03-11T18:35:56.296090+0000","last_became_peered":"2024-03-11T18:35:56.296090+0000","last_unstale":"2024-03-11T20:51:29.294932+0000","last_undegraded":"2024-03-11T20:51:29.294932+0000","last_fullsized":"2024-03-11T20:51:29.294932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:54.777325+0000","last_clean_scrub_stamp":"2024-03-11T18:35:54.777325+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4195508,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":278,"num_write_kb":4222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"30'199","reported_seq":291,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:45.684713+0000","last_change":"2024-03-11T20:51:45.684771+0000","last_active":"2024-03-11T20:51:45.684713+0000","last_peered":"2024-03-11T20:51:45.684713+0000","last_clean":"2024-03-11T20:51:45.684713+0000","last_became_active":"2024-03-11T18:35:57.611097+0000","last_became_peered":"2024-03-11T18:35:57.611097+0000","last_unstale":"2024-03-11T20:51:45.684713+0000","last_undegraded":"2024-03-11T20:51:45.684713+0000","last_fullsized":"2024-03-11T20:51:45.684713+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'199","last_scrub_stamp":"2024-03-11T20:51:45.684534+0000","last_deep_scrub":"30'199","last_deep_scrub_stamp":"2024-03-11T20:51:45.684534+0000","last_clean_scrub_stamp":"2024-03-11T20:51:45.684534+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:57:37.340963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"30'703","reported_seq":835,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:08.264240+0000","last_change":"2024-03-11T18:36:04.590064+0000","last_active":"2024-03-11T20:51:08.264240+0000","last_peered":"2024-03-11T20:51:08.264240+0000","last_clean":"2024-03-11T20:51:08.264240+0000","last_became_active":"2024-03-11T18:36:04.589037+0000","last_became_peered":"2024-03-11T18:36:04.589037+0000","last_unstale":"2024-03-11T20:51:08.264240+0000","last_undegraded":"2024-03-11T20:51:08.264240+0000","last_fullsized":"2024-03-11T20:51:08.264240+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:36:03.099602+0000","last_clean_scrub_stamp":"2024-03-11T18:36:03.099602+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":59,"num_read_kb":23261,"num_write":682,"num_write_kb":69804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"30'217","reported_seq":312,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.961567+0000","last_change":"2024-03-11T18:35:57.614944+0000","last_active":"2024-03-11T20:51:29.961567+0000","last_peered":"2024-03-11T20:51:29.961567+0000","last_clean":"2024-03-11T20:51:29.961567+0000","last_became_active":"2024-03-11T18:35:57.614144+0000","last_became_peered":"2024-03-11T18:35:57.614144+0000","last_unstale":"2024-03-11T20:51:29.961567+0000","last_undegraded":"2024-03-11T20:51:29.961567+0000","last_fullsized":"2024-03-11T20:51:29.961567+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"30'222","reported_seq":317,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.798146+0000","last_change":"2024-03-11T18:35:57.630211+0000","last_active":"2024-03-11T20:51:29.798146+0000","last_peered":"2024-03-11T20:51:29.798146+0000","last_clean":"2024-03-11T20:51:29.798146+0000","last_became_active":"2024-03-11T18:35:57.629599+0000","last_became_peered":"2024-03-11T18:35:57.629599+0000","last_unstale":"2024-03-11T20:51:29.798146+0000","last_undegraded":"2024-03-11T20:51:29.798146+0000","last_fullsized":"2024-03-11T20:51:29.798146+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":222,"log_dups_size":0,"ondisk_log_size":222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"30'236","reported_seq":331,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.624153+0000","last_change":"2024-03-11T18:35:57.688700+0000","last_active":"2024-03-11T20:51:29.624153+0000","last_peered":"2024-03-11T20:51:29.624153+0000","last_clean":"2024-03-11T20:51:29.624153+0000","last_became_active":"2024-03-11T18:35:57.687968+0000","last_became_peered":"2024-03-11T18:35:57.687968+0000","last_unstale":"2024-03-11T20:51:29.624153+0000","last_undegraded":"2024-03-11T20:51:29.624153+0000","last_fullsized":"2024-03-11T20:51:29.624153+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":236,"log_dups_size":0,"ondisk_log_size":236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"30'203","reported_seq":297,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.872308+0000","last_change":"2024-03-11T18:35:57.680730+0000","last_active":"2024-03-11T20:51:29.872308+0000","last_peered":"2024-03-11T20:51:29.872308+0000","last_clean":"2024-03-11T20:51:29.872308+0000","last_became_active":"2024-03-11T18:35:57.679935+0000","last_became_peered":"2024-03-11T18:35:57.679935+0000","last_unstale":"2024-03-11T20:51:29.872308+0000","last_undegraded":"2024-03-11T20:51:29.872308+0000","last_fullsized":"2024-03-11T20:51:29.872308+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"30'184","reported_seq":276,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.764239+0000","last_change":"2024-03-11T18:35:57.690799+0000","last_active":"2024-03-11T20:51:29.764239+0000","last_peered":"2024-03-11T20:51:29.764239+0000","last_clean":"2024-03-11T20:51:29.764239+0000","last_became_active":"2024-03-11T18:35:57.686623+0000","last_became_peered":"2024-03-11T18:35:57.686623+0000","last_unstale":"2024-03-11T20:51:29.764239+0000","last_undegraded":"2024-03-11T20:51:29.764239+0000","last_fullsized":"2024-03-11T20:51:29.764239+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"30'208","reported_seq":300,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.660882+0000","last_change":"2024-03-11T18:35:57.691955+0000","last_active":"2024-03-11T20:51:29.660882+0000","last_peered":"2024-03-11T20:51:29.660882+0000","last_clean":"2024-03-11T20:51:29.660882+0000","last_became_active":"2024-03-11T18:35:57.691413+0000","last_became_peered":"2024-03-11T18:35:57.691413+0000","last_unstale":"2024-03-11T20:51:29.660882+0000","last_undegraded":"2024-03-11T20:51:29.660882+0000","last_fullsized":"2024-03-11T20:51:29.660882+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"30'244","reported_seq":336,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.964042+0000","last_change":"2024-03-11T18:35:57.684695+0000","last_active":"2024-03-11T20:51:29.964042+0000","last_peered":"2024-03-11T20:51:29.964042+0000","last_clean":"2024-03-11T20:51:29.964042+0000","last_became_active":"2024-03-11T18:35:57.684227+0000","last_became_peered":"2024-03-11T18:35:57.684227+0000","last_unstale":"2024-03-11T20:51:29.964042+0000","last_undegraded":"2024-03-11T20:51:29.964042+0000","last_fullsized":"2024-03-11T20:51:29.964042+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":244,"log_dups_size":0,"ondisk_log_size":244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"30'193","reported_seq":285,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.761600+0000","last_change":"2024-03-11T18:35:57.683549+0000","last_active":"2024-03-11T20:51:29.761600+0000","last_peered":"2024-03-11T20:51:29.761600+0000","last_clean":"2024-03-11T20:51:29.761600+0000","last_became_active":"2024-03-11T18:35:57.680610+0000","last_became_peered":"2024-03-11T18:35:57.680610+0000","last_unstale":"2024-03-11T20:51:29.761600+0000","last_undegraded":"2024-03-11T20:51:29.761600+0000","last_fullsized":"2024-03-11T20:51:29.761600+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"30'218","reported_seq":290,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.791942+0000","last_change":"2024-03-11T18:37:04.264086+0000","last_active":"2024-03-11T20:51:29.791942+0000","last_peered":"2024-03-11T20:51:29.791942+0000","last_clean":"2024-03-11T20:51:29.791942+0000","last_became_active":"2024-03-11T18:37:04.263764+0000","last_became_peered":"2024-03-11T18:37:04.263764+0000","last_unstale":"2024-03-11T20:51:29.791942+0000","last_undegraded":"2024-03-11T20:51:29.791942+0000","last_fullsized":"2024-03-11T20:51:29.791942+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"30'205","reported_seq":295,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.764772+0000","last_change":"2024-03-11T18:35:57.681630+0000","last_active":"2024-03-11T20:51:29.764772+0000","last_peered":"2024-03-11T20:51:29.764772+0000","last_clean":"2024-03-11T20:51:29.764772+0000","last_became_active":"2024-03-11T18:35:57.678876+0000","last_became_peered":"2024-03-11T18:35:57.678876+0000","last_unstale":"2024-03-11T20:51:29.764772+0000","last_undegraded":"2024-03-11T20:51:29.764772+0000","last_fullsized":"2024-03-11T20:51:29.764772+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"30'208","reported_seq":306,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:46.579294+0000","last_change":"2024-03-11T20:51:46.579412+0000","last_active":"2024-03-11T20:51:46.579294+0000","last_peered":"2024-03-11T20:51:46.579294+0000","last_clean":"2024-03-11T20:51:46.579294+0000","last_became_active":"2024-03-11T18:35:57.678838+0000","last_became_peered":"2024-03-11T18:35:57.678838+0000","last_unstale":"2024-03-11T20:51:46.579294+0000","last_undegraded":"2024-03-11T20:51:46.579294+0000","last_fullsized":"2024-03-11T20:51:46.579294+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'208","last_scrub_stamp":"2024-03-11T20:51:46.579071+0000","last_deep_scrub":"30'208","last_deep_scrub_stamp":"2024-03-11T20:51:46.579071+0000","last_clean_scrub_stamp":"2024-03-11T20:51:46.579071+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:56:33.377954+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"30'213","reported_seq":303,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.875658+0000","last_change":"2024-03-11T18:35:57.674571+0000","last_active":"2024-03-11T20:51:29.875658+0000","last_peered":"2024-03-11T20:51:29.875658+0000","last_clean":"2024-03-11T20:51:29.875658+0000","last_became_active":"2024-03-11T18:35:57.671573+0000","last_became_peered":"2024-03-11T18:35:57.671573+0000","last_unstale":"2024-03-11T20:51:29.875658+0000","last_undegraded":"2024-03-11T20:51:29.875658+0000","last_fullsized":"2024-03-11T20:51:29.875658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"30'221","reported_seq":311,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.959899+0000","last_change":"2024-03-11T18:35:57.615939+0000","last_active":"2024-03-11T20:51:29.959899+0000","last_peered":"2024-03-11T20:51:29.959899+0000","last_clean":"2024-03-11T20:51:29.959899+0000","last_became_active":"2024-03-11T18:35:57.615579+0000","last_became_peered":"2024-03-11T18:35:57.615579+0000","last_unstale":"2024-03-11T20:51:29.959899+0000","last_undegraded":"2024-03-11T20:51:29.959899+0000","last_fullsized":"2024-03-11T20:51:29.959899+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"30'206","reported_seq":296,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:32.280837+0000","last_change":"2024-03-11T18:35:57.688955+0000","last_active":"2024-03-11T20:51:32.280837+0000","last_peered":"2024-03-11T20:51:32.280837+0000","last_clean":"2024-03-11T20:51:32.280837+0000","last_became_active":"2024-03-11T18:35:57.688365+0000","last_became_peered":"2024-03-11T18:35:57.688365+0000","last_unstale":"2024-03-11T20:51:32.280837+0000","last_undegraded":"2024-03-11T20:51:32.280837+0000","last_fullsized":"2024-03-11T20:51:32.280837+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"30'205","reported_seq":293,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.796108+0000","last_change":"2024-03-11T18:35:57.602044+0000","last_active":"2024-03-11T20:51:29.796108+0000","last_peered":"2024-03-11T20:51:29.796108+0000","last_clean":"2024-03-11T20:51:29.796108+0000","last_became_active":"2024-03-11T18:35:57.601334+0000","last_became_peered":"2024-03-11T18:35:57.601334+0000","last_unstale":"2024-03-11T20:51:29.796108+0000","last_undegraded":"2024-03-11T20:51:29.796108+0000","last_fullsized":"2024-03-11T20:51:29.796108+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"30'206","reported_seq":302,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:45.560705+0000","last_change":"2024-03-11T20:51:45.560764+0000","last_active":"2024-03-11T20:51:45.560705+0000","last_peered":"2024-03-11T20:51:45.560705+0000","last_clean":"2024-03-11T20:51:45.560705+0000","last_became_active":"2024-03-11T18:35:57.676075+0000","last_became_peered":"2024-03-11T18:35:57.676075+0000","last_unstale":"2024-03-11T20:51:45.560705+0000","last_undegraded":"2024-03-11T20:51:45.560705+0000","last_fullsized":"2024-03-11T20:51:45.560705+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'206","last_scrub_stamp":"2024-03-11T20:51:45.560527+0000","last_deep_scrub":"30'206","last_deep_scrub_stamp":"2024-03-11T20:51:45.560527+0000","last_clean_scrub_stamp":"2024-03-11T20:51:45.560527+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:48:10.820914+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"30'219","reported_seq":307,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.630949+0000","last_change":"2024-03-11T18:35:57.641392+0000","last_active":"2024-03-11T20:51:29.630949+0000","last_peered":"2024-03-11T20:51:29.630949+0000","last_clean":"2024-03-11T20:51:29.630949+0000","last_became_active":"2024-03-11T18:35:57.640934+0000","last_became_peered":"2024-03-11T18:35:57.640934+0000","last_unstale":"2024-03-11T20:51:29.630949+0000","last_undegraded":"2024-03-11T20:51:29.630949+0000","last_fullsized":"2024-03-11T20:51:29.630949+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"30'213","reported_seq":301,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.668827+0000","last_change":"2024-03-11T18:35:57.605291+0000","last_active":"2024-03-11T20:51:29.668827+0000","last_peered":"2024-03-11T20:51:29.668827+0000","last_clean":"2024-03-11T20:51:29.668827+0000","last_became_active":"2024-03-11T18:35:57.604645+0000","last_became_peered":"2024-03-11T18:35:57.604645+0000","last_unstale":"2024-03-11T20:51:29.668827+0000","last_undegraded":"2024-03-11T20:51:29.668827+0000","last_fullsized":"2024-03-11T20:51:29.668827+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"30'210","reported_seq":305,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:43.444460+0000","last_change":"2024-03-11T20:51:43.444482+0000","last_active":"2024-03-11T20:51:43.444460+0000","last_peered":"2024-03-11T20:51:43.444460+0000","last_clean":"2024-03-11T20:51:43.444460+0000","last_became_active":"2024-03-11T18:35:57.503362+0000","last_became_peered":"2024-03-11T18:35:57.503362+0000","last_unstale":"2024-03-11T20:51:43.444460+0000","last_undegraded":"2024-03-11T20:51:43.444460+0000","last_fullsized":"2024-03-11T20:51:43.444460+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'210","last_scrub_stamp":"2024-03-11T20:51:43.444395+0000","last_deep_scrub":"30'210","last_deep_scrub_stamp":"2024-03-11T20:51:43.444395+0000","last_clean_scrub_stamp":"2024-03-11T20:51:43.444395+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:38:31.933136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"30'195","reported_seq":261,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.791133+0000","last_change":"2024-03-11T18:37:04.252064+0000","last_active":"2024-03-11T20:51:29.791133+0000","last_peered":"2024-03-11T20:51:29.791133+0000","last_clean":"2024-03-11T20:51:29.791133+0000","last_became_active":"2024-03-11T18:37:04.251092+0000","last_became_peered":"2024-03-11T18:37:04.251092+0000","last_unstale":"2024-03-11T20:51:29.791133+0000","last_undegraded":"2024-03-11T20:51:29.791133+0000","last_fullsized":"2024-03-11T20:51:29.791133+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"30'181","reported_seq":267,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.579866+0000","last_change":"2024-03-11T18:35:57.613374+0000","last_active":"2024-03-11T20:51:29.579866+0000","last_peered":"2024-03-11T20:51:29.579866+0000","last_clean":"2024-03-11T20:51:29.579866+0000","last_became_active":"2024-03-11T18:35:57.613050+0000","last_became_peered":"2024-03-11T18:35:57.613050+0000","last_unstale":"2024-03-11T20:51:29.579866+0000","last_undegraded":"2024-03-11T20:51:29.579866+0000","last_fullsized":"2024-03-11T20:51:29.579866+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"30'186","reported_seq":272,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.959496+0000","last_change":"2024-03-11T18:35:57.636565+0000","last_active":"2024-03-11T20:51:29.959496+0000","last_peered":"2024-03-11T20:51:29.959496+0000","last_clean":"2024-03-11T20:51:29.959496+0000","last_became_active":"2024-03-11T18:35:57.635978+0000","last_became_peered":"2024-03-11T18:35:57.635978+0000","last_unstale":"2024-03-11T20:51:29.959496+0000","last_undegraded":"2024-03-11T20:51:29.959496+0000","last_fullsized":"2024-03-11T20:51:29.959496+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"30'206","reported_seq":292,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.965351+0000","last_change":"2024-03-11T18:35:57.641714+0000","last_active":"2024-03-11T20:51:29.965351+0000","last_peered":"2024-03-11T20:51:29.965351+0000","last_clean":"2024-03-11T20:51:29.965351+0000","last_became_active":"2024-03-11T18:35:57.641417+0000","last_became_peered":"2024-03-11T18:35:57.641417+0000","last_unstale":"2024-03-11T20:51:29.965351+0000","last_undegraded":"2024-03-11T20:51:29.965351+0000","last_fullsized":"2024-03-11T20:51:29.965351+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"30'210","reported_seq":295,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.633974+0000","last_change":"2024-03-11T18:35:57.689703+0000","last_active":"2024-03-11T20:51:29.633974+0000","last_peered":"2024-03-11T20:51:29.633974+0000","last_clean":"2024-03-11T20:51:29.633974+0000","last_became_active":"2024-03-11T18:35:57.689056+0000","last_became_peered":"2024-03-11T18:35:57.689056+0000","last_unstale":"2024-03-11T20:51:29.633974+0000","last_undegraded":"2024-03-11T20:51:29.633974+0000","last_fullsized":"2024-03-11T20:51:29.633974+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"30'194","reported_seq":278,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.773877+0000","last_change":"2024-03-11T18:35:57.684305+0000","last_active":"2024-03-11T20:51:29.773877+0000","last_peered":"2024-03-11T20:51:29.773877+0000","last_clean":"2024-03-11T20:51:29.773877+0000","last_became_active":"2024-03-11T18:35:57.682947+0000","last_became_peered":"2024-03-11T18:35:57.682947+0000","last_unstale":"2024-03-11T20:51:29.773877+0000","last_undegraded":"2024-03-11T20:51:29.773877+0000","last_fullsized":"2024-03-11T20:51:29.773877+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"30'210","reported_seq":294,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.619981+0000","last_change":"2024-03-11T18:35:57.692717+0000","last_active":"2024-03-11T20:51:29.619981+0000","last_peered":"2024-03-11T20:51:29.619981+0000","last_clean":"2024-03-11T20:51:29.619981+0000","last_became_active":"2024-03-11T18:35:57.692504+0000","last_became_peered":"2024-03-11T18:35:57.692504+0000","last_unstale":"2024-03-11T20:51:29.619981+0000","last_undegraded":"2024-03-11T20:51:29.619981+0000","last_fullsized":"2024-03-11T20:51:29.619981+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"30'194","reported_seq":278,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.633607+0000","last_change":"2024-03-11T18:35:57.640483+0000","last_active":"2024-03-11T20:51:29.633607+0000","last_peered":"2024-03-11T20:51:29.633607+0000","last_clean":"2024-03-11T20:51:29.633607+0000","last_became_active":"2024-03-11T18:35:57.639886+0000","last_became_peered":"2024-03-11T18:35:57.639886+0000","last_unstale":"2024-03-11T20:51:29.633607+0000","last_undegraded":"2024-03-11T20:51:29.633607+0000","last_fullsized":"2024-03-11T20:51:29.633607+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"30'212","reported_seq":296,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.635926+0000","last_change":"2024-03-11T18:35:57.687096+0000","last_active":"2024-03-11T20:51:29.635926+0000","last_peered":"2024-03-11T20:51:29.635926+0000","last_clean":"2024-03-11T20:51:29.635926+0000","last_became_active":"2024-03-11T18:35:57.686771+0000","last_became_peered":"2024-03-11T18:35:57.686771+0000","last_unstale":"2024-03-11T20:51:29.635926+0000","last_undegraded":"2024-03-11T20:51:29.635926+0000","last_fullsized":"2024-03-11T20:51:29.635926+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"30'196","reported_seq":293,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.633240+0000","last_change":"2024-03-11T18:35:57.694707+0000","last_active":"2024-03-11T20:51:29.633240+0000","last_peered":"2024-03-11T20:51:29.633240+0000","last_clean":"2024-03-11T20:51:29.633240+0000","last_became_active":"2024-03-11T18:35:57.694104+0000","last_became_peered":"2024-03-11T18:35:57.694104+0000","last_unstale":"2024-03-11T20:51:29.633240+0000","last_undegraded":"2024-03-11T20:51:29.633240+0000","last_fullsized":"2024-03-11T20:51:29.633240+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":196,"log_dups_size":0,"ondisk_log_size":196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"30'191","reported_seq":286,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.921235+0000","last_change":"2024-03-11T18:35:57.538548+0000","last_active":"2024-03-11T20:51:29.921235+0000","last_peered":"2024-03-11T20:51:29.921235+0000","last_clean":"2024-03-11T20:51:29.921235+0000","last_became_active":"2024-03-11T18:35:57.537704+0000","last_became_peered":"2024-03-11T18:35:57.537704+0000","last_unstale":"2024-03-11T20:51:29.921235+0000","last_undegraded":"2024-03-11T20:51:29.921235+0000","last_fullsized":"2024-03-11T20:51:29.921235+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"30'214","reported_seq":309,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.659903+0000","last_change":"2024-03-11T18:35:57.507126+0000","last_active":"2024-03-11T20:51:29.659903+0000","last_peered":"2024-03-11T20:51:29.659903+0000","last_clean":"2024-03-11T20:51:29.659903+0000","last_became_active":"2024-03-11T18:35:57.506429+0000","last_became_peered":"2024-03-11T18:35:57.506429+0000","last_unstale":"2024-03-11T20:51:29.659903+0000","last_undegraded":"2024-03-11T20:51:29.659903+0000","last_fullsized":"2024-03-11T20:51:29.659903+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"30'213","reported_seq":308,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:29.609209+0000","last_change":"2024-03-11T18:35:57.687817+0000","last_active":"2024-03-11T20:51:29.609209+0000","last_peered":"2024-03-11T20:51:29.609209+0000","last_clean":"2024-03-11T20:51:29.609209+0000","last_became_active":"2024-03-11T18:35:57.687532+0000","last_became_peered":"2024-03-11T18:35:57.687532+0000","last_unstale":"2024-03-11T20:51:29.609209+0000","last_undegraded":"2024-03-11T20:51:29.609209+0000","last_fullsized":"2024-03-11T20:51:29.609209+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-11T18:35:55.991674+0000","last_clean_scrub_stamp":"2024-03-11T18:35:55.991674+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2961,"num_read_kb":1579043,"num_write":24379,"num_write_kb":4188254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24824,"ondisk_log_size":24824,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13197,"ondisk_log_size":13197,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":210457699,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":4697,"num_read_kb":5145,"num_write":47510,"num_write_kb":229688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":10000,"num_omap_keys":20,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":421158912,"data_stored":420915398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38335,"ondisk_log_size":38335,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719478381,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":200100,"kb_used_data":78748,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94171740,"statfs":{"total":96636764160,"available":96431861760,"internally_reserved":0,"allocated":80637952,"data_stored":80415478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":124254666},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":16,"seq":68719478381,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":245524,"kb_used_data":61772,"kb_used_omap":1,"kb_used_meta":183742,"kb_avail":94126316,"statfs":{"total":96636764160,"available":96385347584,"internally_reserved":0,"allocated":63254528,"data_stored":63047256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":188152262},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":16,"seq":68719478379,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":200588,"kb_used_data":48708,"kb_used_omap":1,"kb_used_meta":151870,"kb_avail":94171252,"statfs":{"total":96636764160,"available":96431362048,"internally_reserved":0,"allocated":49876992,"data_stored":49644774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":155515334},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":16,"seq":68719478380,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":183188,"kb_used_data":57676,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94188652,"statfs":{"total":96636764160,"available":96449179648,"internally_reserved":0,"allocated":59060224,"data_stored":58852820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424511083,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":218248,"kb_used_data":42816,"kb_used_omap":1,"kb_used_meta":175422,"kb_avail":94153592,"statfs":{"total":96636764160,"available":96413278208,"internally_reserved":0,"allocated":43843584,"data_stored":43631469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":179632583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424511083,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":152012,"kb_used_data":22340,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94219828,"statfs":{"total":96636764160,"available":96481103872,"internally_reserved":0,"allocated":22876160,"data_stored":22659915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424511084,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":228216,"kb_used_data":56880,"kb_used_omap":1,"kb_used_meta":171326,"kb_avail":94143624,"statfs":{"total":96636764160,"available":96403070976,"internally_reserved":0,"allocated":58245120,"data_stored":58030400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":175438280},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129543785,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149276,"kb_used_data":45972,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94222564,"statfs":{"total":96636764160,"available":96483905536,"internally_reserved":0,"allocated":47075328,"data_stored":46859238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":46166016,"data_stored":46138218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57929728,"data_stored":57899748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":22560768,"data_stored":22529263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":43528192,"data_stored":43500817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":58744832,"data_stored":58722168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49561600,"data_stored":49514122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":62939136,"data_stored":62916604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":79728640,"data_stored":79694458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-11T20:51:47.821 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.822 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-03-11T18:36:03.099602+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=36, tm_sec=3, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.822 INFO:tasks.ceph:pgid 2.1e last_scrub_stamp 2024-03-11T18:35:54.777325+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=54, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.822 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.822 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2024-03-11T18:36:03.099602+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=36, tm_sec=3, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.822 INFO:tasks.ceph:pgid 2.1d last_scrub_stamp 2024-03-11T18:35:54.777325+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=54, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.822 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.822 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-03-11T18:36:03.099602+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=36, tm_sec=3, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.822 INFO:tasks.ceph:pgid 2.1c last_scrub_stamp 2024-03-11T18:35:54.777325+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=54, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.822 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.823 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-03-11T18:36:03.099602+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=36, tm_sec=3, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.823 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.823 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2024-03-11T18:35:54.777325+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=54, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.823 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-03-11T18:36:03.099602+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=36, tm_sec=3, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.823 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2024-03-11T18:36:03.099602+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=36, tm_sec=3, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.823 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2024-03-11T18:36:03.099602+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=36, tm_sec=3, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.823 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.823 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-03-11T18:35:54.777325+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=54, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.823 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2024-03-11T18:36:03.099602+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=36, tm_sec=3, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.823 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.824 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-03-11T18:35:54.777325+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=54, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.824 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2024-03-11T18:36:03.099602+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=36, tm_sec=3, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.824 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.824 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-03-11T18:35:54.777325+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=54, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.824 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-03-11T18:36:03.099602+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=36, tm_sec=3, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.824 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.824 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-03-11T18:35:54.777325+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=54, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.824 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-03-11T18:36:03.099602+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=36, tm_sec=3, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.824 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.825 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2024-03-11T18:36:03.099602+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=36, tm_sec=3, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.825 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-03-11T18:35:54.777325+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=54, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.825 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2024-03-11T18:36:03.099602+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=36, tm_sec=3, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.825 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.825 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-03-11T18:36:03.099602+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=36, tm_sec=3, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.825 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.825 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-03-11T18:35:54.777325+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=54, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.825 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-03-11T18:36:03.099602+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=36, tm_sec=3, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.825 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.825 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-03-11T18:35:54.777325+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=54, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.826 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-03-11T18:35:16.207500+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=16, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.826 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2024-03-11T18:36:03.099602+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=36, tm_sec=3, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.826 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.826 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2024-03-11T18:36:03.099602+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=36, tm_sec=3, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.826 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.826 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-03-11T18:36:03.099602+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=36, tm_sec=3, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.826 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.826 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-03-11T18:36:03.099602+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=36, tm_sec=3, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.826 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.827 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-03-11T18:36:03.099602+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=36, tm_sec=3, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.827 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.827 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-03-11T18:36:03.099602+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=36, tm_sec=3, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.827 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.827 INFO:tasks.ceph:pgid 2.11 last_scrub_stamp 2024-03-11T18:35:54.777325+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=54, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.827 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-03-11T18:36:03.099602+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=36, tm_sec=3, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.827 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.827 INFO:tasks.ceph:pgid 2.12 last_scrub_stamp 2024-03-11T18:35:54.777325+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=54, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.827 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-03-11T18:36:03.099602+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=36, tm_sec=3, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.827 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-03-11T18:36:03.099602+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=36, tm_sec=3, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.828 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.828 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-03-11T18:36:03.099602+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=36, tm_sec=3, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.828 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.828 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2024-03-11T18:36:03.099602+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=36, tm_sec=3, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.828 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.828 INFO:tasks.ceph:pgid 2.16 last_scrub_stamp 2024-03-11T18:35:54.777325+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=54, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.828 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-03-11T18:36:03.099602+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=36, tm_sec=3, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.828 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.828 INFO:tasks.ceph:pgid 2.17 last_scrub_stamp 2024-03-11T18:35:54.777325+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=54, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.828 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-03-11T18:36:03.099602+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=36, tm_sec=3, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.829 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-03-11T18:36:03.099602+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=36, tm_sec=3, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.829 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.829 INFO:tasks.ceph:pgid 2.18 last_scrub_stamp 2024-03-11T18:35:54.777325+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=54, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.829 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-03-11T18:36:03.099602+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=36, tm_sec=3, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.829 INFO:tasks.ceph:pgid 2.19 last_scrub_stamp 2024-03-11T18:35:54.777325+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=54, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.829 INFO:tasks.ceph:pgid 2.1a last_scrub_stamp 2024-03-11T18:35:54.777325+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=54, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.829 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.829 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-03-11T18:36:03.099602+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=36, tm_sec=3, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.829 INFO:tasks.ceph:pgid 2.1b last_scrub_stamp 2024-03-11T18:35:54.777325+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=54, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.830 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-03-11T18:36:03.099602+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=36, tm_sec=3, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.830 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.830 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.830 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.830 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.830 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.830 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.830 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.830 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.831 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.831 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.831 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.831 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.831 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.831 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.831 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.831 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.831 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.831 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.832 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.832 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.832 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.832 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.832 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.832 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.832 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.832 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.832 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.832 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.833 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-03-11T18:35:55.991674+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=18, tm_min=35, tm_sec=55, tm_wday=0, tm_yday=71, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=11, tm_hour=20, tm_min=51, tm_sec=38, tm_wday=0, tm_yday=71, tm_isdst=0) 2024-03-11T20:51:47.833 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-03-11T20:52:01.708 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:52:01.711 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:52:07.834 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-11T20:52:07.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.960+0000 7fc63eb6b640 1 Processor -- start 2024-03-11T20:52:07.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.964+0000 7fc63eb6b640 1 -- start start 2024-03-11T20:52:07.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.964+0000 7fc63eb6b640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fc6381541c0 0x7fc6380caf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:07.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.964+0000 7fc63eb6b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fc6380cfda0 0x7fc6380ca180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:07.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.964+0000 7fc63eb6b640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6380cb5b0 con 0x7fc6380c6b20 2024-03-11T20:52:07.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.964+0000 7fc63eb6b640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6380b5410 con 0x7fc6381541c0 2024-03-11T20:52:07.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.964+0000 7fc63eb6b640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc638159e10 con 0x7fc6380cfda0 2024-03-11T20:52:07.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.964+0000 7fc63c907640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fc6381541c0 0x7fc6380caf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:52:07.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.964+0000 7fc63d108640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7fc6380c6b20 0x7fc63815d340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:55990/0 (socket says 172.21.15.90:55990) 2024-03-11T20:52:07.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.964+0000 7fc63d108640 1 -- 172.21.15.90:0/410013128 learned_addr learned my addr 172.21.15.90:0/410013128 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T20:52:07.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.964+0000 7fc637fff640 1 --2- 172.21.15.90:0/410013128 >> v2:172.21.15.92:3300/0 conn(0x7fc6380cfda0 0x7fc6380ca180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:52:07.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.968+0000 7fc63c907640 1 -- 172.21.15.90:0/410013128 >> v2:172.21.15.92:3300/0 conn(0x7fc6380cfda0 msgr2=0x7fc6380ca180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:52:07.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.968+0000 7fc63c907640 1 --2- 172.21.15.90:0/410013128 >> v2:172.21.15.92:3300/0 conn(0x7fc6380cfda0 0x7fc6380ca180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:07.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.968+0000 7fc63c907640 1 -- 172.21.15.90:0/410013128 >> v1:172.21.15.90:6789/0 conn(0x7fc6380c6b20 legacy=0x7fc63815d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:07.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.968+0000 7fc63c907640 1 -- 172.21.15.90:0/410013128 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc63815a0f0 con 0x7fc6381541c0 2024-03-11T20:52:07.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.972+0000 7fc63c907640 1 --2- 172.21.15.90:0/410013128 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6381541c0 0x7fc6380caf00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc638157930 tx=0x7fc63001f110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=650751fdf3145d4c server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:52:07.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.980+0000 7fc6377fe640 1 -- 172.21.15.90:0/410013128 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6300af600 con 0x7fc6381541c0 2024-03-11T20:52:07.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.980+0000 7fc6377fe640 1 -- 172.21.15.90:0/410013128 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6300ab050 con 0x7fc6381541c0 2024-03-11T20:52:07.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.980+0000 7fc6377fe640 1 -- 172.21.15.90:0/410013128 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6300af980 con 0x7fc6381541c0 2024-03-11T20:52:07.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.980+0000 7fc63eb6b640 1 -- 172.21.15.90:0/410013128 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6381541c0 msgr2=0x7fc6380caf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:07.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.980+0000 7fc63eb6b640 1 --2- 172.21.15.90:0/410013128 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6381541c0 0x7fc6380caf00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc638157930 tx=0x7fc63001f110 comp rx=0 tx=0).stop 2024-03-11T20:52:07.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.980+0000 7fc63eb6b640 1 -- 172.21.15.90:0/410013128 shutdown_connections 2024-03-11T20:52:07.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.980+0000 7fc63eb6b640 1 --2- 172.21.15.90:0/410013128 >> v2:172.21.15.92:3300/0 conn(0x7fc6380cfda0 0x7fc6380ca180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:07.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.980+0000 7fc63eb6b640 1 --2- 172.21.15.90:0/410013128 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6381541c0 0x7fc6380caf00 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:07.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.980+0000 7fc63eb6b640 1 -- 172.21.15.90:0/410013128 >> 172.21.15.90:0/410013128 conn(0x7fc63806b620 msgr2=0x7fc63806ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:52:07.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.980+0000 7fc63eb6b640 1 -- 172.21.15.90:0/410013128 shutdown_connections 2024-03-11T20:52:07.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.980+0000 7fc63eb6b640 1 -- 172.21.15.90:0/410013128 wait complete. 2024-03-11T20:52:07.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.984+0000 7fc63eb6b640 1 Processor -- start 2024-03-11T20:52:07.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.984+0000 7fc63eb6b640 1 -- start start 2024-03-11T20:52:07.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.984+0000 7fc63eb6b640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc6380c6b20 0x7fc6380c5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:07.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.984+0000 7fc63eb6b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6380cfda0 0x7fc6381532c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:07.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.984+0000 7fc63eb6b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6381541c0 0x7fc6381566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:07.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.984+0000 7fc63eb6b640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6380c4250 con 0x7fc6381541c0 2024-03-11T20:52:07.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.984+0000 7fc63eb6b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6381f3990 con 0x7fc6380cfda0 2024-03-11T20:52:07.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.984+0000 7fc63eb6b640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc63815cf50 con 0x7fc6380c6b20 2024-03-11T20:52:07.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.984+0000 7fc63d108640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6381541c0 0x7fc6381566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:52:07.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.984+0000 7fc63c907640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc6380c6b20 0x7fc6380c5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:52:07.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.984+0000 7fc63d108640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6381541c0 0x7fc6381566f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60142/0 (socket says 172.21.15.90:60142) 2024-03-11T20:52:07.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.984+0000 7fc63d108640 1 -- 172.21.15.90:0/2476051801 learned_addr learned my addr 172.21.15.90:0/2476051801 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:52:07.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.984+0000 7fc637fff640 1 --2- 172.21.15.90:0/2476051801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6380cfda0 0x7fc6381532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:52:07.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.988+0000 7fc63d108640 1 -- 172.21.15.90:0/2476051801 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc6380c6b20 msgr2=0x7fc6380c5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:07.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.988+0000 7fc63d108640 1 --2- 172.21.15.90:0/2476051801 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc6380c6b20 0x7fc6380c5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:07.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.988+0000 7fc63d108640 1 -- 172.21.15.90:0/2476051801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6380cfda0 msgr2=0x7fc6381532c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:07.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.988+0000 7fc63d108640 1 --2- 172.21.15.90:0/2476051801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6380cfda0 0x7fc6381532c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:07.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.988+0000 7fc63d108640 1 -- 172.21.15.90:0/2476051801 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc630009a70 con 0x7fc6381541c0 2024-03-11T20:52:07.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.992+0000 7fc63d108640 1 --2- 172.21.15.90:0/2476051801 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6381541c0 0x7fc6381566f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc62800c3c0 tx=0x7fc62800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:52:07.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.996+0000 7fc6357fa640 1 -- 172.21.15.90:0/2476051801 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc628005cf0 con 0x7fc6381541c0 2024-03-11T20:52:07.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.996+0000 7fc6357fa640 1 -- 172.21.15.90:0/2476051801 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc628005e50 con 0x7fc6381541c0 2024-03-11T20:52:07.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.996+0000 7fc6357fa640 1 -- 172.21.15.90:0/2476051801 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc628007310 con 0x7fc6381541c0 2024-03-11T20:52:07.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.996+0000 7fc63eb6b640 1 -- 172.21.15.90:0/2476051801 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6380ce260 con 0x7fc6381541c0 2024-03-11T20:52:07.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.996+0000 7fc63eb6b640 1 -- 172.21.15.90:0/2476051801 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6381f3060 con 0x7fc6381541c0 2024-03-11T20:52:07.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:07.996+0000 7fc63eb6b640 1 -- 172.21.15.90:0/2476051801 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6381f3b40 con 0x7fc6381541c0 2024-03-11T20:52:08.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.264+0000 7fc6357fa640 1 -- 172.21.15.90:0/2476051801 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fc628007470 con 0x7fc6381541c0 2024-03-11T20:52:08.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.264+0000 7fc6357fa640 1 --2- 172.21.15.90:0/2476051801 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fc624066e40 0x7fc624069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:08.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.264+0000 7fc63c907640 1 --2- 172.21.15.90:0/2476051801 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fc624066e40 0x7fc624069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:52:08.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.264+0000 7fc63c907640 1 --2- 172.21.15.90:0/2476051801 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fc624066e40 0x7fc624069300 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc6300b1000 tx=0x7fc6300aa000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:52:08.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.276+0000 7fc6357fa640 1 -- 172.21.15.90:0/2476051801 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fc628007720 con 0x7fc6381541c0 2024-03-11T20:52:08.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.276+0000 7fc6357fa640 1 -- 172.21.15.90:0/2476051801 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc628051310 con 0x7fc6381541c0 2024-03-11T20:52:08.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.436+0000 7fc63eb6b640 1 -- 172.21.15.90:0/2476051801 --> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc6380c70c0 con 0x7fc624066e40 2024-03-11T20:52:08.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.448+0000 7fc6357fa640 1 -- 172.21.15.90:0/2476051801 <== mgr.4101 v2:172.21.15.90:6800/1999720161 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+335465 (secure 0 0 0) 0x7fc6380c70c0 con 0x7fc624066e40 2024-03-11T20:52:08.450 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T20:52:08.455 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-03-11T20:52:08.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.456+0000 7fc63eb6b640 1 -- 172.21.15.90:0/2476051801 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fc624066e40 msgr2=0x7fc624069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:08.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.456+0000 7fc63eb6b640 1 --2- 172.21.15.90:0/2476051801 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fc624066e40 0x7fc624069300 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc6300b1000 tx=0x7fc6300aa000 comp rx=0 tx=0).stop 2024-03-11T20:52:08.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.456+0000 7fc63eb6b640 1 -- 172.21.15.90:0/2476051801 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6381541c0 msgr2=0x7fc6381566f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:08.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.456+0000 7fc63eb6b640 1 --2- 172.21.15.90:0/2476051801 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6381541c0 0x7fc6381566f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc62800c3c0 tx=0x7fc62800c880 comp rx=0 tx=0).stop 2024-03-11T20:52:08.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.456+0000 7fc63eb6b640 1 -- 172.21.15.90:0/2476051801 shutdown_connections 2024-03-11T20:52:08.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.456+0000 7fc63eb6b640 1 --2- 172.21.15.90:0/2476051801 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fc624066e40 0x7fc624069300 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:08.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.456+0000 7fc63eb6b640 1 --2- 172.21.15.90:0/2476051801 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6381541c0 0x7fc6381566f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:08.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.456+0000 7fc63eb6b640 1 --2- 172.21.15.90:0/2476051801 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fc6380cfda0 0x7fc6381532c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:08.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.456+0000 7fc63eb6b640 1 --2- 172.21.15.90:0/2476051801 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fc6380c6b20 0x7fc6380c5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:08.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.456+0000 7fc63eb6b640 1 -- 172.21.15.90:0/2476051801 >> 172.21.15.90:0/2476051801 conn(0x7fc63806b620 msgr2=0x7fc6381525b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:52:08.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.456+0000 7fc63eb6b640 1 -- 172.21.15.90:0/2476051801 shutdown_connections 2024-03-11T20:52:08.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.456+0000 7fc63eb6b640 1 -- 172.21.15.90:0/2476051801 wait complete. 2024-03-11T20:52:08.474 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":4131,"stamp":"2024-03-11T20:52:07.463803+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":211048067,"num_objects":150,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":150,"num_whiteouts":0,"num_read":7774,"num_read_kb":1584288,"num_write":72048,"num_write_kb":4420358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":25000,"num_omap_keys":50,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76552,"ondisk_log_size":76552,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":1560512,"kb_used_data":414912,"kb_used_omap":12,"kb_used_meta":1145523,"kb_avail":753414208,"statfs":{"total":773094113280,"available":771496148992,"internally_reserved":0,"allocated":424869888,"data_stored":423141350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":1173016116},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005171"},"pg_stats":[{"pgid":"2.1f","version":"30'689","reported_seq":794,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:42.501287+0000","last_change":"2024-03-11T20:51:42.501371+0000","last_active":"2024-03-11T20:51:42.501287+0000","last_peered":"2024-03-11T20:51:42.501287+0000","last_clean":"2024-03-11T20:51:42.501287+0000","last_became_active":"2024-03-11T18:35:56.289178+0000","last_became_peered":"2024-03-11T18:35:56.289178+0000","last_unstale":"2024-03-11T20:51:42.501287+0000","last_undegraded":"2024-03-11T20:51:42.501287+0000","last_fullsized":"2024-03-11T20:51:42.501287+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'689","last_scrub_stamp":"2024-03-11T20:51:42.501128+0000","last_deep_scrub":"30'689","last_deep_scrub_stamp":"2024-03-11T20:51:42.501128+0000","last_clean_scrub_stamp":"2024-03-11T20:51:42.501128+0000","objects_scrubbed":8,"log_size":689,"log_dups_size":0,"ondisk_log_size":689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:44:39.698011+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389280,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":643,"num_write_kb":8491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"30'202","reported_seq":307,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:52.115353+0000","last_change":"2024-03-11T20:51:52.115414+0000","last_active":"2024-03-11T20:51:52.115353+0000","last_peered":"2024-03-11T20:51:52.115353+0000","last_clean":"2024-03-11T20:51:52.115353+0000","last_became_active":"2024-03-11T18:35:57.675105+0000","last_became_peered":"2024-03-11T18:35:57.675105+0000","last_unstale":"2024-03-11T20:51:52.115353+0000","last_undegraded":"2024-03-11T20:51:52.115353+0000","last_fullsized":"2024-03-11T20:51:52.115353+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'202","last_scrub_stamp":"2024-03-11T20:51:52.115182+0000","last_deep_scrub":"30'202","last_deep_scrub_stamp":"2024-03-11T20:51:52.115182+0000","last_clean_scrub_stamp":"2024-03-11T20:51:52.115182+0000","objects_scrubbed":0,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:42:27.779350+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"30'811","reported_seq":1012,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:52:01.101862+0000","last_change":"2024-03-11T20:52:01.101990+0000","last_active":"2024-03-11T20:52:01.101862+0000","last_peered":"2024-03-11T20:52:01.101862+0000","last_clean":"2024-03-11T20:52:01.101862+0000","last_became_active":"2024-03-11T18:36:04.599562+0000","last_became_peered":"2024-03-11T18:36:04.599562+0000","last_unstale":"2024-03-11T20:52:01.101862+0000","last_undegraded":"2024-03-11T20:52:01.101862+0000","last_fullsized":"2024-03-11T20:52:01.101862+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'811","last_scrub_stamp":"2024-03-11T20:52:01.101616+0000","last_deep_scrub":"30'811","last_deep_scrub_stamp":"2024-03-11T20:52:01.101616+0000","last_clean_scrub_stamp":"2024-03-11T20:52:01.101616+0000","objects_scrubbed":0,"log_size":811,"log_dups_size":0,"ondisk_log_size":811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:48:40.827789+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":119,"num_read_kb":64172,"num_write":795,"num_write_kb":180282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"30'656","reported_seq":763,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:48.124774+0000","last_change":"2024-03-11T20:51:48.124846+0000","last_active":"2024-03-11T20:51:48.124774+0000","last_peered":"2024-03-11T20:51:48.124774+0000","last_clean":"2024-03-11T20:51:48.124774+0000","last_became_active":"2024-03-11T18:35:56.290862+0000","last_became_peered":"2024-03-11T18:35:56.290862+0000","last_unstale":"2024-03-11T20:51:48.124774+0000","last_undegraded":"2024-03-11T20:51:48.124774+0000","last_fullsized":"2024-03-11T20:51:48.124774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'656","last_scrub_stamp":"2024-03-11T20:51:48.124590+0000","last_deep_scrub":"30'656","last_deep_scrub_stamp":"2024-03-11T20:51:48.124590+0000","last_clean_scrub_stamp":"2024-03-11T20:51:48.124590+0000","objects_scrubbed":2,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:06:49.329169+0000","scrub_duration":36,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":603,"num_write_kb":8489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"30'187","reported_seq":295,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:56.366901+0000","last_change":"2024-03-11T20:51:56.366963+0000","last_active":"2024-03-11T20:51:56.366901+0000","last_peered":"2024-03-11T20:51:56.366901+0000","last_clean":"2024-03-11T20:51:56.366901+0000","last_became_active":"2024-03-11T18:37:04.142880+0000","last_became_peered":"2024-03-11T18:37:04.142880+0000","last_unstale":"2024-03-11T20:51:56.366901+0000","last_undegraded":"2024-03-11T20:51:56.366901+0000","last_fullsized":"2024-03-11T20:51:56.366901+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'187","last_scrub_stamp":"2024-03-11T20:51:56.366733+0000","last_deep_scrub":"30'187","last_deep_scrub_stamp":"2024-03-11T20:51:56.366733+0000","last_clean_scrub_stamp":"2024-03-11T20:51:56.366733+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:58:16.997739+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"30'785","reported_seq":981,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:58.640422+0000","last_change":"2024-03-11T20:51:58.640487+0000","last_active":"2024-03-11T20:51:58.640422+0000","last_peered":"2024-03-11T20:51:58.640422+0000","last_clean":"2024-03-11T20:51:58.640422+0000","last_became_active":"2024-03-11T18:37:04.274018+0000","last_became_peered":"2024-03-11T18:37:04.274018+0000","last_unstale":"2024-03-11T20:51:58.640422+0000","last_undegraded":"2024-03-11T20:51:58.640422+0000","last_fullsized":"2024-03-11T20:51:58.640422+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'785","last_scrub_stamp":"2024-03-11T20:51:58.640215+0000","last_deep_scrub":"30'785","last_deep_scrub_stamp":"2024-03-11T20:51:58.640215+0000","last_clean_scrub_stamp":"2024-03-11T20:51:58.640215+0000","objects_scrubbed":0,"log_size":785,"log_dups_size":0,"ondisk_log_size":785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:21:41.621104+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":106,"num_read_kb":55696,"num_write":774,"num_write_kb":160695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2,3],"acting":[4,6,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"30'1745","reported_seq":1839,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:54.598452+0000","last_change":"2024-03-11T20:51:54.598650+0000","last_active":"2024-03-11T20:51:54.598452+0000","last_peered":"2024-03-11T20:51:54.598452+0000","last_clean":"2024-03-11T20:51:54.598452+0000","last_became_active":"2024-03-11T18:35:56.354388+0000","last_became_peered":"2024-03-11T18:35:56.354388+0000","last_unstale":"2024-03-11T20:51:54.598452+0000","last_undegraded":"2024-03-11T20:51:54.598452+0000","last_fullsized":"2024-03-11T20:51:54.598452+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'1745","last_scrub_stamp":"2024-03-11T20:51:54.598164+0000","last_deep_scrub":"30'1745","last_deep_scrub_stamp":"2024-03-11T20:51:54.598164+0000","last_clean_scrub_stamp":"2024-03-11T20:51:54.598164+0000","objects_scrubbed":11,"log_size":1745,"log_dups_size":0,"ondisk_log_size":1745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:21:37.579052+0000","scrub_duration":87,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1669,"num_write_kb":17224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"30'190","reported_seq":282,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:47.590616+0000","last_change":"2024-03-11T20:51:47.590683+0000","last_active":"2024-03-11T20:51:47.590616+0000","last_peered":"2024-03-11T20:51:47.590616+0000","last_clean":"2024-03-11T20:51:47.590616+0000","last_became_active":"2024-03-11T18:35:57.641052+0000","last_became_peered":"2024-03-11T18:35:57.641052+0000","last_unstale":"2024-03-11T20:51:47.590616+0000","last_undegraded":"2024-03-11T20:51:47.590616+0000","last_fullsized":"2024-03-11T20:51:47.590616+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'190","last_scrub_stamp":"2024-03-11T20:51:47.590441+0000","last_deep_scrub":"30'190","last_deep_scrub_stamp":"2024-03-11T20:51:47.590441+0000","last_clean_scrub_stamp":"2024-03-11T20:51:47.590441+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:38:29.612967+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"30'808","reported_seq":982,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:53.729846+0000","last_change":"2024-03-11T20:51:53.729912+0000","last_active":"2024-03-11T20:51:53.729846+0000","last_peered":"2024-03-11T20:51:53.729846+0000","last_clean":"2024-03-11T20:51:53.729846+0000","last_became_active":"2024-03-11T18:36:04.624110+0000","last_became_peered":"2024-03-11T18:36:04.624110+0000","last_unstale":"2024-03-11T20:51:53.729846+0000","last_undegraded":"2024-03-11T20:51:53.729846+0000","last_fullsized":"2024-03-11T20:51:53.729846+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'808","last_scrub_stamp":"2024-03-11T20:51:53.729674+0000","last_deep_scrub":"30'808","last_deep_scrub_stamp":"2024-03-11T20:51:53.729674+0000","last_clean_scrub_stamp":"2024-03-11T20:51:53.729674+0000","objects_scrubbed":0,"log_size":808,"log_dups_size":0,"ondisk_log_size":808,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:14:52.955887+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":93,"num_read_kb":49009,"num_write":797,"num_write_kb":196902,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"30'836","reported_seq":930,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:49.583437+0000","last_change":"2024-03-11T20:51:49.583512+0000","last_active":"2024-03-11T20:51:49.583437+0000","last_peered":"2024-03-11T20:51:49.583437+0000","last_clean":"2024-03-11T20:51:49.583437+0000","last_became_active":"2024-03-11T18:35:56.297582+0000","last_became_peered":"2024-03-11T18:35:56.297582+0000","last_unstale":"2024-03-11T20:51:49.583437+0000","last_undegraded":"2024-03-11T20:51:49.583437+0000","last_fullsized":"2024-03-11T20:51:49.583437+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'836","last_scrub_stamp":"2024-03-11T20:51:49.583269+0000","last_deep_scrub":"30'836","last_deep_scrub_stamp":"2024-03-11T20:51:49.583269+0000","last_clean_scrub_stamp":"2024-03-11T20:51:49.583269+0000","objects_scrubbed":4,"log_size":836,"log_dups_size":0,"ondisk_log_size":836,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:14:48.809483+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":777,"num_write_kb":4502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"30'203","reported_seq":295,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:53.644311+0000","last_change":"2024-03-11T20:51:53.644373+0000","last_active":"2024-03-11T20:51:53.644311+0000","last_peered":"2024-03-11T20:51:53.644311+0000","last_clean":"2024-03-11T20:51:53.644311+0000","last_became_active":"2024-03-11T18:35:57.639770+0000","last_became_peered":"2024-03-11T18:35:57.639770+0000","last_unstale":"2024-03-11T20:51:53.644311+0000","last_undegraded":"2024-03-11T20:51:53.644311+0000","last_fullsized":"2024-03-11T20:51:53.644311+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'203","last_scrub_stamp":"2024-03-11T20:51:53.644137+0000","last_deep_scrub":"30'203","last_deep_scrub_stamp":"2024-03-11T20:51:53.644137+0000","last_clean_scrub_stamp":"2024-03-11T20:51:53.644137+0000","objects_scrubbed":0,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:31:28.689890+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"30'729","reported_seq":834,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:52.753558+0000","last_change":"2024-03-11T20:51:52.753690+0000","last_active":"2024-03-11T20:51:52.753558+0000","last_peered":"2024-03-11T20:51:52.753558+0000","last_clean":"2024-03-11T20:51:52.753558+0000","last_became_active":"2024-03-11T18:36:04.626108+0000","last_became_peered":"2024-03-11T18:36:04.626108+0000","last_unstale":"2024-03-11T20:51:52.753558+0000","last_undegraded":"2024-03-11T20:51:52.753558+0000","last_fullsized":"2024-03-11T20:51:52.753558+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'729","last_scrub_stamp":"2024-03-11T20:51:52.753328+0000","last_deep_scrub":"30'729","last_deep_scrub_stamp":"2024-03-11T20:51:52.753328+0000","last_clean_scrub_stamp":"2024-03-11T20:51:52.753328+0000","objects_scrubbed":0,"log_size":729,"log_dups_size":0,"ondisk_log_size":729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:15:59.483579+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":20804,"num_write":712,"num_write_kb":76413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"30'216","reported_seq":316,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:54.440631+0000","last_change":"2024-03-11T20:51:54.440729+0000","last_active":"2024-03-11T20:51:54.440631+0000","last_peered":"2024-03-11T20:51:54.440631+0000","last_clean":"2024-03-11T20:51:54.440631+0000","last_became_active":"2024-03-11T18:35:57.616398+0000","last_became_peered":"2024-03-11T18:35:57.616398+0000","last_unstale":"2024-03-11T20:51:54.440631+0000","last_undegraded":"2024-03-11T20:51:54.440631+0000","last_fullsized":"2024-03-11T20:51:54.440631+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'216","last_scrub_stamp":"2024-03-11T20:51:54.440459+0000","last_deep_scrub":"30'216","last_deep_scrub_stamp":"2024-03-11T20:51:54.440459+0000","last_clean_scrub_stamp":"2024-03-11T20:51:54.440459+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:38:36.462986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"30'1453","reported_seq":1553,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:52.646019+0000","last_change":"2024-03-11T20:51:52.646103+0000","last_active":"2024-03-11T20:51:52.646019+0000","last_peered":"2024-03-11T20:51:52.646019+0000","last_clean":"2024-03-11T20:51:52.646019+0000","last_became_active":"2024-03-11T18:35:56.302803+0000","last_became_peered":"2024-03-11T18:35:56.302803+0000","last_unstale":"2024-03-11T20:51:52.646019+0000","last_undegraded":"2024-03-11T20:51:52.646019+0000","last_fullsized":"2024-03-11T20:51:52.646019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'1453","last_scrub_stamp":"2024-03-11T20:51:52.645849+0000","last_deep_scrub":"30'1453","last_deep_scrub_stamp":"2024-03-11T20:51:52.645849+0000","last_clean_scrub_stamp":"2024-03-11T20:51:52.645849+0000","objects_scrubbed":5,"log_size":1453,"log_dups_size":0,"ondisk_log_size":1453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:38:34.668375+0000","scrub_duration":67,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1389,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"30'793","reported_seq":974,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:51.739032+0000","last_change":"2024-03-11T20:51:51.739158+0000","last_active":"2024-03-11T20:51:51.739032+0000","last_peered":"2024-03-11T20:51:51.739032+0000","last_clean":"2024-03-11T20:51:51.739032+0000","last_became_active":"2024-03-11T18:36:04.626337+0000","last_became_peered":"2024-03-11T18:36:04.626337+0000","last_unstale":"2024-03-11T20:51:51.739032+0000","last_undegraded":"2024-03-11T20:51:51.739032+0000","last_fullsized":"2024-03-11T20:51:51.739032+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'793","last_scrub_stamp":"2024-03-11T20:51:51.738792+0000","last_deep_scrub":"30'793","last_deep_scrub_stamp":"2024-03-11T20:51:51.738792+0000","last_clean_scrub_stamp":"2024-03-11T20:51:51.738792+0000","objects_scrubbed":0,"log_size":793,"log_dups_size":0,"ondisk_log_size":793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:45:18.798317+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":94,"num_read_kb":46134,"num_write":781,"num_write_kb":151379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"30'211","reported_seq":309,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:44.001936+0000","last_change":"2024-03-11T20:51:44.002049+0000","last_active":"2024-03-11T20:51:44.001936+0000","last_peered":"2024-03-11T20:51:44.001936+0000","last_clean":"2024-03-11T20:51:44.001936+0000","last_became_active":"2024-03-11T18:35:57.689039+0000","last_became_peered":"2024-03-11T18:35:57.689039+0000","last_unstale":"2024-03-11T20:51:44.001936+0000","last_undegraded":"2024-03-11T20:51:44.001936+0000","last_fullsized":"2024-03-11T20:51:44.001936+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'211","last_scrub_stamp":"2024-03-11T20:51:44.001743+0000","last_deep_scrub":"30'211","last_deep_scrub_stamp":"2024-03-11T20:51:44.001743+0000","last_clean_scrub_stamp":"2024-03-11T20:51:44.001743+0000","objects_scrubbed":0,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:14:43.227957+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"30'1239","reported_seq":1341,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:44.157527+0000","last_change":"2024-03-11T20:51:44.157640+0000","last_active":"2024-03-11T20:51:44.157527+0000","last_peered":"2024-03-11T20:51:44.157527+0000","last_clean":"2024-03-11T20:51:44.157527+0000","last_became_active":"2024-03-11T18:35:56.290504+0000","last_became_peered":"2024-03-11T18:35:56.290504+0000","last_unstale":"2024-03-11T20:51:44.157527+0000","last_undegraded":"2024-03-11T20:51:44.157527+0000","last_fullsized":"2024-03-11T20:51:44.157527+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'1239","last_scrub_stamp":"2024-03-11T20:51:44.157352+0000","last_deep_scrub":"30'1239","last_deep_scrub_stamp":"2024-03-11T20:51:44.157352+0000","last_clean_scrub_stamp":"2024-03-11T20:51:44.157352+0000","objects_scrubbed":4,"log_size":1239,"log_dups_size":0,"ondisk_log_size":1239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:34:55.996207+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1168,"num_write_kb":8796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"30'848","reported_seq":1071,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:50.732208+0000","last_change":"2024-03-11T20:51:50.732335+0000","last_active":"2024-03-11T20:51:50.732208+0000","last_peered":"2024-03-11T20:51:50.732208+0000","last_clean":"2024-03-11T20:51:50.732208+0000","last_became_active":"2024-03-11T18:36:04.625084+0000","last_became_peered":"2024-03-11T18:36:04.625084+0000","last_unstale":"2024-03-11T20:51:50.732208+0000","last_undegraded":"2024-03-11T20:51:50.732208+0000","last_fullsized":"2024-03-11T20:51:50.732208+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'848","last_scrub_stamp":"2024-03-11T20:51:50.731995+0000","last_deep_scrub":"30'848","last_deep_scrub_stamp":"2024-03-11T20:51:50.731995+0000","last_clean_scrub_stamp":"2024-03-11T20:51:50.731995+0000","objects_scrubbed":0,"log_size":848,"log_dups_size":0,"ondisk_log_size":848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:31:26.443960+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":136,"num_read_kb":82099,"num_write":832,"num_write_kb":151378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"30'209","reported_seq":309,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:45.022737+0000","last_change":"2024-03-11T20:51:45.022797+0000","last_active":"2024-03-11T20:51:45.022737+0000","last_peered":"2024-03-11T20:51:45.022737+0000","last_clean":"2024-03-11T20:51:45.022737+0000","last_became_active":"2024-03-11T18:35:57.681098+0000","last_became_peered":"2024-03-11T18:35:57.681098+0000","last_unstale":"2024-03-11T20:51:45.022737+0000","last_undegraded":"2024-03-11T20:51:45.022737+0000","last_fullsized":"2024-03-11T20:51:45.022737+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'209","last_scrub_stamp":"2024-03-11T20:51:45.022563+0000","last_deep_scrub":"30'209","last_deep_scrub_stamp":"2024-03-11T20:51:45.022563+0000","last_clean_scrub_stamp":"2024-03-11T20:51:45.022563+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:48:10.282950+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"30'488","reported_seq":590,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:43.173760+0000","last_change":"2024-03-11T20:51:43.173812+0000","last_active":"2024-03-11T20:51:43.173760+0000","last_peered":"2024-03-11T20:51:43.173760+0000","last_clean":"2024-03-11T20:51:43.173760+0000","last_became_active":"2024-03-11T18:35:56.290006+0000","last_became_peered":"2024-03-11T18:35:56.290006+0000","last_unstale":"2024-03-11T20:51:43.173760+0000","last_undegraded":"2024-03-11T20:51:43.173760+0000","last_fullsized":"2024-03-11T20:51:43.173760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'488","last_scrub_stamp":"2024-03-11T20:51:43.173661+0000","last_deep_scrub":"30'488","last_deep_scrub_stamp":"2024-03-11T20:51:43.173661+0000","last_clean_scrub_stamp":"2024-03-11T20:51:43.173661+0000","objects_scrubbed":5,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:25:36.710996+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194394,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":433,"num_write_kb":4314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"30'749","reported_seq":904,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:54.153547+0000","last_change":"2024-03-11T20:51:54.153611+0000","last_active":"2024-03-11T20:51:54.153547+0000","last_peered":"2024-03-11T20:51:54.153547+0000","last_clean":"2024-03-11T20:51:54.153547+0000","last_became_active":"2024-03-11T18:36:04.620282+0000","last_became_peered":"2024-03-11T18:36:04.620282+0000","last_unstale":"2024-03-11T20:51:54.153547+0000","last_undegraded":"2024-03-11T20:51:54.153547+0000","last_fullsized":"2024-03-11T20:51:54.153547+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'749","last_scrub_stamp":"2024-03-11T20:51:54.153357+0000","last_deep_scrub":"30'749","last_deep_scrub_stamp":"2024-03-11T20:51:54.153357+0000","last_clean_scrub_stamp":"2024-03-11T20:51:54.153357+0000","objects_scrubbed":0,"log_size":749,"log_dups_size":0,"ondisk_log_size":749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:25:47.118957+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":68,"num_read_kb":34102,"num_write":731,"num_write_kb":109760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"30'224","reported_seq":324,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:48.762277+0000","last_change":"2024-03-11T20:51:48.762340+0000","last_active":"2024-03-11T20:51:48.762277+0000","last_peered":"2024-03-11T20:51:48.762277+0000","last_clean":"2024-03-11T20:51:48.762277+0000","last_became_active":"2024-03-11T18:35:57.614885+0000","last_became_peered":"2024-03-11T18:35:57.614885+0000","last_unstale":"2024-03-11T20:51:48.762277+0000","last_undegraded":"2024-03-11T20:51:48.762277+0000","last_fullsized":"2024-03-11T20:51:48.762277+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'224","last_scrub_stamp":"2024-03-11T20:51:48.762107+0000","last_deep_scrub":"30'224","last_deep_scrub_stamp":"2024-03-11T20:51:48.762107+0000","last_clean_scrub_stamp":"2024-03-11T20:51:48.762107+0000","objects_scrubbed":0,"log_size":224,"log_dups_size":0,"ondisk_log_size":224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:29:50.976139+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"30'350","reported_seq":452,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:51.554461+0000","last_change":"2024-03-11T20:51:51.554602+0000","last_active":"2024-03-11T20:51:51.554461+0000","last_peered":"2024-03-11T20:51:51.554461+0000","last_clean":"2024-03-11T20:51:51.554461+0000","last_became_active":"2024-03-11T18:35:56.301698+0000","last_became_peered":"2024-03-11T18:35:56.301698+0000","last_unstale":"2024-03-11T20:51:51.554461+0000","last_undegraded":"2024-03-11T20:51:51.554461+0000","last_fullsized":"2024-03-11T20:51:51.554461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'350","last_scrub_stamp":"2024-03-11T20:51:51.554176+0000","last_deep_scrub":"30'350","last_deep_scrub_stamp":"2024-03-11T20:51:51.554176+0000","last_clean_scrub_stamp":"2024-03-11T20:51:51.554176+0000","objects_scrubbed":2,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:07:33.873459+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":289,"num_write_kb":4232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"30'714","reported_seq":916,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:57.676264+0000","last_change":"2024-03-11T20:51:57.676330+0000","last_active":"2024-03-11T20:51:57.676264+0000","last_peered":"2024-03-11T20:51:57.676264+0000","last_clean":"2024-03-11T20:51:57.676264+0000","last_became_active":"2024-03-11T18:37:04.271646+0000","last_became_peered":"2024-03-11T18:37:04.271646+0000","last_unstale":"2024-03-11T20:51:57.676264+0000","last_undegraded":"2024-03-11T20:51:57.676264+0000","last_fullsized":"2024-03-11T20:51:57.676264+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'714","last_scrub_stamp":"2024-03-11T20:51:57.676068+0000","last_deep_scrub":"30'714","last_deep_scrub_stamp":"2024-03-11T20:51:57.676068+0000","last_clean_scrub_stamp":"2024-03-11T20:51:57.676068+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:06:58.880647+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":108,"num_read_kb":62192,"num_write":690,"num_write_kb":100980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,2],"acting":[4,3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"30'216","reported_seq":316,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:44.451539+0000","last_change":"2024-03-11T20:51:44.451565+0000","last_active":"2024-03-11T20:51:44.451539+0000","last_peered":"2024-03-11T20:51:44.451539+0000","last_clean":"2024-03-11T20:51:44.451539+0000","last_became_active":"2024-03-11T18:35:57.506038+0000","last_became_peered":"2024-03-11T18:35:57.506038+0000","last_unstale":"2024-03-11T20:51:44.451539+0000","last_undegraded":"2024-03-11T20:51:44.451539+0000","last_fullsized":"2024-03-11T20:51:44.451539+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'216","last_scrub_stamp":"2024-03-11T20:51:44.451464+0000","last_deep_scrub":"30'216","last_deep_scrub_stamp":"2024-03-11T20:51:44.451464+0000","last_clean_scrub_stamp":"2024-03-11T20:51:44.451464+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:38:19.242937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"30'418","reported_seq":520,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:50.583247+0000","last_change":"2024-03-11T20:51:50.583416+0000","last_active":"2024-03-11T20:51:50.583247+0000","last_peered":"2024-03-11T20:51:50.583247+0000","last_clean":"2024-03-11T20:51:50.583247+0000","last_became_active":"2024-03-11T18:35:56.363731+0000","last_became_peered":"2024-03-11T18:35:56.363731+0000","last_unstale":"2024-03-11T20:51:50.583247+0000","last_undegraded":"2024-03-11T20:51:50.583247+0000","last_fullsized":"2024-03-11T20:51:50.583247+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'418","last_scrub_stamp":"2024-03-11T20:51:50.582955+0000","last_deep_scrub":"30'418","last_deep_scrub_stamp":"2024-03-11T20:51:50.582955+0000","last_clean_scrub_stamp":"2024-03-11T20:51:50.582955+0000","objects_scrubbed":5,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:52:35.214985+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":342,"num_write_kb":4261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"30'829","reported_seq":1065,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:55.674015+0000","last_change":"2024-03-11T20:51:55.674169+0000","last_active":"2024-03-11T20:51:55.674015+0000","last_peered":"2024-03-11T20:51:55.674015+0000","last_clean":"2024-03-11T20:51:55.674015+0000","last_became_active":"2024-03-11T18:36:04.642133+0000","last_became_peered":"2024-03-11T18:36:04.642133+0000","last_unstale":"2024-03-11T20:51:55.674015+0000","last_undegraded":"2024-03-11T20:51:55.674015+0000","last_fullsized":"2024-03-11T20:51:55.674015+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'829","last_scrub_stamp":"2024-03-11T20:51:55.673745+0000","last_deep_scrub":"30'829","last_deep_scrub_stamp":"2024-03-11T20:51:55.673745+0000","last_clean_scrub_stamp":"2024-03-11T20:51:55.673745+0000","objects_scrubbed":0,"log_size":829,"log_dups_size":0,"ondisk_log_size":829,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:56:42.472629+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":144,"num_read_kb":78357,"num_write":819,"num_write_kb":238245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"30'168","reported_seq":268,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:51.111346+0000","last_change":"2024-03-11T20:51:51.111407+0000","last_active":"2024-03-11T20:51:51.111346+0000","last_peered":"2024-03-11T20:51:51.111346+0000","last_clean":"2024-03-11T20:51:51.111346+0000","last_became_active":"2024-03-11T18:35:57.674472+0000","last_became_peered":"2024-03-11T18:35:57.674472+0000","last_unstale":"2024-03-11T20:51:51.111346+0000","last_undegraded":"2024-03-11T20:51:51.111346+0000","last_fullsized":"2024-03-11T20:51:51.111346+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'168","last_scrub_stamp":"2024-03-11T20:51:51.111158+0000","last_deep_scrub":"30'168","last_deep_scrub_stamp":"2024-03-11T20:51:51.111158+0000","last_clean_scrub_stamp":"2024-03-11T20:51:51.111158+0000","objects_scrubbed":0,"log_size":168,"log_dups_size":0,"ondisk_log_size":168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:25:44.076758+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"30'6437","reported_seq":6539,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:47.177892+0000","last_change":"2024-03-11T20:51:47.177930+0000","last_active":"2024-03-11T20:51:47.177892+0000","last_peered":"2024-03-11T20:51:47.177892+0000","last_clean":"2024-03-11T20:51:47.177892+0000","last_became_active":"2024-03-11T18:35:56.348040+0000","last_became_peered":"2024-03-11T18:35:56.348040+0000","last_unstale":"2024-03-11T20:51:47.177892+0000","last_undegraded":"2024-03-11T20:51:47.177892+0000","last_fullsized":"2024-03-11T20:51:47.177892+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'6437","last_scrub_stamp":"2024-03-11T20:51:47.177815+0000","last_deep_scrub":"30'6437","last_deep_scrub_stamp":"2024-03-11T20:51:47.177815+0000","last_clean_scrub_stamp":"2024-03-11T20:51:47.177815+0000","objects_scrubbed":4,"log_size":6437,"log_dups_size":0,"ondisk_log_size":6437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:05:42.070866+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17421,"num_write_kb":6249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"30'799","reported_seq":1004,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:52:00.075200+0000","last_change":"2024-03-11T20:52:00.075263+0000","last_active":"2024-03-11T20:52:00.075200+0000","last_peered":"2024-03-11T20:52:00.075200+0000","last_clean":"2024-03-11T20:52:00.075200+0000","last_became_active":"2024-03-11T18:36:04.599065+0000","last_became_peered":"2024-03-11T18:36:04.599065+0000","last_unstale":"2024-03-11T20:52:00.075200+0000","last_undegraded":"2024-03-11T20:52:00.075200+0000","last_fullsized":"2024-03-11T20:52:00.075200+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'799","last_scrub_stamp":"2024-03-11T20:52:00.075008+0000","last_deep_scrub":"30'799","last_deep_scrub_stamp":"2024-03-11T20:52:00.075008+0000","last_clean_scrub_stamp":"2024-03-11T20:52:00.075008+0000","objects_scrubbed":0,"log_size":799,"log_dups_size":0,"ondisk_log_size":799,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:08:05.596802+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":111,"num_read_kb":58643,"num_write":786,"num_write_kb":122786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"30'199","reported_seq":302,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:50.164697+0000","last_change":"2024-03-11T20:51:50.164759+0000","last_active":"2024-03-11T20:51:50.164697+0000","last_peered":"2024-03-11T20:51:50.164697+0000","last_clean":"2024-03-11T20:51:50.164697+0000","last_became_active":"2024-03-11T18:35:57.537706+0000","last_became_peered":"2024-03-11T18:35:57.537706+0000","last_unstale":"2024-03-11T20:51:50.164697+0000","last_undegraded":"2024-03-11T20:51:50.164697+0000","last_fullsized":"2024-03-11T20:51:50.164697+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'199","last_scrub_stamp":"2024-03-11T20:51:50.164532+0000","last_deep_scrub":"30'199","last_deep_scrub_stamp":"2024-03-11T20:51:50.164532+0000","last_clean_scrub_stamp":"2024-03-11T20:51:50.164532+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:38:32.187059+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"30'990","reported_seq":1095,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:50.588011+0000","last_change":"2024-03-11T20:51:50.588011+0000","last_active":"2024-03-11T20:51:50.588011+0000","last_peered":"2024-03-11T20:51:50.588011+0000","last_clean":"2024-03-11T20:51:50.588011+0000","last_became_active":"2024-03-11T18:35:56.301819+0000","last_became_peered":"2024-03-11T18:35:56.301819+0000","last_unstale":"2024-03-11T20:51:50.588011+0000","last_undegraded":"2024-03-11T20:51:50.588011+0000","last_fullsized":"2024-03-11T20:51:50.588011+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'990","last_scrub_stamp":"2024-03-11T20:51:50.587655+0000","last_deep_scrub":"30'990","last_deep_scrub_stamp":"2024-03-11T20:51:50.587655+0000","last_clean_scrub_stamp":"2024-03-11T20:51:50.587655+0000","objects_scrubbed":5,"log_size":990,"log_dups_size":0,"ondisk_log_size":990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:48:15.848042+0000","scrub_duration":43,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":935,"num_write_kb":8676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"30'751","reported_seq":922,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:53.342590+0000","last_change":"2024-03-11T20:51:53.342658+0000","last_active":"2024-03-11T20:51:53.342590+0000","last_peered":"2024-03-11T20:51:53.342590+0000","last_clean":"2024-03-11T20:51:53.342590+0000","last_became_active":"2024-03-11T18:36:04.608260+0000","last_became_peered":"2024-03-11T18:36:04.608260+0000","last_unstale":"2024-03-11T20:51:53.342590+0000","last_undegraded":"2024-03-11T20:51:53.342590+0000","last_fullsized":"2024-03-11T20:51:53.342590+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'751","last_scrub_stamp":"2024-03-11T20:51:53.342402+0000","last_deep_scrub":"30'751","last_deep_scrub_stamp":"2024-03-11T20:51:53.342402+0000","last_clean_scrub_stamp":"2024-03-11T20:51:53.342402+0000","objects_scrubbed":0,"log_size":751,"log_dups_size":0,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:12:12.260486+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":79,"num_read_kb":54998,"num_write":738,"num_write_kb":142673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"30'218","reported_seq":321,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:58.739624+0000","last_change":"2024-03-11T20:51:58.739624+0000","last_active":"2024-03-11T20:51:58.739624+0000","last_peered":"2024-03-11T20:51:58.739624+0000","last_clean":"2024-03-11T20:51:58.739624+0000","last_became_active":"2024-03-11T18:35:57.688209+0000","last_became_peered":"2024-03-11T18:35:57.688209+0000","last_unstale":"2024-03-11T20:51:58.739624+0000","last_undegraded":"2024-03-11T20:51:58.739624+0000","last_fullsized":"2024-03-11T20:51:58.739624+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'218","last_scrub_stamp":"2024-03-11T20:51:58.739340+0000","last_deep_scrub":"30'218","last_deep_scrub_stamp":"2024-03-11T20:51:58.739340+0000","last_clean_scrub_stamp":"2024-03-11T20:51:58.739340+0000","objects_scrubbed":0,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:53:46.789083+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"30'374","reported_seq":479,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:45.115684+0000","last_change":"2024-03-11T20:51:45.115773+0000","last_active":"2024-03-11T20:51:45.115684+0000","last_peered":"2024-03-11T20:51:45.115684+0000","last_clean":"2024-03-11T20:51:45.115684+0000","last_became_active":"2024-03-11T18:35:56.293276+0000","last_became_peered":"2024-03-11T18:35:56.293276+0000","last_unstale":"2024-03-11T20:51:45.115684+0000","last_undegraded":"2024-03-11T20:51:45.115684+0000","last_fullsized":"2024-03-11T20:51:45.115684+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'374","last_scrub_stamp":"2024-03-11T20:51:45.115571+0000","last_deep_scrub":"30'374","last_deep_scrub_stamp":"2024-03-11T20:51:45.115571+0000","last_clean_scrub_stamp":"2024-03-11T20:51:45.115571+0000","objects_scrubbed":1,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:52:29.747602+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":309,"num_write_kb":4248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"30'742","reported_seq":890,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:56.181423+0000","last_change":"2024-03-11T20:51:56.181485+0000","last_active":"2024-03-11T20:51:56.181423+0000","last_peered":"2024-03-11T20:51:56.181423+0000","last_clean":"2024-03-11T20:51:56.181423+0000","last_became_active":"2024-03-11T18:36:04.625663+0000","last_became_peered":"2024-03-11T18:36:04.625663+0000","last_unstale":"2024-03-11T20:51:56.181423+0000","last_undegraded":"2024-03-11T20:51:56.181423+0000","last_fullsized":"2024-03-11T20:51:56.181423+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'742","last_scrub_stamp":"2024-03-11T20:51:56.181242+0000","last_deep_scrub":"30'742","last_deep_scrub_stamp":"2024-03-11T20:51:56.181242+0000","last_clean_scrub_stamp":"2024-03-11T20:51:56.181242+0000","objects_scrubbed":0,"log_size":742,"log_dups_size":0,"ondisk_log_size":742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:31:31.226996+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":56,"num_read_kb":20684,"num_write":725,"num_write_kb":55759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"30'199","reported_seq":302,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:46.668384+0000","last_change":"2024-03-11T20:51:46.668442+0000","last_active":"2024-03-11T20:51:46.668384+0000","last_peered":"2024-03-11T20:51:46.668384+0000","last_clean":"2024-03-11T20:51:46.668384+0000","last_became_active":"2024-03-11T18:35:57.611509+0000","last_became_peered":"2024-03-11T18:35:57.611509+0000","last_unstale":"2024-03-11T20:51:46.668384+0000","last_undegraded":"2024-03-11T20:51:46.668384+0000","last_fullsized":"2024-03-11T20:51:46.668384+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'199","last_scrub_stamp":"2024-03-11T20:51:46.668213+0000","last_deep_scrub":"30'199","last_deep_scrub_stamp":"2024-03-11T20:51:46.668213+0000","last_clean_scrub_stamp":"2024-03-11T20:51:46.668213+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:15:59.701206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"30'1194","reported_seq":1299,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:50.626823+0000","last_change":"2024-03-11T20:51:50.626902+0000","last_active":"2024-03-11T20:51:50.626823+0000","last_peered":"2024-03-11T20:51:50.626823+0000","last_clean":"2024-03-11T20:51:50.626823+0000","last_became_active":"2024-03-11T18:35:56.297747+0000","last_became_peered":"2024-03-11T18:35:56.297747+0000","last_unstale":"2024-03-11T20:51:50.626823+0000","last_undegraded":"2024-03-11T20:51:50.626823+0000","last_fullsized":"2024-03-11T20:51:50.626823+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'1194","last_scrub_stamp":"2024-03-11T20:51:50.626619+0000","last_deep_scrub":"30'1194","last_deep_scrub_stamp":"2024-03-11T20:51:50.626619+0000","last_clean_scrub_stamp":"2024-03-11T20:51:50.626619+0000","objects_scrubbed":4,"log_size":1194,"log_dups_size":0,"ondisk_log_size":1194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:41:43.748854+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389196,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1123,"num_write_kb":8732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"30'749","reported_seq":972,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:57.180704+0000","last_change":"2024-03-11T20:51:57.180772+0000","last_active":"2024-03-11T20:51:57.180704+0000","last_peered":"2024-03-11T20:51:57.180704+0000","last_clean":"2024-03-11T20:51:57.180704+0000","last_became_active":"2024-03-11T18:36:04.635261+0000","last_became_peered":"2024-03-11T18:36:04.635261+0000","last_unstale":"2024-03-11T20:51:57.180704+0000","last_undegraded":"2024-03-11T20:51:57.180704+0000","last_fullsized":"2024-03-11T20:51:57.180704+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'749","last_scrub_stamp":"2024-03-11T20:51:57.180507+0000","last_deep_scrub":"30'749","last_deep_scrub_stamp":"2024-03-11T20:51:57.180507+0000","last_clean_scrub_stamp":"2024-03-11T20:51:57.180507+0000","objects_scrubbed":0,"log_size":749,"log_dups_size":0,"ondisk_log_size":749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:08:42.646766+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":131,"num_read_kb":65646,"num_write":733,"num_write_kb":179820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"30'211","reported_seq":316,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:49.149784+0000","last_change":"2024-03-11T20:51:49.149901+0000","last_active":"2024-03-11T20:51:49.149784+0000","last_peered":"2024-03-11T20:51:49.149784+0000","last_clean":"2024-03-11T20:51:49.149784+0000","last_became_active":"2024-03-11T18:35:57.537717+0000","last_became_peered":"2024-03-11T18:35:57.537717+0000","last_unstale":"2024-03-11T20:51:49.149784+0000","last_undegraded":"2024-03-11T20:51:49.149784+0000","last_fullsized":"2024-03-11T20:51:49.149784+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'211","last_scrub_stamp":"2024-03-11T20:51:49.149491+0000","last_deep_scrub":"30'211","last_deep_scrub_stamp":"2024-03-11T20:51:49.149491+0000","last_clean_scrub_stamp":"2024-03-11T20:51:49.149491+0000","objects_scrubbed":0,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:56:35.948375+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"30'4764","reported_seq":9566,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:44.821888+0000","last_change":"2024-03-11T20:51:44.821936+0000","last_active":"2024-03-11T20:51:44.821888+0000","last_peered":"2024-03-11T20:51:44.821888+0000","last_clean":"2024-03-11T20:51:44.821888+0000","last_became_active":"2024-03-11T18:35:56.294096+0000","last_became_peered":"2024-03-11T18:35:56.294096+0000","last_unstale":"2024-03-11T20:51:44.821888+0000","last_undegraded":"2024-03-11T20:51:44.821888+0000","last_fullsized":"2024-03-11T20:51:44.821888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'4764","last_scrub_stamp":"2024-03-11T20:51:44.821783+0000","last_deep_scrub":"30'4764","last_deep_scrub_stamp":"2024-03-11T20:51:44.821783+0000","last_clean_scrub_stamp":"2024-03-11T20:51:44.821783+0000","objects_scrubbed":4,"log_size":4764,"log_dups_size":0,"ondisk_log_size":4764,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:27:59.083928+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1555811,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4697,"num_read_kb":5145,"num_write":4704,"num_write_kb":5149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"30'682","reported_seq":811,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:55.174822+0000","last_change":"2024-03-11T20:51:55.174873+0000","last_active":"2024-03-11T20:51:55.174822+0000","last_peered":"2024-03-11T20:51:55.174822+0000","last_clean":"2024-03-11T20:51:55.174822+0000","last_became_active":"2024-03-11T18:36:04.620356+0000","last_became_peered":"2024-03-11T18:36:04.620356+0000","last_unstale":"2024-03-11T20:51:55.174822+0000","last_undegraded":"2024-03-11T20:51:55.174822+0000","last_fullsized":"2024-03-11T20:51:55.174822+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'682","last_scrub_stamp":"2024-03-11T20:51:55.174682+0000","last_deep_scrub":"30'682","last_deep_scrub_stamp":"2024-03-11T20:51:55.174682+0000","last_clean_scrub_stamp":"2024-03-11T20:51:55.174682+0000","objects_scrubbed":0,"log_size":682,"log_dups_size":0,"ondisk_log_size":682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:42:30.838851+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":17597,"num_write":672,"num_write_kb":47486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"30'198","reported_seq":301,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:47.443092+0000","last_change":"2024-03-11T20:51:47.443163+0000","last_active":"2024-03-11T20:51:47.443092+0000","last_peered":"2024-03-11T20:51:47.443092+0000","last_clean":"2024-03-11T20:51:47.443092+0000","last_became_active":"2024-03-11T18:35:57.602060+0000","last_became_peered":"2024-03-11T18:35:57.602060+0000","last_unstale":"2024-03-11T20:51:47.443092+0000","last_undegraded":"2024-03-11T20:51:47.443092+0000","last_fullsized":"2024-03-11T20:51:47.443092+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'198","last_scrub_stamp":"2024-03-11T20:51:47.442888+0000","last_deep_scrub":"30'198","last_deep_scrub_stamp":"2024-03-11T20:51:47.442888+0000","last_clean_scrub_stamp":"2024-03-11T20:51:47.442888+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:08:25.755264+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"30'1025","reported_seq":1132,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:53.595000+0000","last_change":"2024-03-11T20:51:53.595072+0000","last_active":"2024-03-11T20:51:53.595000+0000","last_peered":"2024-03-11T20:51:53.595000+0000","last_clean":"2024-03-11T20:51:53.595000+0000","last_became_active":"2024-03-11T18:35:56.303063+0000","last_became_peered":"2024-03-11T18:35:56.303063+0000","last_unstale":"2024-03-11T20:51:53.595000+0000","last_undegraded":"2024-03-11T20:51:53.595000+0000","last_fullsized":"2024-03-11T20:51:53.595000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'1025","last_scrub_stamp":"2024-03-11T20:51:53.594829+0000","last_deep_scrub":"30'1025","last_deep_scrub_stamp":"2024-03-11T20:51:53.594829+0000","last_clean_scrub_stamp":"2024-03-11T20:51:53.594829+0000","objects_scrubbed":4,"log_size":1025,"log_dups_size":0,"ondisk_log_size":1025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:06:54.799408+0000","scrub_duration":55,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":963,"num_write_kb":12732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"30'852","reported_seq":1026,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:55.331571+0000","last_change":"2024-03-11T20:51:55.331701+0000","last_active":"2024-03-11T20:51:55.331571+0000","last_peered":"2024-03-11T20:51:55.331571+0000","last_clean":"2024-03-11T20:51:55.331571+0000","last_became_active":"2024-03-11T18:36:04.610765+0000","last_became_peered":"2024-03-11T18:36:04.610765+0000","last_unstale":"2024-03-11T20:51:55.331571+0000","last_undegraded":"2024-03-11T20:51:55.331571+0000","last_fullsized":"2024-03-11T20:51:55.331571+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'852","last_scrub_stamp":"2024-03-11T20:51:55.331350+0000","last_deep_scrub":"30'852","last_deep_scrub_stamp":"2024-03-11T20:51:55.331350+0000","last_clean_scrub_stamp":"2024-03-11T20:51:55.331350+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:01:35.861498+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":85,"num_read_kb":37203,"num_write":843,"num_write_kb":103063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"30'185","reported_seq":288,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:55.127804+0000","last_change":"2024-03-11T20:51:55.127885+0000","last_active":"2024-03-11T20:51:55.127804+0000","last_peered":"2024-03-11T20:51:55.127804+0000","last_clean":"2024-03-11T20:51:55.127804+0000","last_became_active":"2024-03-11T18:35:57.682653+0000","last_became_peered":"2024-03-11T18:35:57.682653+0000","last_unstale":"2024-03-11T20:51:55.127804+0000","last_undegraded":"2024-03-11T20:51:55.127804+0000","last_fullsized":"2024-03-11T20:51:55.127804+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'185","last_scrub_stamp":"2024-03-11T20:51:55.127596+0000","last_deep_scrub":"30'185","last_deep_scrub_stamp":"2024-03-11T20:51:55.127596+0000","last_clean_scrub_stamp":"2024-03-11T20:51:55.127596+0000","objects_scrubbed":0,"log_size":185,"log_dups_size":0,"ondisk_log_size":185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:29:00.938520+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"30'377","reported_seq":482,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:48.592195+0000","last_change":"2024-03-11T20:51:48.592273+0000","last_active":"2024-03-11T20:51:48.592195+0000","last_peered":"2024-03-11T20:51:48.592195+0000","last_clean":"2024-03-11T20:51:48.592195+0000","last_became_active":"2024-03-11T18:35:56.296705+0000","last_became_peered":"2024-03-11T18:35:56.296705+0000","last_unstale":"2024-03-11T20:51:48.592195+0000","last_undegraded":"2024-03-11T20:51:48.592195+0000","last_fullsized":"2024-03-11T20:51:48.592195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'377","last_scrub_stamp":"2024-03-11T20:51:48.592035+0000","last_deep_scrub":"30'377","last_deep_scrub_stamp":"2024-03-11T20:51:48.592035+0000","last_clean_scrub_stamp":"2024-03-11T20:51:48.592035+0000","objects_scrubbed":6,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:06:49.796614+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194372,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":305,"num_write_kb":4243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"30'119","reported_seq":265,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:48.521358+0000","last_change":"2024-03-11T20:51:48.521358+0000","last_active":"2024-03-11T20:51:48.521358+0000","last_peered":"2024-03-11T20:51:48.521358+0000","last_clean":"2024-03-11T20:51:48.521358+0000","last_became_active":"2024-03-11T18:35:18.197797+0000","last_became_peered":"2024-03-11T18:35:18.197797+0000","last_unstale":"2024-03-11T20:51:48.521358+0000","last_undegraded":"2024-03-11T20:51:48.521358+0000","last_fullsized":"2024-03-11T20:51:48.521358+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'119","last_scrub_stamp":"2024-03-11T20:51:48.521103+0000","last_deep_scrub":"30'119","last_deep_scrub_stamp":"2024-03-11T20:51:48.521103+0000","last_clean_scrub_stamp":"2024-03-11T20:51:48.521103+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:15:55.251354+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"30'785","reported_seq":962,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:58.550708+0000","last_change":"2024-03-11T20:51:58.550817+0000","last_active":"2024-03-11T20:51:58.550708+0000","last_peered":"2024-03-11T20:51:58.550708+0000","last_clean":"2024-03-11T20:51:58.550708+0000","last_became_active":"2024-03-11T18:36:04.598777+0000","last_became_peered":"2024-03-11T18:36:04.598777+0000","last_unstale":"2024-03-11T20:51:58.550708+0000","last_undegraded":"2024-03-11T20:51:58.550708+0000","last_fullsized":"2024-03-11T20:51:58.550708+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'785","last_scrub_stamp":"2024-03-11T20:51:58.550530+0000","last_deep_scrub":"30'785","last_deep_scrub_stamp":"2024-03-11T20:51:58.550530+0000","last_clean_scrub_stamp":"2024-03-11T20:51:58.550530+0000","objects_scrubbed":0,"log_size":785,"log_dups_size":0,"ondisk_log_size":785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:25:51.516130+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":86,"num_read_kb":36797,"num_write":762,"num_write_kb":114420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"30'207","reported_seq":286,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:49.859233+0000","last_change":"2024-03-11T20:51:49.859233+0000","last_active":"2024-03-11T20:51:49.859233+0000","last_peered":"2024-03-11T20:51:49.859233+0000","last_clean":"2024-03-11T20:51:49.859233+0000","last_became_active":"2024-03-11T18:37:04.141663+0000","last_became_peered":"2024-03-11T18:37:04.141663+0000","last_unstale":"2024-03-11T20:51:49.859233+0000","last_undegraded":"2024-03-11T20:51:49.859233+0000","last_fullsized":"2024-03-11T20:51:49.859233+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'207","last_scrub_stamp":"2024-03-11T20:51:49.858997+0000","last_deep_scrub":"30'207","last_deep_scrub_stamp":"2024-03-11T20:51:49.858997+0000","last_clean_scrub_stamp":"2024-03-11T20:51:49.858997+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:33:30.872533+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"30'289","reported_seq":389,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:45.834207+0000","last_change":"2024-03-11T20:51:45.834271+0000","last_active":"2024-03-11T20:51:45.834207+0000","last_peered":"2024-03-11T20:51:45.834207+0000","last_clean":"2024-03-11T20:51:45.834207+0000","last_became_active":"2024-03-11T18:35:56.293961+0000","last_became_peered":"2024-03-11T18:35:56.293961+0000","last_unstale":"2024-03-11T20:51:45.834207+0000","last_undegraded":"2024-03-11T20:51:45.834207+0000","last_fullsized":"2024-03-11T20:51:45.834207+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'289","last_scrub_stamp":"2024-03-11T20:51:45.834055+0000","last_deep_scrub":"30'289","last_deep_scrub_stamp":"2024-03-11T20:51:45.834055+0000","last_clean_scrub_stamp":"2024-03-11T20:51:45.834055+0000","objects_scrubbed":4,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:08:44.795933+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4196056,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":223,"num_write_kb":4226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"30'822","reported_seq":996,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:52:00.554874+0000","last_change":"2024-03-11T20:52:00.555006+0000","last_active":"2024-03-11T20:52:00.554874+0000","last_peered":"2024-03-11T20:52:00.554874+0000","last_clean":"2024-03-11T20:52:00.554874+0000","last_became_active":"2024-03-11T18:36:04.618908+0000","last_became_peered":"2024-03-11T18:36:04.618908+0000","last_unstale":"2024-03-11T20:52:00.554874+0000","last_undegraded":"2024-03-11T20:52:00.554874+0000","last_fullsized":"2024-03-11T20:52:00.554874+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'822","last_scrub_stamp":"2024-03-11T20:52:00.554587+0000","last_deep_scrub":"30'822","last_deep_scrub_stamp":"2024-03-11T20:52:00.554587+0000","last_clean_scrub_stamp":"2024-03-11T20:52:00.554587+0000","objects_scrubbed":0,"log_size":822,"log_dups_size":0,"ondisk_log_size":822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:31:35.600341+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":85,"num_read_kb":42087,"num_write":807,"num_write_kb":152305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"30'221","reported_seq":319,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:57.717461+0000","last_change":"2024-03-11T20:51:57.717518+0000","last_active":"2024-03-11T20:51:57.717461+0000","last_peered":"2024-03-11T20:51:57.717461+0000","last_clean":"2024-03-11T20:51:57.717461+0000","last_became_active":"2024-03-11T18:35:57.685828+0000","last_became_peered":"2024-03-11T18:35:57.685828+0000","last_unstale":"2024-03-11T20:51:57.717461+0000","last_undegraded":"2024-03-11T20:51:57.717461+0000","last_fullsized":"2024-03-11T20:51:57.717461+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'221","last_scrub_stamp":"2024-03-11T20:51:57.717298+0000","last_deep_scrub":"30'221","last_deep_scrub_stamp":"2024-03-11T20:51:57.717298+0000","last_clean_scrub_stamp":"2024-03-11T20:51:57.717298+0000","objects_scrubbed":0,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:06:51.179842+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"30'408","reported_seq":508,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:46.107531+0000","last_change":"2024-03-11T20:51:46.107582+0000","last_active":"2024-03-11T20:51:46.107531+0000","last_peered":"2024-03-11T20:51:46.107531+0000","last_clean":"2024-03-11T20:51:46.107531+0000","last_became_active":"2024-03-11T18:35:56.295123+0000","last_became_peered":"2024-03-11T18:35:56.295123+0000","last_unstale":"2024-03-11T20:51:46.107531+0000","last_undegraded":"2024-03-11T20:51:46.107531+0000","last_fullsized":"2024-03-11T20:51:46.107531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'408","last_scrub_stamp":"2024-03-11T20:51:46.107427+0000","last_deep_scrub":"30'408","last_deep_scrub_stamp":"2024-03-11T20:51:46.107427+0000","last_clean_scrub_stamp":"2024-03-11T20:51:46.107427+0000","objects_scrubbed":2,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:29:26.738334+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":350,"num_write_kb":4273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"30'821","reported_seq":1030,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:54.336037+0000","last_change":"2024-03-11T20:51:54.336163+0000","last_active":"2024-03-11T20:51:54.336037+0000","last_peered":"2024-03-11T20:51:54.336037+0000","last_clean":"2024-03-11T20:51:54.336037+0000","last_became_active":"2024-03-11T18:36:04.616338+0000","last_became_peered":"2024-03-11T18:36:04.616338+0000","last_unstale":"2024-03-11T20:51:54.336037+0000","last_undegraded":"2024-03-11T20:51:54.336037+0000","last_fullsized":"2024-03-11T20:51:54.336037+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'821","last_scrub_stamp":"2024-03-11T20:51:54.335794+0000","last_deep_scrub":"30'821","last_deep_scrub_stamp":"2024-03-11T20:51:54.335794+0000","last_clean_scrub_stamp":"2024-03-11T20:51:54.335794+0000","objects_scrubbed":0,"log_size":821,"log_dups_size":0,"ondisk_log_size":821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:58:59.450767+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":122,"num_read_kb":70346,"num_write":827,"num_write_kb":188969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"30'199","reported_seq":297,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:52:00.545256+0000","last_change":"2024-03-11T20:52:00.545319+0000","last_active":"2024-03-11T20:52:00.545256+0000","last_peered":"2024-03-11T20:52:00.545256+0000","last_clean":"2024-03-11T20:52:00.545256+0000","last_became_active":"2024-03-11T18:35:57.689424+0000","last_became_peered":"2024-03-11T18:35:57.689424+0000","last_unstale":"2024-03-11T20:52:00.545256+0000","last_undegraded":"2024-03-11T20:52:00.545256+0000","last_fullsized":"2024-03-11T20:52:00.545256+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'199","last_scrub_stamp":"2024-03-11T20:52:00.545084+0000","last_deep_scrub":"30'199","last_deep_scrub_stamp":"2024-03-11T20:52:00.545084+0000","last_clean_scrub_stamp":"2024-03-11T20:52:00.545084+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:29:06.356007+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"30'864","reported_seq":964,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:43.780270+0000","last_change":"2024-03-11T20:51:43.780331+0000","last_active":"2024-03-11T20:51:43.780270+0000","last_peered":"2024-03-11T20:51:43.780270+0000","last_clean":"2024-03-11T20:51:43.780270+0000","last_became_active":"2024-03-11T18:35:56.293585+0000","last_became_peered":"2024-03-11T18:35:56.293585+0000","last_unstale":"2024-03-11T20:51:43.780270+0000","last_undegraded":"2024-03-11T20:51:43.780270+0000","last_fullsized":"2024-03-11T20:51:43.780270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'864","last_scrub_stamp":"2024-03-11T20:51:43.780155+0000","last_deep_scrub":"30'864","last_deep_scrub_stamp":"2024-03-11T20:51:43.780155+0000","last_clean_scrub_stamp":"2024-03-11T20:51:43.780155+0000","objects_scrubbed":3,"log_size":864,"log_dups_size":0,"ondisk_log_size":864,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:45:57.220940+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":804,"num_write_kb":4496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"30'804","reported_seq":991,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:52:02.658857+0000","last_change":"2024-03-11T20:52:02.658925+0000","last_active":"2024-03-11T20:52:02.658857+0000","last_peered":"2024-03-11T20:52:02.658857+0000","last_clean":"2024-03-11T20:52:02.658857+0000","last_became_active":"2024-03-11T18:36:04.626596+0000","last_became_peered":"2024-03-11T18:36:04.626596+0000","last_unstale":"2024-03-11T20:52:02.658857+0000","last_undegraded":"2024-03-11T20:52:02.658857+0000","last_fullsized":"2024-03-11T20:52:02.658857+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'804","last_scrub_stamp":"2024-03-11T20:52:02.658665+0000","last_deep_scrub":"30'804","last_deep_scrub_stamp":"2024-03-11T20:52:02.658665+0000","last_clean_scrub_stamp":"2024-03-11T20:52:02.658665+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:11:16.254755+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":98,"num_read_kb":46229,"num_write":782,"num_write_kb":114631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"30'184","reported_seq":282,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:52:01.557424+0000","last_change":"2024-03-11T20:52:01.557484+0000","last_active":"2024-03-11T20:52:01.557424+0000","last_peered":"2024-03-11T20:52:01.557424+0000","last_clean":"2024-03-11T20:52:01.557424+0000","last_became_active":"2024-03-11T18:35:57.690223+0000","last_became_peered":"2024-03-11T18:35:57.690223+0000","last_unstale":"2024-03-11T20:52:01.557424+0000","last_undegraded":"2024-03-11T20:52:01.557424+0000","last_fullsized":"2024-03-11T20:52:01.557424+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'184","last_scrub_stamp":"2024-03-11T20:52:01.557239+0000","last_deep_scrub":"30'184","last_deep_scrub_stamp":"2024-03-11T20:52:01.557239+0000","last_clean_scrub_stamp":"2024-03-11T20:52:01.557239+0000","objects_scrubbed":0,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:53:14.098434+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"30'818","reported_seq":918,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:44.730724+0000","last_change":"2024-03-11T20:51:44.730811+0000","last_active":"2024-03-11T20:51:44.730724+0000","last_peered":"2024-03-11T20:51:44.730724+0000","last_clean":"2024-03-11T20:51:44.730724+0000","last_became_active":"2024-03-11T18:35:56.256454+0000","last_became_peered":"2024-03-11T18:35:56.256454+0000","last_unstale":"2024-03-11T20:51:44.730724+0000","last_undegraded":"2024-03-11T20:51:44.730724+0000","last_fullsized":"2024-03-11T20:51:44.730724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'818","last_scrub_stamp":"2024-03-11T20:51:44.730537+0000","last_deep_scrub":"30'818","last_deep_scrub_stamp":"2024-03-11T20:51:44.730537+0000","last_clean_scrub_stamp":"2024-03-11T20:51:44.730537+0000","objects_scrubbed":3,"log_size":818,"log_dups_size":0,"ondisk_log_size":818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:30:31.543385+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":757,"num_write_kb":4472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"30'833","reported_seq":1029,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:56.710752+0000","last_change":"2024-03-11T20:51:56.710817+0000","last_active":"2024-03-11T20:51:56.710752+0000","last_peered":"2024-03-11T20:51:56.710752+0000","last_clean":"2024-03-11T20:51:56.710752+0000","last_became_active":"2024-03-11T18:36:04.642079+0000","last_became_peered":"2024-03-11T18:36:04.642079+0000","last_unstale":"2024-03-11T20:51:56.710752+0000","last_undegraded":"2024-03-11T20:51:56.710752+0000","last_fullsized":"2024-03-11T20:51:56.710752+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'833","last_scrub_stamp":"2024-03-11T20:51:56.710552+0000","last_deep_scrub":"30'833","last_deep_scrub_stamp":"2024-03-11T20:51:56.710552+0000","last_clean_scrub_stamp":"2024-03-11T20:51:56.710552+0000","objects_scrubbed":0,"log_size":833,"log_dups_size":0,"ondisk_log_size":833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:38:38.733079+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":107,"num_read_kb":63565,"num_write":823,"num_write_kb":172061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"30'239","reported_seq":335,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:47.481690+0000","last_change":"2024-03-11T20:51:47.481804+0000","last_active":"2024-03-11T20:51:47.481690+0000","last_peered":"2024-03-11T20:51:47.481690+0000","last_clean":"2024-03-11T20:51:47.481690+0000","last_became_active":"2024-03-11T18:35:57.686850+0000","last_became_peered":"2024-03-11T18:35:57.686850+0000","last_unstale":"2024-03-11T20:51:47.481690+0000","last_undegraded":"2024-03-11T20:51:47.481690+0000","last_fullsized":"2024-03-11T20:51:47.481690+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'239","last_scrub_stamp":"2024-03-11T20:51:47.481427+0000","last_deep_scrub":"30'239","last_deep_scrub_stamp":"2024-03-11T20:51:47.481427+0000","last_clean_scrub_stamp":"2024-03-11T20:51:47.481427+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:45:14.540953+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"30'909","reported_seq":1007,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:42.786561+0000","last_change":"2024-03-11T20:51:42.786622+0000","last_active":"2024-03-11T20:51:42.786561+0000","last_peered":"2024-03-11T20:51:42.786561+0000","last_clean":"2024-03-11T20:51:42.786561+0000","last_became_active":"2024-03-11T18:35:56.290944+0000","last_became_peered":"2024-03-11T18:35:56.290944+0000","last_unstale":"2024-03-11T20:51:42.786561+0000","last_undegraded":"2024-03-11T20:51:42.786561+0000","last_fullsized":"2024-03-11T20:51:42.786561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'909","last_scrub_stamp":"2024-03-11T20:51:42.786457+0000","last_deep_scrub":"30'909","last_deep_scrub_stamp":"2024-03-11T20:51:42.786457+0000","last_clean_scrub_stamp":"2024-03-11T20:51:42.786457+0000","objects_scrubbed":10,"log_size":909,"log_dups_size":0,"ondisk_log_size":909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:32:37.677873+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":846,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"30'791","reported_seq":977,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:51.381152+0000","last_change":"2024-03-11T20:51:51.381219+0000","last_active":"2024-03-11T20:51:51.381152+0000","last_peered":"2024-03-11T20:51:51.381152+0000","last_clean":"2024-03-11T20:51:51.381152+0000","last_became_active":"2024-03-11T18:36:04.601224+0000","last_became_peered":"2024-03-11T18:36:04.601224+0000","last_unstale":"2024-03-11T20:51:51.381152+0000","last_undegraded":"2024-03-11T20:51:51.381152+0000","last_fullsized":"2024-03-11T20:51:51.381152+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'791","last_scrub_stamp":"2024-03-11T20:51:51.380947+0000","last_deep_scrub":"30'791","last_deep_scrub_stamp":"2024-03-11T20:51:51.380947+0000","last_clean_scrub_stamp":"2024-03-11T20:51:51.380947+0000","objects_scrubbed":0,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:25:26.612727+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":59304,"num_write":781,"num_write_kb":152027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"30'219","reported_seq":315,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:51.554328+0000","last_change":"2024-03-11T20:51:51.554387+0000","last_active":"2024-03-11T20:51:51.554328+0000","last_peered":"2024-03-11T20:51:51.554328+0000","last_clean":"2024-03-11T20:51:51.554328+0000","last_became_active":"2024-03-11T18:35:57.610955+0000","last_became_peered":"2024-03-11T18:35:57.610955+0000","last_unstale":"2024-03-11T20:51:51.554328+0000","last_undegraded":"2024-03-11T20:51:51.554328+0000","last_fullsized":"2024-03-11T20:51:51.554328+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'219","last_scrub_stamp":"2024-03-11T20:51:51.554153+0000","last_deep_scrub":"30'219","last_deep_scrub_stamp":"2024-03-11T20:51:51.554153+0000","last_clean_scrub_stamp":"2024-03-11T20:51:51.554153+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:46:49.626948+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"30'62","reported_seq":160,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:49.609219+0000","last_change":"2024-03-11T20:51:49.609291+0000","last_active":"2024-03-11T20:51:49.609219+0000","last_peered":"2024-03-11T20:51:49.609219+0000","last_clean":"2024-03-11T20:51:49.609219+0000","last_became_active":"2024-03-11T18:35:56.361141+0000","last_became_peered":"2024-03-11T18:35:56.361141+0000","last_unstale":"2024-03-11T20:51:49.609219+0000","last_undegraded":"2024-03-11T20:51:49.609219+0000","last_fullsized":"2024-03-11T20:51:49.609219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'62","last_scrub_stamp":"2024-03-11T20:51:49.609012+0000","last_deep_scrub":"30'62","last_deep_scrub_stamp":"2024-03-11T20:51:49.609012+0000","last_clean_scrub_stamp":"2024-03-11T20:51:49.609012+0000","objects_scrubbed":3,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:31:25.320976+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"30'743","reported_seq":903,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:59.120938+0000","last_change":"2024-03-11T20:51:59.121084+0000","last_active":"2024-03-11T20:51:59.120938+0000","last_peered":"2024-03-11T20:51:59.120938+0000","last_clean":"2024-03-11T20:51:59.120938+0000","last_became_active":"2024-03-11T18:36:04.592304+0000","last_became_peered":"2024-03-11T18:36:04.592304+0000","last_unstale":"2024-03-11T20:51:59.120938+0000","last_undegraded":"2024-03-11T20:51:59.120938+0000","last_fullsized":"2024-03-11T20:51:59.120938+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'743","last_scrub_stamp":"2024-03-11T20:51:59.120717+0000","last_deep_scrub":"30'743","last_deep_scrub_stamp":"2024-03-11T20:51:59.120717+0000","last_clean_scrub_stamp":"2024-03-11T20:51:59.120717+0000","objects_scrubbed":0,"log_size":743,"log_dups_size":0,"ondisk_log_size":743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:56:09.076196+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":77,"num_read_kb":52017,"num_write":733,"num_write_kb":82372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"30'206","reported_seq":302,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:58.517187+0000","last_change":"2024-03-11T20:51:58.517302+0000","last_active":"2024-03-11T20:51:58.517187+0000","last_peered":"2024-03-11T20:51:58.517187+0000","last_clean":"2024-03-11T20:51:58.517187+0000","last_became_active":"2024-03-11T18:35:57.689139+0000","last_became_peered":"2024-03-11T18:35:57.689139+0000","last_unstale":"2024-03-11T20:51:58.517187+0000","last_undegraded":"2024-03-11T20:51:58.517187+0000","last_fullsized":"2024-03-11T20:51:58.517187+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'206","last_scrub_stamp":"2024-03-11T20:51:58.516868+0000","last_deep_scrub":"30'206","last_deep_scrub_stamp":"2024-03-11T20:51:58.516868+0000","last_clean_scrub_stamp":"2024-03-11T20:51:58.516868+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:31:33.562622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"30'63","reported_seq":161,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:49.552380+0000","last_change":"2024-03-11T20:51:49.552448+0000","last_active":"2024-03-11T20:51:49.552380+0000","last_peered":"2024-03-11T20:51:49.552380+0000","last_clean":"2024-03-11T20:51:49.552380+0000","last_became_active":"2024-03-11T18:35:56.296426+0000","last_became_peered":"2024-03-11T18:35:56.296426+0000","last_unstale":"2024-03-11T20:51:49.552380+0000","last_undegraded":"2024-03-11T20:51:49.552380+0000","last_fullsized":"2024-03-11T20:51:49.552380+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'63","last_scrub_stamp":"2024-03-11T20:51:49.552210+0000","last_deep_scrub":"30'63","last_deep_scrub_stamp":"2024-03-11T20:51:49.552210+0000","last_clean_scrub_stamp":"2024-03-11T20:51:49.552210+0000","objects_scrubbed":3,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:21:32.533098+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"30'724","reported_seq":932,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:58.071436+0000","last_change":"2024-03-11T20:51:58.071573+0000","last_active":"2024-03-11T20:51:58.071436+0000","last_peered":"2024-03-11T20:51:58.071436+0000","last_clean":"2024-03-11T20:51:58.071436+0000","last_became_active":"2024-03-11T18:36:04.590014+0000","last_became_peered":"2024-03-11T18:36:04.590014+0000","last_unstale":"2024-03-11T20:51:58.071436+0000","last_undegraded":"2024-03-11T20:51:58.071436+0000","last_fullsized":"2024-03-11T20:51:58.071436+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'724","last_scrub_stamp":"2024-03-11T20:51:58.071201+0000","last_deep_scrub":"30'724","last_deep_scrub_stamp":"2024-03-11T20:51:58.071201+0000","last_clean_scrub_stamp":"2024-03-11T20:51:58.071201+0000","objects_scrubbed":0,"log_size":724,"log_dups_size":0,"ondisk_log_size":724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:53:46.120944+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":124,"num_read_kb":69166,"num_write":714,"num_write_kb":127024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"30'208","reported_seq":304,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:40.519149+0000","last_change":"2024-03-11T20:51:40.519194+0000","last_active":"2024-03-11T20:51:40.519149+0000","last_peered":"2024-03-11T20:51:40.519149+0000","last_clean":"2024-03-11T20:51:40.519149+0000","last_became_active":"2024-03-11T18:35:57.670893+0000","last_became_peered":"2024-03-11T18:35:57.670893+0000","last_unstale":"2024-03-11T20:51:40.519149+0000","last_undegraded":"2024-03-11T20:51:40.519149+0000","last_fullsized":"2024-03-11T20:51:40.519149+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'208","last_scrub_stamp":"2024-03-11T20:51:40.519026+0000","last_deep_scrub":"30'208","last_deep_scrub_stamp":"2024-03-11T20:51:40.519026+0000","last_clean_scrub_stamp":"2024-03-11T20:51:40.519026+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:28:40.054929+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"30'600","reported_seq":698,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:41.519078+0000","last_change":"2024-03-11T20:51:41.519148+0000","last_active":"2024-03-11T20:51:41.519078+0000","last_peered":"2024-03-11T20:51:41.519078+0000","last_clean":"2024-03-11T20:51:41.519078+0000","last_became_active":"2024-03-11T18:35:56.251287+0000","last_became_peered":"2024-03-11T18:35:56.251287+0000","last_unstale":"2024-03-11T20:51:41.519078+0000","last_undegraded":"2024-03-11T20:51:41.519078+0000","last_fullsized":"2024-03-11T20:51:41.519078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'600","last_scrub_stamp":"2024-03-11T20:51:41.518947+0000","last_deep_scrub":"30'600","last_deep_scrub_stamp":"2024-03-11T20:51:41.518947+0000","last_clean_scrub_stamp":"2024-03-11T20:51:41.518947+0000","objects_scrubbed":8,"log_size":600,"log_dups_size":0,"ondisk_log_size":600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:41:08.769039+0000","scrub_duration":36,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388676,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":533,"num_write_kb":8445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"30'738","reported_seq":855,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:52:01.696262+0000","last_change":"2024-03-11T20:52:01.696327+0000","last_active":"2024-03-11T20:52:01.696262+0000","last_peered":"2024-03-11T20:52:01.696262+0000","last_clean":"2024-03-11T20:52:01.696262+0000","last_became_active":"2024-03-11T18:36:04.618911+0000","last_became_peered":"2024-03-11T18:36:04.618911+0000","last_unstale":"2024-03-11T20:52:01.696262+0000","last_undegraded":"2024-03-11T20:52:01.696262+0000","last_fullsized":"2024-03-11T20:52:01.696262+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'738","last_scrub_stamp":"2024-03-11T20:52:01.696081+0000","last_deep_scrub":"30'738","last_deep_scrub_stamp":"2024-03-11T20:52:01.696081+0000","last_clean_scrub_stamp":"2024-03-11T20:52:01.696081+0000","objects_scrubbed":0,"log_size":738,"log_dups_size":0,"ondisk_log_size":738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:48:41.422254+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":10131,"num_write":722,"num_write_kb":109498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"30'204","reported_seq":298,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:52:03.541233+0000","last_change":"2024-03-11T20:52:03.541295+0000","last_active":"2024-03-11T20:52:03.541233+0000","last_peered":"2024-03-11T20:52:03.541233+0000","last_clean":"2024-03-11T20:52:03.541233+0000","last_became_active":"2024-03-11T18:35:57.697263+0000","last_became_peered":"2024-03-11T18:35:57.697263+0000","last_unstale":"2024-03-11T20:52:03.541233+0000","last_undegraded":"2024-03-11T20:52:03.541233+0000","last_fullsized":"2024-03-11T20:52:03.541233+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'204","last_scrub_stamp":"2024-03-11T20:52:03.541042+0000","last_deep_scrub":"30'204","last_deep_scrub_stamp":"2024-03-11T20:52:03.541042+0000","last_clean_scrub_stamp":"2024-03-11T20:52:03.541042+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:53:51.590784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"30'2209","reported_seq":2307,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:46.633274+0000","last_change":"2024-03-11T20:51:46.633274+0000","last_active":"2024-03-11T20:51:46.633274+0000","last_peered":"2024-03-11T20:51:46.633274+0000","last_clean":"2024-03-11T20:51:46.633274+0000","last_became_active":"2024-03-11T18:35:56.354827+0000","last_became_peered":"2024-03-11T18:35:56.354827+0000","last_unstale":"2024-03-11T20:51:46.633274+0000","last_undegraded":"2024-03-11T20:51:46.633274+0000","last_fullsized":"2024-03-11T20:51:46.633274+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'2209","last_scrub_stamp":"2024-03-11T20:51:46.632881+0000","last_deep_scrub":"30'2209","last_deep_scrub_stamp":"2024-03-11T20:51:46.632881+0000","last_clean_scrub_stamp":"2024-03-11T20:51:46.632881+0000","objects_scrubbed":8,"log_size":2209,"log_dups_size":0,"ondisk_log_size":2209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:28:00.681266+0000","scrub_duration":59,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583584,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2157,"num_write_kb":13630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"30'771","reported_seq":933,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:52:03.618750+0000","last_change":"2024-03-11T20:52:03.618817+0000","last_active":"2024-03-11T20:52:03.618750+0000","last_peered":"2024-03-11T20:52:03.618750+0000","last_clean":"2024-03-11T20:52:03.618750+0000","last_became_active":"2024-03-11T18:37:04.259051+0000","last_became_peered":"2024-03-11T18:37:04.259051+0000","last_unstale":"2024-03-11T20:52:03.618750+0000","last_undegraded":"2024-03-11T20:52:03.618750+0000","last_fullsized":"2024-03-11T20:52:03.618750+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'771","last_scrub_stamp":"2024-03-11T20:52:03.618573+0000","last_deep_scrub":"30'771","last_deep_scrub_stamp":"2024-03-11T20:52:03.618573+0000","last_clean_scrub_stamp":"2024-03-11T20:52:03.618573+0000","objects_scrubbed":0,"log_size":771,"log_dups_size":0,"ondisk_log_size":771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:48:07.318672+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":84,"num_read_kb":37526,"num_write":755,"num_write_kb":85949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2,0],"acting":[5,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"30'201","reported_seq":297,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:49.755274+0000","last_change":"2024-03-11T20:51:49.755274+0000","last_active":"2024-03-11T20:51:49.755274+0000","last_peered":"2024-03-11T20:51:49.755274+0000","last_clean":"2024-03-11T20:51:49.755274+0000","last_became_active":"2024-03-11T18:35:57.678565+0000","last_became_peered":"2024-03-11T18:35:57.678565+0000","last_unstale":"2024-03-11T20:51:49.755274+0000","last_undegraded":"2024-03-11T20:51:49.755274+0000","last_fullsized":"2024-03-11T20:51:49.755274+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'201","last_scrub_stamp":"2024-03-11T20:51:49.754887+0000","last_deep_scrub":"30'201","last_deep_scrub_stamp":"2024-03-11T20:51:49.754887+0000","last_clean_scrub_stamp":"2024-03-11T20:51:49.754887+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:55:08.062806+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"30'3441","reported_seq":3537,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:42.219110+0000","last_change":"2024-03-11T20:51:42.219190+0000","last_active":"2024-03-11T20:51:42.219110+0000","last_peered":"2024-03-11T20:51:42.219110+0000","last_clean":"2024-03-11T20:51:42.219110+0000","last_became_active":"2024-03-11T18:35:56.287846+0000","last_became_peered":"2024-03-11T18:35:56.287846+0000","last_unstale":"2024-03-11T20:51:42.219110+0000","last_undegraded":"2024-03-11T20:51:42.219110+0000","last_fullsized":"2024-03-11T20:51:42.219110+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'3441","last_scrub_stamp":"2024-03-11T20:51:42.218945+0000","last_deep_scrub":"30'3441","last_deep_scrub_stamp":"2024-03-11T20:51:42.218945+0000","last_clean_scrub_stamp":"2024-03-11T20:51:42.218945+0000","objects_scrubbed":6,"log_size":3441,"log_dups_size":0,"ondisk_log_size":3441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:29:44.432977+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7567240,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3391,"num_write_kb":10463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"30'736","reported_seq":911,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:54.697451+0000","last_change":"2024-03-11T20:51:54.697519+0000","last_active":"2024-03-11T20:51:54.697451+0000","last_peered":"2024-03-11T20:51:54.697451+0000","last_clean":"2024-03-11T20:51:54.697451+0000","last_became_active":"2024-03-11T18:36:04.642374+0000","last_became_peered":"2024-03-11T18:36:04.642374+0000","last_unstale":"2024-03-11T20:51:54.697451+0000","last_undegraded":"2024-03-11T20:51:54.697451+0000","last_fullsized":"2024-03-11T20:51:54.697451+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'736","last_scrub_stamp":"2024-03-11T20:51:54.697269+0000","last_deep_scrub":"30'736","last_deep_scrub_stamp":"2024-03-11T20:51:54.697269+0000","last_clean_scrub_stamp":"2024-03-11T20:51:54.697269+0000","objects_scrubbed":0,"log_size":736,"log_dups_size":0,"ondisk_log_size":736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:48:19.957656+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":40504,"num_write":721,"num_write_kb":107102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"30'197","reported_seq":291,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:48.415266+0000","last_change":"2024-03-11T20:51:48.415351+0000","last_active":"2024-03-11T20:51:48.415266+0000","last_peered":"2024-03-11T20:51:48.415266+0000","last_clean":"2024-03-11T20:51:48.415266+0000","last_became_active":"2024-03-11T18:35:57.636093+0000","last_became_peered":"2024-03-11T18:35:57.636093+0000","last_unstale":"2024-03-11T20:51:48.415266+0000","last_undegraded":"2024-03-11T20:51:48.415266+0000","last_fullsized":"2024-03-11T20:51:48.415266+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'197","last_scrub_stamp":"2024-03-11T20:51:48.415020+0000","last_deep_scrub":"30'197","last_deep_scrub_stamp":"2024-03-11T20:51:48.415020+0000","last_clean_scrub_stamp":"2024-03-11T20:51:48.415020+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:14:54.064170+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"30'1242","reported_seq":1338,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:52.665259+0000","last_change":"2024-03-11T20:51:52.665329+0000","last_active":"2024-03-11T20:51:52.665259+0000","last_peered":"2024-03-11T20:51:52.665259+0000","last_clean":"2024-03-11T20:51:52.665259+0000","last_became_active":"2024-03-11T18:35:56.354759+0000","last_became_peered":"2024-03-11T18:35:56.354759+0000","last_unstale":"2024-03-11T20:51:52.665259+0000","last_undegraded":"2024-03-11T20:51:52.665259+0000","last_fullsized":"2024-03-11T20:51:52.665259+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'1242","last_scrub_stamp":"2024-03-11T20:51:52.665113+0000","last_deep_scrub":"30'1242","last_deep_scrub_stamp":"2024-03-11T20:51:52.665113+0000","last_clean_scrub_stamp":"2024-03-11T20:51:52.665113+0000","objects_scrubbed":4,"log_size":1242,"log_dups_size":0,"ondisk_log_size":1242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:42:28.329281+0000","scrub_duration":55,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1180,"num_write_kb":8784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"30'803","reported_seq":948,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:52:03.124231+0000","last_change":"2024-03-11T20:52:03.124295+0000","last_active":"2024-03-11T20:52:03.124231+0000","last_peered":"2024-03-11T20:52:03.124231+0000","last_clean":"2024-03-11T20:52:03.124231+0000","last_became_active":"2024-03-11T18:36:04.613159+0000","last_became_peered":"2024-03-11T18:36:04.613159+0000","last_unstale":"2024-03-11T20:52:03.124231+0000","last_undegraded":"2024-03-11T20:52:03.124231+0000","last_fullsized":"2024-03-11T20:52:03.124231+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'803","last_scrub_stamp":"2024-03-11T20:52:03.124043+0000","last_deep_scrub":"30'803","last_deep_scrub_stamp":"2024-03-11T20:52:03.124043+0000","last_clean_scrub_stamp":"2024-03-11T20:52:03.124043+0000","objects_scrubbed":0,"log_size":803,"log_dups_size":0,"ondisk_log_size":803,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:48:06.824143+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":38200,"num_write":794,"num_write_kb":117163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"30'218","reported_seq":312,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:56.716265+0000","last_change":"2024-03-11T20:51:56.716324+0000","last_active":"2024-03-11T20:51:56.716265+0000","last_peered":"2024-03-11T20:51:56.716265+0000","last_clean":"2024-03-11T20:51:56.716265+0000","last_became_active":"2024-03-11T18:35:57.683150+0000","last_became_peered":"2024-03-11T18:35:57.683150+0000","last_unstale":"2024-03-11T20:51:56.716265+0000","last_undegraded":"2024-03-11T20:51:56.716265+0000","last_fullsized":"2024-03-11T20:51:56.716265+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'218","last_scrub_stamp":"2024-03-11T20:51:56.716071+0000","last_deep_scrub":"30'218","last_deep_scrub_stamp":"2024-03-11T20:51:56.716071+0000","last_clean_scrub_stamp":"2024-03-11T20:51:56.716071+0000","objects_scrubbed":0,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:53:09.257266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"30'64","reported_seq":160,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:48.611819+0000","last_change":"2024-03-11T20:51:48.611884+0000","last_active":"2024-03-11T20:51:48.611819+0000","last_peered":"2024-03-11T20:51:48.611819+0000","last_clean":"2024-03-11T20:51:48.611819+0000","last_became_active":"2024-03-11T18:35:56.359629+0000","last_became_peered":"2024-03-11T18:35:56.359629+0000","last_unstale":"2024-03-11T20:51:48.611819+0000","last_undegraded":"2024-03-11T20:51:48.611819+0000","last_fullsized":"2024-03-11T20:51:48.611819+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'64","last_scrub_stamp":"2024-03-11T20:51:48.611718+0000","last_deep_scrub":"30'64","last_deep_scrub_stamp":"2024-03-11T20:51:48.611718+0000","last_clean_scrub_stamp":"2024-03-11T20:51:48.611718+0000","objects_scrubbed":3,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:55:06.919638+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"30'836","reported_seq":1045,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:52:02.086158+0000","last_change":"2024-03-11T20:52:02.086221+0000","last_active":"2024-03-11T20:52:02.086158+0000","last_peered":"2024-03-11T20:52:02.086158+0000","last_clean":"2024-03-11T20:52:02.086158+0000","last_became_active":"2024-03-11T18:36:04.607063+0000","last_became_peered":"2024-03-11T18:36:04.607063+0000","last_unstale":"2024-03-11T20:52:02.086158+0000","last_undegraded":"2024-03-11T20:52:02.086158+0000","last_fullsized":"2024-03-11T20:52:02.086158+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'836","last_scrub_stamp":"2024-03-11T20:52:02.085970+0000","last_deep_scrub":"30'836","last_deep_scrub_stamp":"2024-03-11T20:52:02.085970+0000","last_clean_scrub_stamp":"2024-03-11T20:52:02.085970+0000","objects_scrubbed":0,"log_size":836,"log_dups_size":0,"ondisk_log_size":836,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:11:15.682060+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":124,"num_read_kb":69905,"num_write":818,"num_write_kb":155514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"30'815","reported_seq":1025,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:52.391686+0000","last_change":"2024-03-11T20:51:52.391751+0000","last_active":"2024-03-11T20:51:52.391686+0000","last_peered":"2024-03-11T20:51:52.391686+0000","last_clean":"2024-03-11T20:51:52.391686+0000","last_became_active":"2024-03-11T18:36:04.607109+0000","last_became_peered":"2024-03-11T18:36:04.607109+0000","last_unstale":"2024-03-11T20:51:52.391686+0000","last_undegraded":"2024-03-11T20:51:52.391686+0000","last_fullsized":"2024-03-11T20:51:52.391686+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'815","last_scrub_stamp":"2024-03-11T20:51:52.391490+0000","last_deep_scrub":"30'815","last_deep_scrub_stamp":"2024-03-11T20:51:52.391490+0000","last_clean_scrub_stamp":"2024-03-11T20:51:52.391490+0000","objects_scrubbed":0,"log_size":815,"log_dups_size":0,"ondisk_log_size":815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:57:13.486210+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":116,"num_read_kb":66679,"num_write":801,"num_write_kb":181203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"30'218","reported_seq":312,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:52.109055+0000","last_change":"2024-03-11T20:51:52.109055+0000","last_active":"2024-03-11T20:51:52.109055+0000","last_peered":"2024-03-11T20:51:52.109055+0000","last_clean":"2024-03-11T20:51:52.109055+0000","last_became_active":"2024-03-11T18:35:57.679642+0000","last_became_peered":"2024-03-11T18:35:57.679642+0000","last_unstale":"2024-03-11T20:51:52.109055+0000","last_undegraded":"2024-03-11T20:51:52.109055+0000","last_fullsized":"2024-03-11T20:51:52.109055+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'218","last_scrub_stamp":"2024-03-11T20:51:52.108736+0000","last_deep_scrub":"30'218","last_deep_scrub_stamp":"2024-03-11T20:51:52.108736+0000","last_clean_scrub_stamp":"2024-03-11T20:51:52.108736+0000","objects_scrubbed":0,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:11:13.125964+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"30'1321","reported_seq":1417,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:51.674392+0000","last_change":"2024-03-11T20:51:51.674468+0000","last_active":"2024-03-11T20:51:51.674392+0000","last_peered":"2024-03-11T20:51:51.674392+0000","last_clean":"2024-03-11T20:51:51.674392+0000","last_became_active":"2024-03-11T18:35:56.302758+0000","last_became_peered":"2024-03-11T18:35:56.302758+0000","last_unstale":"2024-03-11T20:51:51.674392+0000","last_undegraded":"2024-03-11T20:51:51.674392+0000","last_fullsized":"2024-03-11T20:51:51.674392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'1321","last_scrub_stamp":"2024-03-11T20:51:51.674216+0000","last_deep_scrub":"30'1321","last_deep_scrub_stamp":"2024-03-11T20:51:51.674216+0000","last_clean_scrub_stamp":"2024-03-11T20:51:51.674216+0000","objects_scrubbed":4,"log_size":1321,"log_dups_size":0,"ondisk_log_size":1321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:25:44.639817+0000","scrub_duration":55,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583494,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1254,"num_write_kb":12926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"30'766","reported_seq":990,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:59.594378+0000","last_change":"2024-03-11T20:51:59.594441+0000","last_active":"2024-03-11T20:51:59.594378+0000","last_peered":"2024-03-11T20:51:59.594378+0000","last_clean":"2024-03-11T20:51:59.594378+0000","last_became_active":"2024-03-11T18:36:04.617868+0000","last_became_peered":"2024-03-11T18:36:04.617868+0000","last_unstale":"2024-03-11T20:51:59.594378+0000","last_undegraded":"2024-03-11T20:51:59.594378+0000","last_fullsized":"2024-03-11T20:51:59.594378+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'766","last_scrub_stamp":"2024-03-11T20:51:59.594187+0000","last_deep_scrub":"30'766","last_deep_scrub_stamp":"2024-03-11T20:51:59.594187+0000","last_clean_scrub_stamp":"2024-03-11T20:51:59.594187+0000","objects_scrubbed":0,"log_size":766,"log_dups_size":0,"ondisk_log_size":766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:42:35.258355+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":132,"num_read_kb":68723,"num_write":745,"num_write_kb":131559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"30'225","reported_seq":319,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:46.057566+0000","last_change":"2024-03-11T20:51:46.057622+0000","last_active":"2024-03-11T20:51:46.057566+0000","last_peered":"2024-03-11T20:51:46.057566+0000","last_clean":"2024-03-11T20:51:46.057566+0000","last_became_active":"2024-03-11T18:35:57.689146+0000","last_became_peered":"2024-03-11T18:35:57.689146+0000","last_unstale":"2024-03-11T20:51:46.057566+0000","last_undegraded":"2024-03-11T20:51:46.057566+0000","last_fullsized":"2024-03-11T20:51:46.057566+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'225","last_scrub_stamp":"2024-03-11T20:51:46.057403+0000","last_deep_scrub":"30'225","last_deep_scrub_stamp":"2024-03-11T20:51:46.057403+0000","last_clean_scrub_stamp":"2024-03-11T20:51:46.057403+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:07:28.376685+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"30'837","reported_seq":933,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:50.091342+0000","last_change":"2024-03-11T20:51:50.091342+0000","last_active":"2024-03-11T20:51:50.091342+0000","last_peered":"2024-03-11T20:51:50.091342+0000","last_clean":"2024-03-11T20:51:50.091342+0000","last_became_active":"2024-03-11T18:35:56.347631+0000","last_became_peered":"2024-03-11T18:35:56.347631+0000","last_unstale":"2024-03-11T20:51:50.091342+0000","last_undegraded":"2024-03-11T20:51:50.091342+0000","last_fullsized":"2024-03-11T20:51:50.091342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'837","last_scrub_stamp":"2024-03-11T20:51:50.090992+0000","last_deep_scrub":"30'837","last_deep_scrub_stamp":"2024-03-11T20:51:50.090992+0000","last_clean_scrub_stamp":"2024-03-11T20:51:50.090992+0000","objects_scrubbed":3,"log_size":837,"log_dups_size":0,"ondisk_log_size":837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:41:43.213226+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":793,"num_write_kb":4496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"30'1916","reported_seq":2010,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:47.694991+0000","last_change":"2024-03-11T20:51:47.695078+0000","last_active":"2024-03-11T20:51:47.694991+0000","last_peered":"2024-03-11T20:51:47.694991+0000","last_clean":"2024-03-11T20:51:47.694991+0000","last_became_active":"2024-03-11T18:35:56.360580+0000","last_became_peered":"2024-03-11T18:35:56.360580+0000","last_unstale":"2024-03-11T20:51:47.694991+0000","last_undegraded":"2024-03-11T20:51:47.694991+0000","last_fullsized":"2024-03-11T20:51:47.694991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'1916","last_scrub_stamp":"2024-03-11T20:51:47.694811+0000","last_deep_scrub":"30'1916","last_deep_scrub_stamp":"2024-03-11T20:51:47.694811+0000","last_clean_scrub_stamp":"2024-03-11T20:51:47.694811+0000","objects_scrubbed":7,"log_size":1916,"log_dups_size":0,"ondisk_log_size":1916,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:29:49.908843+0000","scrub_duration":75,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1850,"num_write_kb":17317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"30'183","reported_seq":275,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:46.421015+0000","last_change":"2024-03-11T20:51:46.421038+0000","last_active":"2024-03-11T20:51:46.421015+0000","last_peered":"2024-03-11T20:51:46.421015+0000","last_clean":"2024-03-11T20:51:46.421015+0000","last_became_active":"2024-03-11T18:35:57.601990+0000","last_became_peered":"2024-03-11T18:35:57.601990+0000","last_unstale":"2024-03-11T20:51:46.421015+0000","last_undegraded":"2024-03-11T20:51:46.421015+0000","last_fullsized":"2024-03-11T20:51:46.421015+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'183","last_scrub_stamp":"2024-03-11T20:51:46.420935+0000","last_deep_scrub":"30'183","last_deep_scrub_stamp":"2024-03-11T20:51:46.420935+0000","last_clean_scrub_stamp":"2024-03-11T20:51:46.420935+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:22:13.821583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"30'692","reported_seq":839,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:48.853972+0000","last_change":"2024-03-11T20:51:48.854110+0000","last_active":"2024-03-11T20:51:48.853972+0000","last_peered":"2024-03-11T20:51:48.853972+0000","last_clean":"2024-03-11T20:51:48.853972+0000","last_became_active":"2024-03-11T18:36:04.604823+0000","last_became_peered":"2024-03-11T18:36:04.604823+0000","last_unstale":"2024-03-11T20:51:48.853972+0000","last_undegraded":"2024-03-11T20:51:48.853972+0000","last_fullsized":"2024-03-11T20:51:48.853972+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'692","last_scrub_stamp":"2024-03-11T20:51:48.853747+0000","last_deep_scrub":"30'692","last_deep_scrub_stamp":"2024-03-11T20:51:48.853747+0000","last_clean_scrub_stamp":"2024-03-11T20:51:48.853747+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:20:08.035779+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":66,"num_read_kb":37267,"num_write":684,"num_write_kb":109032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"30'334","reported_seq":428,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:49.124711+0000","last_change":"2024-03-11T20:51:49.124793+0000","last_active":"2024-03-11T20:51:49.124711+0000","last_peered":"2024-03-11T20:51:49.124711+0000","last_clean":"2024-03-11T20:51:49.124711+0000","last_became_active":"2024-03-11T18:35:56.296090+0000","last_became_peered":"2024-03-11T18:35:56.296090+0000","last_unstale":"2024-03-11T20:51:49.124711+0000","last_undegraded":"2024-03-11T20:51:49.124711+0000","last_fullsized":"2024-03-11T20:51:49.124711+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'334","last_scrub_stamp":"2024-03-11T20:51:49.124519+0000","last_deep_scrub":"30'334","last_deep_scrub_stamp":"2024-03-11T20:51:49.124519+0000","last_clean_scrub_stamp":"2024-03-11T20:51:49.124519+0000","objects_scrubbed":3,"log_size":334,"log_dups_size":0,"ondisk_log_size":334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:21:32.105408+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4195508,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":278,"num_write_kb":4222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"30'199","reported_seq":291,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:45.684713+0000","last_change":"2024-03-11T20:51:45.684771+0000","last_active":"2024-03-11T20:51:45.684713+0000","last_peered":"2024-03-11T20:51:45.684713+0000","last_clean":"2024-03-11T20:51:45.684713+0000","last_became_active":"2024-03-11T18:35:57.611097+0000","last_became_peered":"2024-03-11T18:35:57.611097+0000","last_unstale":"2024-03-11T20:51:45.684713+0000","last_undegraded":"2024-03-11T20:51:45.684713+0000","last_fullsized":"2024-03-11T20:51:45.684713+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'199","last_scrub_stamp":"2024-03-11T20:51:45.684534+0000","last_deep_scrub":"30'199","last_deep_scrub_stamp":"2024-03-11T20:51:45.684534+0000","last_clean_scrub_stamp":"2024-03-11T20:51:45.684534+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:57:37.340963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"30'703","reported_seq":843,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:57.074428+0000","last_change":"2024-03-11T20:51:57.074570+0000","last_active":"2024-03-11T20:51:57.074428+0000","last_peered":"2024-03-11T20:51:57.074428+0000","last_clean":"2024-03-11T20:51:57.074428+0000","last_became_active":"2024-03-11T18:36:04.589037+0000","last_became_peered":"2024-03-11T18:36:04.589037+0000","last_unstale":"2024-03-11T20:51:57.074428+0000","last_undegraded":"2024-03-11T20:51:57.074428+0000","last_fullsized":"2024-03-11T20:51:57.074428+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'703","last_scrub_stamp":"2024-03-11T20:51:57.074201+0000","last_deep_scrub":"30'703","last_deep_scrub_stamp":"2024-03-11T20:51:57.074201+0000","last_clean_scrub_stamp":"2024-03-11T20:51:57.074201+0000","objects_scrubbed":0,"log_size":703,"log_dups_size":0,"ondisk_log_size":703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:06:50.536745+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":59,"num_read_kb":23261,"num_write":682,"num_write_kb":69804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"30'217","reported_seq":320,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:52.511873+0000","last_change":"2024-03-11T20:51:52.511940+0000","last_active":"2024-03-11T20:51:52.511873+0000","last_peered":"2024-03-11T20:51:52.511873+0000","last_clean":"2024-03-11T20:51:52.511873+0000","last_became_active":"2024-03-11T18:35:57.614144+0000","last_became_peered":"2024-03-11T18:35:57.614144+0000","last_unstale":"2024-03-11T20:51:52.511873+0000","last_undegraded":"2024-03-11T20:51:52.511873+0000","last_fullsized":"2024-03-11T20:51:52.511873+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'217","last_scrub_stamp":"2024-03-11T20:51:52.511696+0000","last_deep_scrub":"30'217","last_deep_scrub_stamp":"2024-03-11T20:51:52.511696+0000","last_clean_scrub_stamp":"2024-03-11T20:51:52.511696+0000","objects_scrubbed":0,"log_size":217,"log_dups_size":0,"ondisk_log_size":217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:48:17.772083+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"30'222","reported_seq":325,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:47.867510+0000","last_change":"2024-03-11T20:51:47.867577+0000","last_active":"2024-03-11T20:51:47.867510+0000","last_peered":"2024-03-11T20:51:47.867510+0000","last_clean":"2024-03-11T20:51:47.867510+0000","last_became_active":"2024-03-11T18:35:57.629599+0000","last_became_peered":"2024-03-11T18:35:57.629599+0000","last_unstale":"2024-03-11T20:51:47.867510+0000","last_undegraded":"2024-03-11T20:51:47.867510+0000","last_fullsized":"2024-03-11T20:51:47.867510+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'222","last_scrub_stamp":"2024-03-11T20:51:47.867389+0000","last_deep_scrub":"30'222","last_deep_scrub_stamp":"2024-03-11T20:51:47.867389+0000","last_clean_scrub_stamp":"2024-03-11T20:51:47.867389+0000","objects_scrubbed":0,"log_size":222,"log_dups_size":0,"ondisk_log_size":222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T07:27:35.272906+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"30'236","reported_seq":339,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:59.704316+0000","last_change":"2024-03-11T20:51:59.704375+0000","last_active":"2024-03-11T20:51:59.704316+0000","last_peered":"2024-03-11T20:51:59.704316+0000","last_clean":"2024-03-11T20:51:59.704316+0000","last_became_active":"2024-03-11T18:35:57.687968+0000","last_became_peered":"2024-03-11T18:35:57.687968+0000","last_unstale":"2024-03-11T20:51:59.704316+0000","last_undegraded":"2024-03-11T20:51:59.704316+0000","last_fullsized":"2024-03-11T20:51:59.704316+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'236","last_scrub_stamp":"2024-03-11T20:51:59.704136+0000","last_deep_scrub":"30'236","last_deep_scrub_stamp":"2024-03-11T20:51:59.704136+0000","last_clean_scrub_stamp":"2024-03-11T20:51:59.704136+0000","objects_scrubbed":0,"log_size":236,"log_dups_size":0,"ondisk_log_size":236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:56:09.659615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"30'203","reported_seq":306,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:50.414108+0000","last_change":"2024-03-11T20:51:50.414108+0000","last_active":"2024-03-11T20:51:50.414108+0000","last_peered":"2024-03-11T20:51:50.414108+0000","last_clean":"2024-03-11T20:51:50.414108+0000","last_became_active":"2024-03-11T18:35:57.679935+0000","last_became_peered":"2024-03-11T18:35:57.679935+0000","last_unstale":"2024-03-11T20:51:50.414108+0000","last_undegraded":"2024-03-11T20:51:50.414108+0000","last_fullsized":"2024-03-11T20:51:50.414108+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'203","last_scrub_stamp":"2024-03-11T20:51:50.413873+0000","last_deep_scrub":"30'203","last_deep_scrub_stamp":"2024-03-11T20:51:50.413873+0000","last_clean_scrub_stamp":"2024-03-11T20:51:50.413873+0000","objects_scrubbed":0,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:21:25.487871+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"30'184","reported_seq":284,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:56.080238+0000","last_change":"2024-03-11T20:51:56.080305+0000","last_active":"2024-03-11T20:51:56.080238+0000","last_peered":"2024-03-11T20:51:56.080238+0000","last_clean":"2024-03-11T20:51:56.080238+0000","last_became_active":"2024-03-11T18:35:57.686623+0000","last_became_peered":"2024-03-11T18:35:57.686623+0000","last_unstale":"2024-03-11T20:51:56.080238+0000","last_undegraded":"2024-03-11T20:51:56.080238+0000","last_fullsized":"2024-03-11T20:51:56.080238+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'184","last_scrub_stamp":"2024-03-11T20:51:56.080062+0000","last_deep_scrub":"30'184","last_deep_scrub_stamp":"2024-03-11T20:51:56.080062+0000","last_clean_scrub_stamp":"2024-03-11T20:51:56.080062+0000","objects_scrubbed":0,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:53:08.621257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"30'208","reported_seq":308,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:53.126268+0000","last_change":"2024-03-11T20:51:53.126395+0000","last_active":"2024-03-11T20:51:53.126268+0000","last_peered":"2024-03-11T20:51:53.126268+0000","last_clean":"2024-03-11T20:51:53.126268+0000","last_became_active":"2024-03-11T18:35:57.691413+0000","last_became_peered":"2024-03-11T18:35:57.691413+0000","last_unstale":"2024-03-11T20:51:53.126268+0000","last_undegraded":"2024-03-11T20:51:53.126268+0000","last_fullsized":"2024-03-11T20:51:53.126268+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'208","last_scrub_stamp":"2024-03-11T20:51:53.125944+0000","last_deep_scrub":"30'208","last_deep_scrub_stamp":"2024-03-11T20:51:53.125944+0000","last_clean_scrub_stamp":"2024-03-11T20:51:53.125944+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:41:46.248179+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"30'244","reported_seq":344,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:57.503744+0000","last_change":"2024-03-11T20:51:57.503859+0000","last_active":"2024-03-11T20:51:57.503744+0000","last_peered":"2024-03-11T20:51:57.503744+0000","last_clean":"2024-03-11T20:51:57.503744+0000","last_became_active":"2024-03-11T18:35:57.684227+0000","last_became_peered":"2024-03-11T18:35:57.684227+0000","last_unstale":"2024-03-11T20:51:57.503744+0000","last_undegraded":"2024-03-11T20:51:57.503744+0000","last_fullsized":"2024-03-11T20:51:57.503744+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'244","last_scrub_stamp":"2024-03-11T20:51:57.503532+0000","last_deep_scrub":"30'244","last_deep_scrub_stamp":"2024-03-11T20:51:57.503532+0000","last_clean_scrub_stamp":"2024-03-11T20:51:57.503532+0000","objects_scrubbed":0,"log_size":244,"log_dups_size":0,"ondisk_log_size":244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:41:50.625767+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"30'193","reported_seq":293,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:54.175103+0000","last_change":"2024-03-11T20:51:54.175216+0000","last_active":"2024-03-11T20:51:54.175103+0000","last_peered":"2024-03-11T20:51:54.175103+0000","last_clean":"2024-03-11T20:51:54.175103+0000","last_became_active":"2024-03-11T18:35:57.680610+0000","last_became_peered":"2024-03-11T18:35:57.680610+0000","last_unstale":"2024-03-11T20:51:54.175103+0000","last_undegraded":"2024-03-11T20:51:54.175103+0000","last_fullsized":"2024-03-11T20:51:54.175103+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'193","last_scrub_stamp":"2024-03-11T20:51:54.174798+0000","last_deep_scrub":"30'193","last_deep_scrub_stamp":"2024-03-11T20:51:54.174798+0000","last_clean_scrub_stamp":"2024-03-11T20:51:54.174798+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:08:39.641057+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"30'218","reported_seq":298,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:51.898123+0000","last_change":"2024-03-11T20:51:51.898181+0000","last_active":"2024-03-11T20:51:51.898123+0000","last_peered":"2024-03-11T20:51:51.898123+0000","last_clean":"2024-03-11T20:51:51.898123+0000","last_became_active":"2024-03-11T18:37:04.263764+0000","last_became_peered":"2024-03-11T18:37:04.263764+0000","last_unstale":"2024-03-11T20:51:51.898123+0000","last_undegraded":"2024-03-11T20:51:51.898123+0000","last_fullsized":"2024-03-11T20:51:51.898123+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'218","last_scrub_stamp":"2024-03-11T20:51:51.897942+0000","last_deep_scrub":"30'218","last_deep_scrub_stamp":"2024-03-11T20:51:51.897942+0000","last_clean_scrub_stamp":"2024-03-11T20:51:51.897942+0000","objects_scrubbed":0,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:06:48.490620+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"30'205","reported_seq":303,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:53.141453+0000","last_change":"2024-03-11T20:51:53.141513+0000","last_active":"2024-03-11T20:51:53.141453+0000","last_peered":"2024-03-11T20:51:53.141453+0000","last_clean":"2024-03-11T20:51:53.141453+0000","last_became_active":"2024-03-11T18:35:57.678876+0000","last_became_peered":"2024-03-11T18:35:57.678876+0000","last_unstale":"2024-03-11T20:51:53.141453+0000","last_undegraded":"2024-03-11T20:51:53.141453+0000","last_fullsized":"2024-03-11T20:51:53.141453+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'205","last_scrub_stamp":"2024-03-11T20:51:53.141278+0000","last_deep_scrub":"30'205","last_deep_scrub_stamp":"2024-03-11T20:51:53.141278+0000","last_clean_scrub_stamp":"2024-03-11T20:51:53.141278+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T06:31:28.187032+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"30'208","reported_seq":306,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:46.579294+0000","last_change":"2024-03-11T20:51:46.579412+0000","last_active":"2024-03-11T20:51:46.579294+0000","last_peered":"2024-03-11T20:51:46.579294+0000","last_clean":"2024-03-11T20:51:46.579294+0000","last_became_active":"2024-03-11T18:35:57.678838+0000","last_became_peered":"2024-03-11T18:35:57.678838+0000","last_unstale":"2024-03-11T20:51:46.579294+0000","last_undegraded":"2024-03-11T20:51:46.579294+0000","last_fullsized":"2024-03-11T20:51:46.579294+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'208","last_scrub_stamp":"2024-03-11T20:51:46.579071+0000","last_deep_scrub":"30'208","last_deep_scrub_stamp":"2024-03-11T20:51:46.579071+0000","last_clean_scrub_stamp":"2024-03-11T20:51:46.579071+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T22:56:33.377954+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"30'213","reported_seq":311,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:49.435024+0000","last_change":"2024-03-11T20:51:49.435082+0000","last_active":"2024-03-11T20:51:49.435024+0000","last_peered":"2024-03-11T20:51:49.435024+0000","last_clean":"2024-03-11T20:51:49.435024+0000","last_became_active":"2024-03-11T18:35:57.671573+0000","last_became_peered":"2024-03-11T18:35:57.671573+0000","last_unstale":"2024-03-11T20:51:49.435024+0000","last_undegraded":"2024-03-11T20:51:49.435024+0000","last_fullsized":"2024-03-11T20:51:49.435024+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'213","last_scrub_stamp":"2024-03-11T20:51:49.434844+0000","last_deep_scrub":"30'213","last_deep_scrub_stamp":"2024-03-11T20:51:49.434844+0000","last_clean_scrub_stamp":"2024-03-11T20:51:49.434844+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:19:33.351470+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"30'221","reported_seq":319,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:53.486041+0000","last_change":"2024-03-11T20:51:53.486100+0000","last_active":"2024-03-11T20:51:53.486041+0000","last_peered":"2024-03-11T20:51:53.486041+0000","last_clean":"2024-03-11T20:51:53.486041+0000","last_became_active":"2024-03-11T18:35:57.615579+0000","last_became_peered":"2024-03-11T18:35:57.615579+0000","last_unstale":"2024-03-11T20:51:53.486041+0000","last_undegraded":"2024-03-11T20:51:53.486041+0000","last_fullsized":"2024-03-11T20:51:53.486041+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'221","last_scrub_stamp":"2024-03-11T20:51:53.485866+0000","last_deep_scrub":"30'221","last_deep_scrub_stamp":"2024-03-11T20:51:53.485866+0000","last_clean_scrub_stamp":"2024-03-11T20:51:53.485866+0000","objects_scrubbed":0,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:07:35.805149+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"30'206","reported_seq":304,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:57.525275+0000","last_change":"2024-03-11T20:51:57.525333+0000","last_active":"2024-03-11T20:51:57.525275+0000","last_peered":"2024-03-11T20:51:57.525275+0000","last_clean":"2024-03-11T20:51:57.525275+0000","last_became_active":"2024-03-11T18:35:57.688365+0000","last_became_peered":"2024-03-11T18:35:57.688365+0000","last_unstale":"2024-03-11T20:51:57.525275+0000","last_undegraded":"2024-03-11T20:51:57.525275+0000","last_fullsized":"2024-03-11T20:51:57.525275+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'206","last_scrub_stamp":"2024-03-11T20:51:57.525078+0000","last_deep_scrub":"30'206","last_deep_scrub_stamp":"2024-03-11T20:51:57.525078+0000","last_clean_scrub_stamp":"2024-03-11T20:51:57.525078+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:42:33.189247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"30'205","reported_seq":301,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:46.853595+0000","last_change":"2024-03-11T20:51:46.853717+0000","last_active":"2024-03-11T20:51:46.853595+0000","last_peered":"2024-03-11T20:51:46.853595+0000","last_clean":"2024-03-11T20:51:46.853595+0000","last_became_active":"2024-03-11T18:35:57.601334+0000","last_became_peered":"2024-03-11T18:35:57.601334+0000","last_unstale":"2024-03-11T20:51:46.853595+0000","last_undegraded":"2024-03-11T20:51:46.853595+0000","last_fullsized":"2024-03-11T20:51:46.853595+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'205","last_scrub_stamp":"2024-03-11T20:51:46.853316+0000","last_deep_scrub":"30'205","last_deep_scrub_stamp":"2024-03-11T20:51:46.853316+0000","last_clean_scrub_stamp":"2024-03-11T20:51:46.853316+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:00:08.391833+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"30'206","reported_seq":302,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:45.560705+0000","last_change":"2024-03-11T20:51:45.560764+0000","last_active":"2024-03-11T20:51:45.560705+0000","last_peered":"2024-03-11T20:51:45.560705+0000","last_clean":"2024-03-11T20:51:45.560705+0000","last_became_active":"2024-03-11T18:35:57.676075+0000","last_became_peered":"2024-03-11T18:35:57.676075+0000","last_unstale":"2024-03-11T20:51:45.560705+0000","last_undegraded":"2024-03-11T20:51:45.560705+0000","last_fullsized":"2024-03-11T20:51:45.560705+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'206","last_scrub_stamp":"2024-03-11T20:51:45.560527+0000","last_deep_scrub":"30'206","last_deep_scrub_stamp":"2024-03-11T20:51:45.560527+0000","last_clean_scrub_stamp":"2024-03-11T20:51:45.560527+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:48:10.820914+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"30'219","reported_seq":315,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:55.723427+0000","last_change":"2024-03-11T20:51:55.723487+0000","last_active":"2024-03-11T20:51:55.723427+0000","last_peered":"2024-03-11T20:51:55.723427+0000","last_clean":"2024-03-11T20:51:55.723427+0000","last_became_active":"2024-03-11T18:35:57.640934+0000","last_became_peered":"2024-03-11T18:35:57.640934+0000","last_unstale":"2024-03-11T20:51:55.723427+0000","last_undegraded":"2024-03-11T20:51:55.723427+0000","last_fullsized":"2024-03-11T20:51:55.723427+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'219","last_scrub_stamp":"2024-03-11T20:51:55.723224+0000","last_deep_scrub":"30'219","last_deep_scrub_stamp":"2024-03-11T20:51:55.723224+0000","last_clean_scrub_stamp":"2024-03-11T20:51:55.723224+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:29:01.534148+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"30'213","reported_seq":309,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:51.139783+0000","last_change":"2024-03-11T20:51:51.139841+0000","last_active":"2024-03-11T20:51:51.139783+0000","last_peered":"2024-03-11T20:51:51.139783+0000","last_clean":"2024-03-11T20:51:51.139783+0000","last_became_active":"2024-03-11T18:35:57.604645+0000","last_became_peered":"2024-03-11T18:35:57.604645+0000","last_unstale":"2024-03-11T20:51:51.139783+0000","last_undegraded":"2024-03-11T20:51:51.139783+0000","last_fullsized":"2024-03-11T20:51:51.139783+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'213","last_scrub_stamp":"2024-03-11T20:51:51.139617+0000","last_deep_scrub":"30'213","last_deep_scrub_stamp":"2024-03-11T20:51:51.139617+0000","last_clean_scrub_stamp":"2024-03-11T20:51:51.139617+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:39:53.106969+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"30'210","reported_seq":306,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:43.444482+0000","last_change":"2024-03-11T20:51:43.444482+0000","last_active":"2024-03-11T20:51:43.444482+0000","last_peered":"2024-03-11T20:51:43.444482+0000","last_clean":"2024-03-11T20:51:43.444482+0000","last_became_active":"2024-03-11T18:35:57.503362+0000","last_became_peered":"2024-03-11T18:35:57.503362+0000","last_unstale":"2024-03-11T20:51:43.444482+0000","last_undegraded":"2024-03-11T20:51:43.444482+0000","last_fullsized":"2024-03-11T20:51:43.444482+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'210","last_scrub_stamp":"2024-03-11T20:51:43.444395+0000","last_deep_scrub":"30'210","last_deep_scrub_stamp":"2024-03-11T20:51:43.444395+0000","last_clean_scrub_stamp":"2024-03-11T20:51:43.444395+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T03:38:31.933136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"30'195","reported_seq":269,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:50.889206+0000","last_change":"2024-03-11T20:51:50.889265+0000","last_active":"2024-03-11T20:51:50.889206+0000","last_peered":"2024-03-11T20:51:50.889206+0000","last_clean":"2024-03-11T20:51:50.889206+0000","last_became_active":"2024-03-11T18:37:04.251092+0000","last_became_peered":"2024-03-11T18:37:04.251092+0000","last_unstale":"2024-03-11T20:51:50.889206+0000","last_undegraded":"2024-03-11T20:51:50.889206+0000","last_fullsized":"2024-03-11T20:51:50.889206+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'195","last_scrub_stamp":"2024-03-11T20:51:50.889025+0000","last_deep_scrub":"30'195","last_deep_scrub_stamp":"2024-03-11T20:51:50.889025+0000","last_clean_scrub_stamp":"2024-03-11T20:51:50.889025+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:22:22.994011+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"30'181","reported_seq":275,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:47.713174+0000","last_change":"2024-03-11T20:51:47.713234+0000","last_active":"2024-03-11T20:51:47.713174+0000","last_peered":"2024-03-11T20:51:47.713174+0000","last_clean":"2024-03-11T20:51:47.713174+0000","last_became_active":"2024-03-11T18:35:57.613050+0000","last_became_peered":"2024-03-11T18:35:57.613050+0000","last_unstale":"2024-03-11T20:51:47.713174+0000","last_undegraded":"2024-03-11T20:51:47.713174+0000","last_fullsized":"2024-03-11T20:51:47.713174+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'181","last_scrub_stamp":"2024-03-11T20:51:47.713001+0000","last_deep_scrub":"30'181","last_deep_scrub_stamp":"2024-03-11T20:51:47.713001+0000","last_clean_scrub_stamp":"2024-03-11T20:51:47.713001+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T05:28:01.761386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"30'186","reported_seq":280,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:55.466564+0000","last_change":"2024-03-11T20:51:55.466628+0000","last_active":"2024-03-11T20:51:55.466564+0000","last_peered":"2024-03-11T20:51:55.466564+0000","last_clean":"2024-03-11T20:51:55.466564+0000","last_became_active":"2024-03-11T18:35:57.635978+0000","last_became_peered":"2024-03-11T18:35:57.635978+0000","last_unstale":"2024-03-11T20:51:55.466564+0000","last_undegraded":"2024-03-11T20:51:55.466564+0000","last_fullsized":"2024-03-11T20:51:55.466564+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'186","last_scrub_stamp":"2024-03-11T20:51:55.466386+0000","last_deep_scrub":"30'186","last_deep_scrub_stamp":"2024-03-11T20:51:55.466386+0000","last_clean_scrub_stamp":"2024-03-11T20:51:55.466386+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:06:56.670965+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"30'206","reported_seq":300,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:56.512696+0000","last_change":"2024-03-11T20:51:56.512765+0000","last_active":"2024-03-11T20:51:56.512696+0000","last_peered":"2024-03-11T20:51:56.512696+0000","last_clean":"2024-03-11T20:51:56.512696+0000","last_became_active":"2024-03-11T18:35:57.641417+0000","last_became_peered":"2024-03-11T18:35:57.641417+0000","last_unstale":"2024-03-11T20:51:56.512696+0000","last_undegraded":"2024-03-11T20:51:56.512696+0000","last_fullsized":"2024-03-11T20:51:56.512696+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'206","last_scrub_stamp":"2024-03-11T20:51:56.512501+0000","last_deep_scrub":"30'206","last_deep_scrub_stamp":"2024-03-11T20:51:56.512501+0000","last_clean_scrub_stamp":"2024-03-11T20:51:56.512501+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:21:39.493390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"30'210","reported_seq":304,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:59.508436+0000","last_change":"2024-03-11T20:51:59.508577+0000","last_active":"2024-03-11T20:51:59.508436+0000","last_peered":"2024-03-11T20:51:59.508436+0000","last_clean":"2024-03-11T20:51:59.508436+0000","last_became_active":"2024-03-11T18:35:57.689056+0000","last_became_peered":"2024-03-11T18:35:57.689056+0000","last_unstale":"2024-03-11T20:51:59.508436+0000","last_undegraded":"2024-03-11T20:51:59.508436+0000","last_fullsized":"2024-03-11T20:51:59.508436+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'210","last_scrub_stamp":"2024-03-11T20:51:59.508223+0000","last_deep_scrub":"30'210","last_deep_scrub_stamp":"2024-03-11T20:51:59.508223+0000","last_clean_scrub_stamp":"2024-03-11T20:51:59.508223+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:08:44.974483+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"30'194","reported_seq":286,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:47.079919+0000","last_change":"2024-03-11T20:51:47.079974+0000","last_active":"2024-03-11T20:51:47.079919+0000","last_peered":"2024-03-11T20:51:47.079919+0000","last_clean":"2024-03-11T20:51:47.079919+0000","last_became_active":"2024-03-11T18:35:57.682947+0000","last_became_peered":"2024-03-11T18:35:57.682947+0000","last_unstale":"2024-03-11T20:51:47.079919+0000","last_undegraded":"2024-03-11T20:51:47.079919+0000","last_fullsized":"2024-03-11T20:51:47.079919+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'194","last_scrub_stamp":"2024-03-11T20:51:47.079818+0000","last_deep_scrub":"30'194","last_deep_scrub_stamp":"2024-03-11T20:51:47.079818+0000","last_clean_scrub_stamp":"2024-03-11T20:51:47.079818+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:38:29.102345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"30'210","reported_seq":302,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:52:02.553287+0000","last_change":"2024-03-11T20:52:02.553346+0000","last_active":"2024-03-11T20:52:02.553287+0000","last_peered":"2024-03-11T20:52:02.553287+0000","last_clean":"2024-03-11T20:52:02.553287+0000","last_became_active":"2024-03-11T18:35:57.692504+0000","last_became_peered":"2024-03-11T18:35:57.692504+0000","last_unstale":"2024-03-11T20:52:02.553287+0000","last_undegraded":"2024-03-11T20:52:02.553287+0000","last_fullsized":"2024-03-11T20:52:02.553287+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'210","last_scrub_stamp":"2024-03-11T20:52:02.553102+0000","last_deep_scrub":"30'210","last_deep_scrub_stamp":"2024-03-11T20:52:02.553102+0000","last_clean_scrub_stamp":"2024-03-11T20:52:02.553102+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T04:06:56.015646+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"30'194","reported_seq":286,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:54.693155+0000","last_change":"2024-03-11T20:51:54.693284+0000","last_active":"2024-03-11T20:51:54.693155+0000","last_peered":"2024-03-11T20:51:54.693155+0000","last_clean":"2024-03-11T20:51:54.693155+0000","last_became_active":"2024-03-11T18:35:57.639886+0000","last_became_peered":"2024-03-11T18:35:57.639886+0000","last_unstale":"2024-03-11T20:51:54.693155+0000","last_undegraded":"2024-03-11T20:51:54.693155+0000","last_fullsized":"2024-03-11T20:51:54.693155+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'194","last_scrub_stamp":"2024-03-11T20:51:54.692872+0000","last_deep_scrub":"30'194","last_deep_scrub_stamp":"2024-03-11T20:51:54.692872+0000","last_clean_scrub_stamp":"2024-03-11T20:51:54.692872+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T23:08:40.159131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"30'212","reported_seq":304,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:55.538970+0000","last_change":"2024-03-11T20:51:55.539100+0000","last_active":"2024-03-11T20:51:55.538970+0000","last_peered":"2024-03-11T20:51:55.538970+0000","last_clean":"2024-03-11T20:51:55.538970+0000","last_became_active":"2024-03-11T18:35:57.686771+0000","last_became_peered":"2024-03-11T18:35:57.686771+0000","last_unstale":"2024-03-11T20:51:55.538970+0000","last_undegraded":"2024-03-11T20:51:55.538970+0000","last_fullsized":"2024-03-11T20:51:55.538970+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'212","last_scrub_stamp":"2024-03-11T20:51:55.538688+0000","last_deep_scrub":"30'212","last_deep_scrub_stamp":"2024-03-11T20:51:55.538688+0000","last_clean_scrub_stamp":"2024-03-11T20:51:55.538688+0000","objects_scrubbed":0,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T00:41:48.660922+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"30'196","reported_seq":301,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:52:00.723519+0000","last_change":"2024-03-11T20:52:00.723577+0000","last_active":"2024-03-11T20:52:00.723519+0000","last_peered":"2024-03-11T20:52:00.723519+0000","last_clean":"2024-03-11T20:52:00.723519+0000","last_became_active":"2024-03-11T18:35:57.694104+0000","last_became_peered":"2024-03-11T18:35:57.694104+0000","last_unstale":"2024-03-11T20:52:00.723519+0000","last_undegraded":"2024-03-11T20:52:00.723519+0000","last_fullsized":"2024-03-11T20:52:00.723519+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'196","last_scrub_stamp":"2024-03-11T20:52:00.723349+0000","last_deep_scrub":"30'196","last_deep_scrub_stamp":"2024-03-11T20:52:00.723349+0000","last_clean_scrub_stamp":"2024-03-11T20:52:00.723349+0000","objects_scrubbed":0,"log_size":196,"log_dups_size":0,"ondisk_log_size":196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T01:08:06.245143+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"30'191","reported_seq":294,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:45.413016+0000","last_change":"2024-03-11T20:51:45.413040+0000","last_active":"2024-03-11T20:51:45.413016+0000","last_peered":"2024-03-11T20:51:45.413016+0000","last_clean":"2024-03-11T20:51:45.413016+0000","last_became_active":"2024-03-11T18:35:57.537704+0000","last_became_peered":"2024-03-11T18:35:57.537704+0000","last_unstale":"2024-03-11T20:51:45.413016+0000","last_undegraded":"2024-03-11T20:51:45.413016+0000","last_fullsized":"2024-03-11T20:51:45.413016+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'191","last_scrub_stamp":"2024-03-11T20:51:45.412921+0000","last_deep_scrub":"30'191","last_deep_scrub_stamp":"2024-03-11T20:51:45.412921+0000","last_clean_scrub_stamp":"2024-03-11T20:51:45.412921+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-12T21:34:53.054951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"30'214","reported_seq":317,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:48.193012+0000","last_change":"2024-03-11T20:51:48.193122+0000","last_active":"2024-03-11T20:51:48.193012+0000","last_peered":"2024-03-11T20:51:48.193012+0000","last_clean":"2024-03-11T20:51:48.193012+0000","last_became_active":"2024-03-11T18:35:57.506429+0000","last_became_peered":"2024-03-11T18:35:57.506429+0000","last_unstale":"2024-03-11T20:51:48.193012+0000","last_undegraded":"2024-03-11T20:51:48.193012+0000","last_fullsized":"2024-03-11T20:51:48.193012+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'214","last_scrub_stamp":"2024-03-11T20:51:48.192820+0000","last_deep_scrub":"30'214","last_deep_scrub_stamp":"2024-03-11T20:51:48.192820+0000","last_clean_scrub_stamp":"2024-03-11T20:51:48.192820+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T08:31:46.380911+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"30'213","reported_seq":316,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-11T20:51:56.505521+0000","last_change":"2024-03-11T20:51:56.505584+0000","last_active":"2024-03-11T20:51:56.505521+0000","last_peered":"2024-03-11T20:51:56.505521+0000","last_clean":"2024-03-11T20:51:56.505521+0000","last_became_active":"2024-03-11T18:35:57.687532+0000","last_became_peered":"2024-03-11T18:35:57.687532+0000","last_unstale":"2024-03-11T20:51:56.505521+0000","last_undegraded":"2024-03-11T20:51:56.505521+0000","last_fullsized":"2024-03-11T20:51:56.505521+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"30'213","last_scrub_stamp":"2024-03-11T20:51:56.505315+0000","last_deep_scrub":"30'213","last_deep_scrub_stamp":"2024-03-11T20:51:56.505315+0000","last_clean_scrub_stamp":"2024-03-11T20:51:56.505315+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-13T02:25:49.470915+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2961,"num_read_kb":1579043,"num_write":24379,"num_write_kb":4188254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24824,"ondisk_log_size":24824,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13197,"ondisk_log_size":13197,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":210457699,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":4697,"num_read_kb":5145,"num_write":47510,"num_write_kb":229688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":25000,"num_omap_keys":50,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":421158912,"data_stored":420915398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38412,"ondisk_log_size":38412,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719478385,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":200100,"kb_used_data":78748,"kb_used_omap":1,"kb_used_meta":121342,"kb_avail":94171740,"statfs":{"total":96636764160,"available":96431861760,"internally_reserved":0,"allocated":80637952,"data_stored":80415478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":124254666},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":16,"seq":68719478385,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":245524,"kb_used_data":61772,"kb_used_omap":1,"kb_used_meta":183742,"kb_avail":94126316,"statfs":{"total":96636764160,"available":96385347584,"internally_reserved":0,"allocated":63254528,"data_stored":63047256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":188152262},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":16,"seq":68719478383,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":183948,"kb_used_data":48708,"kb_used_omap":1,"kb_used_meta":135230,"kb_avail":94187892,"statfs":{"total":96636764160,"available":96448401408,"internally_reserved":0,"allocated":49876992,"data_stored":49644774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":138475974},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":16,"seq":68719478384,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":183188,"kb_used_data":57676,"kb_used_omap":1,"kb_used_meta":125502,"kb_avail":94188652,"statfs":{"total":96636764160,"available":96449179648,"internally_reserved":0,"allocated":59060224,"data_stored":58852820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":128514501},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424511087,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":218248,"kb_used_data":42816,"kb_used_omap":1,"kb_used_meta":175422,"kb_avail":94153592,"statfs":{"total":96636764160,"available":96413278208,"internally_reserved":0,"allocated":43843584,"data_stored":43631469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":179632583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424511087,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":152012,"kb_used_data":22340,"kb_used_omap":1,"kb_used_meta":129662,"kb_avail":94219828,"statfs":{"total":96636764160,"available":96481103872,"internally_reserved":0,"allocated":22876160,"data_stored":22659915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132774341},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424511088,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":228216,"kb_used_data":56880,"kb_used_omap":1,"kb_used_meta":171326,"kb_avail":94143624,"statfs":{"total":96636764160,"available":96403070976,"internally_reserved":0,"allocated":58245120,"data_stored":58030400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":175438280},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129543789,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149276,"kb_used_data":45972,"kb_used_omap":1,"kb_used_meta":103294,"kb_avail":94222564,"statfs":{"total":96636764160,"available":96483905536,"internally_reserved":0,"allocated":47075328,"data_stored":46859238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105773509},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":46166016,"data_stored":46138218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57929728,"data_stored":57899748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":22560768,"data_stored":22529263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":43528192,"data_stored":43500817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":58744832,"data_stored":58722168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49561600,"data_stored":49514122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":62939136,"data_stored":62916604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":79728640,"data_stored":79694458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-11T20:52:08.481 DEBUG:teuthology.orchestra.run.smithi090:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-03-11T20:52:08.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.648+0000 7f2670840640 1 Processor -- start 2024-03-11T20:52:08.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.648+0000 7f2670840640 1 -- start start 2024-03-11T20:52:08.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.648+0000 7f2670840640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f266c151b30 0x7f266c06e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:08.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.648+0000 7f2670840640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f266c0c4af0 0x7f266c0c4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:08.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.648+0000 7f2670840640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f266c06f430 con 0x7f266c070b00 2024-03-11T20:52:08.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.648+0000 7f2670840640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f266c0c65b0 con 0x7f266c0c4af0 2024-03-11T20:52:08.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.648+0000 7f2670840640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f266c06e820 con 0x7f266c151b30 2024-03-11T20:52:08.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.648+0000 7f266a59c640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f266c151b30 0x7f266c06e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:52:08.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.648+0000 7f266ad9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f266c0c4af0 0x7f266c0c4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:52:08.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.652+0000 7f266ad9d640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f266c0c4af0 0x7f266c0c4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.90:37238/0 (socket says 172.21.15.90:37238) 2024-03-11T20:52:08.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.652+0000 7f266a59c640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f266c151b30 0x7f266c06e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3301/0 says I am v2:172.21.15.90:33722/0 (socket says 172.21.15.90:33722) 2024-03-11T20:52:08.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.652+0000 7f266ad9d640 1 -- 172.21.15.90:0/1739263795 learned_addr learned my addr 172.21.15.90:0/1739263795 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:52:08.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.656+0000 7f266a59c640 1 -- 172.21.15.90:0/1739263795 >> v2:172.21.15.92:3300/0 conn(0x7f266c0c4af0 msgr2=0x7f266c0c4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:52:08.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.656+0000 7f266a59c640 1 --2- 172.21.15.90:0/1739263795 >> v2:172.21.15.92:3300/0 conn(0x7f266c0c4af0 0x7f266c0c4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:08.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.656+0000 7f266a59c640 1 -- 172.21.15.90:0/1739263795 >> v1:172.21.15.92:6789/0 conn(0x7f266c070b00 legacy=0x7f266c0c5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:52:08.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.656+0000 7f266a59c640 1 -- 172.21.15.90:0/1739263795 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f266c06eb00 con 0x7f266c151b30 2024-03-11T20:52:08.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.668+0000 7f266a59c640 1 --2- 172.21.15.90:0/1739263795 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f266c151b30 0x7f266c06e210 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f265c0097c0 tx=0x7f265c01c430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0c1de3e4cc6fc11 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:52:08.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.680+0000 7f266959a640 1 -- 172.21.15.90:0/1739263795 <== mon.2 v2:172.21.15.92:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265c0ab5f0 con 0x7f266c151b30 2024-03-11T20:52:08.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.680+0000 7f266959a640 1 -- 172.21.15.90:0/1739263795 <== mon.2 v2:172.21.15.92:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f265c0ab750 con 0x7f266c151b30 2024-03-11T20:52:08.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.680+0000 7f266959a640 1 -- 172.21.15.90:0/1739263795 <== mon.2 v2:172.21.15.92:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265c0abae0 con 0x7f266c151b30 2024-03-11T20:52:08.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.680+0000 7f2670840640 1 -- 172.21.15.90:0/1739263795 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f266c151b30 msgr2=0x7f266c06e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:08.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.680+0000 7f2670840640 1 --2- 172.21.15.90:0/1739263795 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f266c151b30 0x7f266c06e210 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f265c0097c0 tx=0x7f265c01c430 comp rx=0 tx=0).stop 2024-03-11T20:52:08.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.680+0000 7f2670840640 1 -- 172.21.15.90:0/1739263795 shutdown_connections 2024-03-11T20:52:08.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.680+0000 7f2670840640 1 --2- 172.21.15.90:0/1739263795 >> v2:172.21.15.92:3300/0 conn(0x7f266c0c4af0 0x7f266c0c4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:08.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.680+0000 7f2670840640 1 --2- 172.21.15.90:0/1739263795 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f266c151b30 0x7f266c06e210 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:08.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.680+0000 7f2670840640 1 -- 172.21.15.90:0/1739263795 >> 172.21.15.90:0/1739263795 conn(0x7f266c06ac00 msgr2=0x7f266c06b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:52:08.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.680+0000 7f2670840640 1 -- 172.21.15.90:0/1739263795 shutdown_connections 2024-03-11T20:52:08.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.680+0000 7f2670840640 1 -- 172.21.15.90:0/1739263795 wait complete. 2024-03-11T20:52:08.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.684+0000 7f2670840640 1 Processor -- start 2024-03-11T20:52:08.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.684+0000 7f2670840640 1 -- start start 2024-03-11T20:52:08.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.684+0000 7f2670840640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f266c070b00 0x7f266c0ca4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:08.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.684+0000 7f2670840640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f266c0c4af0 0x7f266c1f0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:08.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.684+0000 7f2670840640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f266c151b30 0x7f266c13ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:08.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.684+0000 7f2670840640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f266c1f46c0 con 0x7f266c0c4af0 2024-03-11T20:52:08.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.684+0000 7f2670840640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f266c1f4b50 con 0x7f266c070b00 2024-03-11T20:52:08.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.684+0000 7f2670840640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f266c14f410 con 0x7f266c151b30 2024-03-11T20:52:08.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.684+0000 7f2669d9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f266c0c4af0 0x7f266c1f0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:52:08.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.684+0000 7f2669d9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f266c0c4af0 0x7f266c1f0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60152/0 (socket says 172.21.15.90:60152) 2024-03-11T20:52:08.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.684+0000 7f2669d9b640 1 -- 172.21.15.90:0/2010721712 learned_addr learned my addr 172.21.15.90:0/2010721712 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:52:08.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.684+0000 7f266a59c640 1 --2- 172.21.15.90:0/2010721712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f266c070b00 0x7f266c0ca4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:52:08.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.688+0000 7f266ad9d640 1 --2- 172.21.15.90:0/2010721712 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f266c151b30 0x7f266c13ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:52:08.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.688+0000 7f2669d9b640 1 -- 172.21.15.90:0/2010721712 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f266c151b30 msgr2=0x7f266c13ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:08.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.688+0000 7f2669d9b640 1 --2- 172.21.15.90:0/2010721712 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f266c151b30 0x7f266c13ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:08.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.688+0000 7f2669d9b640 1 -- 172.21.15.90:0/2010721712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f266c070b00 msgr2=0x7f266c0ca4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:08.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.688+0000 7f2669d9b640 1 --2- 172.21.15.90:0/2010721712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f266c070b00 0x7f266c0ca4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:08.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.688+0000 7f2669d9b640 1 -- 172.21.15.90:0/2010721712 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f265c009420 con 0x7f266c0c4af0 2024-03-11T20:52:08.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.692+0000 7f2669d9b640 1 --2- 172.21.15.90:0/2010721712 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f266c0c4af0 0x7f266c1f0780 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f266000b3e0 tx=0x7f266000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:52:08.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.696+0000 7f265affd640 1 -- 172.21.15.90:0/2010721712 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2660011830 con 0x7f266c0c4af0 2024-03-11T20:52:08.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.696+0000 7f265affd640 1 -- 172.21.15.90:0/2010721712 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26600119b0 con 0x7f266c0c4af0 2024-03-11T20:52:08.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.696+0000 7f265affd640 1 -- 172.21.15.90:0/2010721712 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2660011d60 con 0x7f266c0c4af0 2024-03-11T20:52:08.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.696+0000 7f2670840640 1 -- 172.21.15.90:0/2010721712 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f266c14f6f0 con 0x7f266c0c4af0 2024-03-11T20:52:08.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.696+0000 7f2670840640 1 -- 172.21.15.90:0/2010721712 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f266c1f4070 con 0x7f266c0c4af0 2024-03-11T20:52:08.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.696+0000 7f2670840640 1 -- 172.21.15.90:0/2010721712 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f266c04eef0 con 0x7f266c0c4af0 2024-03-11T20:52:08.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.992+0000 7f265affd640 1 -- 172.21.15.90:0/2010721712 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f266001a400 con 0x7f266c0c4af0 2024-03-11T20:52:08.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.996+0000 7f265affd640 1 --2- 172.21.15.90:0/2010721712 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f264c066ec0 0x7f264c069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:08.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.996+0000 7f266a59c640 1 --2- 172.21.15.90:0/2010721712 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f264c066ec0 0x7f264c069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:52:08.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:08.996+0000 7f266a59c640 1 --2- 172.21.15.90:0/2010721712 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f264c066ec0 0x7f264c069380 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f265c0af000 tx=0x7f265c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:52:09.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.032+0000 7f265affd640 1 -- 172.21.15.90:0/2010721712 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f2660056370 con 0x7f266c0c4af0 2024-03-11T20:52:09.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.032+0000 7f265affd640 1 -- 172.21.15.90:0/2010721712 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2660051f10 con 0x7f266c0c4af0 2024-03-11T20:52:09.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.236+0000 7f2670840640 1 -- 172.21.15.90:0/2010721712 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f266c06b500 con 0x7f266c0c4af0 2024-03-11T20:52:09.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.408+0000 7f265affd640 1 -- 172.21.15.90:0/2010721712 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v10)=0 v10) v1 ==== 123+0+0 (secure 0 0 0) 0x7f2660015070 con 0x7f266c0c4af0 2024-03-11T20:52:09.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.408+0000 7f265affd640 1 -- 172.21.15.90:0/2010721712 <== mon.0 v2:172.21.15.90:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f2660051f10 con 0x7f266c0c4af0 2024-03-11T20:52:09.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.412+0000 7f2670840640 1 -- 172.21.15.90:0/2010721712 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f264c066ec0 msgr2=0x7f264c069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:09.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.412+0000 7f2670840640 1 --2- 172.21.15.90:0/2010721712 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f264c066ec0 0x7f264c069380 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f265c0af000 tx=0x7f265c0a7000 comp rx=0 tx=0).stop 2024-03-11T20:52:09.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.412+0000 7f2670840640 1 -- 172.21.15.90:0/2010721712 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f266c0c4af0 msgr2=0x7f266c1f0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:09.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.412+0000 7f2670840640 1 --2- 172.21.15.90:0/2010721712 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f266c0c4af0 0x7f266c1f0780 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f266000b3e0 tx=0x7f266000b8a0 comp rx=0 tx=0).stop 2024-03-11T20:52:09.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.412+0000 7f2670840640 1 -- 172.21.15.90:0/2010721712 shutdown_connections 2024-03-11T20:52:09.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.412+0000 7f2670840640 1 --2- 172.21.15.90:0/2010721712 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f264c066ec0 0x7f264c069380 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:09.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.412+0000 7f2670840640 1 --2- 172.21.15.90:0/2010721712 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f266c151b30 0x7f266c13ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:09.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.412+0000 7f2670840640 1 --2- 172.21.15.90:0/2010721712 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f266c0c4af0 0x7f266c1f0780 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:09.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.412+0000 7f2670840640 1 --2- 172.21.15.90:0/2010721712 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f266c070b00 0x7f266c0ca4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:09.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.412+0000 7f2670840640 1 -- 172.21.15.90:0/2010721712 >> 172.21.15.90:0/2010721712 conn(0x7f266c06ac00 msgr2=0x7f266c14d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:52:09.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.412+0000 7f2670840640 1 -- 172.21.15.90:0/2010721712 shutdown_connections 2024-03-11T20:52:09.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.412+0000 7f2670840640 1 -- 172.21.15.90:0/2010721712 wait complete. 2024-03-11T20:52:09.429 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-11T20:52:09.429 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T20:52:09.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.552+0000 7f74b749b640 1 Processor -- start 2024-03-11T20:52:09.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.552+0000 7f74b749b640 1 -- start start 2024-03-11T20:52:09.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.556+0000 7f74b749b640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f74b01509f0 0x7f74b0157390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:09.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.556+0000 7f74b749b640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f74b0070b00 0x7f74b00c4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:09.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.556+0000 7f74b749b640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74b00c5de0 con 0x7f74b0150dd0 2024-03-11T20:52:09.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.556+0000 7f74b749b640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74b0156910 con 0x7f74b01509f0 2024-03-11T20:52:09.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.556+0000 7f74b749b640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74b01578d0 con 0x7f74b0070b00 2024-03-11T20:52:09.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.580+0000 7f74b5237640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f74b01509f0 0x7f74b0157390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:52:09.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.580+0000 7f74b4a36640 1 --1- >> v1:172.21.15.92:6789/0 conn(0x7f74b0150dd0 0x7f74b00c28e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.92:6789/0 says I am v1:172.21.15.90:38024/0 (socket says 172.21.15.90:38024) 2024-03-11T20:52:09.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.580+0000 7f74b4a36640 1 -- 172.21.15.90:0/1267324844 learned_addr learned my addr 172.21.15.90:0/1267324844 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T20:52:09.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.588+0000 7f74b5237640 1 -- 172.21.15.90:0/1267324844 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f74b0070b00 msgr2=0x7f74b00c4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:52:09.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.588+0000 7f74b5237640 1 --2- 172.21.15.90:0/1267324844 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f74b0070b00 0x7f74b00c4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:09.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.588+0000 7f74b5237640 1 -- 172.21.15.90:0/1267324844 >> v1:172.21.15.92:6789/0 conn(0x7f74b0150dd0 legacy=0x7f74b00c28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:09.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.588+0000 7f74b5237640 1 -- 172.21.15.90:0/1267324844 --> v2:172.21.15.92:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b00c6030 con 0x7f74b01509f0 2024-03-11T20:52:09.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.596+0000 7f74b5237640 1 --2- 172.21.15.90:0/1267324844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74b01509f0 0x7f74b0157390 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f74a80097c0 tx=0x7f74a801c4c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0fa7f8997afa118 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:52:09.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.604+0000 7f74affff640 1 -- 172.21.15.90:0/1267324844 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a80aa620 con 0x7f74b01509f0 2024-03-11T20:52:09.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.604+0000 7f74affff640 1 -- 172.21.15.90:0/1267324844 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f74a80aa780 con 0x7f74b01509f0 2024-03-11T20:52:09.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.604+0000 7f74affff640 1 -- 172.21.15.90:0/1267324844 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a80aab30 con 0x7f74b01509f0 2024-03-11T20:52:09.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.604+0000 7f74b749b640 1 -- 172.21.15.90:0/1267324844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74b01509f0 msgr2=0x7f74b0157390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:09.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.604+0000 7f74b749b640 1 --2- 172.21.15.90:0/1267324844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74b01509f0 0x7f74b0157390 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f74a80097c0 tx=0x7f74a801c4c0 comp rx=0 tx=0).stop 2024-03-11T20:52:09.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.604+0000 7f74b749b640 1 -- 172.21.15.90:0/1267324844 shutdown_connections 2024-03-11T20:52:09.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.604+0000 7f74b749b640 1 --2- 172.21.15.90:0/1267324844 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f74b0070b00 0x7f74b00c4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:09.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.604+0000 7f74b749b640 1 --2- 172.21.15.90:0/1267324844 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74b01509f0 0x7f74b0157390 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:09.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.604+0000 7f74b749b640 1 -- 172.21.15.90:0/1267324844 >> 172.21.15.90:0/1267324844 conn(0x7f74b006ac00 msgr2=0x7f74b006b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:52:09.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.604+0000 7f74b749b640 1 -- 172.21.15.90:0/1267324844 shutdown_connections 2024-03-11T20:52:09.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.604+0000 7f74b749b640 1 -- 172.21.15.90:0/1267324844 wait complete. 2024-03-11T20:52:09.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.604+0000 7f74b749b640 1 Processor -- start 2024-03-11T20:52:09.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.604+0000 7f74b749b640 1 -- start start 2024-03-11T20:52:09.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.608+0000 7f74b749b640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f74b0070b00 0x7f74b0144250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:09.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.608+0000 7f74b749b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74b01509f0 0x7f74b0152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:09.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.608+0000 7f74b749b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74b0150dd0 0x7f74b0152ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:09.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.608+0000 7f74b749b640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b0153fb0 con 0x7f74b0150dd0 2024-03-11T20:52:09.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.608+0000 7f74b749b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b0152120 con 0x7f74b01509f0 2024-03-11T20:52:09.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.608+0000 7f74b749b640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74b0150090 con 0x7f74b0070b00 2024-03-11T20:52:09.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.608+0000 7f74b4a36640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74b01509f0 0x7f74b0152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:52:09.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.608+0000 7f74b4a36640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74b01509f0 0x7f74b0152ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.92:3300/0 says I am v2:172.21.15.90:37258/0 (socket says 172.21.15.90:37258) 2024-03-11T20:52:09.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.608+0000 7f74b4a36640 1 -- 172.21.15.90:0/1117415209 learned_addr learned my addr 172.21.15.90:0/1117415209 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:52:09.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.608+0000 7f74b5237640 1 --2- 172.21.15.90:0/1117415209 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f74b0070b00 0x7f74b0144250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:52:09.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.612+0000 7f74b4a36640 1 -- 172.21.15.90:0/1117415209 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f74b0070b00 msgr2=0x7f74b0144250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:09.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.612+0000 7f74b4a36640 1 --2- 172.21.15.90:0/1117415209 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f74b0070b00 0x7f74b0144250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:09.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.612+0000 7f74b4a36640 1 -- 172.21.15.90:0/1117415209 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74b0150dd0 msgr2=0x7f74b0152ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:09.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.612+0000 7f74b4a36640 1 --2- 172.21.15.90:0/1117415209 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74b0150dd0 0x7f74b0152ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:09.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.612+0000 7f74b4a36640 1 -- 172.21.15.90:0/1117415209 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a8009420 con 0x7f74b01509f0 2024-03-11T20:52:09.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.616+0000 7f74b4a36640 1 --2- 172.21.15.90:0/1117415209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74b01509f0 0x7f74b0152ab0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f74a4001ec0 tx=0x7f74a400c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:52:09.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.624+0000 7f74adffb640 1 -- 172.21.15.90:0/1117415209 <== mon.1 v2:172.21.15.92:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a4005cf0 con 0x7f74b01509f0 2024-03-11T20:52:09.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.624+0000 7f74adffb640 1 -- 172.21.15.90:0/1117415209 <== mon.1 v2:172.21.15.92:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f74a4005e50 con 0x7f74b01509f0 2024-03-11T20:52:09.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.624+0000 7f74adffb640 1 -- 172.21.15.90:0/1117415209 <== mon.1 v2:172.21.15.92:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a40072f0 con 0x7f74b01509f0 2024-03-11T20:52:09.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.624+0000 7f74b749b640 1 -- 172.21.15.90:0/1117415209 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74b0151e60 con 0x7f74b01509f0 2024-03-11T20:52:09.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.624+0000 7f74b749b640 1 -- 172.21.15.90:0/1117415209 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74b014fa40 con 0x7f74b01509f0 2024-03-11T20:52:09.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.624+0000 7f74b749b640 1 -- 172.21.15.90:0/1117415209 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74b0150220 con 0x7f74b01509f0 2024-03-11T20:52:09.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.644+0000 7f74adffb640 1 -- 172.21.15.90:0/1117415209 <== mon.1 v2:172.21.15.92:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f74a4007450 con 0x7f74b01509f0 2024-03-11T20:52:09.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.644+0000 7f74adffb640 1 --2- 172.21.15.90:0/1117415209 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f749c066e90 0x7f749c069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:09.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.644+0000 7f74b5237640 1 --2- 172.21.15.90:0/1117415209 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f749c066e90 0x7f749c069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:52:09.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.644+0000 7f74b5237640 1 --2- 172.21.15.90:0/1117415209 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f749c066e90 0x7f749c069350 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f74a80ae000 tx=0x7f74a80a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:52:09.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.900+0000 7f74adffb640 1 -- 172.21.15.90:0/1117415209 <== mon.1 v2:172.21.15.92:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f74a40556e0 con 0x7f74b01509f0 2024-03-11T20:52:09.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:09.932+0000 7f74adffb640 1 -- 172.21.15.90:0/1117415209 <== mon.1 v2:172.21.15.92:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f74a4051280 con 0x7f74b01509f0 2024-03-11T20:52:10.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.152+0000 7f74b749b640 1 -- 172.21.15.90:0/1117415209 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74b004eef0 con 0x7f74b01509f0 2024-03-11T20:52:10.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.256+0000 7f74adffb640 1 -- 172.21.15.90:0/1117415209 <== mon.1 v2:172.21.15.92:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+6575 (secure 0 0 0) 0x7f74a4017020 con 0x7f74b01509f0 2024-03-11T20:52:10.256 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T20:52:10.256 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/641527992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":641527992},{"type":"v1","addr":"172.21.15.90:6839","nonce":641527992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-11T18:35:57.363570+0000","modified":"2024-03-11T18:36:51.575661+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4255,"mds_1":4253,"mds_2":4241,"mds_3":4238,"mds_4":4229},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4229":{"gid":4229,"name":"c","rank":4,"incarnation":18,"state":"up:active","state_seq":17,"addr":"172.21.15.90:6835/460417177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":460417177},{"type":"v1","addr":"172.21.15.90:6835","nonce":460417177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4238":{"gid":4238,"name":"d","rank":3,"incarnation":16,"state":"up:active","state_seq":17,"addr":"172.21.15.92:6835/3568438077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":3568438077},{"type":"v1","addr":"172.21.15.92:6835","nonce":3568438077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4241":{"gid":4241,"name":"b","rank":2,"incarnation":14,"state":"up:active","state_seq":16,"addr":"172.21.15.92:6834/2306487548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":2306487548},{"type":"v1","addr":"172.21.15.92:6834","nonce":2306487548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4253":{"gid":4253,"name":"f","rank":1,"incarnation":10,"state":"up:active","state_seq":15,"addr":"172.21.15.92:6837/3209534444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":3209534444},{"type":"v1","addr":"172.21.15.92:6837","nonce":3209534444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4255":{"gid":4255,"name":"e","rank":0,"incarnation":5,"state":"up:active","state_seq":6,"addr":"172.21.15.90:6837/3485943375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3485943375},{"type":"v1","addr":"172.21.15.90:6837","nonce":3485943375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T20:52:10.256 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 19 2024-03-11T20:52:10.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.260+0000 7f74b749b640 1 -- 172.21.15.90:0/1117415209 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f749c066e90 msgr2=0x7f749c069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:10.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.260+0000 7f74b749b640 1 --2- 172.21.15.90:0/1117415209 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f749c066e90 0x7f749c069350 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f74a80ae000 tx=0x7f74a80a7000 comp rx=0 tx=0).stop 2024-03-11T20:52:10.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.260+0000 7f74b749b640 1 -- 172.21.15.90:0/1117415209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74b01509f0 msgr2=0x7f74b0152ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:10.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.260+0000 7f74b749b640 1 --2- 172.21.15.90:0/1117415209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74b01509f0 0x7f74b0152ab0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f74a4001ec0 tx=0x7f74a400c890 comp rx=0 tx=0).stop 2024-03-11T20:52:10.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.260+0000 7f74b749b640 1 -- 172.21.15.90:0/1117415209 shutdown_connections 2024-03-11T20:52:10.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.260+0000 7f74b749b640 1 --2- 172.21.15.90:0/1117415209 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f749c066e90 0x7f749c069350 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:10.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.260+0000 7f74b749b640 1 --2- 172.21.15.90:0/1117415209 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74b0150dd0 0x7f74b0152ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:10.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.260+0000 7f74b749b640 1 --2- 172.21.15.90:0/1117415209 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f74b01509f0 0x7f74b0152ab0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:10.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.260+0000 7f74b749b640 1 --2- 172.21.15.90:0/1117415209 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f74b0070b00 0x7f74b0144250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:10.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.260+0000 7f74b749b640 1 -- 172.21.15.90:0/1117415209 >> 172.21.15.90:0/1117415209 conn(0x7f74b006ac00 msgr2=0x7f74b006afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:52:10.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.260+0000 7f74b749b640 1 -- 172.21.15.90:0/1117415209 shutdown_connections 2024-03-11T20:52:10.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.260+0000 7f74b749b640 1 -- 172.21.15.90:0/1117415209 wait complete. 2024-03-11T20:52:10.283 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-11T20:52:10.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.408+0000 7fbb661d4640 1 Processor -- start 2024-03-11T20:52:10.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.408+0000 7fbb661d4640 1 -- start start 2024-03-11T20:52:10.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.412+0000 7fbb661d4640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fbb60153cf0 0x7fbb6015d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:10.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.412+0000 7fbb661d4640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fbb600c4b60 0x7fbb6015a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:10.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.412+0000 7fbb661d4640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb6015a0b0 con 0x7fbb600c42f0 2024-03-11T20:52:10.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.412+0000 7fbb661d4640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb6015e120 con 0x7fbb60153cf0 2024-03-11T20:52:10.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.412+0000 7fbb661d4640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb60159b30 con 0x7fbb600c4b60 2024-03-11T20:52:10.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.428+0000 7fbb5ffff640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7fbb600c42f0 0x7fbb600ca370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:56004/0 (socket says 172.21.15.90:56004) 2024-03-11T20:52:10.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.428+0000 7fbb5ffff640 1 -- 172.21.15.90:0/2498203390 learned_addr learned my addr 172.21.15.90:0/2498203390 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T20:52:10.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.428+0000 7fbb5f7fe640 1 --2- 172.21.15.90:0/2498203390 >> v2:172.21.15.90:3300/0 conn(0x7fbb60153cf0 0x7fbb6015d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:52:10.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.432+0000 7fbb5f7fe640 1 -- 172.21.15.90:0/2498203390 >> v2:172.21.15.92:3300/0 conn(0x7fbb600c4b60 msgr2=0x7fbb6015a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:52:10.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.432+0000 7fbb5f7fe640 1 --2- 172.21.15.90:0/2498203390 >> v2:172.21.15.92:3300/0 conn(0x7fbb600c4b60 0x7fbb6015a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:10.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.432+0000 7fbb5f7fe640 1 -- 172.21.15.90:0/2498203390 >> v1:172.21.15.90:6789/0 conn(0x7fbb600c42f0 legacy=0x7fbb600ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:10.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.432+0000 7fbb5f7fe640 1 -- 172.21.15.90:0/2498203390 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb6015aa80 con 0x7fbb60153cf0 2024-03-11T20:52:10.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.436+0000 7fbb5f7fe640 1 --2- 172.21.15.90:0/2498203390 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb60153cf0 0x7fbb6015d120 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbb60157760 tx=0x7fbb5401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0ce79f30419ffea server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:52:10.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.444+0000 7fbb5e7fc640 1 -- 172.21.15.90:0/2498203390 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb540ab600 con 0x7fbb60153cf0 2024-03-11T20:52:10.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.444+0000 7fbb5e7fc640 1 -- 172.21.15.90:0/2498203390 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fbb540a7050 con 0x7fbb60153cf0 2024-03-11T20:52:10.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.444+0000 7fbb5e7fc640 1 -- 172.21.15.90:0/2498203390 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb540ab9a0 con 0x7fbb60153cf0 2024-03-11T20:52:10.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.444+0000 7fbb661d4640 1 -- 172.21.15.90:0/2498203390 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb60153cf0 msgr2=0x7fbb6015d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:10.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.444+0000 7fbb661d4640 1 --2- 172.21.15.90:0/2498203390 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb60153cf0 0x7fbb6015d120 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbb60157760 tx=0x7fbb5401c480 comp rx=0 tx=0).stop 2024-03-11T20:52:10.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.444+0000 7fbb661d4640 1 -- 172.21.15.90:0/2498203390 shutdown_connections 2024-03-11T20:52:10.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.444+0000 7fbb661d4640 1 --2- 172.21.15.90:0/2498203390 >> v2:172.21.15.92:3300/0 conn(0x7fbb600c4b60 0x7fbb6015a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:10.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.444+0000 7fbb661d4640 1 --2- 172.21.15.90:0/2498203390 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb60153cf0 0x7fbb6015d120 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:10.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.444+0000 7fbb661d4640 1 -- 172.21.15.90:0/2498203390 >> 172.21.15.90:0/2498203390 conn(0x7fbb6006b640 msgr2=0x7fbb6006ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:52:10.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.444+0000 7fbb661d4640 1 -- 172.21.15.90:0/2498203390 shutdown_connections 2024-03-11T20:52:10.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.444+0000 7fbb661d4640 1 -- 172.21.15.90:0/2498203390 wait complete. 2024-03-11T20:52:10.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.444+0000 7fbb661d4640 1 Processor -- start 2024-03-11T20:52:10.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.444+0000 7fbb661d4640 1 -- start start 2024-03-11T20:52:10.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.448+0000 7fbb661d4640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb600c42f0 0x7fbb60156260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:10.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.448+0000 7fbb661d4640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbb600c4b60 0x7fbb6015c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:10.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.448+0000 7fbb661d4640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb600c8b80 0x7fbb600c65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:10.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.448+0000 7fbb661d4640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb60158300 con 0x7fbb600c8b80 2024-03-11T20:52:10.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.448+0000 7fbb661d4640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb600c95b0 con 0x7fbb600c42f0 2024-03-11T20:52:10.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.448+0000 7fbb661d4640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb600c92d0 con 0x7fbb600c4b60 2024-03-11T20:52:10.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.448+0000 7fbb5ffff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb600c8b80 0x7fbb600c65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:52:10.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.448+0000 7fbb5ffff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb600c8b80 0x7fbb600c65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60184/0 (socket says 172.21.15.90:60184) 2024-03-11T20:52:10.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.448+0000 7fbb5ffff640 1 -- 172.21.15.90:0/258636484 learned_addr learned my addr 172.21.15.90:0/258636484 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:52:10.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.452+0000 7fbb5ffff640 1 -- 172.21.15.90:0/258636484 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbb600c4b60 msgr2=0x7fbb6015c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:10.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.452+0000 7fbb5ffff640 1 --2- 172.21.15.90:0/258636484 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbb600c4b60 0x7fbb6015c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:10.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.452+0000 7fbb5ffff640 1 -- 172.21.15.90:0/258636484 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb600c42f0 msgr2=0x7fbb60156260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:10.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.452+0000 7fbb5ffff640 1 --2- 172.21.15.90:0/258636484 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb600c42f0 0x7fbb60156260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:10.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.452+0000 7fbb5ffff640 1 -- 172.21.15.90:0/258636484 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb54009420 con 0x7fbb600c8b80 2024-03-11T20:52:10.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.456+0000 7fbb5ffff640 1 --2- 172.21.15.90:0/258636484 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb600c8b80 0x7fbb600c65a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fbb5800cbc0 tx=0x7fbb5800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:52:10.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.456+0000 7fbb47fff640 1 -- 172.21.15.90:0/258636484 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5800ad40 con 0x7fbb600c8b80 2024-03-11T20:52:10.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.456+0000 7fbb661d4640 1 -- 172.21.15.90:0/258636484 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb601579d0 con 0x7fbb600c8b80 2024-03-11T20:52:10.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.460+0000 7fbb661d4640 1 -- 172.21.15.90:0/258636484 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb60157f90 con 0x7fbb600c8b80 2024-03-11T20:52:10.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.460+0000 7fbb661d4640 1 -- 172.21.15.90:0/258636484 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb6015d050 con 0x7fbb600c8b80 2024-03-11T20:52:10.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.460+0000 7fbb47fff640 1 -- 172.21.15.90:0/258636484 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fbb5801c440 con 0x7fbb600c8b80 2024-03-11T20:52:10.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.460+0000 7fbb47fff640 1 -- 172.21.15.90:0/258636484 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5800aea0 con 0x7fbb600c8b80 2024-03-11T20:52:10.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.744+0000 7fbb47fff640 1 -- 172.21.15.90:0/258636484 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fbb58012400 con 0x7fbb600c8b80 2024-03-11T20:52:10.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.744+0000 7fbb47fff640 1 --2- 172.21.15.90:0/258636484 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fbb4c066e40 0x7fbb4c069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:10.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.744+0000 7fbb5f7fe640 1 --2- 172.21.15.90:0/258636484 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fbb4c066e40 0x7fbb4c069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:52:10.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.744+0000 7fbb5f7fe640 1 --2- 172.21.15.90:0/258636484 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fbb4c066e40 0x7fbb4c069300 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fbb540ad000 tx=0x7fbb54004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:52:10.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.804+0000 7fbb47fff640 1 -- 172.21.15.90:0/258636484 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fbb58007ab0 con 0x7fbb600c8b80 2024-03-11T20:52:10.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:10.804+0000 7fbb47fff640 1 -- 172.21.15.90:0/258636484 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbb58052090 con 0x7fbb600c8b80 2024-03-11T20:52:11.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.012+0000 7fbb661d4640 1 -- 172.21.15.90:0/258636484 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbb6004eef0 con 0x7fbb600c8b80 2024-03-11T20:52:11.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.048+0000 7fbb47fff640 1 -- 172.21.15.90:0/258636484 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+6575 (secure 0 0 0) 0x7fbb58017090 con 0x7fbb600c8b80 2024-03-11T20:52:11.046 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T20:52:11.046 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/641527992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":641527992},{"type":"v1","addr":"172.21.15.90:6839","nonce":641527992}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-11T18:35:57.363570+0000","modified":"2024-03-11T18:36:51.575661+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4255,"mds_1":4253,"mds_2":4241,"mds_3":4238,"mds_4":4229},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4229":{"gid":4229,"name":"c","rank":4,"incarnation":18,"state":"up:active","state_seq":17,"addr":"172.21.15.90:6835/460417177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":460417177},{"type":"v1","addr":"172.21.15.90:6835","nonce":460417177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4238":{"gid":4238,"name":"d","rank":3,"incarnation":16,"state":"up:active","state_seq":17,"addr":"172.21.15.92:6835/3568438077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6833","nonce":3568438077},{"type":"v1","addr":"172.21.15.92:6835","nonce":3568438077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4241":{"gid":4241,"name":"b","rank":2,"incarnation":14,"state":"up:active","state_seq":16,"addr":"172.21.15.92:6834/2306487548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6832","nonce":2306487548},{"type":"v1","addr":"172.21.15.92:6834","nonce":2306487548}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4253":{"gid":4253,"name":"f","rank":1,"incarnation":10,"state":"up:active","state_seq":15,"addr":"172.21.15.92:6837/3209534444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6836","nonce":3209534444},{"type":"v1","addr":"172.21.15.92:6837","nonce":3209534444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_4255":{"gid":4255,"name":"e","rank":0,"incarnation":5,"state":"up:active","state_seq":6,"addr":"172.21.15.90:6837/3485943375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":3485943375},{"type":"v1","addr":"172.21.15.90:6837","nonce":3485943375}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-11T20:52:11.046 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 19 2024-03-11T20:52:11.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.048+0000 7fbb661d4640 1 -- 172.21.15.90:0/258636484 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fbb4c066e40 msgr2=0x7fbb4c069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:11.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.048+0000 7fbb661d4640 1 --2- 172.21.15.90:0/258636484 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fbb4c066e40 0x7fbb4c069300 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fbb540ad000 tx=0x7fbb54004740 comp rx=0 tx=0).stop 2024-03-11T20:52:11.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.048+0000 7fbb661d4640 1 -- 172.21.15.90:0/258636484 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb600c8b80 msgr2=0x7fbb600c65a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:11.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.048+0000 7fbb661d4640 1 --2- 172.21.15.90:0/258636484 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb600c8b80 0x7fbb600c65a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fbb5800cbc0 tx=0x7fbb5800a430 comp rx=0 tx=0).stop 2024-03-11T20:52:11.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.048+0000 7fbb661d4640 1 -- 172.21.15.90:0/258636484 shutdown_connections 2024-03-11T20:52:11.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.048+0000 7fbb661d4640 1 --2- 172.21.15.90:0/258636484 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fbb4c066e40 0x7fbb4c069300 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:11.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.048+0000 7fbb661d4640 1 --2- 172.21.15.90:0/258636484 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb600c8b80 0x7fbb600c65a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:11.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.048+0000 7fbb661d4640 1 --2- 172.21.15.90:0/258636484 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fbb600c4b60 0x7fbb6015c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:11.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.048+0000 7fbb661d4640 1 --2- 172.21.15.90:0/258636484 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fbb600c42f0 0x7fbb60156260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:11.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.048+0000 7fbb661d4640 1 -- 172.21.15.90:0/258636484 >> 172.21.15.90:0/258636484 conn(0x7fbb6006b640 msgr2=0x7fbb60158840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:52:11.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.048+0000 7fbb661d4640 1 -- 172.21.15.90:0/258636484 shutdown_connections 2024-03-11T20:52:11.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.048+0000 7fbb661d4640 1 -- 172.21.15.90:0/258636484 wait complete. 2024-03-11T20:52:11.067 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-11T20:52:11.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.188+0000 7f29da7f3640 1 Processor -- start 2024-03-11T20:52:11.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.188+0000 7f29da7f3640 1 -- start start 2024-03-11T20:52:11.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.192+0000 7f29da7f3640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f29d40c8e20 0x7f29d415a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:11.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.192+0000 7f29da7f3640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f29d40afb60 0x7f29d406c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:11.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.192+0000 7f29da7f3640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29d415abb0 con 0x7f29d4153f10 2024-03-11T20:52:11.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.192+0000 7f29da7f3640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29d406c2d0 con 0x7f29d40c8e20 2024-03-11T20:52:11.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.192+0000 7f29da7f3640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29d40c81e0 con 0x7f29d40afb60 2024-03-11T20:52:11.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.192+0000 7f29d37fe640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f29d40c8e20 0x7f29d415a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:52:11.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.192+0000 7f29d37fe640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f29d40c8e20 0x7f29d415a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60188/0 (socket says 172.21.15.90:60188) 2024-03-11T20:52:11.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.192+0000 7f29d37fe640 1 -- 172.21.15.90:0/2923212387 learned_addr learned my addr 172.21.15.90:0/2923212387 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:52:11.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.196+0000 7f29d37fe640 1 -- 172.21.15.90:0/2923212387 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f29d40afb60 msgr2=0x7f29d406c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:52:11.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.196+0000 7f29d37fe640 1 --2- 172.21.15.90:0/2923212387 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f29d40afb60 0x7f29d406c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:11.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.196+0000 7f29d37fe640 1 -- 172.21.15.90:0/2923212387 >> v1:172.21.15.92:6789/0 conn(0x7f29d4153f10 legacy=0x7f29d415db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:52:11.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.196+0000 7f29d37fe640 1 -- 172.21.15.90:0/2923212387 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d406bd10 con 0x7f29d40c8e20 2024-03-11T20:52:11.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.196+0000 7f29d37fe640 1 --2- 172.21.15.90:0/2923212387 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29d40c8e20 0x7f29d415a670 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f29d406e2a0 tx=0x7f29bc01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b568514251bf4bfe server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:52:11.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.204+0000 7f29d2ffd640 1 -- 172.21.15.90:0/2923212387 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc0aa620 con 0x7f29d40c8e20 2024-03-11T20:52:11.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.204+0000 7f29d2ffd640 1 -- 172.21.15.90:0/2923212387 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f29bc0aa780 con 0x7f29d40c8e20 2024-03-11T20:52:11.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.204+0000 7f29d2ffd640 1 -- 172.21.15.90:0/2923212387 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc0aab30 con 0x7f29d40c8e20 2024-03-11T20:52:11.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.204+0000 7f29da7f3640 1 -- 172.21.15.90:0/2923212387 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29d40c8e20 msgr2=0x7f29d415a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:11.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.204+0000 7f29da7f3640 1 --2- 172.21.15.90:0/2923212387 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29d40c8e20 0x7f29d415a670 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f29d406e2a0 tx=0x7f29bc01c500 comp rx=0 tx=0).stop 2024-03-11T20:52:11.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.204+0000 7f29da7f3640 1 -- 172.21.15.90:0/2923212387 shutdown_connections 2024-03-11T20:52:11.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.204+0000 7f29da7f3640 1 --2- 172.21.15.90:0/2923212387 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f29d40afb60 0x7f29d406c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:11.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.204+0000 7f29da7f3640 1 --2- 172.21.15.90:0/2923212387 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29d40c8e20 0x7f29d415a670 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:11.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.204+0000 7f29da7f3640 1 -- 172.21.15.90:0/2923212387 >> 172.21.15.90:0/2923212387 conn(0x7f29d4070e80 msgr2=0x7f29d4066990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:52:11.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.204+0000 7f29da7f3640 1 -- 172.21.15.90:0/2923212387 shutdown_connections 2024-03-11T20:52:11.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.204+0000 7f29da7f3640 1 -- 172.21.15.90:0/2923212387 wait complete. 2024-03-11T20:52:11.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.204+0000 7f29da7f3640 1 Processor -- start 2024-03-11T20:52:11.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.204+0000 7f29da7f3640 1 -- start start 2024-03-11T20:52:11.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.208+0000 7f29da7f3640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f29d40afb60 0x7f29d41f6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:11.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.208+0000 7f29da7f3640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29d40c8e20 0x7f29d4158260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:11.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.208+0000 7f29da7f3640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29d4153f10 0x7f29d4144770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:11.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.208+0000 7f29da7f3640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d4157dd0 con 0x7f29d4153f10 2024-03-11T20:52:11.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.208+0000 7f29da7f3640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d415c270 con 0x7f29d40c8e20 2024-03-11T20:52:11.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.208+0000 7f29da7f3640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29d41f6af0 con 0x7f29d40afb60 2024-03-11T20:52:11.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.208+0000 7f29d8d90640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29d4153f10 0x7f29d4144770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:52:11.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.208+0000 7f29d8d90640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29d4153f10 0x7f29d4144770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60192/0 (socket says 172.21.15.90:60192) 2024-03-11T20:52:11.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.208+0000 7f29d8d90640 1 -- 172.21.15.90:0/455242087 learned_addr learned my addr 172.21.15.90:0/455242087 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:52:11.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.212+0000 7f29d8d90640 1 -- 172.21.15.90:0/455242087 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f29d40afb60 msgr2=0x7f29d41f6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:11.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.212+0000 7f29d8d90640 1 --2- 172.21.15.90:0/455242087 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f29d40afb60 0x7f29d41f6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:11.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.212+0000 7f29d8d90640 1 -- 172.21.15.90:0/455242087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29d40c8e20 msgr2=0x7f29d4158260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:11.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.212+0000 7f29d8d90640 1 --2- 172.21.15.90:0/455242087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29d40c8e20 0x7f29d4158260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:11.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.212+0000 7f29d8d90640 1 -- 172.21.15.90:0/455242087 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29bc0094d0 con 0x7f29d4153f10 2024-03-11T20:52:11.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.212+0000 7f29d8d90640 1 --2- 172.21.15.90:0/455242087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29d4153f10 0x7f29d4144770 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f29c400b430 tx=0x7f29c400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:52:11.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.216+0000 7f29d0ff9640 1 -- 172.21.15.90:0/455242087 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c4011830 con 0x7f29d4153f10 2024-03-11T20:52:11.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.216+0000 7f29da7f3640 1 -- 172.21.15.90:0/455242087 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29d41f6dd0 con 0x7f29d4153f10 2024-03-11T20:52:11.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.216+0000 7f29da7f3640 1 -- 172.21.15.90:0/455242087 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29d415c550 con 0x7f29d4153f10 2024-03-11T20:52:11.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.216+0000 7f29da7f3640 1 -- 172.21.15.90:0/455242087 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29d404eef0 con 0x7f29d4153f10 2024-03-11T20:52:11.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.220+0000 7f29d0ff9640 1 -- 172.21.15.90:0/455242087 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f29c4011990 con 0x7f29d4153f10 2024-03-11T20:52:11.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.508+0000 7f29d0ff9640 1 -- 172.21.15.90:0/455242087 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c4009280 con 0x7f29d4153f10 2024-03-11T20:52:11.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.508+0000 7f29d0ff9640 1 -- 172.21.15.90:0/455242087 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f29c401a400 con 0x7f29d4153f10 2024-03-11T20:52:11.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.512+0000 7f29d0ff9640 1 --2- 172.21.15.90:0/455242087 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f29b4066e70 0x7f29b4069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:11.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.512+0000 7f29d3fff640 1 --2- 172.21.15.90:0/455242087 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f29b4066e70 0x7f29b4069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:52:11.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.512+0000 7f29d3fff640 1 --2- 172.21.15.90:0/455242087 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f29b4066e70 0x7f29b4069330 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f29cc00abb0 tx=0x7f29cc009250 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:52:11.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.544+0000 7f29d0ff9640 1 -- 172.21.15.90:0/455242087 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f29c4056510 con 0x7f29d4153f10 2024-03-11T20:52:11.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.544+0000 7f29d0ff9640 1 -- 172.21.15.90:0/455242087 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f29c4052020 con 0x7f29d4153f10 2024-03-11T20:52:11.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.736+0000 7f29da7f3640 1 -- 172.21.15.90:0/455242087 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f29d40c84f0 con 0x7f29d4153f10 2024-03-11T20:52:11.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.824+0000 7f29d0ff9640 1 -- 172.21.15.90:0/455242087 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+18819 (secure 0 0 0) 0x7f29c4053f50 con 0x7f29d4153f10 2024-03-11T20:52:11.822 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-11T20:52:11.822 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":30,"fsid":"76d56b10-6c29-476a-804b-8482ea90f9e2","created":"2024-03-11T18:34:54.521741+0000","modified":"2024-03-11T18:37:03.877940+0000","last_up_change":"2024-03-11T18:35:16.207500+0000","last_in_change":"2024-03-11T18:35:05.213634+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-11T18:35:16.193842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-11T18:35:54.001469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-11T18:35:55.917321+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-11T18:36:02.210943+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"34decff9-7280-4d84-9665-d6af8ea6de84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6819","nonce":1755667921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6821","nonce":1755667921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6825","nonce":1755667921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1755667921},{"type":"v1","addr":"172.21.15.90:6823","nonce":1755667921}]},"public_addr":"172.21.15.90:6819/1755667921","cluster_addr":"172.21.15.90:6821/1755667921","heartbeat_back_addr":"172.21.15.90:6825/1755667921","heartbeat_front_addr":"172.21.15.90:6823/1755667921","state":["exists","up"]},{"osd":1,"uuid":"8ecdce9e-5486-415a-a609-dd16e945440e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6807","nonce":438682081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6809","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6811","nonce":438682081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6817","nonce":438682081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":438682081},{"type":"v1","addr":"172.21.15.90:6815","nonce":438682081}]},"public_addr":"172.21.15.90:6807/438682081","cluster_addr":"172.21.15.90:6811/438682081","heartbeat_back_addr":"172.21.15.90:6817/438682081","heartbeat_front_addr":"172.21.15.90:6815/438682081","state":["exists","up"]},{"osd":2,"uuid":"737486d4-0e90-4c82-9e79-bf37c812e5ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6827","nonce":2837704738}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6829","nonce":2837704738}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6833","nonce":2837704738}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":2837704738},{"type":"v1","addr":"172.21.15.90:6831","nonce":2837704738}]},"public_addr":"172.21.15.90:6827/2837704738","cluster_addr":"172.21.15.90:6829/2837704738","heartbeat_back_addr":"172.21.15.90:6833/2837704738","heartbeat_front_addr":"172.21.15.90:6831/2837704738","state":["exists","up"]},{"osd":3,"uuid":"698bd661-bbef-4699-a619-f7bfe23ce12c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6803","nonce":240987971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6805","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6806","nonce":240987971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6813","nonce":240987971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":240987971},{"type":"v1","addr":"172.21.15.90:6810","nonce":240987971}]},"public_addr":"172.21.15.90:6803/240987971","cluster_addr":"172.21.15.90:6806/240987971","heartbeat_back_addr":"172.21.15.90:6813/240987971","heartbeat_front_addr":"172.21.15.90:6810/240987971","state":["exists","up"]},{"osd":4,"uuid":"aba9e31f-73d9-43fe-a10b-2dc5832968b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6801","nonce":502685275}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6803","nonce":502685275}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6807","nonce":502685275}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":502685275},{"type":"v1","addr":"172.21.15.92:6805","nonce":502685275}]},"public_addr":"172.21.15.92:6801/502685275","cluster_addr":"172.21.15.92:6803/502685275","heartbeat_back_addr":"172.21.15.92:6807/502685275","heartbeat_front_addr":"172.21.15.92:6805/502685275","state":["exists","up"]},{"osd":5,"uuid":"bdd56766-4b28-4007-8de3-b5b08cf335d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6809","nonce":200142070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6811","nonce":200142070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6815","nonce":200142070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6812","nonce":200142070},{"type":"v1","addr":"172.21.15.92:6813","nonce":200142070}]},"public_addr":"172.21.15.92:6809/200142070","cluster_addr":"172.21.15.92:6811/200142070","heartbeat_back_addr":"172.21.15.92:6815/200142070","heartbeat_front_addr":"172.21.15.92:6813/200142070","state":["exists","up"]},{"osd":6,"uuid":"6c580a59-610f-455e-a96b-033299dcc118","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6816","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6817","nonce":1919010204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6819","nonce":1919010204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6823","nonce":1919010204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6820","nonce":1919010204},{"type":"v1","addr":"172.21.15.92:6821","nonce":1919010204}]},"public_addr":"172.21.15.92:6817/1919010204","cluster_addr":"172.21.15.92:6819/1919010204","heartbeat_back_addr":"172.21.15.92:6823/1919010204","heartbeat_front_addr":"172.21.15.92:6821/1919010204","state":["exists","up"]},{"osd":7,"uuid":"958184cc-aebf-4c5b-91a0-2a3e38a6b42b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6825","nonce":3475925719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6827","nonce":3475925719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6830","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6831","nonce":3475925719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6828","nonce":3475925719},{"type":"v1","addr":"172.21.15.92:6829","nonce":3475925719}]},"public_addr":"172.21.15.92:6825/3475925719","cluster_addr":"172.21.15.92:6827/3475925719","heartbeat_back_addr":"172.21.15.92:6831/3475925719","heartbeat_front_addr":"172.21.15.92:6829/3475925719","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.816294+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.048084+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.137975+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.968246+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.016829+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:10.931676+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.041673+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-11T18:35:11.079404+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.33","mappings":[{"from":3,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-11T20:52:11.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.824+0000 7f29da7f3640 1 -- 172.21.15.90:0/455242087 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f29b4066e70 msgr2=0x7f29b4069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:11.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.824+0000 7f29da7f3640 1 --2- 172.21.15.90:0/455242087 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f29b4066e70 0x7f29b4069330 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f29cc00abb0 tx=0x7f29cc009250 comp rx=0 tx=0).stop 2024-03-11T20:52:11.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.824+0000 7f29da7f3640 1 -- 172.21.15.90:0/455242087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29d4153f10 msgr2=0x7f29d4144770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:11.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.824+0000 7f29da7f3640 1 --2- 172.21.15.90:0/455242087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29d4153f10 0x7f29d4144770 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f29c400b430 tx=0x7f29c400b8f0 comp rx=0 tx=0).stop 2024-03-11T20:52:11.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.824+0000 7f29da7f3640 1 -- 172.21.15.90:0/455242087 shutdown_connections 2024-03-11T20:52:11.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.824+0000 7f29da7f3640 1 --2- 172.21.15.90:0/455242087 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f29b4066e70 0x7f29b4069330 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:11.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.824+0000 7f29da7f3640 1 --2- 172.21.15.90:0/455242087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f29d4153f10 0x7f29d4144770 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:11.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.824+0000 7f29da7f3640 1 --2- 172.21.15.90:0/455242087 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f29d40c8e20 0x7f29d4158260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:11.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.824+0000 7f29da7f3640 1 --2- 172.21.15.90:0/455242087 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f29d40afb60 0x7f29d41f6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:11.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.824+0000 7f29da7f3640 1 -- 172.21.15.90:0/455242087 >> 172.21.15.90:0/455242087 conn(0x7f29d4070e80 msgr2=0x7f29d414b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:52:11.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.824+0000 7f29da7f3640 1 -- 172.21.15.90:0/455242087 shutdown_connections 2024-03-11T20:52:11.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.824+0000 7f29da7f3640 1 -- 172.21.15.90:0/455242087 wait complete. 2024-03-11T20:52:11.847 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-11T20:52:11.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.976+0000 7f28d6cc1640 1 Processor -- start 2024-03-11T20:52:11.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.976+0000 7f28d6cc1640 1 -- start start 2024-03-11T20:52:11.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.976+0000 7f28d6cc1640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f28d0153d90 0x7f28d00c48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:11.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.976+0000 7f28d6cc1640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28d00cc1a0 con 0x7f28d00ca310 2024-03-11T20:52:11.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.976+0000 7f28d6cc1640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28d00cb8e0 con 0x7f28d00cc630 2024-03-11T20:52:11.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.976+0000 7f28d6cc1640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28d00c4320 con 0x7f28d0153d90 2024-03-11T20:52:11.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.976+0000 7f28d4a5d640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f28d0153d90 0x7f28d00c48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:52:11.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.976+0000 7f28cffff640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f28d00ca310 0x7f28d015db20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:56016/0 (socket says 172.21.15.90:56016) 2024-03-11T20:52:11.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.976+0000 7f28cffff640 1 -- 172.21.15.90:0/744788858 learned_addr learned my addr 172.21.15.90:0/744788858 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T20:52:11.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.980+0000 7f28d4a5d640 1 -- 172.21.15.90:0/744788858 >> v1:172.21.15.92:6789/0 conn(0x7f28d00cc630 legacy=0x7f28d00cf370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:11.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.980+0000 7f28d4a5d640 1 -- 172.21.15.90:0/744788858 >> v1:172.21.15.90:6789/0 conn(0x7f28d00ca310 legacy=0x7f28d015db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:11.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.980+0000 7f28d4a5d640 1 -- 172.21.15.90:0/744788858 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d00c45e0 con 0x7f28d0153d90 2024-03-11T20:52:11.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.984+0000 7f28d4a5d640 1 --2- 172.21.15.90:0/744788858 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f28d0153d90 0x7f28d00c48c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f28b80097c0 tx=0x7f28b801ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30f3d592ecc8b84 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:52:11.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.988+0000 7f28cf7fe640 1 -- 172.21.15.90:0/744788858 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b80a8990 con 0x7f28d0153d90 2024-03-11T20:52:11.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.988+0000 7f28cf7fe640 1 -- 172.21.15.90:0/744788858 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f28b80a8af0 con 0x7f28d0153d90 2024-03-11T20:52:11.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.988+0000 7f28cf7fe640 1 -- 172.21.15.90:0/744788858 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b80a8ea0 con 0x7f28d0153d90 2024-03-11T20:52:11.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.988+0000 7f28d6cc1640 1 -- 172.21.15.90:0/744788858 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f28d0153d90 msgr2=0x7f28d00c48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:11.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.988+0000 7f28d6cc1640 1 --2- 172.21.15.90:0/744788858 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f28d0153d90 0x7f28d00c48c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f28b80097c0 tx=0x7f28b801ec90 comp rx=0 tx=0).stop 2024-03-11T20:52:11.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.992+0000 7f28d6cc1640 1 -- 172.21.15.90:0/744788858 shutdown_connections 2024-03-11T20:52:11.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.992+0000 7f28d6cc1640 1 --2- 172.21.15.90:0/744788858 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f28d0153d90 0x7f28d00c48c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:11.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.992+0000 7f28d6cc1640 1 -- 172.21.15.90:0/744788858 >> 172.21.15.90:0/744788858 conn(0x7f28d006d770 msgr2=0x7f28d006b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:52:11.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.992+0000 7f28d6cc1640 1 -- 172.21.15.90:0/744788858 shutdown_connections 2024-03-11T20:52:11.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.992+0000 7f28d6cc1640 1 -- 172.21.15.90:0/744788858 wait complete. 2024-03-11T20:52:11.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.992+0000 7f28d6cc1640 1 Processor -- start 2024-03-11T20:52:11.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.992+0000 7f28d6cc1640 1 -- start start 2024-03-11T20:52:11.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.992+0000 7f28d6cc1640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f28d00ca310 0x7f28d0148340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:11.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.992+0000 7f28d6cc1640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28d00cc630 0x7f28d01f33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:11.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.992+0000 7f28d6cc1640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f28d0153d90 0x7f28d01525e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:11.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.992+0000 7f28d6cc1640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d01f06e0 con 0x7f28d0153d90 2024-03-11T20:52:11.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.992+0000 7f28d6cc1640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d01579b0 con 0x7f28d00cc630 2024-03-11T20:52:11.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.992+0000 7f28d6cc1640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28d014afb0 con 0x7f28d00ca310 2024-03-11T20:52:11.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.992+0000 7f28d525e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f28d0153d90 0x7f28d01525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:52:11.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.992+0000 7f28d525e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f28d0153d90 0x7f28d01525e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60202/0 (socket says 172.21.15.90:60202) 2024-03-11T20:52:11.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.992+0000 7f28d525e640 1 -- 172.21.15.90:0/988503115 learned_addr learned my addr 172.21.15.90:0/988503115 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:52:11.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.992+0000 7f28d4a5d640 1 --2- 172.21.15.90:0/988503115 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f28d00ca310 0x7f28d0148340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:52:11.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.996+0000 7f28cffff640 1 --2- 172.21.15.90:0/988503115 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28d00cc630 0x7f28d01f33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:52:11.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.996+0000 7f28d525e640 1 -- 172.21.15.90:0/988503115 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f28d00ca310 msgr2=0x7f28d0148340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:11.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.996+0000 7f28d525e640 1 --2- 172.21.15.90:0/988503115 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f28d00ca310 0x7f28d0148340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:11.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.996+0000 7f28d525e640 1 -- 172.21.15.90:0/988503115 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28d00cc630 msgr2=0x7f28d01f33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:11.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.996+0000 7f28d525e640 1 --2- 172.21.15.90:0/988503115 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28d00cc630 0x7f28d01f33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:11.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:11.996+0000 7f28d525e640 1 -- 172.21.15.90:0/988503115 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28b8009420 con 0x7f28d0153d90 2024-03-11T20:52:11.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:12.000+0000 7f28d525e640 1 --2- 172.21.15.90:0/988503115 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f28d0153d90 0x7f28d01525e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f28c800c710 tx=0x7f28c800cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:52:12.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:12.004+0000 7f28cd7fa640 1 -- 172.21.15.90:0/988503115 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c8006d60 con 0x7f28d0153d90 2024-03-11T20:52:12.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:12.004+0000 7f28cd7fa640 1 -- 172.21.15.90:0/988503115 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f28c800ce20 con 0x7f28d0153d90 2024-03-11T20:52:12.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:12.004+0000 7f28cd7fa640 1 -- 172.21.15.90:0/988503115 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c8003be0 con 0x7f28d0153d90 2024-03-11T20:52:12.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:12.004+0000 7f28d6cc1640 1 -- 172.21.15.90:0/988503115 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28d01efb60 con 0x7f28d0153d90 2024-03-11T20:52:12.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:12.004+0000 7f28d6cc1640 1 -- 172.21.15.90:0/988503115 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28d01f0120 con 0x7f28d0153d90 2024-03-11T20:52:12.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:12.004+0000 7f28d6cc1640 1 -- 172.21.15.90:0/988503115 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28d004eef0 con 0x7f28d0153d90 2024-03-11T20:52:12.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:12.292+0000 7f28cd7fa640 1 -- 172.21.15.90:0/988503115 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f28c8003d40 con 0x7f28d0153d90 2024-03-11T20:52:12.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:12.292+0000 7f28cd7fa640 1 --2- 172.21.15.90:0/988503115 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f28b0066e90 0x7f28b0069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:12.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:12.292+0000 7f28d4a5d640 1 --2- 172.21.15.90:0/988503115 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f28b0066e90 0x7f28b0069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:52:12.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:12.292+0000 7f28d4a5d640 1 --2- 172.21.15.90:0/988503115 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f28b0066e90 0x7f28b0069350 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f28b80b2000 tx=0x7f28b80aa070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:52:12.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:12.324+0000 7f28cd7fa640 1 -- 172.21.15.90:0/988503115 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f28c8055f30 con 0x7f28d0153d90 2024-03-11T20:52:12.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:12.328+0000 7f28cd7fa640 1 -- 172.21.15.90:0/988503115 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f28c80521d0 con 0x7f28d0153d90 2024-03-11T20:52:12.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:12.520+0000 7f28d6cc1640 1 -- 172.21.15.90:0/988503115 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f28d015a7c0 con 0x7f28d0153d90 2024-03-11T20:52:12.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:12.924+0000 7f28cd7fa640 1 -- 172.21.15.90:0/988503115 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v20) v1 ==== 161+0+0 (secure 0 0 0) 0x7f28c8055890 con 0x7f28d0153d90 2024-03-11T20:52:12.925 INFO:teuthology.orchestra.run.smithi090.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-11T20:52:12.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:12.928+0000 7f28d6cc1640 1 -- 172.21.15.90:0/988503115 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f28b0066e90 msgr2=0x7f28b0069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:12.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:12.928+0000 7f28d6cc1640 1 --2- 172.21.15.90:0/988503115 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f28b0066e90 0x7f28b0069350 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f28b80b2000 tx=0x7f28b80aa070 comp rx=0 tx=0).stop 2024-03-11T20:52:12.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:12.928+0000 7f28d6cc1640 1 -- 172.21.15.90:0/988503115 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f28d0153d90 msgr2=0x7f28d01525e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:12.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:12.928+0000 7f28d6cc1640 1 --2- 172.21.15.90:0/988503115 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f28d0153d90 0x7f28d01525e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f28c800c710 tx=0x7f28c800cbd0 comp rx=0 tx=0).stop 2024-03-11T20:52:12.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:12.928+0000 7f28d6cc1640 1 -- 172.21.15.90:0/988503115 shutdown_connections 2024-03-11T20:52:12.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:12.928+0000 7f28d6cc1640 1 --2- 172.21.15.90:0/988503115 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f28b0066e90 0x7f28b0069350 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:12.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:12.928+0000 7f28d6cc1640 1 --2- 172.21.15.90:0/988503115 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f28d0153d90 0x7f28d01525e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:12.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:12.928+0000 7f28d6cc1640 1 --2- 172.21.15.90:0/988503115 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f28d00cc630 0x7f28d01f33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:12.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:12.928+0000 7f28d6cc1640 1 --2- 172.21.15.90:0/988503115 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f28d00ca310 0x7f28d0148340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:12.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:12.928+0000 7f28d6cc1640 1 -- 172.21.15.90:0/988503115 >> 172.21.15.90:0/988503115 conn(0x7f28d006d770 msgr2=0x7f28d0149d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:52:12.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:12.928+0000 7f28d6cc1640 1 -- 172.21.15.90:0/988503115 shutdown_connections 2024-03-11T20:52:12.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:12.932+0000 7f28d6cc1640 1 -- 172.21.15.90:0/988503115 wait complete. 2024-03-11T20:52:12.953 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-11T20:52:13.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.088+0000 7f095d132640 1 Processor -- start 2024-03-11T20:52:13.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.088+0000 7f095d132640 1 -- start start 2024-03-11T20:52:13.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.088+0000 7f095d132640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7f09580c65b0 0x7f09580c37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:13.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.088+0000 7f095d132640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f09580c3d30 0x7f09580cc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:13.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.088+0000 7f095d132640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09580c5e60 con 0x7f0958150a90 2024-03-11T20:52:13.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.088+0000 7f095d132640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0958156a60 con 0x7f09580c65b0 2024-03-11T20:52:13.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.088+0000 7f095d132640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09580c55c0 con 0x7f09580c3d30 2024-03-11T20:52:13.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.088+0000 7f0956d9d640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f0958150a90 0x7f0958157220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:56030/0 (socket says 172.21.15.90:56030) 2024-03-11T20:52:13.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.088+0000 7f0956d9d640 1 -- 172.21.15.90:0/3496350675 learned_addr learned my addr 172.21.15.90:0/3496350675 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T20:52:13.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.092+0000 7f0955d9b640 1 -- 172.21.15.90:0/3496350675 <== mon.0 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1724223141 0 0) 0x7f09580c5e60 con 0x7f0958150a90 2024-03-11T20:52:13.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.092+0000 7f0955d9b640 1 -- 172.21.15.90:0/3496350675 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0944003530 con 0x7f0958150a90 2024-03-11T20:52:13.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.096+0000 7f095659c640 1 --2- 172.21.15.90:0/3496350675 >> v2:172.21.15.92:3300/0 conn(0x7f09580c65b0 0x7f09580c37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:52:13.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.100+0000 7f0955d9b640 1 -- 172.21.15.90:0/3496350675 <== mon.0 v1:172.21.15.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2690218723 0 0) 0x7f0944003530 con 0x7f0958150a90 2024-03-11T20:52:13.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.100+0000 7f0955d9b640 1 -- 172.21.15.90:0/3496350675 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f09580c3d30 msgr2=0x7f09580cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:52:13.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.100+0000 7f0955d9b640 1 --2- 172.21.15.90:0/3496350675 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f09580c3d30 0x7f09580cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:13.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.100+0000 7f0955d9b640 1 -- 172.21.15.90:0/3496350675 >> v2:172.21.15.92:3300/0 conn(0x7f09580c65b0 msgr2=0x7f09580c37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:52:13.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.100+0000 7f0955d9b640 1 --2- 172.21.15.90:0/3496350675 >> v2:172.21.15.92:3300/0 conn(0x7f09580c65b0 0x7f09580c37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:13.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.100+0000 7f0955d9b640 1 -- 172.21.15.90:0/3496350675 --> v1:172.21.15.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09580c58a0 con 0x7f0958150a90 2024-03-11T20:52:13.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.100+0000 7f0955d9b640 1 -- 172.21.15.90:0/3496350675 <== mon.0 v1:172.21.15.90:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 691735435 0 0) 0x7f0950001680 con 0x7f0958150a90 2024-03-11T20:52:13.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.100+0000 7f0955d9b640 1 -- 172.21.15.90:0/3496350675 >> v1:172.21.15.90:6789/0 conn(0x7f0958150a90 legacy=0x7f0958157220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:13.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.100+0000 7f0955d9b640 1 --2- 172.21.15.90:0/3496350675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0944003980 0x7f0944003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:13.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.100+0000 7f0955d9b640 1 --2- 172.21.15.90:0/3496350675 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09440047d0 0x7f0944004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:13.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.100+0000 7f0955d9b640 1 --2- 172.21.15.90:0/3496350675 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f09580c3d30 0x7f0944005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:13.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.100+0000 7f0955d9b640 1 -- 172.21.15.90:0/3496350675 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0958156a60 con 0x7f09440047d0 2024-03-11T20:52:13.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.100+0000 7f0955d9b640 1 -- 172.21.15.90:0/3496350675 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09580c55c0 con 0x7f0944003980 2024-03-11T20:52:13.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.100+0000 7f0955d9b640 1 -- 172.21.15.90:0/3496350675 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f094400baf0 con 0x7f09580c3d30 2024-03-11T20:52:13.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.100+0000 7f095659c640 1 --2- 172.21.15.90:0/3496350675 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09440047d0 0x7f0944004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:52:13.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.108+0000 7f095659c640 1 -- 172.21.15.90:0/3496350675 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f09580c3d30 msgr2=0x7f0944005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:52:13.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.108+0000 7f095659c640 1 --2- 172.21.15.90:0/3496350675 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f09580c3d30 0x7f0944005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:13.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.108+0000 7f095659c640 1 -- 172.21.15.90:0/3496350675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0944003980 msgr2=0x7f0944003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:52:13.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.108+0000 7f095659c640 1 --2- 172.21.15.90:0/3496350675 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f0944003980 0x7f0944003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:13.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.108+0000 7f095659c640 1 -- 172.21.15.90:0/3496350675 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0944003530 con 0x7f09440047d0 2024-03-11T20:52:13.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.112+0000 7f095659c640 1 --2- 172.21.15.90:0/3496350675 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09440047d0 0x7f0944004c50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f09580c69b0 tx=0x7f094001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be84cd6e8785ffb4 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:52:13.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.116+0000 7f0955d9b640 1 -- 172.21.15.90:0/3496350675 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09400a5620 con 0x7f09440047d0 2024-03-11T20:52:13.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.116+0000 7f0955d9b640 1 -- 172.21.15.90:0/3496350675 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f09400a5780 con 0x7f09440047d0 2024-03-11T20:52:13.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.116+0000 7f0955d9b640 1 -- 172.21.15.90:0/3496350675 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09400a5b90 con 0x7f09440047d0 2024-03-11T20:52:13.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.116+0000 7f095d132640 1 -- 172.21.15.90:0/3496350675 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09440047d0 msgr2=0x7f0944004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:13.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.116+0000 7f095d132640 1 --2- 172.21.15.90:0/3496350675 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09440047d0 0x7f0944004c50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f09580c69b0 tx=0x7f094001ca30 comp rx=0 tx=0).stop 2024-03-11T20:52:13.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.116+0000 7f095759e640 1 -- 172.21.15.90:0/3496350675 reap_dead start 2024-03-11T20:52:13.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.116+0000 7f095d132640 1 -- 172.21.15.90:0/3496350675 shutdown_connections 2024-03-11T20:52:13.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.116+0000 7f095d132640 1 -- 172.21.15.90:0/3496350675 >> 172.21.15.90:0/3496350675 conn(0x7f0958071aa0 msgr2=0x7f0958071eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:52:13.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.116+0000 7f095d132640 1 -- 172.21.15.90:0/3496350675 shutdown_connections 2024-03-11T20:52:13.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.116+0000 7f095d132640 1 -- 172.21.15.90:0/3496350675 wait complete. 2024-03-11T20:52:13.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.120+0000 7f095d132640 1 Processor -- start 2024-03-11T20:52:13.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.120+0000 7f095d132640 1 -- start start 2024-03-11T20:52:13.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.120+0000 7f095d132640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09440047d0 0x7f09581e4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:13.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.120+0000 7f095d132640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f095814f980 0x7f0958144ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:13.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.120+0000 7f095d132640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f095814ecd0 0x7f0958141a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:13.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.120+0000 7f095d132640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0958145b60 con 0x7f09440047d0 2024-03-11T20:52:13.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.120+0000 7f095d132640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09581501c0 con 0x7f095814ecd0 2024-03-11T20:52:13.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.120+0000 7f095d132640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09581e48b0 con 0x7f095814f980 2024-03-11T20:52:13.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.120+0000 7f0956d9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09440047d0 0x7f09581e4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:52:13.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.120+0000 7f095759e640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f095814ecd0 0x7f0958141a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:52:13.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.120+0000 7f0956d9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09440047d0 0x7f09581e4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60214/0 (socket says 172.21.15.90:60214) 2024-03-11T20:52:13.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.120+0000 7f0956d9d640 1 -- 172.21.15.90:0/1528412207 learned_addr learned my addr 172.21.15.90:0/1528412207 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:52:13.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.124+0000 7f0956d9d640 1 -- 172.21.15.90:0/1528412207 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f095814f980 msgr2=0x7f0958144ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:13.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.124+0000 7f0956d9d640 1 --2- 172.21.15.90:0/1528412207 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f095814f980 0x7f0958144ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:13.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.124+0000 7f0956d9d640 1 -- 172.21.15.90:0/1528412207 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f095814ecd0 msgr2=0x7f0958141a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:13.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.124+0000 7f0956d9d640 1 --2- 172.21.15.90:0/1528412207 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f095814ecd0 0x7f0958141a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:13.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.124+0000 7f0956d9d640 1 -- 172.21.15.90:0/1528412207 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09581e3ee0 con 0x7f09440047d0 2024-03-11T20:52:13.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.128+0000 7f0956d9d640 1 --2- 172.21.15.90:0/1528412207 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09440047d0 0x7f09581e4370 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f095000f790 tx=0x7f095000fc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:52:13.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.136+0000 7f094f7fe640 1 -- 172.21.15.90:0/1528412207 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0950007af0 con 0x7f09440047d0 2024-03-11T20:52:13.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.136+0000 7f094f7fe640 1 -- 172.21.15.90:0/1528412207 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f0950007c50 con 0x7f09440047d0 2024-03-11T20:52:13.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.136+0000 7f094f7fe640 1 -- 172.21.15.90:0/1528412207 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095000a0d0 con 0x7f09440047d0 2024-03-11T20:52:13.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.136+0000 7f095d132640 1 -- 172.21.15.90:0/1528412207 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f095814e3d0 con 0x7f09440047d0 2024-03-11T20:52:13.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.136+0000 7f095d132640 1 -- 172.21.15.90:0/1528412207 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f095814e670 con 0x7f09440047d0 2024-03-11T20:52:13.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.136+0000 7f095d132640 1 -- 172.21.15.90:0/1528412207 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09581e4070 con 0x7f09440047d0 2024-03-11T20:52:13.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.172+0000 7f094f7fe640 1 -- 172.21.15.90:0/1528412207 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f095000a230 con 0x7f09440047d0 2024-03-11T20:52:13.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.172+0000 7f094f7fe640 1 --2- 172.21.15.90:0/1528412207 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0938066e90 0x7f0938069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:13.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.172+0000 7f095659c640 1 --2- 172.21.15.90:0/1528412207 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0938066e90 0x7f0938069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:52:13.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.172+0000 7f095659c640 1 --2- 172.21.15.90:0/1528412207 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0938066e90 0x7f0938069350 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f09400af000 tx=0x7f0940004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:52:13.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.456+0000 7f094f7fe640 1 -- 172.21.15.90:0/1528412207 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 7165+0+0 (secure 0 0 0) 0x7f09500580e0 con 0x7f09440047d0 2024-03-11T20:52:13.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.456+0000 7f094f7fe640 1 -- 172.21.15.90:0/1528412207 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0950054380 con 0x7f09440047d0 2024-03-11T20:52:13.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:13.680+0000 7f095d132640 1 -- 172.21.15.90:0/1528412207 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f095804eef0 con 0x7f09440047d0 2024-03-11T20:52:14.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.844+0000 7f094f7fe640 1 -- 172.21.15.90:0/1528412207 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7f095001b030 con 0x7f09440047d0 2024-03-11T20:52:14.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.844+0000 7f095d132640 1 -- 172.21.15.90:0/1528412207 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0938066e90 msgr2=0x7f0938069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:14.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.844+0000 7f095d132640 1 --2- 172.21.15.90:0/1528412207 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0938066e90 0x7f0938069350 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f09400af000 tx=0x7f0940004740 comp rx=0 tx=0).stop 2024-03-11T20:52:14.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.844+0000 7f095d132640 1 -- 172.21.15.90:0/1528412207 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09440047d0 msgr2=0x7f09581e4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:14.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.844+0000 7f095d132640 1 --2- 172.21.15.90:0/1528412207 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09440047d0 0x7f09581e4370 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f095000f790 tx=0x7f095000fc50 comp rx=0 tx=0).stop 2024-03-11T20:52:14.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.844+0000 7f095d132640 1 -- 172.21.15.90:0/1528412207 shutdown_connections 2024-03-11T20:52:14.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.844+0000 7f095d132640 1 --2- 172.21.15.90:0/1528412207 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f0938066e90 0x7f0938069350 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:14.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.844+0000 7f095d132640 1 --2- 172.21.15.90:0/1528412207 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f095814ecd0 0x7f0958141a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:14.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.844+0000 7f095d132640 1 --2- 172.21.15.90:0/1528412207 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f095814f980 0x7f0958144ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:14.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.844+0000 7f095d132640 1 --2- 172.21.15.90:0/1528412207 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09440047d0 0x7f09581e4370 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:14.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.844+0000 7f095d132640 1 -- 172.21.15.90:0/1528412207 >> 172.21.15.90:0/1528412207 conn(0x7f0958071aa0 msgr2=0x7f09580c7830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:52:14.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.844+0000 7f095d132640 1 -- 172.21.15.90:0/1528412207 shutdown_connections 2024-03-11T20:52:14.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.844+0000 7f095d132640 1 -- 172.21.15.90:0/1528412207 wait complete. 2024-03-11T20:52:14.862 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-11T20:52:14.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.980+0000 7fb2e2b3f640 1 Processor -- start 2024-03-11T20:52:14.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.980+0000 7fb2e2b3f640 1 -- start start 2024-03-11T20:52:14.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.980+0000 7fb2e2b3f640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fb2dc0cb5a0 0x7fb2dc153af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:14.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.980+0000 7fb2e2b3f640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb2dc0c97f0 0x7fb2dc154030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:14.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.980+0000 7fb2e2b3f640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2dc072930 con 0x7fb2dc0cb1c0 2024-03-11T20:52:14.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.980+0000 7fb2e2b3f640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2dc071d60 con 0x7fb2dc0cb5a0 2024-03-11T20:52:14.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.980+0000 7fb2e2b3f640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc072000 con 0x7fb2dc0c97f0 2024-03-11T20:52:14.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.980+0000 7fb2dbfff640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fb2dc0cb5a0 0x7fb2dc153af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:52:14.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.980+0000 7fb2dbfff640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fb2dc0cb5a0 0x7fb2dc153af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34608/0 (socket says 172.21.15.90:34608) 2024-03-11T20:52:14.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.980+0000 7fb2dbfff640 1 -- 172.21.15.90:0/2620588999 learned_addr learned my addr 172.21.15.90:0/2620588999 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:52:14.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.980+0000 7fb2e10dc640 1 --2- 172.21.15.90:0/2620588999 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb2dc0c97f0 0x7fb2dc154030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:52:14.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.984+0000 7fb2dbfff640 1 -- 172.21.15.90:0/2620588999 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb2dc0c97f0 msgr2=0x7fb2dc154030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:52:14.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.984+0000 7fb2dbfff640 1 --2- 172.21.15.90:0/2620588999 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb2dc0c97f0 0x7fb2dc154030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:14.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.984+0000 7fb2dbfff640 1 -- 172.21.15.90:0/2620588999 >> v1:172.21.15.92:6789/0 conn(0x7fb2dc0cb1c0 legacy=0x7fb2dc071220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:14.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.984+0000 7fb2dbfff640 1 -- 172.21.15.90:0/2620588999 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2dc0722e0 con 0x7fb2dc0cb5a0 2024-03-11T20:52:14.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.988+0000 7fb2dbfff640 1 --2- 172.21.15.90:0/2620588999 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2dc0cb5a0 0x7fb2dc153af0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb2d00098a0 tx=0x7fb2d001ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=563f6425de93c40d server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:52:14.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.992+0000 7fb2db7fe640 1 -- 172.21.15.90:0/2620588999 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d00a8a30 con 0x7fb2dc0cb5a0 2024-03-11T20:52:14.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.992+0000 7fb2db7fe640 1 -- 172.21.15.90:0/2620588999 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fb2d00a8b90 con 0x7fb2dc0cb5a0 2024-03-11T20:52:14.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.992+0000 7fb2db7fe640 1 -- 172.21.15.90:0/2620588999 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d00b0440 con 0x7fb2dc0cb5a0 2024-03-11T20:52:14.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.992+0000 7fb2e2b3f640 1 -- 172.21.15.90:0/2620588999 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2dc0cb5a0 msgr2=0x7fb2dc153af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:14.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.992+0000 7fb2e2b3f640 1 --2- 172.21.15.90:0/2620588999 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2dc0cb5a0 0x7fb2dc153af0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb2d00098a0 tx=0x7fb2d001ed30 comp rx=0 tx=0).stop 2024-03-11T20:52:14.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.992+0000 7fb2e2b3f640 1 -- 172.21.15.90:0/2620588999 shutdown_connections 2024-03-11T20:52:14.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.992+0000 7fb2e2b3f640 1 --2- 172.21.15.90:0/2620588999 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb2dc0c97f0 0x7fb2dc154030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:14.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.992+0000 7fb2e2b3f640 1 --2- 172.21.15.90:0/2620588999 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2dc0cb5a0 0x7fb2dc153af0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:14.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.992+0000 7fb2e2b3f640 1 -- 172.21.15.90:0/2620588999 >> 172.21.15.90:0/2620588999 conn(0x7fb2dc0cba10 msgr2=0x7fb2dc0cbe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:52:14.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.992+0000 7fb2e2b3f640 1 -- 172.21.15.90:0/2620588999 shutdown_connections 2024-03-11T20:52:14.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.992+0000 7fb2e2b3f640 1 -- 172.21.15.90:0/2620588999 wait complete. 2024-03-11T20:52:14.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.996+0000 7fb2e2b3f640 1 Processor -- start 2024-03-11T20:52:14.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.996+0000 7fb2e2b3f640 1 -- start start 2024-03-11T20:52:14.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.996+0000 7fb2e2b3f640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2dc0c97f0 0x7fb2dc152c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:14.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.996+0000 7fb2e2b3f640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb2dc0cb1c0 0x7fb2dc0c7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:14.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.996+0000 7fb2e2b3f640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2dc0cb5a0 0x7fb2dc0c7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:14.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.996+0000 7fb2e2b3f640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc0c68e0 con 0x7fb2dc0cb5a0 2024-03-11T20:52:14.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.996+0000 7fb2e2b3f640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc1f3910 con 0x7fb2dc0c97f0 2024-03-11T20:52:14.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.996+0000 7fb2e2b3f640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc0c7e70 con 0x7fb2dc0cb1c0 2024-03-11T20:52:14.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.996+0000 7fb2e10dc640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2dc0cb5a0 0x7fb2dc0c7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:52:14.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.996+0000 7fb2dbfff640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb2dc0cb1c0 0x7fb2dc0c7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:52:14.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.996+0000 7fb2e10dc640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2dc0cb5a0 0x7fb2dc0c7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34622/0 (socket says 172.21.15.90:34622) 2024-03-11T20:52:14.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.996+0000 7fb2e10dc640 1 -- 172.21.15.90:0/2902710906 learned_addr learned my addr 172.21.15.90:0/2902710906 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:52:14.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:14.996+0000 7fb2e08db640 1 --2- 172.21.15.90:0/2902710906 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2dc0c97f0 0x7fb2dc152c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:52:14.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:15.000+0000 7fb2e10dc640 1 -- 172.21.15.90:0/2902710906 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb2dc0cb1c0 msgr2=0x7fb2dc0c7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:14.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:15.000+0000 7fb2e10dc640 1 --2- 172.21.15.90:0/2902710906 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb2dc0cb1c0 0x7fb2dc0c7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:14.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:15.000+0000 7fb2e10dc640 1 -- 172.21.15.90:0/2902710906 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2dc0c97f0 msgr2=0x7fb2dc152c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:14.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:15.000+0000 7fb2e10dc640 1 --2- 172.21.15.90:0/2902710906 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2dc0c97f0 0x7fb2dc152c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:14.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:15.000+0000 7fb2e10dc640 1 -- 172.21.15.90:0/2902710906 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d00094d0 con 0x7fb2dc0cb5a0 2024-03-11T20:52:15.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:15.000+0000 7fb2e10dc640 1 --2- 172.21.15.90:0/2902710906 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2dc0cb5a0 0x7fb2dc0c7670 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4007e60 tx=0x7fb2d400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:52:15.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:15.004+0000 7fb2d97fa640 1 -- 172.21.15.90:0/2902710906 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d4014620 con 0x7fb2dc0cb5a0 2024-03-11T20:52:15.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:15.004+0000 7fb2d97fa640 1 -- 172.21.15.90:0/2902710906 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fb2d4014780 con 0x7fb2dc0cb5a0 2024-03-11T20:52:15.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:15.004+0000 7fb2d97fa640 1 -- 172.21.15.90:0/2902710906 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d4014b50 con 0x7fb2dc0cb5a0 2024-03-11T20:52:15.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:15.004+0000 7fb2e2b3f640 1 -- 172.21.15.90:0/2902710906 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2dc0c4ef0 con 0x7fb2dc0cb5a0 2024-03-11T20:52:15.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:15.004+0000 7fb2e2b3f640 1 -- 172.21.15.90:0/2902710906 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2dc157ac0 con 0x7fb2dc0cb5a0 2024-03-11T20:52:15.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:15.008+0000 7fb2e2b3f640 1 -- 172.21.15.90:0/2902710906 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2dc157890 con 0x7fb2dc0cb5a0 2024-03-11T20:52:15.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:15.024+0000 7fb2d97fa640 1 -- 172.21.15.90:0/2902710906 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fb2d4014cb0 con 0x7fb2dc0cb5a0 2024-03-11T20:52:15.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:15.028+0000 7fb2d97fa640 1 --2- 172.21.15.90:0/2902710906 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fb2c0066ec0 0x7fb2c0069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:15.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:15.028+0000 7fb2e08db640 1 --2- 172.21.15.90:0/2902710906 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fb2c0066ec0 0x7fb2c0069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:52:15.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:15.028+0000 7fb2e08db640 1 --2- 172.21.15.90:0/2902710906 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fb2c0066ec0 0x7fb2c0069380 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8001810 tx=0x7fb2c8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:52:15.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:15.292+0000 7fb2d97fa640 1 -- 172.21.15.90:0/2902710906 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 7165+0+0 (secure 0 0 0) 0x7fb2d4056610 con 0x7fb2dc0cb5a0 2024-03-11T20:52:15.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:15.292+0000 7fb2d97fa640 1 -- 172.21.15.90:0/2902710906 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb2d40528b0 con 0x7fb2dc0cb5a0 2024-03-11T20:52:15.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:15.452+0000 7fb2e2b3f640 1 -- 172.21.15.90:0/2902710906 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb2dc157c50 con 0x7fb2dc0cb5a0 2024-03-11T20:52:16.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.164+0000 7fb2d97fa640 1 -- 172.21.15.90:0/2902710906 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v32) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb2d4055f70 con 0x7fb2dc0cb5a0 2024-03-11T20:52:16.164 INFO:teuthology.orchestra.run.smithi090.stderr:pool 'cephfs_metadata' removed 2024-03-11T20:52:16.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.168+0000 7fb2e2b3f640 1 -- 172.21.15.90:0/2902710906 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fb2c0066ec0 msgr2=0x7fb2c0069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:16.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.168+0000 7fb2e2b3f640 1 --2- 172.21.15.90:0/2902710906 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fb2c0066ec0 0x7fb2c0069380 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8001810 tx=0x7fb2c8009040 comp rx=0 tx=0).stop 2024-03-11T20:52:16.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.168+0000 7fb2e2b3f640 1 -- 172.21.15.90:0/2902710906 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2dc0cb5a0 msgr2=0x7fb2dc0c7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:16.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.168+0000 7fb2e2b3f640 1 --2- 172.21.15.90:0/2902710906 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2dc0cb5a0 0x7fb2dc0c7670 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4007e60 tx=0x7fb2d400e4b0 comp rx=0 tx=0).stop 2024-03-11T20:52:16.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.168+0000 7fb2e2b3f640 1 -- 172.21.15.90:0/2902710906 shutdown_connections 2024-03-11T20:52:16.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.168+0000 7fb2e2b3f640 1 --2- 172.21.15.90:0/2902710906 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fb2c0066ec0 0x7fb2c0069380 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:16.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.168+0000 7fb2e2b3f640 1 --2- 172.21.15.90:0/2902710906 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2dc0cb5a0 0x7fb2dc0c7670 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:16.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.168+0000 7fb2e2b3f640 1 --2- 172.21.15.90:0/2902710906 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fb2dc0cb1c0 0x7fb2dc0c7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:16.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.168+0000 7fb2e2b3f640 1 --2- 172.21.15.90:0/2902710906 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fb2dc0c97f0 0x7fb2dc152c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:16.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.168+0000 7fb2e2b3f640 1 -- 172.21.15.90:0/2902710906 >> 172.21.15.90:0/2902710906 conn(0x7fb2dc0cba10 msgr2=0x7fb2dc14d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:52:16.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.168+0000 7fb2e2b3f640 1 -- 172.21.15.90:0/2902710906 shutdown_connections 2024-03-11T20:52:16.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.168+0000 7fb2e2b3f640 1 -- 172.21.15.90:0/2902710906 wait complete. 2024-03-11T20:52:16.197 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-11T20:52:16.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.376+0000 7fef43f22640 1 Processor -- start 2024-03-11T20:52:16.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.376+0000 7fef43f22640 1 -- start start 2024-03-11T20:52:16.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.380+0000 7fef43f22640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fef3c0cf5f0 0x7fef3c15a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:16.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.380+0000 7fef43f22640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fef3c0cf9d0 0x7fef3c0ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:16.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.380+0000 7fef43f22640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef3c0cbbe0 con 0x7fef3c153a30 2024-03-11T20:52:16.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.380+0000 7fef43f22640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef3c0cadd0 con 0x7fef3c0cf9d0 2024-03-11T20:52:16.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.380+0000 7fef43f22640 1 -- --> v2:172.21.15.92:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef3c15abb0 con 0x7fef3c0cf5f0 2024-03-11T20:52:16.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.380+0000 7fef414bd640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fef3c0cf9d0 0x7fef3c0ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:52:16.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.380+0000 7fef41cbe640 1 --2- >> v2:172.21.15.92:3300/0 conn(0x7fef3c0cf5f0 0x7fef3c15a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:52:16.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.380+0000 7fef424bf640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7fef3c153a30 0x7fef3c0b3740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:60786/0 (socket says 172.21.15.90:60786) 2024-03-11T20:52:16.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.380+0000 7fef424bf640 1 -- 172.21.15.90:0/2291519603 learned_addr learned my addr 172.21.15.90:0/2291519603 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T20:52:16.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.384+0000 7fef414bd640 1 -- 172.21.15.90:0/2291519603 >> v2:172.21.15.92:3300/0 conn(0x7fef3c0cf5f0 msgr2=0x7fef3c15a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:52:16.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.384+0000 7fef414bd640 1 --2- 172.21.15.90:0/2291519603 >> v2:172.21.15.92:3300/0 conn(0x7fef3c0cf5f0 0x7fef3c15a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:16.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.384+0000 7fef414bd640 1 -- 172.21.15.90:0/2291519603 >> v1:172.21.15.90:6789/0 conn(0x7fef3c153a30 legacy=0x7fef3c0b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:16.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.384+0000 7fef414bd640 1 -- 172.21.15.90:0/2291519603 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef3c0cab30 con 0x7fef3c0cf9d0 2024-03-11T20:52:16.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.392+0000 7fef414bd640 1 --2- 172.21.15.90:0/2291519603 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef3c0cf9d0 0x7fef3c0ca5f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fef3c0cbde0 tx=0x7fef2c01ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4f0277ab1a22158 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:52:16.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.396+0000 7fef40cbc640 1 -- 172.21.15.90:0/2291519603 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2c0a8970 con 0x7fef3c0cf9d0 2024-03-11T20:52:16.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.396+0000 7fef40cbc640 1 -- 172.21.15.90:0/2291519603 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fef2c0a8ad0 con 0x7fef3c0cf9d0 2024-03-11T20:52:16.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.396+0000 7fef40cbc640 1 -- 172.21.15.90:0/2291519603 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2c0a8e80 con 0x7fef3c0cf9d0 2024-03-11T20:52:16.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.396+0000 7fef43f22640 1 -- 172.21.15.90:0/2291519603 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef3c0cf9d0 msgr2=0x7fef3c0ca5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:16.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.396+0000 7fef43f22640 1 --2- 172.21.15.90:0/2291519603 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef3c0cf9d0 0x7fef3c0ca5f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fef3c0cbde0 tx=0x7fef2c01ec70 comp rx=0 tx=0).stop 2024-03-11T20:52:16.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.396+0000 7fef43f22640 1 -- 172.21.15.90:0/2291519603 shutdown_connections 2024-03-11T20:52:16.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.396+0000 7fef43f22640 1 --2- 172.21.15.90:0/2291519603 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef3c0cf9d0 0x7fef3c0ca5f0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:16.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.396+0000 7fef43f22640 1 --2- 172.21.15.90:0/2291519603 >> v2:172.21.15.92:3300/0 conn(0x7fef3c0cf5f0 0x7fef3c15a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:16.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.396+0000 7fef43f22640 1 -- 172.21.15.90:0/2291519603 >> 172.21.15.90:0/2291519603 conn(0x7fef3c06d770 msgr2=0x7fef3c06b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:52:16.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.396+0000 7fef43f22640 1 -- 172.21.15.90:0/2291519603 shutdown_connections 2024-03-11T20:52:16.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.396+0000 7fef43f22640 1 -- 172.21.15.90:0/2291519603 wait complete. 2024-03-11T20:52:16.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.400+0000 7fef43f22640 1 Processor -- start 2024-03-11T20:52:16.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.400+0000 7fef43f22640 1 -- start start 2024-03-11T20:52:16.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.400+0000 7fef43f22640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef3c0cf5f0 0x7fef3c14ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:16.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.400+0000 7fef43f22640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fef3c0cf9d0 0x7fef3c158c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:16.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.400+0000 7fef43f22640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef3c153a30 0x7fef3c1efa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:16.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.400+0000 7fef43f22640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef3c14f010 con 0x7fef3c153a30 2024-03-11T20:52:16.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.400+0000 7fef43f22640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef3c15cef0 con 0x7fef3c0cf5f0 2024-03-11T20:52:16.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.400+0000 7fef43f22640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef3c1ef520 con 0x7fef3c0cf9d0 2024-03-11T20:52:16.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.400+0000 7fef424bf640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef3c153a30 0x7fef3c1efa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:52:16.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.400+0000 7fef424bf640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef3c153a30 0x7fef3c1efa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34638/0 (socket says 172.21.15.90:34638) 2024-03-11T20:52:16.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.400+0000 7fef424bf640 1 -- 172.21.15.90:0/826773495 learned_addr learned my addr 172.21.15.90:0/826773495 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:52:16.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.400+0000 7fef41cbe640 1 --2- 172.21.15.90:0/826773495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef3c0cf5f0 0x7fef3c14ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:52:16.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.400+0000 7fef414bd640 1 --2- 172.21.15.90:0/826773495 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fef3c0cf9d0 0x7fef3c158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:52:16.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.404+0000 7fef424bf640 1 -- 172.21.15.90:0/826773495 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fef3c0cf9d0 msgr2=0x7fef3c158c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:16.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.404+0000 7fef424bf640 1 --2- 172.21.15.90:0/826773495 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fef3c0cf9d0 0x7fef3c158c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:16.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.404+0000 7fef424bf640 1 -- 172.21.15.90:0/826773495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef3c0cf5f0 msgr2=0x7fef3c14ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:16.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.404+0000 7fef424bf640 1 --2- 172.21.15.90:0/826773495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef3c0cf5f0 0x7fef3c14ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:16.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.404+0000 7fef424bf640 1 -- 172.21.15.90:0/826773495 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef2c009420 con 0x7fef3c153a30 2024-03-11T20:52:16.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.408+0000 7fef424bf640 1 --2- 172.21.15.90:0/826773495 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef3c153a30 0x7fef3c1efa00 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fef3400cab0 tx=0x7fef3400cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:52:16.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.408+0000 7fef327fc640 1 -- 172.21.15.90:0/826773495 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3400ac30 con 0x7fef3c153a30 2024-03-11T20:52:16.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.408+0000 7fef327fc640 1 -- 172.21.15.90:0/826773495 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fef3400ad90 con 0x7fef3c153a30 2024-03-11T20:52:16.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.408+0000 7fef327fc640 1 -- 172.21.15.90:0/826773495 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef34012600 con 0x7fef3c153a30 2024-03-11T20:52:16.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.408+0000 7fef43f22640 1 -- 172.21.15.90:0/826773495 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef3c1f34a0 con 0x7fef3c153a30 2024-03-11T20:52:16.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.408+0000 7fef43f22640 1 -- 172.21.15.90:0/826773495 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef3c0c6700 con 0x7fef3c153a30 2024-03-11T20:52:16.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.412+0000 7fef43f22640 1 -- 172.21.15.90:0/826773495 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef3c1ef6e0 con 0x7fef3c153a30 2024-03-11T20:52:16.703 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-03-11T20:52:16.703 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~0s 2024-03-11T20:52:16.703 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-03-11T20:52:16.703 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-03-11T20:52:16.703 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~0s 2024-03-11T20:52:16.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.720+0000 7fef327fc640 1 -- 172.21.15.90:0/826773495 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fef34007070 con 0x7fef3c153a30 2024-03-11T20:52:16.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.720+0000 7fef327fc640 1 --2- 172.21.15.90:0/826773495 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fef20066ec0 0x7fef20069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:16.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.720+0000 7fef41cbe640 1 --2- 172.21.15.90:0/826773495 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fef20066ec0 0x7fef20069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:52:16.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.720+0000 7fef41cbe640 1 --2- 172.21.15.90:0/826773495 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fef20066ec0 0x7fef20069380 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fef38009b60 tx=0x7fef38009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:52:16.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.756+0000 7fef327fc640 1 -- 172.21.15.90:0/826773495 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6727+0+0 (secure 0 0 0) 0x7fef340559f0 con 0x7fef3c153a30 2024-03-11T20:52:16.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.756+0000 7fef327fc640 1 -- 172.21.15.90:0/826773495 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fef34051c90 con 0x7fef3c153a30 2024-03-11T20:52:16.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:16.928+0000 7fef43f22640 1 -- 172.21.15.90:0/826773495 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fef3c04efd0 con 0x7fef3c153a30 2024-03-11T20:52:17.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.452+0000 7fef327fc640 1 -- 172.21.15.90:0/826773495 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v33) v1 ==== 169+0+0 (secure 0 0 0) 0x7fef34055350 con 0x7fef3c153a30 2024-03-11T20:52:17.452 INFO:teuthology.orchestra.run.smithi090.stderr:pool 'cephfs_data' removed 2024-03-11T20:52:17.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.456+0000 7fef43f22640 1 -- 172.21.15.90:0/826773495 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fef20066ec0 msgr2=0x7fef20069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:17.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.456+0000 7fef43f22640 1 --2- 172.21.15.90:0/826773495 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fef20066ec0 0x7fef20069380 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fef38009b60 tx=0x7fef38009190 comp rx=0 tx=0).stop 2024-03-11T20:52:17.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.456+0000 7fef43f22640 1 -- 172.21.15.90:0/826773495 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef3c153a30 msgr2=0x7fef3c1efa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:17.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.456+0000 7fef43f22640 1 --2- 172.21.15.90:0/826773495 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef3c153a30 0x7fef3c1efa00 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fef3400cab0 tx=0x7fef3400cf70 comp rx=0 tx=0).stop 2024-03-11T20:52:17.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.456+0000 7fef43f22640 1 -- 172.21.15.90:0/826773495 shutdown_connections 2024-03-11T20:52:17.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.456+0000 7fef43f22640 1 --2- 172.21.15.90:0/826773495 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7fef20066ec0 0x7fef20069380 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:17.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.456+0000 7fef43f22640 1 --2- 172.21.15.90:0/826773495 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef3c153a30 0x7fef3c1efa00 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:17.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.456+0000 7fef43f22640 1 --2- 172.21.15.90:0/826773495 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7fef3c0cf9d0 0x7fef3c158c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:17.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.456+0000 7fef43f22640 1 --2- 172.21.15.90:0/826773495 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7fef3c0cf5f0 0x7fef3c14ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:17.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.456+0000 7fef43f22640 1 -- 172.21.15.90:0/826773495 >> 172.21.15.90:0/826773495 conn(0x7fef3c06d770 msgr2=0x7fef3c0cbd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:52:17.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.456+0000 7fef43f22640 1 -- 172.21.15.90:0/826773495 shutdown_connections 2024-03-11T20:52:17.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.456+0000 7fef43f22640 1 -- 172.21.15.90:0/826773495 wait complete. 2024-03-11T20:52:17.494 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-03-11T20:52:17.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.616+0000 7f10c8d5b640 1 Processor -- start 2024-03-11T20:52:17.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.616+0000 7f10c8d5b640 1 -- start start 2024-03-11T20:52:17.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.616+0000 7f10c8d5b640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f10c4151ca0 0x7f10c415ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:17.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.616+0000 7f10c8d5b640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10c415bf60 con 0x7f10c406b700 2024-03-11T20:52:17.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.616+0000 7f10c8d5b640 1 -- --> v1:172.21.15.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10c40c28e0 con 0x7f10c40c60e0 2024-03-11T20:52:17.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.616+0000 7f10c8d5b640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10c40c3490 con 0x7f10c4151ca0 2024-03-11T20:52:17.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.616+0000 7f10c1d9b640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f10c406b700 0x7f10c40c3920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:60798/0 (socket says 172.21.15.90:60798) 2024-03-11T20:52:17.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.616+0000 7f10c1d9b640 1 -- 172.21.15.90:0/3661977935 learned_addr learned my addr 172.21.15.90:0/3661977935 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-11T20:52:17.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.616+0000 7f10c259c640 1 --2- 172.21.15.90:0/3661977935 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f10c4151ca0 0x7f10c415ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:52:17.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.620+0000 7f10c159a640 1 -- 172.21.15.90:0/3661977935 <== mon.0 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1800244643 0 0) 0x7f10b8001120 con 0x7f10c406b700 2024-03-11T20:52:17.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.620+0000 7f10c159a640 1 -- 172.21.15.90:0/3661977935 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10ac003530 con 0x7f10c406b700 2024-03-11T20:52:17.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.624+0000 7f10c159a640 1 -- 172.21.15.90:0/3661977935 <== mon.1 v1:172.21.15.92:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 604236803 0 0) 0x7f10c40c28e0 con 0x7f10c40c60e0 2024-03-11T20:52:17.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.624+0000 7f10c159a640 1 -- 172.21.15.90:0/3661977935 --> v1:172.21.15.92:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10b8001120 con 0x7f10c40c60e0 2024-03-11T20:52:17.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.624+0000 7f10c159a640 1 -- 172.21.15.90:0/3661977935 <== mon.0 v1:172.21.15.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 140154405 0 0) 0x7f10ac003530 con 0x7f10c406b700 2024-03-11T20:52:17.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.624+0000 7f10c159a640 1 -- 172.21.15.90:0/3661977935 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f10c4151ca0 msgr2=0x7f10c415ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:52:17.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.624+0000 7f10c159a640 1 --2- 172.21.15.90:0/3661977935 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f10c4151ca0 0x7f10c415ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:17.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.624+0000 7f10c159a640 1 -- 172.21.15.90:0/3661977935 >> v1:172.21.15.92:6789/0 conn(0x7f10c40c60e0 legacy=0x7f10c406ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:17.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.624+0000 7f10c159a640 1 -- 172.21.15.90:0/3661977935 --> v1:172.21.15.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10c40c6670 con 0x7f10c406b700 2024-03-11T20:52:17.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.624+0000 7f10c159a640 1 -- 172.21.15.90:0/3661977935 <== mon.0 v1:172.21.15.90:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 691735435 0 0) 0x7f10b8001810 con 0x7f10c406b700 2024-03-11T20:52:17.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.624+0000 7f10c159a640 1 -- 172.21.15.90:0/3661977935 >> v1:172.21.15.90:6789/0 conn(0x7f10c406b700 legacy=0x7f10c40c3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:17.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.624+0000 7f10c159a640 1 --2- 172.21.15.90:0/3661977935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10ac003b40 0x7f10ac003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:17.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.624+0000 7f10c159a640 1 --2- 172.21.15.90:0/3661977935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10ac0047d0 0x7f10ac004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:17.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.624+0000 7f10c159a640 1 --2- 172.21.15.90:0/3661977935 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f10c4151ca0 0x7f10ac005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:17.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.624+0000 7f10c159a640 1 -- 172.21.15.90:0/3661977935 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c40c3490 con 0x7f10ac003b40 2024-03-11T20:52:17.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.624+0000 7f10c159a640 1 -- 172.21.15.90:0/3661977935 --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac00fe50 con 0x7f10ac0047d0 2024-03-11T20:52:17.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.624+0000 7f10c159a640 1 -- 172.21.15.90:0/3661977935 --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac010010 con 0x7f10c4151ca0 2024-03-11T20:52:17.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.628+0000 7f10c259c640 1 --2- 172.21.15.90:0/3661977935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10ac003b40 0x7f10ac003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:52:17.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.628+0000 7f10c1d9b640 1 --2- 172.21.15.90:0/3661977935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10ac0047d0 0x7f10ac004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:52:17.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.632+0000 7f10c259c640 1 -- 172.21.15.90:0/3661977935 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f10c4151ca0 msgr2=0x7f10ac005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:52:17.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.632+0000 7f10c259c640 1 --2- 172.21.15.90:0/3661977935 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f10c4151ca0 0x7f10ac005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:17.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.632+0000 7f10c259c640 1 -- 172.21.15.90:0/3661977935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10ac0047d0 msgr2=0x7f10ac004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-11T20:52:17.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.632+0000 7f10c259c640 1 --2- 172.21.15.90:0/3661977935 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10ac0047d0 0x7f10ac004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:17.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.632+0000 7f10c259c640 1 -- 172.21.15.90:0/3661977935 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10ac003530 con 0x7f10ac003b40 2024-03-11T20:52:17.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.636+0000 7f10c259c640 1 --2- 172.21.15.90:0/3661977935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10ac003b40 0x7f10ac003f40 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f10b4009ae0 tx=0x7f10b401ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e18b33456d7cda6e server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:52:17.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.640+0000 7f10c159a640 1 -- 172.21.15.90:0/3661977935 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b40a5560 con 0x7f10ac003b40 2024-03-11T20:52:17.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.640+0000 7f10c159a640 1 -- 172.21.15.90:0/3661977935 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f10b40a56c0 con 0x7f10ac003b40 2024-03-11T20:52:17.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.640+0000 7f10c159a640 1 -- 172.21.15.90:0/3661977935 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b40a5a90 con 0x7f10ac003b40 2024-03-11T20:52:17.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.640+0000 7f10c8d5b640 1 -- 172.21.15.90:0/3661977935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10ac003b40 msgr2=0x7f10ac003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:17.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.640+0000 7f10c8d5b640 1 --2- 172.21.15.90:0/3661977935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10ac003b40 0x7f10ac003f40 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f10b4009ae0 tx=0x7f10b401ca80 comp rx=0 tx=0).stop 2024-03-11T20:52:17.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.640+0000 7f10c2d9d640 1 -- 172.21.15.90:0/3661977935 reap_dead start 2024-03-11T20:52:17.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.640+0000 7f10c8d5b640 1 -- 172.21.15.90:0/3661977935 shutdown_connections 2024-03-11T20:52:17.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.640+0000 7f10c8d5b640 1 -- 172.21.15.90:0/3661977935 >> 172.21.15.90:0/3661977935 conn(0x7f10c406d230 msgr2=0x7f10c406e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:52:17.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.640+0000 7f10c8d5b640 1 -- 172.21.15.90:0/3661977935 shutdown_connections 2024-03-11T20:52:17.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.640+0000 7f10c8d5b640 1 -- 172.21.15.90:0/3661977935 wait complete. 2024-03-11T20:52:17.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.640+0000 7f10c8d5b640 1 Processor -- start 2024-03-11T20:52:17.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.640+0000 7f10c8d5b640 1 -- start start 2024-03-11T20:52:17.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.644+0000 7f10c8d5b640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10ac003b40 0x7f10c40c7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:17.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.644+0000 7f10c8d5b640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f10c40c81e0 0x7f10c41eebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:17.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.644+0000 7f10c8d5b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10c4150ff0 0x7f10c41ef110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:17.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.644+0000 7f10c8d5b640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c4142aa0 con 0x7f10c4150ff0 2024-03-11T20:52:17.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.644+0000 7f10c8d5b640 1 -- --> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c41f1e00 con 0x7f10ac003b40 2024-03-11T20:52:17.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.644+0000 7f10c8d5b640 1 -- --> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10c41ee6d0 con 0x7f10c40c81e0 2024-03-11T20:52:17.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.644+0000 7f10c2d9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10c4150ff0 0x7f10c41ef110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:52:17.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.644+0000 7f10c259c640 1 --2- >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10ac003b40 0x7f10c40c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:52:17.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.644+0000 7f10c1d9b640 1 --2- >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f10c40c81e0 0x7f10c41eebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:52:17.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.644+0000 7f10c2d9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10c4150ff0 0x7f10c41ef110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34644/0 (socket says 172.21.15.90:34644) 2024-03-11T20:52:17.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.644+0000 7f10c2d9d640 1 -- 172.21.15.90:0/935291115 learned_addr learned my addr 172.21.15.90:0/935291115 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-11T20:52:17.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.648+0000 7f10c2d9d640 1 -- 172.21.15.90:0/935291115 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f10c40c81e0 msgr2=0x7f10c41eebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:17.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.648+0000 7f10c2d9d640 1 --2- 172.21.15.90:0/935291115 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f10c40c81e0 0x7f10c41eebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:17.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.648+0000 7f10c2d9d640 1 -- 172.21.15.90:0/935291115 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10ac003b40 msgr2=0x7f10c40c7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:17.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.648+0000 7f10c2d9d640 1 --2- 172.21.15.90:0/935291115 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10ac003b40 0x7f10c40c7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:17.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.648+0000 7f10c2d9d640 1 -- 172.21.15.90:0/935291115 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10c41f20c0 con 0x7f10c4150ff0 2024-03-11T20:52:17.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.652+0000 7f10c2d9d640 1 --2- 172.21.15.90:0/935291115 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10c4150ff0 0x7f10c41ef110 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f10c406be10 tx=0x7f10bc00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:52:17.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.656+0000 7f10b2ffd640 1 -- 172.21.15.90:0/935291115 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10bc005d50 con 0x7f10c4150ff0 2024-03-11T20:52:17.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.656+0000 7f10b2ffd640 1 -- 172.21.15.90:0/935291115 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f10bc00a610 con 0x7f10c4150ff0 2024-03-11T20:52:17.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.656+0000 7f10b2ffd640 1 -- 172.21.15.90:0/935291115 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10bc00a9e0 con 0x7f10c4150ff0 2024-03-11T20:52:17.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.656+0000 7f10c8d5b640 1 -- 172.21.15.90:0/935291115 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10c41f23a0 con 0x7f10c4150ff0 2024-03-11T20:52:17.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.656+0000 7f10c8d5b640 1 -- 172.21.15.90:0/935291115 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10c41f2960 con 0x7f10c4150ff0 2024-03-11T20:52:17.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.656+0000 7f10c8d5b640 1 -- 172.21.15.90:0/935291115 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10c41ee890 con 0x7f10c4150ff0 2024-03-11T20:52:17.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.948+0000 7f10b2ffd640 1 -- 172.21.15.90:0/935291115 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f10bc00ab40 con 0x7f10c4150ff0 2024-03-11T20:52:17.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.952+0000 7f10b2ffd640 1 --2- 172.21.15.90:0/935291115 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f10a4066e90 0x7f10a4069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-11T20:52:17.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.952+0000 7f10c259c640 1 --2- 172.21.15.90:0/935291115 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f10a4066e90 0x7f10a4069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-11T20:52:17.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.952+0000 7f10c259c640 1 --2- 172.21.15.90:0/935291115 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f10a4066e90 0x7f10a4069350 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f10b40af000 tx=0x7f10b40046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-11T20:52:17.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.984+0000 7f10b2ffd640 1 -- 172.21.15.90:0/935291115 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6197+0+0 (secure 0 0 0) 0x7f10bc055090 con 0x7f10c4150ff0 2024-03-11T20:52:17.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:17.988+0000 7f10b2ffd640 1 -- 172.21.15.90:0/935291115 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f10bc0512a0 con 0x7f10c4150ff0 2024-03-11T20:52:18.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:18.168+0000 7f10c8d5b640 1 -- 172.21.15.90:0/935291115 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f10c41f2b20 con 0x7f10c4150ff0 2024-03-11T20:52:18.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:18.584+0000 7f10b2ffd640 1 -- 172.21.15.90:0/935291115 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v34) v1 ==== 178+0+0 (secure 0 0 0) 0x7f10bc017070 con 0x7f10c4150ff0 2024-03-11T20:52:18.584 INFO:teuthology.orchestra.run.smithi090.stderr:pool 'cephfs_data_ec' removed 2024-03-11T20:52:18.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:18.588+0000 7f10c8d5b640 1 -- 172.21.15.90:0/935291115 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f10a4066e90 msgr2=0x7f10a4069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:18.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:18.588+0000 7f10c8d5b640 1 --2- 172.21.15.90:0/935291115 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f10a4066e90 0x7f10a4069350 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f10b40af000 tx=0x7f10b40046d0 comp rx=0 tx=0).stop 2024-03-11T20:52:18.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:18.588+0000 7f10c8d5b640 1 -- 172.21.15.90:0/935291115 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10c4150ff0 msgr2=0x7f10c41ef110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-11T20:52:18.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:18.588+0000 7f10c8d5b640 1 --2- 172.21.15.90:0/935291115 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10c4150ff0 0x7f10c41ef110 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f10c406be10 tx=0x7f10bc00cb90 comp rx=0 tx=0).stop 2024-03-11T20:52:18.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:18.588+0000 7f10c8d5b640 1 -- 172.21.15.90:0/935291115 shutdown_connections 2024-03-11T20:52:18.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:18.588+0000 7f10c8d5b640 1 --2- 172.21.15.90:0/935291115 >> [v2:172.21.15.90:6800/1999720161,v1:172.21.15.90:6801/1999720161] conn(0x7f10a4066e90 0x7f10a4069350 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:18.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:18.588+0000 7f10c8d5b640 1 --2- 172.21.15.90:0/935291115 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10c4150ff0 0x7f10c41ef110 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:18.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:18.588+0000 7f10c8d5b640 1 --2- 172.21.15.90:0/935291115 >> [v2:172.21.15.92:3301/0,v1:172.21.15.92:6790/0] conn(0x7f10c40c81e0 0x7f10c41eebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:18.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:18.588+0000 7f10c8d5b640 1 --2- 172.21.15.90:0/935291115 >> [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] conn(0x7f10ac003b40 0x7f10c40c7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-11T20:52:18.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:18.588+0000 7f10c8d5b640 1 -- 172.21.15.90:0/935291115 >> 172.21.15.90:0/935291115 conn(0x7f10c406d230 msgr2=0x7f10c40712f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-11T20:52:18.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:18.588+0000 7f10c8d5b640 1 -- 172.21.15.90:0/935291115 shutdown_connections 2024-03-11T20:52:18.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-11T20:52:18.588+0000 7f10c8d5b640 1 -- 172.21.15.90:0/935291115 wait complete. 2024-03-11T20:52:18.612 INFO:teuthology.misc:Shutting down mds daemons... 2024-03-11T20:52:18.612 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-11T20:52:18.612 INFO:teuthology.orchestra.run:waiting for 300 2024-03-11T20:52:18.634 INFO:tasks.ceph.mds.a.smithi090.stderr:2024-03-11T20:52:18.620+0000 b9d9640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mds.a.log --time-stamp=yes --vgdb=yes --tool=memcheck ceph-mds -f --cluster ceph -i a (PID: 33320) UID: 0 2024-03-11T20:52:18.634 INFO:tasks.ceph.mds.a.smithi090.stderr:2024-03-11T20:52:18.620+0000 b9d9640 -1 mds.a *** got signal Terminated *** 2024-03-11T20:52:23.013 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-03-11T20:52:23.014 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~6s 2024-03-11T20:52:23.014 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~6s 2024-03-11T20:52:23.014 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~6s 2024-03-11T20:52:23.014 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~6s 2024-03-11T20:52:23.014 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~6s 2024-03-11T20:52:24.714 INFO:tasks.ceph.mds.a:Stopped 2024-03-11T20:52:24.714 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-11T20:52:24.714 INFO:teuthology.orchestra.run:waiting for 300 2024-03-11T20:52:24.714 INFO:tasks.ceph.mds.c:Stopped 2024-03-11T20:52:24.714 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2024-03-11T20:52:24.715 INFO:teuthology.orchestra.run:waiting for 300 2024-03-11T20:52:24.715 INFO:tasks.ceph.mds.e:Stopped 2024-03-11T20:52:24.715 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-11T20:52:24.715 INFO:teuthology.orchestra.run:waiting for 300 2024-03-11T20:52:24.715 INFO:tasks.ceph.mds.b:Stopped 2024-03-11T20:52:24.715 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-11T20:52:24.715 INFO:teuthology.orchestra.run:waiting for 300 2024-03-11T20:52:24.715 INFO:tasks.ceph.mds.d:Stopped 2024-03-11T20:52:24.715 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2024-03-11T20:52:24.715 INFO:teuthology.orchestra.run:waiting for 300 2024-03-11T20:52:24.716 INFO:tasks.ceph.mds.f:Stopped 2024-03-11T20:52:24.716 INFO:teuthology.misc:Shutting down osd daemons... 2024-03-11T20:52:24.716 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-03-11T20:52:24.716 INFO:teuthology.orchestra.run:waiting for 300 2024-03-11T20:52:24.718 INFO:tasks.ceph.osd.0.smithi090.stderr:2024-03-11T20:52:24.720+0000 7fe9a4041640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term ceph-osd -f --cluster ceph -i 0 (PID: 27854) UID: 0 2024-03-11T20:52:24.718 INFO:tasks.ceph.osd.0.smithi090.stderr:2024-03-11T20:52:24.720+0000 7fe9a4041640 -1 osd.0 35 *** Got signal Terminated *** 2024-03-11T20:52:29.225 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-03-11T20:52:29.226 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-03-11T20:52:29.226 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e has been restored 2024-03-11T20:52:29.226 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-03-11T20:52:29.226 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-03-11T20:52:29.226 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f has been restored 2024-03-11T20:52:29.226 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-03-11T20:52:30.818 INFO:tasks.ceph.osd.0:Stopped 2024-03-11T20:52:30.818 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-03-11T20:52:30.819 INFO:teuthology.orchestra.run:waiting for 300 2024-03-11T20:52:30.820 INFO:tasks.ceph.osd.1.smithi090.stderr:2024-03-11T20:52:30.820+0000 7f6ff6131640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term ceph-osd -f --cluster ceph -i 1 (PID: 27856) UID: 0 2024-03-11T20:52:30.820 INFO:tasks.ceph.osd.1.smithi090.stderr:2024-03-11T20:52:30.820+0000 7f6ff6131640 -1 osd.1 37 *** Got signal Terminated *** 2024-03-11T20:52:31.787 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:52:31.790 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:52:35.334 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-03-11T20:52:35.335 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-03-11T20:52:36.920 INFO:tasks.ceph.osd.1:Stopped 2024-03-11T20:52:36.921 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-03-11T20:52:36.921 INFO:teuthology.orchestra.run:waiting for 300 2024-03-11T20:52:36.922 INFO:tasks.ceph.osd.2.smithi090.stderr:2024-03-11T20:52:36.924+0000 7f754f0e1640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term ceph-osd -f --cluster ceph -i 2 (PID: 27858) UID: 0 2024-03-11T20:52:36.922 INFO:tasks.ceph.osd.2.smithi090.stderr:2024-03-11T20:52:36.924+0000 7f754f0e1640 -1 osd.2 38 *** Got signal Terminated *** 2024-03-11T20:52:41.342 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-03-11T20:52:41.343 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-03-11T20:52:43.022 INFO:tasks.ceph.osd.2:Stopped 2024-03-11T20:52:43.023 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-03-11T20:52:43.023 INFO:teuthology.orchestra.run:waiting for 300 2024-03-11T20:52:43.024 INFO:tasks.ceph.osd.3.smithi090.stderr:2024-03-11T20:52:43.024+0000 7f36c4d2a640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term ceph-osd -f --cluster ceph -i 3 (PID: 27860) UID: 0 2024-03-11T20:52:43.024 INFO:tasks.ceph.osd.3.smithi090.stderr:2024-03-11T20:52:43.024+0000 7f36c4d2a640 -1 osd.3 39 *** Got signal Terminated *** 2024-03-11T20:52:47.250 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-03-11T20:52:47.251 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-03-11T20:52:49.125 INFO:tasks.ceph.osd.3:Stopped 2024-03-11T20:52:49.125 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-03-11T20:52:49.126 INFO:teuthology.orchestra.run:waiting for 300 2024-03-11T20:52:49.127 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-03-11T20:52:49.129+0000 7f2e77853640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term ceph-osd -f --cluster ceph -i 4 (PID: 27957) UID: 0 2024-03-11T20:52:49.127 INFO:tasks.ceph.osd.4.smithi092.stderr:2024-03-11T20:52:49.129+0000 7f2e77853640 -1 osd.4 40 *** Got signal Terminated *** 2024-03-11T20:52:53.059 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2024-03-11T20:52:53.059 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-03-11T20:52:55.227 INFO:tasks.ceph.osd.4:Stopped 2024-03-11T20:52:55.227 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-03-11T20:52:55.227 INFO:teuthology.orchestra.run:waiting for 300 2024-03-11T20:52:55.229 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-03-11T20:52:55.229+0000 7f0506f23640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term ceph-osd -f --cluster ceph -i 5 (PID: 27959) UID: 0 2024-03-11T20:52:55.229 INFO:tasks.ceph.osd.5.smithi092.stderr:2024-03-11T20:52:55.229+0000 7f0506f23640 -1 osd.5 41 *** Got signal Terminated *** 2024-03-11T20:52:58.764 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-03-11T20:52:58.764 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2024-03-11T20:53:01.329 INFO:tasks.ceph.osd.5:Stopped 2024-03-11T20:53:01.330 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-03-11T20:53:01.330 INFO:teuthology.orchestra.run:waiting for 300 2024-03-11T20:53:01.331 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-03-11T20:53:01.333+0000 7f366c890640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term ceph-osd -f --cluster ceph -i 6 (PID: 27961) UID: 0 2024-03-11T20:53:01.331 INFO:tasks.ceph.osd.6.smithi092.stderr:2024-03-11T20:53:01.333+0000 7f366c890640 -1 osd.6 42 *** Got signal Terminated *** 2024-03-11T20:53:01.841 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:53:01.845 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:53:04.369 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-03-11T20:53:04.370 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-03-11T20:53:07.432 INFO:tasks.ceph.osd.6:Stopped 2024-03-11T20:53:07.432 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-03-11T20:53:07.432 INFO:teuthology.orchestra.run:waiting for 300 2024-03-11T20:53:07.433 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-03-11T20:53:07.433+0000 7f3e1c097640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term ceph-osd -f --cluster ceph -i 7 (PID: 27963) UID: 0 2024-03-11T20:53:07.434 INFO:tasks.ceph.osd.7.smithi092.stderr:2024-03-11T20:53:07.433+0000 7f3e1c097640 -1 osd.7 42 *** Got signal Terminated *** 2024-03-11T20:53:09.875 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-03-11T20:53:09.875 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-03-11T20:53:13.534 INFO:tasks.ceph.osd.7:Stopped 2024-03-11T20:53:13.534 INFO:teuthology.misc:Shutting down mgr daemons... 2024-03-11T20:53:13.535 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-03-11T20:53:13.535 INFO:teuthology.orchestra.run:waiting for 300 2024-03-11T20:53:13.536 INFO:tasks.ceph.mgr.y.smithi090.stderr:2024-03-11T20:53:13.536+0000 7f7a88ef5640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term ceph-mgr -f --cluster ceph -i y (PID: 27661) UID: 0 2024-03-11T20:53:13.537 INFO:tasks.ceph.mgr.y.smithi090.stderr:2024-03-11T20:53:13.536+0000 7f7a88ef5640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-03-11T20:53:13.565 INFO:tasks.ceph.mgr.y:Stopped 2024-03-11T20:53:13.565 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-03-11T20:53:13.565 INFO:teuthology.orchestra.run:waiting for 300 2024-03-11T20:53:13.602 INFO:tasks.ceph.mgr.x:Stopped 2024-03-11T20:53:13.602 INFO:teuthology.misc:Shutting down mon daemons... 2024-03-11T20:53:13.602 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-03-11T20:53:13.602 INFO:teuthology.orchestra.run:waiting for 300 2024-03-11T20:53:13.628 INFO:tasks.ceph.mon.a.smithi090.stderr:2024-03-11T20:53:13.616+0000 a37a640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a (PID: 27656) UID: 0 2024-03-11T20:53:13.630 INFO:tasks.ceph.mon.a.smithi090.stderr:2024-03-11T20:53:13.616+0000 a37a640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-11T20:53:15.078 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2024-03-11T20:53:15.078 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-03-11T20:53:19.704 INFO:tasks.ceph.mon.a:Stopped 2024-03-11T20:53:19.705 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-03-11T20:53:19.705 INFO:teuthology.orchestra.run:waiting for 300 2024-03-11T20:53:19.745 INFO:tasks.ceph.mon.b.smithi092.stderr:2024-03-11T20:53:19.717+0000 a37a640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b (PID: 27630) UID: 0 2024-03-11T20:53:19.746 INFO:tasks.ceph.mon.b.smithi092.stderr:2024-03-11T20:53:19.721+0000 a37a640 -1 mon.b@1(peon) e1 *** Got Signal Terminated *** 2024-03-11T20:53:20.280 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2024-03-11T20:53:25.382 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2024-03-11T20:53:25.807 INFO:tasks.ceph.mon.b:Stopped 2024-03-11T20:53:25.807 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-03-11T20:53:25.807 INFO:teuthology.orchestra.run:waiting for 300 2024-03-11T20:53:25.825 INFO:tasks.ceph.mon.c.smithi092.stderr:2024-03-11T20:53:25.816+0000 a37a640 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c (PID: 27632) UID: 0 2024-03-11T20:53:25.827 INFO:tasks.ceph.mon.c.smithi092.stderr:2024-03-11T20:53:25.816+0000 a37a640 -1 mon.c@2(probing) e1 *** Got Signal Terminated *** 2024-03-11T20:53:30.383 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2024-03-11T20:53:30.384 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2024-03-11T20:53:31.894 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:53:31.897 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:53:31.908 INFO:tasks.ceph.mon.c:Stopped 2024-03-11T20:53:31.909 INFO:tasks.ceph:Checking cluster log for badness... 2024-03-11T20:53:31.909 DEBUG:teuthology.orchestra.run.smithi090:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow request' | egrep -v SLOW_OPS | egrep -v MON_DOWN | head -n 1 2024-03-11T20:53:31.958 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi090.front.sepia.ceph.com 2024-03-11T20:53:31.958 DEBUG:teuthology.orchestra.run.smithi090:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-03-11T20:53:32.247 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi090.front.sepia.ceph.com 2024-03-11T20:53:32.248 DEBUG:teuthology.orchestra.run.smithi090:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-03-11T20:53:32.493 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi090.front.sepia.ceph.com 2024-03-11T20:53:32.494 DEBUG:teuthology.orchestra.run.smithi090:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-03-11T20:53:32.765 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi090.front.sepia.ceph.com 2024-03-11T20:53:32.765 DEBUG:teuthology.orchestra.run.smithi090:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-03-11T20:53:33.043 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi092.front.sepia.ceph.com 2024-03-11T20:53:33.043 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-03-11T20:53:33.257 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi092.front.sepia.ceph.com 2024-03-11T20:53:33.257 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-03-11T20:53:33.455 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi092.front.sepia.ceph.com 2024-03-11T20:53:33.455 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-03-11T20:53:33.660 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi092.front.sepia.ceph.com 2024-03-11T20:53:33.660 DEBUG:teuthology.orchestra.run.smithi092:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-03-11T20:53:33.812 INFO:tasks.ceph:Archiving mon data... 2024-03-11T20:53:33.813 DEBUG:teuthology.misc:Transferring archived files from smithi090:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591231/data/mon.a.tgz 2024-03-11T20:53:33.813 DEBUG:teuthology.orchestra.run.smithi090:> mktemp 2024-03-11T20:53:33.819 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/tmp.6cesdJ8a1F 2024-03-11T20:53:33.820 DEBUG:teuthology.orchestra.run.smithi090:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.6cesdJ8a1F 2024-03-11T20:53:33.948 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 0666 /tmp/tmp.6cesdJ8a1F 2024-03-11T20:53:34.009 DEBUG:teuthology.orchestra.remote:smithi090:/tmp/tmp.6cesdJ8a1F is 414KB 2024-03-11T20:53:34.027 DEBUG:teuthology.orchestra.run.smithi090:> rm -fr /tmp/tmp.6cesdJ8a1F 2024-03-11T20:53:34.032 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591231/data/mon.b.tgz 2024-03-11T20:53:34.032 DEBUG:teuthology.orchestra.run.smithi092:> mktemp 2024-03-11T20:53:34.038 INFO:teuthology.orchestra.run.smithi092.stdout:/tmp/tmp.gUdAq6wryV 2024-03-11T20:53:34.038 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.gUdAq6wryV 2024-03-11T20:53:34.172 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0666 /tmp/tmp.gUdAq6wryV 2024-03-11T20:53:34.235 DEBUG:teuthology.orchestra.remote:smithi092:/tmp/tmp.gUdAq6wryV is 434KB 2024-03-11T20:53:34.297 DEBUG:teuthology.orchestra.run.smithi092:> rm -fr /tmp/tmp.gUdAq6wryV 2024-03-11T20:53:34.302 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591231/data/mon.c.tgz 2024-03-11T20:53:34.302 DEBUG:teuthology.orchestra.run.smithi092:> mktemp 2024-03-11T20:53:34.347 INFO:teuthology.orchestra.run.smithi092.stdout:/tmp/tmp.sqbqAAJLXl 2024-03-11T20:53:34.348 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.sqbqAAJLXl 2024-03-11T20:53:34.483 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0666 /tmp/tmp.sqbqAAJLXl 2024-03-11T20:53:34.547 DEBUG:teuthology.orchestra.remote:smithi092:/tmp/tmp.sqbqAAJLXl is 434KB 2024-03-11T20:53:34.609 DEBUG:teuthology.orchestra.run.smithi092:> rm -fr /tmp/tmp.sqbqAAJLXl 2024-03-11T20:53:34.614 INFO:tasks.ceph:Cleaning ceph cluster... 2024-03-11T20:53:34.614 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-11T20:53:34.616 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-11T20:53:35.277 DEBUG:tasks.ceph:valgrind exception message: valgrind error: Leak_StillReachable calloc calloc _dl_check_map_versions 2024-03-11T20:53:35.385 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2024-03-11T20:53:35.684 INFO:tasks.ceph:Archiving crash dumps... 2024-03-11T20:53:35.856 DEBUG:teuthology.misc:Transferring archived files from smithi090:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591231/remote/smithi090/crash 2024-03-11T20:53:35.886 DEBUG:teuthology.orchestra.run.smithi090:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-11T20:53:35.916 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591231/remote/smithi092/crash 2024-03-11T20:53:35.918 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-11T20:53:35.932 INFO:tasks.ceph:Shutting down logrotate 2024-03-11T20:53:35.933 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:53:35.968 DEBUG:teuthology.orchestra.run.smithi092:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-11T20:53:36.132 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-11T20:53:36.149 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-11T20:53:36.162 INFO:tasks.ceph:Compressing logs... 2024-03-11T20:53:36.162 DEBUG:teuthology.orchestra.run.smithi090:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-11T20:53:36.195 DEBUG:teuthology.orchestra.run.smithi092:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-11T20:53:36.216 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34285.log 2024-03-11T20:53:36.217 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34864.log 2024-03-11T20:53:36.217 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.34285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35681.log 2024-03-11T20:53:36.220 INFO:teuthology.orchestra.run.smithi090.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34285.log.gz 2024-03-11T20:53:36.220 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.34864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38136.log 2024-03-11T20:53:36.220 INFO:teuthology.orchestra.run.smithi090.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34864.log.gz 2024-03-11T20:53:36.220 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35112.log 2024-03-11T20:53:36.220 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.38136.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35681.log.gz 2024-03-11T20:53:36.220 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-03-11T20:53:36.221 INFO:teuthology.orchestra.run.smithi090.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.38136.log.gz 2024-03-11T20:53:36.221 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37518.log 2024-03-11T20:53:36.221 INFO:teuthology.orchestra.run.smithi090.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35112.log.gz 2024-03-11T20:53:36.221 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35498.log 2024-03-11T20:53:36.221 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.37518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33444.log 2024-03-11T20:53:36.221 INFO:teuthology.orchestra.run.smithi090.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37518.log.gz 2024-03-11T20:53:36.221 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34631.log 2024-03-11T20:53:36.221 INFO:teuthology.orchestra.run.smithi090.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35498.log.gz 2024-03-11T20:53:36.224 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34181.log 2024-03-11T20:53:36.224 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.33444.log: /var/log/ceph/ceph-client.admin.34631.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-03-11T20:53:36.224 INFO:teuthology.orchestra.run.smithi090.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33444.log.gz 2024-03-11T20:53:36.224 INFO:teuthology.orchestra.run.smithi090.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34631.log.gz 2024-03-11T20:53:36.224 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.34181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38345.log 2024-03-11T20:53:36.224 INFO:teuthology.orchestra.run.smithi090.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34181.log.gz 2024-03-11T20:53:36.224 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.36299.log 2024-03-11T20:53:36.225 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.38345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33651.log 2024-03-11T20:53:36.225 INFO:teuthology.orchestra.run.smithi090.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38345.log.gz 2024-03-11T20:53:36.225 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.0.36299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33360.log 2024-03-11T20:53:36.225 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.33651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33599.log 2024-03-11T20:53:36.225 INFO:teuthology.orchestra.run.smithi090.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33651.log.gz 2024-03-11T20:53:36.225 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.33360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38162.log 2024-03-11T20:53:36.225 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.33599.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33360.log.gz 2024-03-11T20:53:36.225 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37571.log 2024-03-11T20:53:36.225 INFO:teuthology.orchestra.run.smithi090.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33599.log.gz 2024-03-11T20:53:36.228 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.38162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34579.log 2024-03-11T20:53:36.228 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.37571.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38162.log.gz 2024-03-11T20:53:36.228 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37649.log 2024-03-11T20:53:36.228 INFO:teuthology.orchestra.run.smithi090.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37571.log.gz 2024-03-11T20:53:36.228 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.34579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34468.log 2024-03-11T20:53:36.228 INFO:teuthology.orchestra.run.smithi090.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34579.log.gz 2024-03-11T20:53:36.228 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.37649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36118.log 2024-03-11T20:53:36.228 INFO:teuthology.orchestra.run.smithi090.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37649.log.gz 2024-03-11T20:53:36.228 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.34468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34683.log 2024-03-11T20:53:36.228 INFO:teuthology.orchestra.run.smithi090.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34468.log.gz 2024-03-11T20:53:36.228 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.36118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34551.log 2024-03-11T20:53:36.228 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33386.log 2024-03-11T20:53:36.228 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.34683.log: /var/log/ceph/ceph-client.admin.34551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33287.log 2024-03-11T20:53:36.229 INFO:teuthology.orchestra.run.smithi090.stderr: 90.8% 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36118.log.gz 2024-03-11T20:53:36.229 INFO:teuthology.orchestra.run.smithi090.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34683.log.gz 2024-03-11T20:53:36.229 INFO:teuthology.orchestra.run.smithi090.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34551.log.gz 2024-03-11T20:53:36.238 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.33386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27978.log 2024-03-11T20:53:36.238 INFO:teuthology.orchestra.run.smithi090.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33386.log.gz 2024-03-11T20:53:36.238 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.33287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37597.log 2024-03-11T20:53:36.238 INFO:teuthology.orchestra.run.smithi090.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33287.log.gz 2024-03-11T20:53:36.238 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37544.log 2024-03-11T20:53:36.238 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.27978.log: /var/log/ceph/ceph-client.admin.37597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37980.log 2024-03-11T20:53:36.238 INFO:teuthology.orchestra.run.smithi090.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.27978.log.gz 2024-03-11T20:53:36.238 INFO:teuthology.orchestra.run.smithi090.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37597.log.gz 2024-03-11T20:53:36.238 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.37544.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-03-11T20:53:36.238 INFO:teuthology.orchestra.run.smithi090.stderr: 87.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.37544.log.gz 2024-03-11T20:53:36.239 INFO:teuthology.orchestra.run.smithi090.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.33547.log 2024-03-11T20:53:36.239 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.37980.log: /var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38188.log 2024-03-11T20:53:36.239 INFO:teuthology.orchestra.run.smithi090.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37980.log.gz 2024-03-11T20:53:36.239 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.33547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35524.log 2024-03-11T20:53:36.239 INFO:teuthology.orchestra.run.smithi090.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33547.log.gz 2024-03-11T20:53:36.239 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.38188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35786.log 2024-03-11T20:53:36.239 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35864.log 2024-03-11T20:53:36.239 INFO:teuthology.orchestra.run.smithi090.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38188.log.gz 2024-03-11T20:53:36.239 INFO:teuthology.orchestra.run.smithi090.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35524.log.gz 2024-03-11T20:53:36.239 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37413.log 2024-03-11T20:53:36.239 INFO:teuthology.orchestra.run.smithi090.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35786.log.gz 2024-03-11T20:53:36.239 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35099.log 2024-03-11T20:53:36.239 INFO:teuthology.orchestra.run.smithi090.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35864.log.gz 2024-03-11T20:53:36.239 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.37413.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-03-11T20:53:36.240 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35812.log 2024-03-11T20:53:36.240 INFO:teuthology.orchestra.run.smithi090.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37413.log.gz 2024-03-11T20:53:36.240 INFO:teuthology.orchestra.run.smithi090.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35099.log.gz 2024-03-11T20:53:36.240 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35317.log 2024-03-11T20:53:36.240 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33480.log 2024-03-11T20:53:36.240 INFO:teuthology.orchestra.run.smithi090.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35812.log.gz 2024-03-11T20:53:36.240 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34001.log 2024-03-11T20:53:36.240 INFO:teuthology.orchestra.run.smithi090.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35317.log.gz 2024-03-11T20:53:36.240 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.33480.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33480.log.gz 2024-03-11T20:53:36.240 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35760.log 2024-03-11T20:53:36.240 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.34001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35629.log 2024-03-11T20:53:36.240 INFO:teuthology.orchestra.run.smithi090.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34001.log.gz/var/log/ceph/ceph-client.admin.35760.log: 2024-03-11T20:53:36.240 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37466.log 2024-03-11T20:53:36.240 INFO:teuthology.orchestra.run.smithi090.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35760.log.gz 2024-03-11T20:53:36.240 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34856.log 2024-03-11T20:53:36.252 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27904.log 2024-03-11T20:53:36.252 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27780.log 2024-03-11T20:53:36.252 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27828.log 2024-03-11T20:53:36.252 INFO:teuthology.orchestra.run.smithi092.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.27904.log.gz 2024-03-11T20:53:36.252 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27780.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-03-11T20:53:36.252 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27828.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-03-11T20:53:36.252 INFO:teuthology.orchestra.run.smithi092.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.27780.log.gz 2024-03-11T20:53:36.252 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-osd.7.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.27828.log.gz 2024-03-11T20:53:36.253 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-03-11T20:53:36.253 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27804.log 2024-03-11T20:53:36.253 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-03-11T20:53:36.253 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27804.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.f.log 2024-03-11T20:53:36.253 INFO:teuthology.orchestra.run.smithi092.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.27804.log.gz 2024-03-11T20:53:36.253 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-11T20:53:36.253 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27877.log 2024-03-11T20:53:36.253 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27749.log 2024-03-11T20:53:36.253 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27877.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-03-11T20:53:36.253 INFO:teuthology.orchestra.run.smithi092.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.27877.log.gz 2024-03-11T20:53:36.253 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27930.log 2024-03-11T20:53:36.253 INFO:teuthology.orchestra.run.smithi092.stderr: 88.0%/var/log/ceph/ceph-mon.c.log: -- replaced with /var/log/ceph/ceph-client.admin.27749.log.gz 2024-03-11T20:53:36.253 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-11T20:53:36.253 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27930.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.27930.log.gz 2024-03-11T20:53:36.253 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27853.log 2024-03-11T20:53:36.254 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph.audit.log: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-11T20:53:36.254 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-03-11T20:53:36.254 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-client.admin.27853.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.27853.log.gz 2024-03-11T20:53:36.254 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/valgrind/mon.c.log 2024-03-11T20:53:36.260 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.37466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34813.log 2024-03-11T20:53:36.260 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35629.log: /var/log/ceph/ceph-client.admin.34856.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37466.log.gz 2024-03-11T20:53:36.260 INFO:teuthology.orchestra.run.smithi090.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35629.log.gz 2024-03-11T20:53:36.260 INFO:teuthology.orchestra.run.smithi090.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34856.log.gz 2024-03-11T20:53:36.260 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37798.log 2024-03-11T20:53:36.260 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.34813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33521.log 2024-03-11T20:53:36.260 INFO:teuthology.orchestra.run.smithi090.stderr: 88.2%/var/log/ceph/ceph-client.admin.37798.log: -- replaced with /var/log/ceph/ceph-client.admin.34813.log.gz 2024-03-11T20:53:36.261 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38449.log 2024-03-11T20:53:36.261 INFO:teuthology.orchestra.run.smithi090.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37798.log.gz 2024-03-11T20:53:36.261 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.33521.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33521.log.gz 2024-03-11T20:53:36.261 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37824.log 2024-03-11T20:53:36.261 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.38449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38084.log 2024-03-11T20:53:36.261 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.37824.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38449.log.gz 2024-03-11T20:53:36.261 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37928.log 2024-03-11T20:53:36.261 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38110.log 2024-03-11T20:53:36.261 INFO:teuthology.orchestra.run.smithi090.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37824.log.gz 2024-03-11T20:53:36.261 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.38084.log: /var/log/ceph/ceph-client.admin.37928.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38084.log.gz 2024-03-11T20:53:36.261 INFO:teuthology.orchestra.run.smithi090.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37928.log.gz 2024-03-11T20:53:36.261 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35115.log 2024-03-11T20:53:36.261 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.38110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34862.log 2024-03-11T20:53:36.261 INFO:teuthology.orchestra.run.smithi090.stderr: 88.1%/var/log/ceph/ceph-client.admin.35115.log: -- replaced with /var/log/ceph/ceph-client.admin.38110.log.gz 2024-03-11T20:53:36.262 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34761.log 2024-03-11T20:53:36.262 INFO:teuthology.orchestra.run.smithi090.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35115.log.gz 2024-03-11T20:53:36.262 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.34862.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34862.log.gz 2024-03-11T20:53:36.262 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34364.log 2024-03-11T20:53:36.262 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.34761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33704.log 2024-03-11T20:53:36.262 INFO:teuthology.orchestra.run.smithi090.stderr: 89.3%/var/log/ceph/ceph-client.admin.34364.log: -- replaced with /var/log/ceph/ceph-client.admin.34761.log.gz 2024-03-11T20:53:36.262 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35019.log 2024-03-11T20:53:36.262 INFO:teuthology.orchestra.run.smithi090.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34364.log.gz 2024-03-11T20:53:36.262 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.33704.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33704.log.gz 2024-03-11T20:53:36.262 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-03-11T20:53:36.262 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34861.log 2024-03-11T20:53:36.262 INFO:teuthology.orchestra.run.smithi090.stderr: 88.0%/var/log/ceph/ceph-osd.3.log: -- replaced with /var/log/ceph/ceph-client.admin.35019.log.gz 2024-03-11T20:53:36.262 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37492.log 2024-03-11T20:53:36.262 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.34861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35550.log 2024-03-11T20:53:36.263 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.37492.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34861.log.gz 2024-03-11T20:53:36.263 INFO:teuthology.orchestra.run.smithi090.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37492.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34709.log 2024-03-11T20:53:36.263 INFO:teuthology.orchestra.run.smithi090.stderr: 2024-03-11T20:53:36.263 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35550.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35550.log.gz 2024-03-11T20:53:36.263 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38371.log 2024-03-11T20:53:36.263 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.34709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38006.log 2024-03-11T20:53:36.263 INFO:teuthology.orchestra.run.smithi090.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34709.log.gz/var/log/ceph/ceph-client.admin.38371.log: 2024-03-11T20:53:36.263 INFO:teuthology.orchestra.run.smithi090.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38371.log.gz 2024-03-11T20:53:36.263 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37701.log 2024-03-11T20:53:36.263 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38501.log 2024-03-11T20:53:36.263 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.38006.log: /var/log/ceph/ceph-client.admin.37701.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38006.log.gz 2024-03-11T20:53:36.263 INFO:teuthology.orchestra.run.smithi090.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37701.log.gz 2024-03-11T20:53:36.263 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34416.log 2024-03-11T20:53:36.263 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.38501.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38501.log.gz 2024-03-11T20:53:36.263 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38527.log 2024-03-11T20:53:36.264 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34787.log 2024-03-11T20:53:36.264 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.34416.log: /var/log/ceph/ceph-client.admin.38527.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34416.log.gz 2024-03-11T20:53:36.264 INFO:teuthology.orchestra.run.smithi090.stderr: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34850.log 2024-03-11T20:53:36.264 INFO:teuthology.orchestra.run.smithi090.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38527.log.gz 2024-03-11T20:53:36.264 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.34787.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34787.log.gz 2024-03-11T20:53:36.264 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-03-11T20:53:36.264 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38240.log 2024-03-11T20:53:36.264 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.34850.log: /var/log/ceph/ceph-osd.1.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34850.log.gz 2024-03-11T20:53:36.264 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35359.log 2024-03-11T20:53:36.266 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/valgrind/mds.f.log 2024-03-11T20:53:36.267 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/valgrind/mon.c.log: 91.6% -- replaced with /var/log/ceph/valgrind/mon.c.log.gz 2024-03-11T20:53:36.267 INFO:teuthology.orchestra.run.smithi092.stderr: 94.1% -- replaced with /var/log/ceph/ceph.log.gz 2024-03-11T20:53:36.267 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/valgrind/mds.b.log 2024-03-11T20:53:36.268 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.38240.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38240.log.gz 2024-03-11T20:53:36.269 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35255.log 2024-03-11T20:53:36.269 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35359.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35359.log.gz 2024-03-11T20:53:36.269 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34207.log 2024-03-11T20:53:36.269 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35255.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35255.log.gz 2024-03-11T20:53:36.269 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35655.log 2024-03-11T20:53:36.269 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.34207.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34207.log.gz 2024-03-11T20:53:36.269 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35734.log 2024-03-11T20:53:36.269 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35655.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35655.log.gz 2024-03-11T20:53:36.269 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34866.log 2024-03-11T20:53:36.269 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35734.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35734.log.gz 2024-03-11T20:53:36.269 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36018.log 2024-03-11T20:53:36.269 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.34866.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34866.log.gz 2024-03-11T20:53:36.269 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38214.log 2024-03-11T20:53:36.271 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/valgrind/mds.f.log: 95.4% -- replaced with /var/log/ceph/valgrind/mds.f.log.gz 2024-03-11T20:53:36.271 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/valgrind/mds.d.log 2024-03-11T20:53:36.272 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.36018.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.36018.log.gz 2024-03-11T20:53:36.272 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38397.log 2024-03-11T20:53:36.272 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.38214.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38214.log.gz 2024-03-11T20:53:36.272 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35577.log 2024-03-11T20:53:36.272 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.38397.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38397.log.gz 2024-03-11T20:53:36.273 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37902.log 2024-03-11T20:53:36.273 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35577.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35577.log.gz 2024-03-11T20:53:36.274 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-11T20:53:36.274 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.37902.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37902.log.gz 2024-03-11T20:53:36.274 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/valgrind/mds.b.log: 95.1% -- replaced with /var/log/ceph/valgrind/mds.b.log.gz 2024-03-11T20:53:36.274 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/valgrind/mon.b.log 2024-03-11T20:53:36.275 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35838.log 2024-03-11T20:53:36.277 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/valgrind/mds.d.log: 95.4% -- replaced with /var/log/ceph/valgrind/mds.d.log.gz 2024-03-11T20:53:36.277 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-03-11T20:53:36.279 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/valgrind/mon.b.log: 91.6% -- replaced with /var/log/ceph/valgrind/mon.b.log.gz 2024-03-11T20:53:36.279 INFO:teuthology.orchestra.run.smithi092.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-03-11T20:53:36.288 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph.log: 88.9% -- replaced with /var/log/ceph/ceph.log.gz 2024-03-11T20:53:36.289 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35465.log 2024-03-11T20:53:36.290 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35838.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35838.log.gz 2024-03-11T20:53:36.290 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34499.log 2024-03-11T20:53:36.291 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-03-11T20:53:36.292 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35465.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35465.log.gz 2024-03-11T20:53:36.292 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33677.log 2024-03-11T20:53:36.292 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.34499.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34499.log.gz 2024-03-11T20:53:36.293 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27663.log 2024-03-11T20:53:36.293 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.33677.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.33677.log.gz 2024-03-11T20:53:36.294 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38319.log 2024-03-11T20:53:36.295 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.27663.log: 93.2% -- replaced with /var/log/ceph/ceph-client.admin.27663.log.gz 2024-03-11T20:53:36.295 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34657.log 2024-03-11T20:53:36.296 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.38319.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38319.log.gz 2024-03-11T20:53:36.296 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35059.log 2024-03-11T20:53:36.297 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.34657.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34657.log.gz 2024-03-11T20:53:36.297 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34525.log 2024-03-11T20:53:36.298 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35059.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35059.log.gz 2024-03-11T20:53:36.299 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35042.log 2024-03-11T20:53:36.299 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.34525.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34525.log.gz 2024-03-11T20:53:36.300 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37439.log 2024-03-11T20:53:36.300 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35042.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35042.log.gz 2024-03-11T20:53:36.301 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33767.log 2024-03-11T20:53:36.301 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.37439.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37439.log.gz 2024-03-11T20:53:36.302 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33899.log 2024-03-11T20:53:36.303 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.33767.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33767.log.gz 2024-03-11T20:53:36.303 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37675.log 2024-03-11T20:53:36.304 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.33899.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33899.log.gz 2024-03-11T20:53:36.304 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38032.log 2024-03-11T20:53:36.306 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.37675.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37675.log.gz 2024-03-11T20:53:36.306 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33951.log 2024-03-11T20:53:36.307 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.38032.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38032.log.gz 2024-03-11T20:53:36.307 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35890.log 2024-03-11T20:53:36.309 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.33951.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.33951.log.gz 2024-03-11T20:53:36.309 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33222.log 2024-03-11T20:53:36.310 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35890.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35890.log.gz 2024-03-11T20:53:36.310 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33925.log 2024-03-11T20:53:36.311 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.33222.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33222.log.gz 2024-03-11T20:53:36.312 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37954.log 2024-03-11T20:53:36.313 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.33925.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33925.log.gz 2024-03-11T20:53:36.313 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34311.log 2024-03-11T20:53:36.314 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.37954.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37954.log.gz 2024-03-11T20:53:36.314 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34127.log 2024-03-11T20:53:36.315 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.34311.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34311.log.gz 2024-03-11T20:53:36.316 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35331.log 2024-03-11T20:53:36.318 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.34127.log: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.34127.log.gz 2024-03-11T20:53:36.319 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33821.log 2024-03-11T20:53:36.320 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35331.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35331.log.gz 2024-03-11T20:53:36.320 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37746.log 2024-03-11T20:53:36.321 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.33821.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33821.log.gz 2024-03-11T20:53:36.321 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35708.log 2024-03-11T20:53:36.322 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.37746.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37746.log.gz 2024-03-11T20:53:36.322 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38475.log 2024-03-11T20:53:36.323 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35708.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35708.log.gz 2024-03-11T20:53:36.324 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33625.log 2024-03-11T20:53:36.325 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.38475.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38475.log.gz 2024-03-11T20:53:36.325 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35283.log 2024-03-11T20:53:36.326 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.33625.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33625.log.gz 2024-03-11T20:53:36.326 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-11T20:53:36.327 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35283.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35283.log.gz 2024-03-11T20:53:36.328 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36179.log 2024-03-11T20:53:36.329 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph.audit.log: 89.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-11T20:53:36.330 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-03-11T20:53:36.331 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.36179.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36179.log.gz 2024-03-11T20:53:36.331 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33795.log 2024-03-11T20:53:36.337 INFO:teuthology.orchestra.run.smithi092.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-osd.4.log: 95.3% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-03-11T20:53:36.354 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35259.log 2024-03-11T20:53:36.355 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.33795.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33795.log.gz 2024-03-11T20:53:36.365 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-03-11T20:53:36.366 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35259.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35259.log.gz 2024-03-11T20:53:36.373 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38266.log 2024-03-11T20:53:36.381 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34390.log 2024-03-11T20:53:36.382 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.38266.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.38266.log.gz 2024-03-11T20:53:36.390 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/valgrind/client.0.log 2024-03-11T20:53:36.390 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.34390.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34390.log.gz 2024-03-11T20:53:36.398 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/valgrind/mds.a.log 2024-03-11T20:53:36.398 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/valgrind/client.0.log: 77.9% -- replaced with /var/log/ceph/valgrind/client.0.log.gz 2024-03-11T20:53:36.405 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/valgrind/mon.a.log 2024-03-11T20:53:36.406 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/valgrind/mds.a.log: 57.7% -- replaced with /var/log/ceph/valgrind/mds.a.log.gz 2024-03-11T20:53:36.413 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/valgrind/mds.c.log 2024-03-11T20:53:36.414 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/valgrind/mon.a.log: 91.6% -- replaced with /var/log/ceph/valgrind/mon.a.log.gz 2024-03-11T20:53:36.421 INFO:teuthology.orchestra.run.smithi090.stderr: 93.0%gzip -5 --verbose -- /var/log/ceph/valgrind/mds.e.log 2024-03-11T20:53:36.424 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/valgrind/mds.c.log: 95.2% -- replaced with /var/log/ceph/valgrind/mds.c.log.gz 2024-03-11T20:53:36.429 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34867.log 2024-03-11T20:53:36.433 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/valgrind/mds.e.log: -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-03-11T20:53:36.445 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.22857.log 2024-03-11T20:53:36.446 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.34867.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.34867.log.gz 2024-03-11T20:53:36.449 INFO:teuthology.orchestra.run.smithi090.stderr: 95.3% -- replaced with /var/log/ceph/valgrind/mds.e.log.gz 2024-03-11T20:53:36.453 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34259.log 2024-03-11T20:53:36.454 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph.tmp-client.admin.22857.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.22857.log.gz 2024-03-11T20:53:36.469 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37623.log 2024-03-11T20:53:36.470 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.34259.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34259.log.gz 2024-03-11T20:53:36.481 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36068.log 2024-03-11T20:53:36.482 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.37623.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37623.log.gz 2024-03-11T20:53:36.497 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34605.log 2024-03-11T20:53:36.499 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.36068.log: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.36068.log.gz 2024-03-11T20:53:36.509 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37772.log 2024-03-11T20:53:36.510 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.34605.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34605.log.gz 2024-03-11T20:53:36.534 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34735.log 2024-03-11T20:53:36.534 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.37772.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37772.log.gz 2024-03-11T20:53:36.545 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37850.log 2024-03-11T20:53:36.546 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.34735.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34735.log.gz 2024-03-11T20:53:36.570 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33573.log 2024-03-11T20:53:36.570 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.37850.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37850.log.gz 2024-03-11T20:53:36.582 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35043.log 2024-03-11T20:53:36.582 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.33573.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33573.log.gz 2024-03-11T20:53:36.593 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35916.log 2024-03-11T20:53:36.594 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35043.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35043.log.gz 2024-03-11T20:53:36.610 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33873.log 2024-03-11T20:53:36.611 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35916.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.35916.log.gz 2024-03-11T20:53:36.625 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37876.log 2024-03-11T20:53:36.626 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.33873.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33873.log.gz 2024-03-11T20:53:36.635 INFO:teuthology.orchestra.run.smithi092.stderr: 93.9% -- replaced with /var/log/ceph/ceph-mds.f.log.gz 2024-03-11T20:53:36.642 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38423.log 2024-03-11T20:53:36.646 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.37876.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37876.log.gz 2024-03-11T20:53:36.649 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33329.log 2024-03-11T20:53:36.650 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.38423.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38423.log.gz 2024-03-11T20:53:36.661 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34338.log 2024-03-11T20:53:36.662 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.33329.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33329.log.gz 2024-03-11T20:53:36.668 INFO:teuthology.orchestra.run.smithi092.stderr: 93.8% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-03-11T20:53:36.673 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34865.log 2024-03-11T20:53:36.674 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.34338.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34338.log.gz 2024-03-11T20:53:36.685 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33739.log 2024-03-11T20:53:36.686 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.34865.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34865.log.gz 2024-03-11T20:53:36.691 INFO:teuthology.orchestra.run.smithi092.stderr: 93.8% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-03-11T20:53:36.697 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38058.log 2024-03-11T20:53:36.698 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.33739.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33739.log.gz 2024-03-11T20:53:36.714 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34233.log 2024-03-11T20:53:36.714 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.38058.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38058.log.gz 2024-03-11T20:53:36.716 INFO:teuthology.orchestra.run.smithi090.stderr: 94.0% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-03-11T20:53:36.718 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35968.log 2024-03-11T20:53:36.718 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.34233.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34233.log.gz 2024-03-11T20:53:36.719 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35336.log 2024-03-11T20:53:36.720 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35968.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.35968.log.gz 2024-03-11T20:53:36.720 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35067.log 2024-03-11T20:53:36.721 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35336.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35336.log.gz 2024-03-11T20:53:36.722 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38295.log 2024-03-11T20:53:36.722 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35067.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35067.log.gz 2024-03-11T20:53:36.723 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35603.log 2024-03-11T20:53:36.724 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.38295.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38295.log.gz 2024-03-11T20:53:36.724 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33847.log 2024-03-11T20:53:36.725 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35603.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35603.log.gz 2024-03-11T20:53:36.725 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.e.log 2024-03-11T20:53:36.726 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.33847.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33847.log.gz 2024-03-11T20:53:36.726 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35373.log 2024-03-11T20:53:36.741 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34442.log 2024-03-11T20:53:36.742 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.35373.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35373.log.gz 2024-03-11T20:53:36.761 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27865.log 2024-03-11T20:53:36.762 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.34442.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34442.log.gz 2024-03-11T20:53:36.764 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.27865.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.27865.log.gz 2024-03-11T20:53:37.421 INFO:teuthology.orchestra.run.smithi090.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-03-11T20:53:38.183 INFO:teuthology.orchestra.run.smithi092.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-03-11T20:53:38.632 INFO:teuthology.orchestra.run.smithi092.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-03-11T20:53:40.231 INFO:teuthology.orchestra.run.smithi090.stderr: 89.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-03-11T20:53:48.163 INFO:teuthology.orchestra.run.smithi092.stderr: 91.3% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-03-11T20:53:48.705 INFO:teuthology.orchestra.run.smithi092.stderr: 91.3% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-03-11T20:53:49.434 INFO:teuthology.orchestra.run.smithi090.stderr: 91.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-03-11T20:53:51.290 INFO:teuthology.orchestra.run.smithi092.stderr: 91.7% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-03-11T20:53:51.622 INFO:teuthology.orchestra.run.smithi092.stderr: 91.3% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-03-11T20:53:51.623 INFO:teuthology.orchestra.run.smithi092.stderr: 2024-03-11T20:53:51.623 INFO:teuthology.orchestra.run.smithi092.stderr:real 0m15.413s 2024-03-11T20:53:51.623 INFO:teuthology.orchestra.run.smithi092.stderr:user 0m58.073s 2024-03-11T20:53:51.623 INFO:teuthology.orchestra.run.smithi092.stderr:sys 0m1.919s 2024-03-11T20:53:52.648 INFO:teuthology.orchestra.run.smithi090.stderr: 91.2% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-03-11T20:53:56.310 INFO:teuthology.orchestra.run.smithi090.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.0.36299.log.gz 2024-03-11T20:53:58.237 INFO:teuthology.orchestra.run.smithi090.stderr: 91.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-03-11T20:53:58.354 INFO:teuthology.orchestra.run.smithi090.stderr: 91.2% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-03-11T20:54:33.659 INFO:teuthology.orchestra.run.smithi090.stderr: 92.5% -- replaced with /var/log/ceph/ceph-mds.e.log.gz 2024-03-11T20:54:33.660 INFO:teuthology.orchestra.run.smithi090.stderr: 2024-03-11T20:54:33.660 INFO:teuthology.orchestra.run.smithi090.stderr:real 0m57.459s 2024-03-11T20:54:33.660 INFO:teuthology.orchestra.run.smithi090.stderr:user 2m31.714s 2024-03-11T20:54:33.660 INFO:teuthology.orchestra.run.smithi090.stderr:sys 0m4.681s 2024-03-11T20:54:33.660 INFO:tasks.ceph:Archiving logs... 2024-03-11T20:54:33.661 DEBUG:teuthology.misc:Transferring archived files from smithi090:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591231/remote/smithi090/log 2024-03-11T20:54:33.661 DEBUG:teuthology.orchestra.run.smithi090:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-11T20:54:52.699 DEBUG:teuthology.misc:Transferring archived files from smithi092:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591231/remote/smithi092/log 2024-03-11T20:54:52.700 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-11T20:55:01.118 ERROR:teuthology.run_tasks:Manager failed: ceph Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_b6a4997be2b8bb74411268cc42753f29ae9a78ad/qa/tasks/ceph.py", line 1935, in task mon0_remote.run( File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph_b6a4997be2b8bb74411268cc42753f29ae9a78ad/qa/tasks/ceph.py", line 252, in ceph_log yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_b6a4997be2b8bb74411268cc42753f29ae9a78ad/qa/tasks/ceph.py", line 346, in valgrind_post raise valgrind_exception Exception: valgrind error: Leak_StillReachable calloc calloc _dl_check_map_versions 2024-03-11T20:55:01.119 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-11T20:55:01.128 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_b6a4997be2b8bb74411268cc42753f29ae9a78ad/qa/tasks/ceph.py", line 1935, in task mon0_remote.run( File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph_b6a4997be2b8bb74411268cc42753f29ae9a78ad/qa/tasks/ceph.py", line 252, in ceph_log yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_b6a4997be2b8bb74411268cc42753f29ae9a78ad/qa/tasks/ceph.py", line 346, in valgrind_post raise valgrind_exception Exception: valgrind error: Leak_StillReachable calloc calloc _dl_check_map_versions 2024-03-11T20:55:01.129 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-11T20:55:01.129 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-11T20:55:01.132 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-11T20:55:01.213 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-11T20:55:01.225 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-11T20:55:01.225 DEBUG:teuthology.orchestra.run.smithi090:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-11T20:55:01.227 DEBUG:teuthology.orchestra.run.smithi092:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-11T20:55:01.241 INFO:teuthology.orchestra.run.smithi090.stdout: remote refid st t when poll reach delay offset jitter 2024-03-11T20:55:01.241 INFO:teuthology.orchestra.run.smithi090.stdout:============================================================================== 2024-03-11T20:55:01.241 INFO:teuthology.orchestra.run.smithi090.stdout:*hv01.front.sepi 44.4.53.2 2 u 212 256 377 0.168 +0.518 0.082 2024-03-11T20:55:01.241 INFO:teuthology.orchestra.run.smithi090.stdout:+hv02.front.sepi 63.231.80.2 3 u 10 256 377 0.169 +2.798 0.346 2024-03-11T20:55:01.241 INFO:teuthology.orchestra.run.smithi090.stdout:+hv03.front.sepi 74.6.168.72 3 u 22 256 377 0.183 -1.187 0.900 2024-03-11T20:55:01.241 INFO:teuthology.orchestra.run.smithi090.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 +0.000 0.000 2024-03-11T20:55:01.246 INFO:teuthology.orchestra.run.smithi092.stdout: remote refid st t when poll reach delay offset jitter 2024-03-11T20:55:01.246 INFO:teuthology.orchestra.run.smithi092.stdout:============================================================================== 2024-03-11T20:55:01.246 INFO:teuthology.orchestra.run.smithi092.stdout:*hv01.front.sepi 44.4.53.2 2 u 195 256 377 0.171 +0.541 1.724 2024-03-11T20:55:01.246 INFO:teuthology.orchestra.run.smithi092.stdout:+hv02.front.sepi 63.231.80.2 3 u 216 256 377 0.203 +0.945 0.930 2024-03-11T20:55:01.246 INFO:teuthology.orchestra.run.smithi092.stdout:+hv03.front.sepi 74.6.168.72 3 u 203 256 377 0.182 -1.295 0.754 2024-03-11T20:55:01.246 INFO:teuthology.orchestra.run.smithi092.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 +0.000 0.000 2024-03-11T20:55:01.247 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-11T20:55:01.256 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-11T20:55:01.257 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-11T20:55:01.295 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-11T20:55:01.309 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-11T20:55:01.324 INFO:teuthology.task.internal:Duration was 9396.401775 seconds 2024-03-11T20:55:01.325 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-11T20:55:01.336 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-11T20:55:01.336 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-11T20:55:01.339 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-11T20:55:01.379 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-11T20:55:01.379 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi090.front.sepia.ceph.com 2024-03-11T20:55:01.379 DEBUG:teuthology.orchestra.run.smithi090:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-11T20:55:01.427 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi092.front.sepia.ceph.com 2024-03-11T20:55:01.427 DEBUG:teuthology.orchestra.run.smithi092:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-11T20:55:01.437 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-11T20:55:01.437 DEBUG:teuthology.orchestra.run.smithi090:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-11T20:55:01.471 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-11T20:55:01.510 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-11T20:55:01.511 DEBUG:teuthology.orchestra.run.smithi090:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-11T20:55:01.543 DEBUG:teuthology.orchestra.run.smithi092:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-11T20:55:01.619 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-11T20:55:01.630 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-11T20:55:01.630 DEBUG:teuthology.orchestra.run.smithi090:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-11T20:55:01.641 DEBUG:teuthology.orchestra.run.smithi092:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-11T20:55:01.653 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-11T20:55:01.663 DEBUG:teuthology.orchestra.run.smithi090:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-11T20:55:01.687 DEBUG:teuthology.orchestra.run.smithi092:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-11T20:55:01.696 INFO:teuthology.orchestra.run.smithi090.stdout:kernel.core_pattern = core 2024-03-11T20:55:01.705 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = core 2024-03-11T20:55:01.734 DEBUG:teuthology.orchestra.run.smithi090:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-11T20:55:01.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-11T20:55:01.754 DEBUG:teuthology.orchestra.run.smithi092:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-11T20:55:01.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-11T20:55:01.783 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-11T20:55:01.793 INFO:teuthology.task.internal:Transferring archived files... 2024-03-11T20:55:01.794 DEBUG:teuthology.misc:Transferring archived files from smithi090:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591231/remote/smithi090 2024-03-11T20:55:01.794 DEBUG:teuthology.orchestra.run.smithi090:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-11T20:55:01.819 DEBUG:teuthology.misc:Transferring archived files from smithi092:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591231/remote/smithi092 2024-03-11T20:55:01.819 DEBUG:teuthology.orchestra.run.smithi092:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-11T20:55:01.847 INFO:teuthology.task.internal:Removing archive directory... 2024-03-11T20:55:01.847 DEBUG:teuthology.orchestra.run.smithi090:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-11T20:55:01.859 DEBUG:teuthology.orchestra.run.smithi092:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-11T20:55:01.891 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-11T20:55:01.903 INFO:teuthology.task.internal:Not uploading archives. 2024-03-11T20:55:01.903 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-11T20:55:01.915 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-11T20:55:01.915 DEBUG:teuthology.orchestra.run.smithi090:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-11T20:55:01.918 DEBUG:teuthology.orchestra.run.smithi092:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-11T20:55:01.921 INFO:teuthology.orchestra.run.smithi090.stdout: 658625 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 11 20:55 /home/ubuntu/cephtest 2024-03-11T20:55:01.939 INFO:teuthology.orchestra.run.smithi092.stdout: 658624 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 11 20:55 /home/ubuntu/cephtest 2024-03-11T20:55:01.940 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-11T20:55:01.954 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-11T20:55:01.966 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: Exception: valgrind error: Leak_StillReachable calloc calloc _dl_check_map_versions 2024-03-11T20:55:01.966 INFO:teuthology.nuke:Checking targets against current locks 2024-03-11T20:55:01.992 DEBUG:teuthology.nuke:shortname: smithi090 2024-03-11T20:55:01.992 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-11T20:55:02.014 DEBUG:teuthology.nuke:shortname: smithi092 2024-03-11T20:55:02.014 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-11T20:55:02.019 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi090.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591231', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-11 18:08:48.894740', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIJuV0sxi+snrwXTRt4/zPLb9SRqvHOFAqDX8Znz88gcSqAlDcU59fZUKUi9A3xYhq5fZD0ebLvwMIUVchiTyn8='} 2024-03-11T20:55:02.036 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi092.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-03-11_14:02:35-fs-squid-distro-default-smithi/7591231', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-11 18:08:48.896341', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBElYpEGkVAh1XHw3GH0jCDOU0ZVV0GfXVejBx1pRjf7vK5vqjzj0CWHmynSSiipAFCvhpWZVecthJQz5iDI3x3Y='} 2024-03-11T20:55:02.067 INFO:teuthology.orchestra.console.smithi090:Power off 2024-03-11T20:55:02.067 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-11T20:55:02.091 DEBUG:teuthology.orchestra.console.smithi090:power off output: Chassis Power Control: Down/Off 2024-03-11T20:55:02.091 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-11T20:55:02.106 INFO:teuthology.orchestra.console.smithi092:Power off 2024-03-11T20:55:02.106 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-11T20:55:02.116 DEBUG:teuthology.orchestra.console.smithi090:check power output: Chassis Power is on 2024-03-11T20:55:02.129 DEBUG:teuthology.orchestra.console.smithi092:power off output: Chassis Power Control: Down/Off 2024-03-11T20:55:02.129 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-11T20:55:02.152 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is on 2024-03-11T20:55:06.117 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-11T20:55:06.153 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-11T20:55:06.228 DEBUG:teuthology.orchestra.console.smithi090:check power output: Chassis Power is on 2024-03-11T20:55:06.263 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is on 2024-03-11T20:55:10.230 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-11T20:55:10.264 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-11T20:55:10.342 DEBUG:teuthology.orchestra.console.smithi090:check power output: Chassis Power is off 2024-03-11T20:55:10.374 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is on 2024-03-11T20:55:10.443 INFO:teuthology.orchestra.console.smithi090:Power off completed 2024-03-11T20:55:14.379 DEBUG:teuthology.orchestra.console.smithi092:pexpect command: ipmitool -H smithi092.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-11T20:55:14.490 DEBUG:teuthology.orchestra.console.smithi092:check power output: Chassis Power is off 2024-03-11T20:55:14.590 INFO:teuthology.orchestra.console.smithi092:Power off completed 2024-03-11T20:55:14.696 INFO:teuthology.run:Summary data: description: fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{ubuntu/{overrides ubuntu_latest}} mount/fuse objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/5 tasks/fsstress validater/valgrind} duration: 9396.40177488327 failure_reason: 'valgrind error: Leak_StillReachable calloc calloc _dl_check_map_versions' flavor: default owner: scheduled_teuthology@teuthology status: fail success: false 2024-03-11T20:55:14.696 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-11T20:55:14.782 INFO:teuthology.run:FAIL